Evaluating a Skill Before Install: Trust Signals

+15 Mana ✨
Hermes Skills Trust Levels(DOCUMENTATION)

Introduction

A skill is content that goes directly into your agent's prompt. Installing one is closer to importing a system prompt than to running pip install. That makes evaluation important. Hermes gives you several signals (and a security scanner) to help. Knowing what to look at and what to skip is the difference between sensible reuse and a self-inflicted prompt injection.

Key Concepts

  • Trust level: A classification Hermes attaches to every install source (builtin, official, trusted, community).
  • Security scan: An automatic check Hermes runs on every hub-installed skill, looking for data exfiltration, prompt injection patterns, destructive commands, and supply-chain signals.
  • hermes skills inspect: The CLI command that fetches a skill's metadata, full SKILL.md, and upstream signals without installing.
  • --force: An install flag that overrides non-dangerous policy findings. It does not override a dangerous verdict.

Real World Context

A developer sees a community skill on skills.sh called git-history-rewrite. The description sounds useful. Before installing, they run hermes skills inspect skills-sh/some-author/git-history-rewrite. The output shows the source repo, weekly install count, and the body of SKILL.md. They read the body and notice it instructs the agent to run git push --force after rewriting. They decide that is too aggressive for shared branches and pass.

The whole exchange takes 30 seconds. That is the loop you want to internalize.

Deep Dive

Useful signals fall into four groups.

Provenance. Where does the skill come from? inspect shows the source repo URL, the install identifier, and (for skills.sh entries) a detail page. Skills from known taps (openai/skills, anthropics/skills, huggingface/skills) get trusted status; everything else starts at community.

Popularity. Weekly installs and the audit status surface in the inspect panel. Popularity is not safety, but a skill that thousands of people have installed without complaint has had more eyes on it than a one-author-one-install skill.

Body content. The most reliable signal is reading the actual SKILL.md. Look for:

  • Destructive shell commands you did not expect (rm -rf, force pushes, DROP TABLE).
  • Instructions to upload files or environment variables to external endpoints.
  • Vague instructions that pull in arbitrary URLs at runtime.
  • Hidden meta-instructions in the description trying to override your system prompt.

If the procedure surprises you, do not install it.

Security verdict. Hermes runs a scan and tags each install with safe, caution, warn, or dangerous. --force overrides caution and warn once you have read the report. It does not override dangerous. That gate is hard.

bash
# Inspect before installing
hermes skills inspect skills-sh/vercel-labs/json-render/json-render-react

# Install with default policy (will block dangerous, prompt on warn)
hermes skills install skills-sh/some-author/some-skill

# Override a non-dangerous finding after manually reviewing
hermes skills install skills-sh/some-author/some-skill --force

Official optional skills (official/...) ship with builtin trust and skip the third-party warning panel. The scanner still runs (the policy is one defense among many), but the skills have been reviewed by Hermes maintainers.

Common Pitfalls

  1. Trusting the description: A description like formats your code could hide a procedure that exfiltrates files. Always read the body before installing a community skill.
  2. Reflexively passing --force: --force is for cases where you understand the finding and accept it. Reaching for it on the first prompt defeats the gate.

Best Practices

  1. Triangulate trust: A skill that is trusted source, well-known repo, and reads cleanly is a green light. Any two of those without the third is a yellow light.
  2. Pin to a tap you control: If your org publishes a tap of vetted skills, install from there preferentially. Your review is the trust signal.

Summary

  • Trust levels (builtin, official, trusted, community) classify the source.
  • hermes skills inspect shows the SKILL.md plus upstream metadata before any install.
  • The security scanner classifies findings as safe, caution, warn, or dangerous. --force overrides the first three; dangerous is hard-blocked.
  • The most reliable signal is reading the body of the SKILL.md yourself.

Code Examples

bash
# Full inspection workflow before installing
hermes skills inspect openai/skills/k8s

# Output (illustrative): 
#   name: k8s
#   description: Production-grade Kubernetes workflows
#   source: github (openai/skills)
#   trust: trusted
#   weekly_installs: 4,213
#   audit: safe
#   body: # Full SKILL.md content rendered to terminal

# Only install after reading the body and accepting any findings
hermes skills install openai/skills/k8s
✓ Completed