One agent, three claims, one measuring stick.
Hermes is the most-used AI agent of 2026. Nous Research shipped it in February, and it crossed 100,000 GitHub stars in two months, briefly processing 224 billion tokens in a single day through OpenRouter. Its headline feature is self-evolution. After a task, it writes a SKILL.md, files it away, and claims to get faster and cheaper. The docs put a number on it: skills cut token consumption 30 to 85 percent.
We tested the claims we could reach.
The three claims
Hermes makes three claims we could check.
- It completes the task it is given.
- It remembers across sessions.
- Its skills cut token consumption 30 to 85 percent.
Claim 1: task completion. Held.
Eight diverse tasks. File creation, arithmetic, CSV to JSON, string reversal, sum of squares, a prime check, a file copy, and a multi-step create-and-count. Each with a checkable effect we verified on the filesystem.
Eight correct. Zero wrong. On task completion, Hermes does what it says.
Claim 2: memory. Held.
We planted a preference in one session: name Prakarsh, preferred language Rust. A fresh session recalled both, correctly, from memory.
The “solves AI amnesia” claim held in our test.
Claim 3: token reduction. Did not hold.
This is the number the docs lead with. We measured it three times, cold versus with a preloaded skill.
| task | cold | with skill | change |
|---|---|---|---|
| word count | 41,364 | 54,718 | +32% |
| CSV to JSON | 27,440 | 55,348 | +102% |
| sum of squares | 39,357 | 40,019 | +2% |
Three tasks. Three increases. Zero reductions. The skill adds its text to the prompt and does not cut the spend. The claimed 30 to 85 percent reduction did not appear.
The honest caveat
One thing we could not reach. The skill-writing loop itself is a background process the scripted one-shot mode never runs. So we could not test whether Hermes writes good skills. We could only test the measurable half: does a skill reduce tokens. It did not, in our runs.
What we sign
Verdict: not_falsified, on task completion and memory. The finding documents the token claim: three measurements, three increases, no reduction.
We did not manufacture a catch. Hermes completed every task and remembered what we told it. The interesting part is that its most famous number, the 30 to 85 percent token cut, did not survive a measuring stick. We signed exactly that.
You can check the receipt yourself. The raw JSON is in the public record.