IMPP
A protocol and registry for handing what one AI agent has learned to another, and measuring whether it helped.
Protocol draft (schema v0.2), working reference registry, paper draft. Public repo.
| Role | Year | Kind | Try it |
|---|---|---|---|
| Concept, protocol design, engineering | 2026 | AI infrastructure | impp.sh |
| 544 | live predictions in the first pilot |
| 1 of 4 | model and task combinations got better with shared memory |
| 151 | commits, April 2026 |
The problem
Agents pick up useful things in use: rules of thumb, calibration notes, skills. There is no dependable way to hand that to a different agent, and no way to tell whether it helped once you do.
Two kinds of trust usually get mixed up. A signature tells you who made something and that it has not been tampered with. It does not tell you whether it is any good.
What I built
- A package format. What an agent has learned, packaged as something signed, versioned and testable that another agent can pick up and use.
- Five ways to attach it. The protocol spells out the five ways know-how can be handed to an agent, from slipping it in as instructions to keeping it on a shelf for reference, and what can go wrong with each.
- A referee. It runs the agent with and without the memory, throws hard tests at it (bias, hidden instructions, overfitting, leaked personal data), and signs off on what it found.
- A registry. impp.sh lists the packages, with a command-line tool to install, verify and publish them.
- A trust score. Transfer counts for 50%, adversarial cleanliness 30%, freshness 20%. The weights are provisional.
How it is structured
The verification flow
Submit artifact
Select benchmark
Spawn novice baseline
Spawn referee (disposable, with artifact)
Differential scoring
Adversarial probes
Generate certificate
Update trust score
The eight steps as listed in amm/verification/referee.py.
The artifact: M = (D, K, P, A, H)
| domain | object | Category, subcategory and scope the knowledge applies to. |
| knowledge | object | The content itself, as markdown, JSON or text, with a token count. |
| provenance | object | Which agent and model produced it, when, and from how many training rounds. |
| attestation | object | Measured results: transfer efficiency, expert error, evaluation items, trials. |
| hash | string | SHA-256 content-addressed hash of the artifact. |
| version | string | Schema version. |
Fields of schemas/artifact.json.
In the code
From Beach-Bum/impp, which is public.
def compute_trust_score(
transfer_efficiency: float,
adversarial_score: float,
freshness_pct: float = 100.0,
) -> float:
"""Compute composite trust score.
trust = 0.50 × transfer_efficiency + 0.30 × (100 - adversarial) + 0.20 × freshness
Transfer efficiency is clamped to [-100, 100] — negative TE actively
penalizes the trust score rather than being clipped to 0.
Adversarial and freshness are clamped to [0, 100].
"""
# Allow negative TE to penalize trust score (not clipped to 0)
te_clamped = min(100, max(-100, transfer_efficiency))
return (
TRUST_WEIGHTS["transfer_efficiency"] * te_clamped
+ TRUST_WEIGHTS["adversarial_clean"] * (100 - min(100, max(0, adversarial_score)))
+ TRUST_WEIGHTS["freshness"] * min(100, max(0, freshness_pct))
)
How a run is kept honest
The protocol and the referee are Python with four dependencies (the Anthropic client, cryptography, click, httpx) and adapters for OpenAI and local Ollama models. The registry site is Next.js 15. 151 commits.
A score is only worth something if the thing being scored cannot influence the scoring. Four mechanisms do that.
| Mechanism | What it does |
|---|---|
| A fresh agent every time | Each evaluation starts an agent with no memory, hands it the artifact, runs the tasks and throws the agent away. The same tasks are run by an agent without the artifact; the difference between the two is the transfer score. |
| Hard tests on the artifact itself | Four probes scored by a separate auditor, weighted bias 30%, consistency 25%, hidden instructions 25%, overfitting 20%, plus a pattern check for personal data (emails, phone numbers, card numbers, IP addresses). |
| A signed certificate | The result is signed with an Ed25519 key. The private key file is written readable by its owner only. Certificates expire and can be revoked. |
| A score that can go down | Transfer is allowed to be negative, from -100 to 100. Memory that makes an agent worse lowers the trust score instead of counting as zero. |
How it is tested
36 automated tests in three files.
| Area | Tests | Examples |
|---|---|---|
| The artifact | 7 | The hash is the same for the same content, changes when the content changes, and tampering is detected. |
| Certificates | 10 | A tampered certificate fails, the wrong key fails, an unsigned certificate fails, a signature survives being saved and loaded. |
| Domains, statistics, registry | 19 | Score ranges, confidence intervals, creating and searching listings, proving ownership. |
The pilot's data is in the repository as a reproducibility package: the 40 tasks, their answer keys, the splits, the results and the probe output for a deliberately poisoned artifact. All of it is synthetic.
Two limits, stated plainly. The tests are not yet run automatically on every change; the only automation publishes releases. And 40 synthetic tasks on two models is a pilot, not a benchmark.
What the first test showed
The pilot used Claude and GPT-4o on two problem sets with known answers: 32 DeFi tasks and 8 cybersecurity tasks.
Only one of the four combinations got better. On DeFi, Claude improved by 5.6% and GPT-4o fell by 2.7%. On cybersecurity, Claude fell by 7.6% and GPT-4o by 7.1%.
That is the finding. Handing an agent memory can make it worse, and a signature check would never tell you.
The decision that mattered
The first version, in February 2026, was a marketplace for buying and selling agent memory. This version drops the market and asks a simpler question first: can you package a piece of know-how, hand it to a different agent, and measure whether it helped?
Open questions
Full payloads in the live runs, the six other benchmark conditions, a calibrated trust score, and a package on PyPI.