IMPP


A protocol and registry for handing what one AI agent has learned to another, and measuring whether it helped.

Status

Protocol draft (schema v0.2), working reference registry, paper draft. Public repo.

RoleYearKindTry it
Concept, protocol design, engineering2026AI infrastructureimpp.sh
544live predictions in the first pilot
1 of 4model and task combinations got better with shared memory
151commits, April 2026

The problem

Agents pick up useful things in use: rules of thumb, calibration notes, skills. There is no dependable way to hand that to a different agent, and no way to tell whether it helped once you do.

Two kinds of trust usually get mixed up. A signature tells you who made something and that it has not been tampered with. It does not tell you whether it is any good.

What I built

  • A package format. What an agent has learned, packaged as something signed, versioned and testable that another agent can pick up and use.
  • Five ways to attach it. The protocol spells out the five ways know-how can be handed to an agent, from slipping it in as instructions to keeping it on a shelf for reference, and what can go wrong with each.
  • A referee. It runs the agent with and without the memory, throws hard tests at it (bias, hidden instructions, overfitting, leaked personal data), and signs off on what it found.
  • A registry. impp.sh lists the packages, with a command-line tool to install, verify and publish them.
  • A trust score. Transfer counts for 50%, adversarial cleanliness 30%, freshness 20%. The weights are provisional.

How it is structured

From one agent to another: every artifact passes the referee before it earns a trust score.

The verification flow

Submit artifact

Select benchmark

Spawn novice baseline

Spawn referee (disposable, with artifact)

Differential scoring

Adversarial probes

Generate certificate

Update trust score

The eight steps as listed in amm/verification/referee.py.

The artifact: M = (D, K, P, A, H)

domainobjectCategory, subcategory and scope the knowledge applies to.
knowledgeobjectThe content itself, as markdown, JSON or text, with a token count.
provenanceobjectWhich agent and model produced it, when, and from how many training rounds.
attestationobjectMeasured results: transfer efficiency, expert error, evaluation items, trials.
hashstringSHA-256 content-addressed hash of the artifact.
versionstringSchema version.

Fields of schemas/artifact.json.

In the code

From Beach-Bum/impp, which is public.

artifact.json
certificate.json
impp.artifact.v0.2.schema.json
run_verify.py
amm/verification/referee.py
def compute_trust_score(
    transfer_efficiency: float,
    adversarial_score: float,
    freshness_pct: float = 100.0,
) -> float:
    """Compute composite trust score.

    trust = 0.50 × transfer_efficiency + 0.30 × (100 - adversarial) + 0.20 × freshness

    Transfer efficiency is clamped to [-100, 100] — negative TE actively
    penalizes the trust score rather than being clipped to 0.
    Adversarial and freshness are clamped to [0, 100].
    """
    # Allow negative TE to penalize trust score (not clipped to 0)
    te_clamped = min(100, max(-100, transfer_efficiency))
    return (
        TRUST_WEIGHTS["transfer_efficiency"] * te_clamped
        + TRUST_WEIGHTS["adversarial_clean"] * (100 - min(100, max(0, adversarial_score)))
        + TRUST_WEIGHTS["freshness"] * min(100, max(0, freshness_pct))
    )

How a run is kept honest

The protocol and the referee are Python with four dependencies (the Anthropic client, cryptography, click, httpx) and adapters for OpenAI and local Ollama models. The registry site is Next.js 15. 151 commits.

A score is only worth something if the thing being scored cannot influence the scoring. Four mechanisms do that.

MechanismWhat it does
A fresh agent every timeEach evaluation starts an agent with no memory, hands it the artifact, runs the tasks and throws the agent away. The same tasks are run by an agent without the artifact; the difference between the two is the transfer score.
Hard tests on the artifact itselfFour probes scored by a separate auditor, weighted bias 30%, consistency 25%, hidden instructions 25%, overfitting 20%, plus a pattern check for personal data (emails, phone numbers, card numbers, IP addresses).
A signed certificateThe result is signed with an Ed25519 key. The private key file is written readable by its owner only. Certificates expire and can be revoked.
A score that can go downTransfer is allowed to be negative, from -100 to 100. Memory that makes an agent worse lowers the trust score instead of counting as zero.

How it is tested

36 automated tests in three files.

AreaTestsExamples
The artifact7The hash is the same for the same content, changes when the content changes, and tampering is detected.
Certificates10A tampered certificate fails, the wrong key fails, an unsigned certificate fails, a signature survives being saved and loaded.
Domains, statistics, registry19Score ranges, confidence intervals, creating and searching listings, proving ownership.

The pilot's data is in the repository as a reproducibility package: the 40 tasks, their answer keys, the splits, the results and the probe output for a deliberately poisoned artifact. All of it is synthetic.

Two limits, stated plainly. The tests are not yet run automatically on every change; the only automation publishes releases. And 40 synthetic tasks on two models is a pilot, not a benchmark.

What the first test showed

The pilot used Claude and GPT-4o on two problem sets with known answers: 32 DeFi tasks and 8 cybersecurity tasks.

Only one of the four combinations got better. On DeFi, Claude improved by 5.6% and GPT-4o fell by 2.7%. On cybersecurity, Claude fell by 7.6% and GPT-4o by 7.1%.

That is the finding. Handing an agent memory can make it worse, and a signature check would never tell you.

The decision that mattered

The first version, in February 2026, was a marketplace for buying and selling agent memory. This version drops the market and asks a simpler question first: can you package a piece of know-how, hand it to a different agent, and measure whether it helped?

Open questions

More

Source on GitHub
Open ↗
Project notes on the weblog
Open ↗