opshero

Abstract

An agent reports that the tests pass. They may have passed on an earlier version of the change. opshero runs your check, ties the result to the git tree it ran on, and later answers one question from the recorded facts: did the required checks pass on this exact tree? The answer is PASS, FAIL or UNKNOWN. It is a function of the policy, the content and the recorded evidence, so it can be recomputed later and gives the same answer.

It is a single static binary. It has no account, no telemetry and no machine-learning model. It certifies a process, not the quality of the work (section 5).

1. Problem

Coding agents now write a large share of changes, and the bottleneck moved from writing code to trusting it. Two failures are cheap for a machine to detect and expensive for a reviewer to notice.

1.1 Stale evidence

“Tests pass” was true for an earlier version of the change, not for the one in the pull request. opshero detects this today.

1.2 Unexercised change

The checks that ran never touched the files that were modified. opshero does not detect this yet; it needs the files a run read to be observed, which is not built (section 8).

2. Synopsis

A repository with one file, a.py. The policy requires python3 -B * to pass. The output below is copied from a real run of the v0.0.1 build on Linux.

The tests run, then the verdict is asked for:

$ opshero run -- python3 -B -c "import a"
$ opshero verify
subject: tree 0465605cd01d  (head b432a6fef, clean)
policy:  default
  PASS    tests-on-final-tree: `python3 -B *` passed on the current tree (1 run(s))
verdict: PASS   assurance: A1

One line is added. The tree id, a hash of the whole working tree, changes, so the earlier run no longer counts:

$ echo 'x=2' >> a.py
$ opshero verify
subject: tree ef22ae1c5732  (head b432a6fef, dirty)
policy:  default
  FAIL    tests-on-final-tree: stale evidence: `python3 -B *` last passed on tree
          0465605cd01d, the current tree is ef22ae1c5732
verdict: FAIL   assurance: n/a

Running the tests again on the new tree gives a fresh record, and the verdict returns to PASS with the same command, the same policy and the new tree id.

3. Verdicts and exit status

Exit status 2 is a usage error and 1 an internal error.

4. Assurance levels

A record written on your own machine can be rewritten by whoever controls that machine. Only independent execution resists that, so every verdict carries the level of its weakest required evidence.

Records imported from another machine are capped at A1 on every read.

5. Scope

It certifies that these commands ran, with these exit codes, on this exact content.

It does not certify that the change is correct, that the tests are good, or anything about security or intent. If a sentence on this page claims more, that is a defect in the sentence and worth reporting.

6. Design rules

1. The verdict never depends on the network. The network only moves evidence: opshero evidence push publishes records to a git ref and opshero verify --ci fetches them. A gate therefore does not fail because a service is down, and anyone can recompute a verdict years later.
2. Evidence is bound to content, not names. The subject is a git tree id, computed without touching your index.
3. Fail closed. Anything it cannot establish is UNKNOWN, never a pass.
4. No account, no telemetry, nothing sensitive stored. Command output and environment values are never recorded.
5. No machine-learning model. A verifier has to be more trustworthy than what it verifies, so it is deterministic and auditable.
6. Policy is data. A small TOML file, reviewed like code, not a programming language.

Each rule is a decision record with its alternatives, linked in section 10.

7. Testing and measurements

8. Status

Pre-alpha. Demand for this tool is not proven; the results that would make us stop are written down in advance in the validation plan.

8.1 Implemented

• Evidence records bound to a git tree; PASS, FAIL, UNKNOWN policy verdict
• Claude Code adapter, with an optional gate that stops the agent from finishing on stale evidence
• Flaky commands (--repeat)
• Evidence transport, verify --ci, CI-executed evidence (A2)
• Linux and macOS binaries; a Windows binary (experimental)
• opshero doctor, shell completions, a manual page
• A GitHub Action that runs the gate and writes the verdict to the job summary

8.2 Not implemented

• Detection of unexercised changes (section 1.2)
• Signed attestations (A3)
• Codex and Cursor adapters
• Windows as a supported platform: it installs and passes its tests in CI, but has not been tried with a real agent
• Publication on crates.io

9. Installation

Read the installer before running it. It checks the archive against the release checksums and installs to ~/.local/bin without root.

$ curl -fsSL https://raw.githubusercontent.com/NICE-DEV226/opshero-core/main/scripts/install.sh -o install.sh
$ less install.sh
$ sh install.sh
$ opshero init
$ opshero run -- cargo test
$ opshero verify

opshero init writes .opshero/policy.toml; set the command your project requires, and opshero doctor checks the setup. Release archives carry build provenance: gh attestation verify <archive> --repo NICE-DEV226/opshero-core.

On Windows (experimental), scripts/install.ps1 does the same and installs opshero.exe for the current user.

9.1 In CI

On a pull request, the Action runs the checks the policy requires in the CI job itself (assurance A2), using the policy of the target branch, and fails the step on anything but PASS:

- uses: actions/checkout@v7
  with:
    ref: ${{ github.event.pull_request.head.sha }}
- uses: NICE-DEV226/opshero-core@v0.0.2

Details and threat model: CI gate.

10. References

• Command-line reference
• Using it as a CI gate
• RFC 0001: principles, threat model, assurance levels
• RFC 0002: evidence record and attestation format
• RFC 0004: policy language
• Architecture decision records
• Roadmap
• Releases
• Report a vulnerability privately

Apache-2.0. This page sets no cookies and loads no scripts, fonts or analytics.