How the AI Change Evidence Engine is built
Read this in order. Each chapter starts with the engineering question, explains the idea in plain language, and shows exactly how that idea appears in this product.
Every implementation choice exists to preserve the meaning and provenance of a fact as it crosses those boundaries.
Start with the decision
Understand
A system should begin with an actor and a decision, not a list of technologies. Here the actor is a second-line reviewer. The unit of work is one completed material change. The outcome is a defensible review, not automatic approval.
How we build it
Write the reviewer question and the product boundary before creating a schema. This prevents a general governance dashboard from swallowing the project.
actor = "second-line reviewer"
object = "one completed material change"
outcome = "review-ready evidence trace"Model facts before screens
Understand
GitHub, ServiceNow and monitoring tools use different words. The engine needs one stable EvidenceRecord that preserves source, reference, time, status, lineage and correlation identifiers.
How we build it
Define the domain types as the contract. The UI, database, adapters and tests should all speak this language.
EvidenceRecord {
sourceSystem, sourceRef, observedAt
status, lineage, correlationIds
requirementIds, detail
}Make policy data
Understand
A requirement is configuration, not a hidden if-statement. Each requirement names accepted source statuses, severity and a stable identifier. A profile bundles and versions those requirements.
How we build it
Keep the profile separate from the evidence. That lets the same facts be assessed against a new approved profile without rewriting their history.
{
id: "security.dast",
severity: "required",
acceptedStatuses: ["passed"]
}Use a pure evaluator
Understand
The policy function does no network or database work. It receives a passport and returns an evaluation. Pure functions are easier to test, explain and replay.
How we build it
Use three states: satisfied, incomplete and attention. Missing is not failed. Even complete evidence only becomes ready for human review.
no evidence → incomplete
accepted status → satisfied
other status → attention
unresolved > 0 → human decision requiredDefend the ingress boundary
Understand
Never parse first and verify later. Read the exact raw body, bound its byte size, verify GitHub's HMAC signature, then parse and validate only the supported fields.
How we build it
Fail closed when the secret is missing. Reject invalid signatures. Return retryable errors when storage is unavailable. Do not log the raw payload.
raw bytes
→ size limit
→ HMAC verify
→ JSON parse
→ schema checks
→ normaliseNormalise narrowly
Understand
Merge, review and approved scope are separate claims. A merged pull request creates merge and declared-reference facts; an approved review arrives separately and is tied to its exact commit.
How we build it
Accept only exact workflow names from a demonstration allowlist. Unknown, artefact-named and deployment-named workflows cannot satisfy a requirement from their conclusion alone.
pull_request.closed → code.merge + change.reference
pull_request_review → code.review
allowlisted workflow → typed result
unknown workflow → ignoredDesign retries as normal
Understand
A delivery first enters processing. A retry resumes the same stable identities; a completed duplicate only updates attempt metadata, and conflicting bytes under the same ID are rejected.
How we build it
Use source time for deterministic snapshots. A D1 failure returns 503, while delivery-identity conflict returns 409.
processing → normalised
same delivery + same bytes → converge
same delivery + new bytes → 409
storage failure → 503Correlate without guessing
Understand
Repository, pull request, exact head SHA and an explicit Change-Ref anchor the release. Workflow evidence attaches only when repository and SHA match. Missing identifiers remain visible gaps.
How we build it
Persist typed correlation keys and keep them visible. Never use an LLM or fuzzy match to invent assurance joins.
repository + PR + Change-Ref → release
repository + exact head SHA → workflow link
missing explicit key → unresolvedBuild for review, not reporting
Understand
The first viewport answers the reviewer question, shows coverage and exposes the unresolved requirement. Selecting a row reveals provenance and the deterministic reason for its status.
How we build it
Keep the gap prominent. Do not hide it in a score. Use language that preserves human authority and avoids compliance claims.
summary → evidence trace → provenance
↓
human interpretationBe precise about integrity
Understand
A digest can show that the same canonical bytes produce the same fingerprint. It does not identify who created the record or prove the upstream facts are genuine.
How we build it
Sort object keys recursively, hash the evidence envelope and label the result unsigned. Production authenticity requires asymmetric signing keys and an independent verifier.
canonical JSON → SHA-256 → digest
detects modification ✓
authenticates source ✗Test decisions and abuse cases
Understand
The most valuable tests target business semantics and trust boundaries: missing evidence, failed workflows, modified webhook bodies, explicit reference extraction, data minimisation and digest changes.
How we build it
Unit-test pure logic first. Then validate the worker build, live empty state, Event Inspector, filters and responsive layout. A green UI without policy tests is not enough.
policy + assembler tests
HMAC abuse tests
multi-fact adapter tests
canonical export testsName the production gap
Understand
A credible engineer distinguishes implemented code from a production service. Live use needs tenant isolation, identity, roles, retention, reconciliation, audit access, monitoring, recovery and managed signing keys.
How we build it
Keep a readiness checklist and threat model. Never let a portfolio demonstration silently become a production claim.
demo → design partner → controlled pilot → production
evidence at every gateIf you can explain these five things, you understand the engine.
- Why evidence and policy are separate data structures.
- Why a webhook signature must be checked against the raw body before parsing.
- Why missing evidence is not the same as a failed control.
- Why stable IDs make retries safe.
- Why a digest is not a digital signature.