Deep dive · Part 2 of 2AI accountability

Applying the seven criteria: an assessment of SocioProphet's agent authority chain

An independent assessment of one AI governance architecture against seven procurement criteria for government buyers - including where it does not yet meet the standard.

personKYROGA consultant
schedule21 min read
calendar_today

I have spent more than thirty years in vendor-led technology sales and consulting. I know what the current moment looks like from the other side of the table, because I have been on that side of it.

Here is the play, and it is not cynical to name it. When a category moves as fast as agentic AI has, the most effective sales narrative available is fear. Not invented fear — the risks are real, and the regulatory pressure on regulated organisations is real. But fear is useful commercially in a particular way: it compresses the decision. An organisation that needs to demonstrate risk management, due diligence and testable governance to a board or a regulator needs to demonstrate it this year, and a vendor with a credible answer to this year’s problem will be rewarded for it.

What that framing does not surface — because it is not in the seller’s interest to surface it — is the second question. Not “does this solve the problem in front of me,” but “what am I committing to in two years, and in five.”

The answer, more often than it should be, is total dependency. Not through malice or fine print, but structurally: the governance layer wires itself into the workflows, the evidence lives in the supplier’s systems in the supplier’s format, the knowledge accumulates on the supplier’s side of the boundary, and by year three the cost of leaving is not a migration project but a re-certification of everything the system touches. The commercial terms that made the decision easy in year one are renewed under conditions the buyer no longer sets.

The account manager who sold it may well be gone by then. The contract and the interconnected dependencies will not be.

That is the problem the companion piece was actually about, underneath the procurement language. It set out a three-question test for any AI system a government agency is considering — was the agent permitted to act, under which policy in force at that moment, and can you reproduce that decision exactly, for an auditor, two years from now — and then seven criteria a system would have to meet to answer the third one honestly. Those criteria are not a compliance checklist. They are a test of whether what you build stays yours.

Criteria are easy to write and harder to meet, so this piece does the obvious next thing: applies them to a real architecture and marks each one off, including the ones where this vendor is not distinctive and the market has caught up.

Disclosure: KYROGA AI works with SocioProphet. That is exactly why this piece separates what I verified from the repository record from what I took on the company’s word, and says which is which. A reader can weigh it accordingly. Given what I have just written about vendor incentives, an assessment that did not draw that line would not be worth publishing.

Introducing SocioProphet

SocioProphet builds governed AI infrastructure for organisations that cannot let their reasoning, their data, or their decisions leave their control. It is not a wrapper over a frontier model, and not a governance dashboard bolted onto someone else’s platform. In their architecture the governance is the construction — which is the reason the properties described below exist at all, and equally the reason they could not have been added later.

The company organises its thinking around three commitments that recur throughout this assessment: that an organisation’s knowledge should remain sovereign to it, that governance is infrastructure rather than paperwork, and that the point of both is knowledge which compounds into an owned asset rather than dissipating into a supplier’s platform. The bulk of this article concerns the second of those. The final section explains why the other two are what actually matter.

Two engagement platforms matter to a reader coming from the companion piece. Noetica is the individual user interface. Prophet Platform is the organisational governance and control plane. Both are running software rather than design intent, and the assessment below was made against working deployments as well as against the code.

Running through both is a cross-repository standard the company calls the Prophet Trust Chain, and that standard is the actual subject of this article. Each component owns a declared slice of it: policy profile, route selection, action admission, grant authority. It is not a module that gets enabled and not a product bought separately.

That last point connects directly back to the procurement argument in the companion piece. The trust chain is how the system is constructed, which is why these properties cannot be switched off — and why an architecture built without them cannot acquire them in a later release. This is what it means for evidence capability to be fixed on the day a contract is signed.

What the market has already solved

Before crediting anyone with anything, it is worth marking what is no longer distinctive — and being equally precise about what still isn’t there.

A year ago it was fair to say agent governance was universally bolted on outside the execution path and could therefore only observe. That is no longer true, and any vendor still saying it is either behind or hoping their buyer is.

AWS Bedrock AgentCore Policy is a real in-path admission control, and a more capable one than it was even a few months ago. Every MCP tool call through an AgentCore Gateway is intercepted and evaluated against Cedar before it reaches the tool, on a default-deny basis, with enforcement at the Gateway rather than inside the agent’s own code. AWS has also added Dogwood, an open-source policy language layering session-aware conditions over Cedar’s per-call rules, so a decision can weigh what an agent already did earlier in the same session. Coverage is specifically tool calls — prompt and resource requests are always permitted regardless of policy mode, per AWS’s documentation. Evidence is policy evaluations and decisions recorded through CloudWatch logs and metrics.

Microsoft Entra Agent ID gives agents first-class identities, sponsorship, lifecycle governance, Conditional Access and identity-protection risk signals — and, per current documentation, an explicit migration path for agents built on ordinary service principals, including agents running entirely outside Azure, to be brought under that governance with multi-cloud visibility. That is a stronger position than identity-only or Microsoft-only, and anyone still describing it as either is out of date.

Both are genuine engineering. Neither should be described as narrower than it now is. The question this piece is asking is a different one, and it is not answered by either product getting better at what it does.

Assembled governance, and integrated governance

Here is the distinction that survives every feature release either platform ships.

Attestation is not a capability. It is a claim about completeness — that this record is the whole record, that nothing relevant sits outside it. Making that claim requires a single authority over the entire chain. And that is where assembled governance runs into a structural limit that has nothing to do with the quality of the parts.

Consider what answering question three actually requires on a hyperscaler stack. Admission was decided at a gateway. The decision was logged to a monitoring service. The acting identity was issued and risk-assessed by a separate identity plane, under separate licences. The model version came from a managed inference service. What code was running is in a build system nobody in that chain is looking at. Every one of those components may be excellent. To reconstruct one decision you must join five records, across systems with different identifiers, different clock behaviour, different retention windows and no shared schema — and no vendor owns the join, because the join spans two vendors and your own infrastructure.

The retention point is not hypothetical. Microsoft’s own documentation puts default Entra sign-in log retention at thirty days and recommends a separate SIEM product to extend it. So the identity half of an evidence chain, for a question asked two years later, depends on another product having been purchased, configured and correctly wired before the decision was made. That is a reasonable engineering posture for an identity platform. It is not a foundation for attestation.

The fragmentation is visible in how the capability is sold, which is the clearest available evidence that it is composed rather than integrated. Microsoft’s own licensing guidance lists conditional access for agents, identity protection for agents, identity governance for agents and network controls for agents as separately entitled capabilities, layered on a per-user agent licence and an enterprise base plan. I raise this not as a cost objection — enterprise software is priced how it is priced — but because governance assembled from four separately licensed products and two clouds cannot produce a single attestable record, and no amount of further capability changes that. Each new control is another part to join.

SocioProphet’s architecture is the other shape. Admission, identity scope, policy evaluation, routing, code provenance and the decision record are slices of one declared standard inside one offering. That is why a single decision record can bind inputs, model version, policy version, evaluation result and reasoning trace as one object rather than five correlated ones. Not because the components are individually better than Cedar, which they may well not be. Because there is no seam between them for evidence to fall through, and one authority stands behind the whole record.

This is the difference between a system that can show you five true things and a system that can attest to one complete thing. Auditors ask for the second.

The governing principle: no invisible authority

Probabilistic intelligence requires deterministic authority. An agent’s reasoning can stay probabilistic — that is what makes it useful — but what it is allowed to do cannot be left to depend on it reasoning correctly every time. The design rule underneath SocioProphet’s architecture follows directly from that: no component may hold a risk-bearing capability without declaring it, and no declaration is taken on trust.

Any part of the system that opens a network connection, stores a credential, runs an agent, or routes traffic to a model must declare that capability in a machine-readable file — TRUST_SURFACE.yaml — validated against a schema by a check that runs in continuous integration. A component that quietly acquires a capability and does not declare it does not get a warning. It fails the build.

That detail is worth sitting with, because it is the one I found hardest to find anywhere else. The governance invariant is a build gate, in the same pipeline as the tests, at the same severity. Not a policy document. Not a quarterly review. A build that goes red.

It is also why the rest of the chain can be trusted. Every downstream control depends on knowing what capability actually exists, and this is the mechanism that keeps that inventory honest.

Following one action through the chain

Stage 1 — May this connection exist at all?

Before any call is made, there is a prior question: is this agent permitted to be bound to this tool, this provider, this other agent, at all?

That question gets harder once more than one agent is involved. Consider a human delegating an objective to Agent A, which asks Agent B for information, which asks Agent C to act on an external system. Every hop can be individually valid — each agent holds a legitimate identity, each call is technically well-formed — and the resulting action can still exceed what the human ever authorised. Identity at each hop answers who is acting. It does not answer whether the authority behind the action legitimately propagated from the original objective through every step that produced it. Policy engines can now reason over a lot within one boundary; session-aware conditions mean a gateway is no longer blind to what happened five minutes earlier in the same session. What that doesn’t reach is provenance across a chain of independently governed agents.

The component that owns this decision holds grant, decision and ledger contracts, attestations, and profiles of capability that may never be granted at all — across the range of protocols agents actually use to reach each other, rather than one vendor’s.

What it deliberately is not is a gateway, and that distinction is the point. A gateway sits in the middle of traffic and decides what passes, which means its coverage is exactly the traffic routed through it. An authority model decides whether the binding may be established at all — earlier, not contingent on which path the traffic later takes, and not contingent on how many agents the request passed through to get there.

Stage 2 — Which agent is this, and what is it holding?

A registry of agent specifications, identities, sessions, memory, tool grants and runtime authority, with revocation that takes effect at the point of action rather than whenever a token happens to refresh next. An agent whose authority is withdrawn stops being able to act, not when its credentials expire, but immediately.

Stage 3 — What does policy say about this data?

Data-protection policy expressed as code rather than prose. Classification, handling rules and redaction obligations are versioned, testable and evaluated at the moment of use — rather than written in a standard and interpreted, differently, by whoever reads it.

Stage 4 — Who may see this, across a boundary?

Most access control assumes one organisation with one policy authority. Cross-organisational work breaks that immediately — and for government, where information moves between agencies with genuinely different clearance boundaries, it breaks it on day one.

This layer enforces scope-bounded sharing with a policy veto on merges: a combination of data that would cross a boundary in a way policy forbids is refused, not flagged for someone to review later. Where information legitimately must cross, export is consented and minimised rather than wholesale.

Stage 5 — Admit or refuse

The gate itself. Every decision produces a named, versioned record with its own evidence log, tested across multiple runtime versions, and it fails closed when policy cannot be loaded. The versioning matters more than it looks: a decision captured today has to still be readable in five years, and a record format that drifts silently breaks reproduction even when every byte of data was retained.

That last property deserves its own paragraph, because it is a distinction most discussions skip. There are two different failure cases. The first is policy evaluating and denying — no matching permit, request refused. Default-deny handles that, and Cedar handles it elegantly. The second is the governance layer itself being unavailable: policy store unreachable, document malformed, evaluation timing out. That is the condition present during an incident, and it is where a control and a control-shaped object diverge. This one steps down rather than pressing on: when it cannot verify, it reduces what it is willing to do instead of continuing at full capability on guarantees it can no longer honour. That is a deliberate design principle rather than an error path, and it is the same instinct as a system that declines to answer rather than inventing one.

It is a question worth putting to every vendor in the category, SocioProphet included. They have published their answer. It is reasonable to ask for everyone else’s in writing.

Stage 6 — Route, or decline to route

Inference has to go somewhere, and where it goes is itself a governed decision — weighed against cost, latency, quality, privacy and how much confidence the system has in its own evaluation.

Stated plainly: where the evidence required for a governed routing decision is incomplete, the system does not fall back to a best guess. It declines. That behaviour is captured as a test, so it fails the build if it ever regresses.

Stage 7 — Can you reproduce it exactly?

Everything above governs the action. This governs the aftermath, and it is the part built for question three.

The platform connects to whatever AI tooling an organisation already uses — assistants, development environments, agent frameworks — without bespoke integration work, and every request that comes back carries its evidence with it rather than leaving the record somewhere else to be collected later.

The behaviour worth noting is what it refuses to do. Where a result cannot be grounded, it declines to produce one rather than returning a confident answer with nothing underneath it. Where it does answer, it returns a sealed, checkable package rather than raw reasoning a reader has to take on faith.

The part that matters most is smaller than it sounds, and it is the honest label. Every record states plainly whether that particular decision can be exactly reproduced, or whether only partial evidence exists, or whether reproduction would be best-effort. The system does not present all three as though they were the same thing. That is the direct answer to the replay divergence problem described in the companion piece: rather than handing back a result and leaving the consumer to assume it is reconstructable, it says how much weight the record actually bears.

Most systems cannot make that distinction, which means they assert everything with equal confidence — including the records that would not survive an auditor pulling on them.

The word “exactly” in question three is doing precise work. It is the difference between a system that can describe what it did and one that can run it again.

Underneath — supply chain, provenance and drift

Three things sit beneath the chain, and none of them are visible in normal operation. They matter because criterion five has no floor without them.

The first tracks where knowledge came from. Every input carries its lineage and classification, verified at each point it is promoted into use rather than asserted once at ingestion. An organisation that cannot say where a piece of information entered its system cannot defend a decision that rested on it.

The second answers the question “what code was actually running when this happened” cryptographically rather than administratively. Versions are pinned and hashed, so the software state at the moment of a decision is a verifiable fact rather than a deployment note someone wrote down afterwards. Reproduction is impossible without this — you cannot re-run a decision if you cannot establish what you would be re-running.

The third keeps the whole estate coherent. Each part of the system has a declared jurisdiction and a declared limit on what it is entitled to claim, with a generated report showing where coverage is complete and where it is not. That report is as useful for its gaps as for its confirmations. Alongside it runs an adversarial testing regime whose purpose is to attack the system’s own claims — to find the places where an assertion outruns its evidence, and fail the build when it does.

That last point is the same discipline this article is written under. A system built to make claims defensible has to be willing to have its own claims attacked first.

The seven criteria, marked

Here they are in order, with an honest note on which actually separate anything.

One — admission before action, in the execution path. Met by the admission gate at stage five, and met by others. AgentCore Policy does this well. Table stakes now, and any vendor who cannot answer it should be eliminated early rather than argued with.

Two — fail-closed when the governance layer itself fails. Met, and distinct from default-deny. The gate refuses on policy-load failure rather than proceeding with a warning. A narrower property than it sounds, and a different question. Ask it separately.

Three — policy expressed as versioned code, not documents. Met at stage three. Increasingly common; Cedar is a good answer to the same requirement. Table stakes, trending.

Four — authority declared and validated before deployment. Met by the build-time declaration gate. This axis appears genuinely unoccupied, and the clearest evidence for that comes from Microsoft’s own remediation guidance rather than from anything a competitor omits.

Microsoft documents a four-phase process for bringing existing agents under governance: discover, classify, migrate, validate. Discovery involves scanning service principals for behavioural signals, matching names against patterns like “agent” and “copilot,” reconciling against a configuration database, and — where automated methods fall short — asking developers to self-attest whether their workload is an agent. The documentation notes to expect false positives, and states plainly that there is no in-place conversion: each agent must be migrated deliberately, with new identities created and application code updated.

This is a well-designed process and any large organisation will need something like it. But read what it implies. Agents get built and deployed first, discovered second, governed third. The governance gap between deployment and enrolment is not a defect in the tooling — it is the model. Detection plus manual remediation is what you build when capability can exist before it is declared.

The alternative is to make undeclared capability impossible rather than discoverable. There is no discovery phase because there is nothing to discover — an undeclared agent does not reach deployment. Neither hyperscaler offers an equivalent, and structurally neither can: they govern workloads running on their infrastructure, not the pipelines that produce them. That is not a gap they have overlooked. It is outside what either product is scoped to reach.

Five — decision records that re-derive rather than describe. Met at stage seven, and the one that matters most. The prevailing pattern across the market is admission paired with observability. Observability tells you an action occurred and a policy evaluation returned a result. It does not reliably bind enough to reproduce a decision once a model version shifts underneath it. The honest per-record label answers that as a stated property rather than an assurance.

Six — a cryptographic answer to “what code was running.” Met by pinned, hashed version manifests with independent ledger verification. Without this, criterion five has no floor.

Seven — portability across the estate. Met by an authority model that spans the protocols agents actually use and lives with the workload rather than in a cloud account, so the control travels with what it governs.

Restated as the procurement question: AWS’s gateway, even with session-aware temporal policy, governs what is routed through it. Microsoft’s identity plane, even with its multi-cloud migration path, governs what has been deliberately onboarded to it — and by Microsoft’s own account, onboarding is neither automatic nor retroactive. Both are real. Both are improving faster than most buyers are tracking.

What neither claims, and neither is built to do, is carry a single provable chain of authority across both boundaries at once. For an agency running Microsoft 365, a Bedrock workload, an assistant over MCP and two vendors’ embedded agents, that is the difference between one attestable record and several partial ones with seams between them. And closing that gap fully would require one hyperscaler to govern infrastructure it does not own — a commercial decision neither has made, rather than a technical problem either has failed to solve. Which means the seam is durable, and an agency should plan for it rather than wait for it to close.

Two of seven are table stakes. One is trending that way. Four still separate, and one of those four — reproduction — is the one an auditor will actually ask about.

What I can attest to, and what I can’t

This is the part most vendor-adjacent writing omits, which is precisely why it is here.

What I verified from the repository record: the components described above exist as code, with the specific artifacts identified — the versioned decision record, the declined-routing test, the capability declaration schema and its validator, the build-time workflows, the software bill of materials, the evidence receipts. I assessed these against the code and against working deployments of both platforms.

What I took on the company’s word: that exact reproduction holds under a substituted model version. This is criterion five, it is the load-bearing claim in this entire assessment, and I confirmed it with SocioProphet’s engineering lead rather than by running it myself. Anyone evaluating seriously should ask to see it demonstrated — and I would treat how readily a vendor produces that demonstration as part of the evidence, this one included.

What integration costs, since I have just argued for it: a single integrated trust chain means a single vendor in the critical path of your governance, a smaller ecosystem than either hyperscaler’s, and an integration that has not been independently audited by anyone outside the company. Cedar has formal analysis tooling and a public research trail behind it; the components described here do not, yet. A buyer trading assembled governance for integrated governance is trading joinable-but-fragmented evidence for a complete record from a much smaller supplier, and should price that trade rather than assume it away. The argument I am making is that the complete record is worth more for question three specifically — not that the trade is free.

What nobody should claim yet: that this has been proven at whole-of-government scale. The existence of a component is not the same as conformance to a requirement, and the two should not be blurred. Load characteristics at very large user counts are an open question. There is no sovereign hosting accreditation, no regional data residency, and none of the public sector security certifications a production deployment would require. Those gaps are real and the company does not dispute them.

On that last point one distinction is worth drawing, rather than leaving a reader to draw it unaided. The companion piece observed that government procurement tests security posture, residency and sovereignty first — and measured against that gate, this is not today a buyable proposition for a classified workload. But those gaps and the architectural ones are not the same kind of thing. Accreditation is acquirable on a schedule: the work is known, the cost is bounded, and a vendor without it can obtain it. Reproduction is not acquirable that way. A system built without it has to have its execution path rebuilt, which is the argument the companion piece makes about evidence capability being fixed at signature.

SocioProphet’s certification sequence is defined and gated on a named engagement rather than pursued speculatively — a defensible position for a company at this stage, and a fair thing for a buyer to interrogate directly. The asymmetry is the point. One of these gaps closes with time and money. The other closes with a rewrite.

What I will not assert about anyone else: what competing platforms do when their governance layer is unavailable rather than when policy simply denies. That is a different question from the boundary limits described above — those are documented in each vendor’s own materials; failure-mode behaviour is not, and I have not tested it. It is the right question to ask. It is not one I am going to answer on their behalf.

The reason for setting all this out is that the failure mode this architecture exists to prevent is a system asserting more confidence than its evidence supports. An assessment applying a lower standard to itself than to its subject would be worth nothing.

The wider point

Everything in this article has been about foundations, and foundations are a difficult thing to argue for. They are expensive, invisible once the building is occupied, and nobody has ever walked into a room and admired them. A structure raised without them does not usually fail immediately either. It fails progressively, under its own accumulating weight, at the point where enough has been built on top that correcting it means taking the whole thing down.

That is roughly where enterprise AI is now. The weight is accumulating fast — more agents, more autonomy, more consequential actions taken without a human in the path — and the foundation question is being deferred because deferring it is cheaper this quarter.

But the foundations are not the point of the building, and it would be a strange article that argued otherwise. Nobody commissions a structure for its footings. The return is what accumulates inside.

For an organisation using AI, what accumulates is knowledge — or it doesn’t, and that turns out to be the whole distinction. Every question asked, every document reasoned over, every decision made and rationale recorded is a deposit. Where those deposits land inside the organisation’s own boundary, they compound: the system gets progressively better at that organisation’s particular work in a way no general model can match, because no general model has seen it. Where they dissipate into someone else’s platform, the organisation has paid to improve an asset it does not own, cannot audit, and cannot take with it when the contract ends.

That is why sovereignty is not a security preference. It is the difference between renting intelligence and building one.

And it is where the foundation argument stops being abstract, because compounding requires retention, and retention requires that what accumulated can be stood behind. Knowledge an organisation cannot trace, cannot reconstruct and cannot defend is not an asset — it is a liability that happens to be useful for now. A decision you cannot reproduce is not institutional knowledge. It is a rumour with a timestamp. Governance is what turns the second thing into the first.

So the three commitments named at the outset are not three separate features to evaluate. They are one structure. Sovereignty determines whether anything accumulates. Governance determines whether what accumulated can be relied on. And compounding knowledge is the only one of the three that produces a return — which is precisely why the other two are not optional, and why neither can be retrofitted onto a system that was built without them.

It applies at both levels, deliberately. For the individual, it means the reasoning they do daily builds something that remains theirs and their organisation’s rather than evaporating into a chat history. For the organisation, it means a knowledge position that strengthens with use and cannot be quietly transferred to a supplier’s balance sheet.

Which puts question three in its proper place. Can you reproduce this decision exactly, two years from now sounds like a compliance question, and every vendor will answer it as one. It isn’t. It is the test of whether an organisation is accumulating anything durable, or simply generating output at speed and calling it progress. An agency that can answer it owns what it has built. One that cannot has been renting all along, and will find out at the least convenient moment.

One last thing

SocioProphet is a startup. Not a scale-up with a startup’s culture — a startup, by any definition a procurement officer would use. The client base is interesting, broader than I expected, and genuinely emerging. It is not a roster of major enterprise references, and the company’s ability to deliver and support at that tier is unproven rather than demonstrated. Anyone reading this as a recommendation to place a large regulated workload with them next quarter has read it wrong.

I am aware that this sits awkwardly against everything above. An article arguing that organisations should avoid dependency on suppliers whose future they cannot control, which then points at a company that might not exist in five years, owes the reader an answer rather than a shrug.

Here is the honest one. The architectural properties described here are portable and resident in the customer’s own infrastructure rather than in the vendor’s cloud account — which is the specific property that makes supplier immaturity survivable, because what the organisation built does not evaporate if the supplier does. That is the mitigation, and it is a real one. It is also exactly the thing a buyer should test rather than accept from me: what happens to the knowledge, the evidence and the running system if this company is acquired, pivots, or fails. A vendor who cannot answer that clearly has not earned a pilot, and that standard applies to the subject of this article as much as to anyone competing with it.

The reason to write it down is the same reason the rest of this piece is written the way it is. The failure mode worth avoiding is a claim outrunning its evidence — and “this company can serve you at scale today” is a claim the evidence does not yet support.

Share this article

Read next

See how we keep agents accountable.

Our method keeps seeing, deciding and acting as separate permissions, with a named human owner for every agent.

Or just talk to us