Quality
Measure rule-level and aggregate behavior, including costly error directions rather than hiding them inside one score.
Semantic Policy Enforcement research
Separate the question from the claim.
Can meaning-heavy policies be evaluated with useful quality, bounded human review, operationally meaningful cost, and reproducible evidence? That is the research question. Antaeus is the reference implementation—not an answer by itself.
New to the term? Start withwhat semantic policy enforcement is.
The question
Can semantic policy become an inspectable engineering practice?
The inquiry is broader than whether a model can produce a plausible answer once. It asks whether decisions remain measurable, reviewable, and attributable as policies, cases, evaluators, and operating conditions change.
Review
Treat ambiguity, disagreement, and the burden placed on people as outcomes to measure—not noise to discard.
Operations
Include failure, latency, cost, truncation, and route behavior in the evidence for or against a usable boundary.
Reproducibility
Trace observations to frozen inputs, policy identity, evaluator configuration, and disclosed limits on exact replay.
Four different things
A roadmap is not a protocol. A demo is not a result.
Each layer has a different evidentiary role. Keeping them distinct makes it harder for implementation progress or an encouraging development run to become an unsupported claim.
- 01
Open question
Hypothesis
A specific proposition that may fail and should be stated without treating intent as evidence.
- 02
Methodological commitment
Experiment
A frozen protocol with separated data, declared metrics, comparison rules, and no post-hoc threshold changes on the final test.
- 03
Early implementation available
Reference implementation
Antaeus supplies versioned artifacts and an inspectable execution boundary. Its synthetic fixture verifies plumbing, not semantic quality.
- 04
Not published
Result
An observation supported by the frozen evaluation, reported with operating tradeoffs, uncertainty, threats to validity, and reproducible artifacts where possible.
Evidence standard
Make the eventual claim smaller than the evidence.
A credible publication has to expose the conditions under which its observation holds—and the conditions it did not test.
Freeze what decides behavior
Identify the implementation, policies, cases, evaluator, configuration, and final test material used for the reported run.
Declare the test before the result
Choose primary measures, asymmetric costs, and success thresholds before final evaluation rather than tuning them to the outcome.
Report the whole operating surface
Include review rate, evaluator disagreement, failure, latency, cost, and relevant adversarial behavior beside quality measures.
Preserve reproducibility honestly
Publish artifacts or a privacy-preserving substitute, and disclose where provider mutability or data restrictions prevent exact replay.
Current boundary
What the implementation proves today.
It proves that the local contract can be exercised.
The current open-source path can validate a versioned policy, execute exact synthetic evidence, and return a typed decision through the same provider-neutral boundary used by named regression cases.
It does not prove semantic performance.
A deterministic fixture cannot establish accuracy, calibration, robustness, acceptable review burden, cost, or production suitability. Those questions remain open until evaluated under a frozen protocol.
Implementation context
Inspect the instrument without confusing it for the finding.
Antaeus makes policy identity, named cases, evaluator configuration, explicit outcomes, and decision evidence inspectable. That is useful infrastructure for an eventual experiment—and independently useful open-source work—but it is not empirical validation of the research hypothesis.
Inspect the reference implementation