How to Build Custom Content Moderation Rules for a Marketplace

Custom content moderation rules start with a written policy: what your marketplace permits, what it prohibits, and what needs a person to review it. Turn those requirements into named rules, evaluate each condition against a listing, and return an explicit decision that your application can handle.

Consider a used-goods marketplace. A vintage camera with a clear description and price might be publishable. A counterfeit watch should be rejected. An incomplete listing might need review. A moderation service timeout needs its own handling path.

This tutorial uses the open-source Antaeus engine to represent those rules as a versioned policy. It walks through policy validation, an optional semantic evaluation, and the cases you should test before connecting the result to publication. Antaeus’s semantic evaluators are experimental; this example does not establish moderation accuracy.

Define the marketplace rules

Begin with concrete requirements. For this example, the marketplace prohibits weapons, counterfeit goods, and prescription drugs. Complete, ordinary listings can be allowed. Other resolved cases go to review.

These are example business rules, not a complete marketplace policy. Define your actual categories, exceptions, and review procedures with the people responsible for moderation.

Keep exact checks in application code. A missing required field or a nonpositive numeric price does not need a language model. Semantic evaluation is useful when a condition requires interpreting the seller’s words, such as whether “1:1 replica with fake certificate” describes counterfeit goods.

Write a versioned policy

Save this as listing-policy.yaml:

apiVersion: policy.antaeus.io/v0alpha1
kind: Policy
metadata:
  name: marketplace-listing
  description: Decide whether a marketplace listing can be published.
spec:
  defaultOutcome: review
  rules:
    - id: prohibited-item
      description: Deny listings for items the marketplace does not allow.
      when: The listing offers weapons, counterfeit goods, or prescription drugs.
      outcome: deny
    - id: complete-listing
      description: Publish complete, ordinary listings automatically.
      when: The listing clearly describes a legal item with its condition and price.
      outcome: allow

Each rule has a stable identifier, a natural-language condition, and an authored outcome. The evaluator judges whether the conditions apply. The engine attaches the outcomes and reduces the results deterministically.

A matched deny wins over a matched allow. If no deny matches and a rule remains unresolved, the result is failure. The default review applies only when every rule is resolved and none matches. Rule order does not create a first-match program.

The condition “legal item” is deliberately broad in this introductory example. For your own marketplace, define permitted categories and concrete exceptions rather than expecting a model to establish legal status.

Validate the policy locally

This tutorial uses Antaeus v0.6.0. Install that version using the release archives or installation instructions, then check antaeus version. Validation and fixture execution need no account or provider key.

Run this command from the directory where you saved the policy:

antaeus validate listing-policy.yaml

For the policy above, validation returns:

{"name":"marketplace-listing","digest":"sha256:d1553c7d6e4eead5b362c0b5e3447f4db2e00c0f861f5fe876e775ccf188b0d4","rules":2}

Validation makes no model call. It checks the supported policy structure and invariants, then computes a digest identifying the parsed policy content. It does not prove that the wording is clear or that an evaluator will interpret it accurately.

Changing a condition or outcome changes the policy digest. Retain that digest with every decision so you can identify the exact policy used later.

Get the pinned example files for the remaining commands:

git clone --depth 1 --branch v0.6.0 https://github.com/antaeusio/antaeus.git
cd antaeus
antaeus version
antaeus validate examples/marketplace/listing-policy.yaml

The checked-in marketplace policy has the same content and digest as the policy above. Keep using this checkout for both the optional semantic step and the credential-free fixture commands below.

Optional: evaluate a listing with a semantic evaluator

The public marketplace example includes this synthetic listing as examples/marketplace/replica-watch.json:

{
  "title": "Luxury watch, 1:1 replica",
  "description": "Looks identical to the original brand, comes with fake certificate. Ships worldwide.",
  "price": "95 USD"
}

To try semantic evaluation, use the OpenAI evaluator guide to configure your own provider credential.

The example profile pins a model snapshot and allows at most two attempts. Supply your key through the process environment as described in the guide; Antaeus does not load .env files. Check the credential binding before making a provider call:

antaeus config check --profile examples/openai/profile.json

This preflight makes no model call. Once it passes, the profile command is:

antaeus evaluate-profile \
  --policy examples/marketplace/listing-policy.yaml \
  --input examples/marketplace/replica-watch.json \
  --profile examples/openai/profile.json

This command sends the rule conditions and listing to the configured provider and can incur provider charges. Inspect the profile’s model and timeout settings before running it.

The intended decision for this case is deny: the listing explicitly describes a counterfeit product. Inspect the actual ruleResults, policy digest, evaluator metadata, and outcome. A model can misclassify a case, so an intended outcome is a test expectation rather than a guaranteed response.

Try the checked-in examples/marketplace/vintage-camera.json input next. Compare the evidence for an ordinary listing with the counterfeit example. Both inputs are synthetic.

Handle each decision in the application

The engine returns a decision; your application decides what happens to the listing.

Outcome	Example application handling
allow	Continue through the remaining application checks before publishing.
review	Hold the listing in a review queue with its policy and rule evidence.
deny	Reject publication and show a suitable explanation for the applicable rule.
failure	Hold publication, retain the failure record, and use your defined operational recovery or human-attention path.

Preserve failure even if the application sends the case to a person. It means evaluation did not produce a policy judgment. A review decision means the policy selected review. Those differences matter when diagnosing an outage or measuring review workload.

Do not use a successful CLI exit as permission to publish. Evaluation commands can emit any of the four outcomes successfully. Read and validate the decision itself.

Authentication, authorization, transactions, and publication remain application responsibilities. A semantic decision does not grant the evaluator authority to change marketplace state.

Test ordinary cases and difficult boundaries

Build a small, named dataset before automating publication. Include examples that should be permitted alongside examples that should be rejected.

Case	Expected handling to establish
Used camera with condition and price	Allow under the example policy when both conditions are resolved appropriately.
Watch explicitly advertised as counterfeit	Deny.
Listing with incomplete information	Review when no rule matches and all conditions are resolved.
An accessory with “replica” in its name	Establish whether it is permitted; a word match alone should not settle the case.
Listing that contains instructions to ignore the policy	Test evaluator behavior with hostile input.
Conflicting matched allow and deny rules	Deny under the engine’s reduction rules.
Provider timeout or malformed response	Preserve the operational failure; prevent automatic publication through an unchecked path.

Use two kinds of tests. Deterministic fixtures check policy identity, reduction, serialization, and application routing. Semantic evaluation checks how a particular evaluator interprets the inputs.

The credential-free quickstart demonstrates fixture execution and a regression suite. From the tagged checkout above, run it with your installed binary:

antaeus evaluate \
  --policy contracts/examples/v0alpha1/policy/vendor-onboarding.yaml \
  --input contracts/examples/v0alpha1/input/aggregate-analytics.json \
  --fixture-set contracts/examples/v0alpha1/fixture-set/quickstart.json \
  --case aggregate-analytics

antaeus test \
  --policy contracts/examples/v0alpha1/policy/vendor-onboarding.yaml \
  --suite contracts/examples/v0alpha1/regression-suite/quickstart.json \
  --fixture-set contracts/examples/v0alpha1/fixture-set/quickstart.json

This separate vendor-onboarding fixture returns review, with evaluator.synthetic: true, and its named regression case passes. Its evidence is supplied in advance; it does not classify unseen marketplace listings or measure model quality. The fixture is tied to its own policy and input digests, so it cannot be reused with the marketplace policy.

For this article, the displayed marketplace policy was validated with v0.6.0, and both fixture commands passed on 10 October 2026. The optional OpenAI evaluations were not run as part of this verification; their intended marketplace outcomes remain expectations.

For semantic tests, freeze the inputs and expected judgments, record the policy and evaluator versions, and report failures and unresolved cases alongside incorrect judgments. Repeat the evaluation when policy wording, the model, or routing changes. Keep hostile-input testing in the dataset without treating a few passing examples as proof of general injection resistance.

Keep decisions inspectable as the policy changes

Retain the policy digest, rule results, final outcome, and evaluator execution metadata needed to diagnose a decision. Choose separately whether to retain the input, a redacted snapshot, or a reference under your data-handling requirements.

When a rule changes, review both its wording and its expected cases. A new digest identifies the new policy; it does not explain whether the change improves moderation. That requires evaluation of the changed behavior.

Custom moderation becomes easier to operate when the rule, its evidence, and the action taken are explicit. Start with a bounded policy and a small dataset you understand.

Start by validating the public marketplace example. For the returned fields, read how Antaeus represents a decision. For the broader architecture, see semantic policy enforcement.