Moderation and guardrails are some of the most natural jobs for Jev. They are high-volume, latency-sensitive, and every outcome is already a small set of actions: allow, hide, review, block. TypeSafe lists "score, judge, verify, guardrail, and detect jailbreaks" among Jev's intended uses (launch post), and its docs include a Twitch chat filter cookbook built on the same idea (TypeSafe cookbooks).

Pattern 1: user-generated content

One request per message, several questions in parallel:

{
  "model": "typesafe/jev-1.13",
  "state": { "message": "…", "channel": "kids-gaming" },
  "questions": {
    "category": {
      "type": "choice",
      "instructions": "Which policy area, if any, does `message` fall under?",
      "criteria": {
        "none": "Ordinary, acceptable conversation",
        "harassment": "Insults or attacks aimed at a person",
        "spam": "Ads, links or repeated promotional text",
        "self_harm": "Mentions of hurting oneself"
      }
    },
    "severity": {
      "type": "score",
      "instructions": "How harmful is `message` for the audience of `channel`?",
      "criteria": ["Harmless", "Mildly inappropriate", "Clearly harmful", "Dangerous or illegal"]
    },
    "hide": { "type": "noul", "instructions": "Should `message` be hidden from `channel`?" }
  }
}

Your code then maps answers to actions: auto-allow when category is none with high confidence, auto-hide when hide is above a strict threshold, and send everything in between to a human.

Pattern 2: guardrails around an LLM

Put Jev on both sides of a generative model:

  • Before: is this prompt a jailbreak or prompt-injection attempt? Does it contain personal data that should not leave the system? See semantic PII lint.
  • After: is the draft answer grounded in the retrieved sources? See grounding / hallucination judge.

Because Jev never generates text, the guardrail itself cannot be talked into producing content — the worst case is a wrong probability, which your thresholds handle.

Pattern 3: agent action gates

Before an agent runs a destructive or costly tool, ask a Noul: "Does the user's request authorize this action?" Block below a threshold, ask for confirmation in the middle band. See the action safety gate and refund tool-call check.

Thresholds and human review

  • Use three bands. Act, confirm/review, escalate. TypeSafe's confidence guide recommends this shape (Confidence docs).
  • Set bands from labelled data. Label a few hundred real items, look at where Jev's probabilities fall for correct and incorrect cases, and choose cut-offs that keep the auto-action error rate acceptable.
  • Remember it can be confidently wrong. Schema-safe is not the same as correct. Keep high-stakes actions (account bans, legal reports) behind a human.
  • Probabilities are routing signals, not findings about a person. Don't store a score as a verdict.

Limits to plan for

  • Text and (on 1.14) images only; transcribe audio/video first.
  • English is strongest; test other languages before relying on them (TypeSafe docs).
  • Exact rules (blocked domains, regex-detectable card numbers) are cheaper and more reliable in code. Use Jev for the judgment calls.

Start from a working example

Each linked use case has a live demo and copy-paste curl, Node.js and Python code for Flatkey.

FAQ

Can Jev be used for content moderation?

Yes. TypeSafe lists scoring, judging, verifying, guardrails and jailbreak detection among Jev's intended uses. A typical request asks a Choice for the policy category, a Score for severity and a Noul for whether to hide the message, all in one call.

Can a Jev guardrail be prompt-injected into producing content?

Jev never generates text, so the guardrail cannot be talked into writing content. The worst case is a wrong probability, which confidence thresholds and human review are there to catch.

How should I set moderation thresholds?

Use three bands (act, confirm or review, escalate) and choose the cut-offs from a few hundred labelled real items, keeping the auto-action error rate acceptable. Keep high-stakes actions such as bans behind a human.

What are the limits of Jev for moderation?

It handles text and, on Jev 1.14, images; audio and video must be transcribed first. English is strongest, and exact rules like blocked domains or regex-detectable card numbers are cheaper and more reliable in code.