Gate actions with
probability thresholds.

QUESTION DESIGN

Make every outcome operational.

{
  "requires_review": {
    "type": "noul",
    "instructions": "Does this action need human review?"
  },
  "risk": {
    "type": "score",
    "instructions": "Score the potential impact",
    "criteria": ["Low and reversible", "Material", "High impact"]
  }
}

Good criteria describe what each option means in the workflow. Avoid overlapping labels and do not hide permissions or business rules inside model instructions. The request shape follows the documented question types.

REQUEST AND RESPONSE

Two signals from one state.

A Noul question estimates whether review is needed, and a Score question places the action on an ordered risk scale. Both read the same state in one request. Score levels are indexed from zero here for illustration; confirm the indexing in the provider documentation.

{
  "model": "jev-latest",
  "state": {
    "action": "Send refund of $480 to a new payee",
    "authorized_by": "support agent",
    "reversible": false
  },
  "questions": {
    "requires_review": { "type": "noul", "instructions": "Does this action need human review?" },
    "risk": {
      "type": "score",
      "instructions": "Score the potential impact",
      "criteria": ["Low and reversible", "Material", "High impact"]
    }
  }
}

{
  "model": "jev-1.13.0",
  "answers": {
    "requires_review": { "type": "noul", "noul": 0.93 },
    "risk": {
      "type": "score",
      "score": 1.77,
      "legend": { "0": "Low and reversible", "1": "Material", "2": "High impact" },
      "probabilities": { "0": 0.03, "1": 0.17, "2": 0.8 },
      "confidence": 0.8
    }
  },
  "usage": { "input_tokens": 238, "output_tokens": 30 }
}

Label: illustrative. The field names follow the documented API; the values are invented to show branching, not measured results.

THRESHOLD BANDS

Set the bar by reversibility, not by feel.

Irreversible actions such as payments or external messages deserve a stricter review threshold than a reversible tag or draft. Choose each threshold from the cost of a miss, measured on your own labeled cases.

IMPLEMENTATION

From model answer to safe action.

1. Use Jev as a signal. Never treat the model decision as the permission system itself. Hard rules such as allowlists and spending limits stay in code.
2. Fail safely. Timeouts, malformed responses, and low confidence resolve to review or stop, never to an automatic action.
3. Separate risk dimensions. Privacy, cost, reversibility, and external communication may need distinct questions so one number does not hide the reason.
4. Keep the evidence. Store the state, both answers, the model version, and the reviewer’s final call. That record is how you calibrate thresholds later.
5. Measure false negatives. A guardrail is only useful when checked against real incidents and labeled cases, not just typical traffic.

FAILURE MODES

What goes wrong in practice.

• False negatives. The costliest error is a risky action scored as safe. Test specifically on examples that should have been blocked.
• Untrusted text in the state. Content copied from users or documents can try to steer any model. Treat it as data, keep hard rules in code, and do not rely on the model alone as the only barrier.
• One blended score. A single “risk” number hides which dimension fired. Ask separate questions when the response differs by cause.
• Threshold drift. A cutoff tuned on one model version can shift after an update. Pin the version and re-check after changes.
• Silent fallbacks. If errors quietly default to “allow”, the guardrail is off exactly when it is needed. Default to review.

WHEN NOT TO USE JEV

Cases where a rule or a person is the right control.

• The policy is a hard rule, such as an allowlist, a spending cap, or a required approval. Enforce it in code.
• The stakes are legal or safety-critical and need an accountable human decision.
• You cannot state the policy clearly enough to write it as a question.
• You need an audit explanation in prose. Use an LLM or a person for the explanation.