Why use a declarative YAML constitution rather than ad-hoc code checks?

Because rules in YAML are reviewable by non-engineers, lived in version control, and replaceable without a code change

Because YAML is faster to parse than Python

Because the standard library prefers YAML

What fields must every rule have at minimum?

name, severity, must, explanation

How does the engine compose predicates?

Via all_of, any_of, and not_ which recursively wrap atomic predicates

Only via a single contains_regex per rule

By multiplying confidence scores

What is the role of applies_when?

It gates the must predicate so rules that do not apply are recorded as not_applicable rather than violation

What does the Fixer do?

It applies the per-rule declarative fix (append, prepend, replace_regex) to produce a revised text

It calls a remote model to repair the text

It computes severity from the fix

Why does the demo run the engine a second time after the fixer?

To verify the revised text satisfies the rules and to surface any rule the fixer cannot repair locally

Because the first pass is only a dry run

Capstone 86 — Constitutional Rules Engine

A rule is a name, a predicate, and an explanation. Anything missing one of those three is a vibe, not a rule.

Lesson 84 of 85Build~90 minPython, YAML

Before this: Phase 18 safety lessons, Phase 19 Track A lessons 25 to 29

On this page

Problem

Classifiers cover the recognizable failures. Rules engines cover the contractual ones. A team writing a coding assistant wants a constraint like "every response that contains code must end in either a runnable block or a stated assumption." A team running a customer support bot wants "every refusal must offer a next step." These constraints are not natural classifier targets. They are predicates over the response, the conversation, and the system policy, and they need to be readable by a non-engineer.

The honest representation is a declarative file. A constitution lives in YAML alongside the code, in version control, with a separate review process. Each rule has a name, a severity, an explanation template, and a predicate split into two fields: applies_when decides whether the rule is in scope, and must is the condition the response has to meet. The engine loads the file, evaluates each rule against the candidate output, and returns a structured Violation per rule that fired. The rules engine in this capstone composes predicates with all_of, any_of, and not_ so a single rule can express "if the response contains code, it must end with a runnable block AND not reference an internal-only library."

The other half of the lesson is revision. A rule engine that only blocks is half-built. A rule engine that proposes a fix is operationally useful: the assistant drafts a response, the engine flags violations, a fixer produces a revised response, and the engine confirms the revision satisfies the rules. The lesson ships a minimal fixer (regex replacement per rule) and a structured diff (line-by-line additions, removals, edits) between draft and revised.

Concept

A rule has the shape

- name: end-with-runnable-or-assumption
  severity: medium
  applies_when:
    contains_regex: '```python'
  must:
    any_of:
      - ends_with_regex: '```\s*$'
      - contains_regex: 'assumption:'
  explanation: "Code responses must end in either a closing fence or an explicit assumption."
  fix:
    append_if_missing: "\n\nAssumption: example inputs are valid."

Predicates are atomic: contains_regex, not_contains_regex, ends_with_regex, starts_with_regex, max_words, min_words. Compositions are all_of, any_of, not_. The engine evaluates applies_when first; if the rule does not apply, the violation is recorded as not_applicable. Otherwise the engine evaluates must and produces either pass or violation.

Severities are low, medium, high, mirroring lesson 85. The downstream gate (lesson 87) treats a high rule violation the same as a high classifier verdict: block.

The fixer is a list of declarative operations: append_if_missing, prepend_if_missing, replace_regex. Each operation maps a rule by name to a transform. The fixer is intentionally limited to local edits; structural rewrites belong in a separate refusal-and-help layer not covered here.

The diff is computed against the original and the revised. It is a list of Change records with op (add, remove, edit) and the relevant text. The downstream gate can log the diff so a human reviewer audits the fixer's behavior over time.

cd-constitution-loop

Build It

code/rules.yml holds the constitution. The loader in code/main.py accepts either a YAML file (when PyYAML is available) or a JSON file (built-in). The lesson ships a rules.yml that the lesson tests parse by both code paths. code/main.py defines the Engine and Fixer classes and a diff function. Compositions are evaluated recursively with short-circuiting on any_of.

The constitution as shipped:

no-empty-refusal (medium) - a refusal must include either a suggestion or a redirect
end-with-runnable-or-assumption (medium) - code responses must close cleanly
no-pii-in-examples (high) - example data must not contain emails or phone shapes
cite-when-asserting-fact (low) - lines beginning with "According to" must contain a parenthetical citation
no-internal-library-leak (high) - the words internal-only and policybot-internal must not appear in the output
bounded-length (low) - responses must not exceed 800 words

Use It

python3 main.py. The demo runs three draft responses through the engine, prints violations, runs the fixer, prints the diff, and writes outputs/rules_report.json. One fixture has a non-applicable rule (no code block in the draft), and the report shows not_applicable for that rule so the team sees the engine evaluated it explicitly.

Ship It

outputs/skill-constitutional-rules-engine.md documents the rule grammar and the fixer operations.

Exercises

Add a rule that requires every response to include the phrase "If this is urgent" when the prompt mentions safety. Use composition.
Replace the regex fixer with a templating fixer that takes named slots. Demonstrate one rule rewritten under the new design.
Add a metrics endpoint that, given a corpus of drafts, returns the per-rule violation rate so the team can see which rule is over-firing.

Key Terms

Term	Common usage	Precise meaning
constitution	a vague policy doc	a YAML file of rules with predicates, severities, and explanations
predicate	a check	a callable from text to bool, atomic or composed via all_of/any_of/not_
violation	a failure	a structured record with rule name, severity, explanation, and matched span
fixer	a model fine-tune	a deterministic per-rule transform mapping draft to revised
diff	a string compare	a structured list of add, remove, edit operations between draft and revised

Quick quiz

Why use a declarative YAML constitution rather than ad-hoc code checks?
1. Because YAML supports anchors
2. Because YAML is faster to parse than Python
3. Because rules in YAML are reviewable by non-engineers, lived in version control, and replaceable without a code change
4. Because the standard library prefers YAML
Show the answer
C. Because rules in YAML are reviewable by non-engineers, lived in version control, and replaceable without a code change
What fields must every rule have at minimum?
1. name, regex, fix
2. id, owner, timestamp
3. predicate, severity, owner
4. name, severity, must, explanation
Show the answer
D. name, severity, must, explanation
How does the engine compose predicates?
1. Only via a single contains_regex per rule
2. By multiplying confidence scores
3. Through a neural classifier
4. Via all_of, any_of, and not_ which recursively wrap atomic predicates
Show the answer
D. Via all_of, any_of, and not_ which recursively wrap atomic predicates
What is the role of applies_when?
1. It overrides the severity
2. It picks the fixer operation
3. It gates the must predicate so rules that do not apply are recorded as not_applicable rather than violation
4. It always evaluates to true
Show the answer
C. It gates the must predicate so rules that do not apply are recorded as not_applicable rather than violation
What does the Fixer do?
1. It calls a remote model to repair the text
2. It applies the per-rule declarative fix (append, prepend, replace_regex) to produce a revised text
3. It rewrites the rules file
4. It computes severity from the fix
Show the answer
B. It applies the per-rule declarative fix (append, prepend, replace_regex) to produce a revised text
Why does the demo run the engine a second time after the fixer?
1. Because the first pass is only a dry run
2. To re-load the constitution
3. To inflate the report size
4. To verify the revised text satisfies the rules and to surface any rule the fixer cannot repair locally
Show the answer
D. To verify the revised text satisfies the rules and to surface any rule the fixer cannot repair locally

Questions people ask

What is constitutional AI?

Constitutional AI guides a model with a written set of principles. The principles are used to train the model or to check and revise its outputs. Because the rules are written down, people can read and change them.

Why should AI guardrail rules include explanations?

An explanation says why a rule exists and why it fired. That makes decisions easier to debug, review and defend, and helps people tell whether a block was correct or a false alarm.

Run the code Read it on GitHub

Next lessonCapstone 87 — End-to-End Safety Gate Previous: Capstone 85 — Content Classifier Integration

Lessons from AI Engineering from Scratch by Rohit Ghumare and contributors, MIT license. Notes for owners by Lozo. License