Deterministic test report
- Produced by
- 5D offline harness
- Produced when
- When declared scenarios are run against the selected release and policy conditions.
- Contains
- A structured summary of expected and observed results, with optional per-case results and explicit test limitations.
It makes the tested behavior reviewable before policy is promoted or changed.
Primary users- A pass means the declared expectation matched the observed result.
- It does not prove system robustness, security, integration coverage, or test sufficiency.
Relevant framework references
Articles 9(7) and 15(1), 15(4)
Testing, robustness and cybersecurity
Where this output may help: Deterministic policy tests and explicit failure states can contribute action-level testing evidence.
Limit: Only relevant where these provider duties apply; 5D scenario tests do not establish system accuracy, robustness, cybersecurity or Article 9 test sufficiency.
Official source ↗MEASURE 1.1–1.2; 2.1, 2.3 and 2.6–2.7
Measurement approaches, test sets, performance and risk evaluation
Where this output may help: Release-specific deterministic tests, controlled scenarios and replay can measure a declared policy under stated conditions.
Limit: This is evidence for selected action-policy behavior, not verified satisfaction of the NIST subcategories or evaluation of the full AI system.
Official source ↗ASI01 Agent Goal Hijack — mitigation 8; ASI02 Tool Misuse and Exploitation — mitigations 1–4
Red-team scenarios and bounded tool-use controls
Where this output may help: Hostile scenarios can test one technical measure: pre-action policy on covered tool calls.
Limit: 5D does not mitigate either complete ASI risk; effectiveness depends on realistic scenarios, complete interception and host enforcement.
Official source ↗