A soft checklist before production

This checklist asks whether you are ready. It does not replace the hard review in Acceptance. Hard acceptance proves that interfaces and boundaries work under their contracts; this checklist confirms that you placed them inside a real business and can explain where results come from and where failures fall back.

Every item needs evidence: a log, an evaluation result, a configuration diff or a real call. Do not mark an item complete without it.

Corpus and permissions

Check	Evidence	If it fails
The corpus loads and an empty corpus has an explicit state	Startup log or self-check output	Fix the load path; do not treat empty data as normal
Document, section and chunk ids are unique; revision is non-empty	Validation output	Fix the data; do not skip validation
Permission filters use trusted identity	Code path plus one unauthorized test	Add filtering before reading or fusion
Deletion and rewrites clear indexes and citations	One deletion or rewrite drill	Add cleanup; editing the list is not enough

Configuration and versions

Check	Evidence	If it fails
The effective configuration has a version recorded in logs	One request log	Add the identifier instead of saying "current configuration"
Source and runtime copies match; no hand-edited runtime files	Generated diff or comparison	Edit the source and regenerate
Secrets come from the environment or a secret store	Configuration review	Move them out of the repository and corpus
Model, embedding and index space identifiers agree	Startup self-check	Rebuild the index or fix the configuration

Budgets and capabilities

Check	Evidence	If it fails
Normal requests do not hit budget rejections intermittently	Character counts from real samples	Leave headroom or read in passes
Over-limit requests fail explicitly instead of being truncated	One over-limit call	Reject and explain
Retries have attempt and deadline bounds	Failure-injection result	Add bounds and backoff
Cancellation stops later calls	One cancellation test	Check and propagate signal at every layer

Retrieval and citations

Check	Evidence	If it fails
Source recall on real questions meets the target	Small-sample evaluation	Fix chunking, vocabulary or sections first
The system says "unknown" when no material exists	No-answer sample result	Fix the decision and prompt boundary
Citations point at real sources with matching text	Spot-check answers against sources	Read back and verify revision and visibility
Retrieval failure and "no material" are distinct	Degradation log	Separate the two states
Index updates never return deleted content	One query after deletion	Clear indexes and caches; re-check visibility

Failure paths

Check	Evidence	If it fails
Rate limits, timeouts and authentication failures have explicit behavior	Failure-injection record	Add classification and handling
Failed writes can be reconciled by idempotency key	One timeout drill	Add the key and reconciliation
Unavailable dependencies degrade or fail explicitly	One degradation drill	Remove fallbacks that pretend success
Degradation never widens permissions	Unauthorized-path test	Remove branches that bypass validation

Observability

Check	Evidence	If it fails
A failure can be traced to its stage, tool or retriever	One failure log	Add fields
Success, degradation, failure and cancellation are distinguishable	One sample of each	Unify result classification
Logs contain no secrets, full prompts or unmasked data	Log spot-check	Redact or remove fields
Degradation and failure have separate counters	Metrics or report	Count them instead of watching success rate only

Release and rollback

Check	Evidence	If it fails
A release pins matching corpus, configuration and index versions	Version and build record	Fix versions; do not mix them
The rollback path is explicit and drilled	One rollback walkthrough	Document it and retain the old version
A new version never reads an incompatible old index	Startup self-check	Validate identity and indexRevision
The reason for the change and what was observed are recorded	Change record	Record them before releasing

Regression

Check	Evidence	If it fails
Key protocols and tool selection have offline cases	Test results	Add cases with MockGPT
Failures, cancellation and budget overflow are covered	Test results	Add boundary cases
Real model quality is verified separately	Manual or online evaluation record	Do not substitute offline cases for semantic acceptance

How to write the conclusion

State the version, the samples covered, the observed results, what was not verified and the remaining risk. Do not turn "we found no problems" into "there is no risk", and never present passing offline cases as proof of semantic quality.

Next

• Hard acceptance scope: Acceptance.
• Failure-injection techniques: Offline verification with MockGPT.
• Retrieval evaluation methods: Evaluating retrieval quality.