Skip to documentation
Code Voucherby Pyratz
Open
Browse documentation
How-to guideSetup assistant12 min readFirst-party runtime QA

Set up runtime QA

Use a local coding agent to map runtime services, external integrations, existing test mocks, initialization needs, and browser capability before proposing repository-owned QA files.

Preview a connected repository

Workspace managers can open a connected GitLab repository, select Runtime QA, and choose Generate setup preview. Code Voucher reads the configured default branch at an exact source revision and proposes QA files for supported Django/Postgres and Node/Postgres layouts. The preview lists conflicts and unsupported layouts as blockers, so you can review the proposal before deciding how to proceed.

This is a static preview. It does not run repository code, apply files, publish a merge request, or enable runtime QA. A static-valid result is not evidence that the application can start or that browser scenarios pass.

Validate a frozen proposal

For trusted, allowlisted internal repositories, a manager can choose Validate environment after a preview passes static checks. The validation binds the exact source revision and proposed file hashes, builds and starts the environment, checks database connectivity and repeatable migrations, probes application health, and verifies cleanup.

A Level 0 pass covers those environment checks only. It does not validate browser scenarios, apply repository changes, publish a merge request, or enable Agentic QA. Failed or blocked validations retain their gate results; a new request is a separate attempt against the frozen proposal.

Trusted execution only

This validation executor is restricted to trusted internal repositories. General customer execution remains unavailable until a separate isolated executor is supported. A static preview remains available without runtime validation.

Local assistant workflow

The local skill below analyzes your selected checkout and proposes changes for review. It does not request hosted execution and never publishes changes without explicit approval.

Start with analysis

Invoke the installed repository-local skill from the repository you want to prepare.

Agent prompt
Use $agentic-qa-setup to analyze this repository and propose a safe, deterministic Agentic QA setup.
  1. 1

    Map the runtime

    Statically inventory entrypoints, infrastructure, external integrations, initialization, and existing test assets without executing repository code.
  2. 2

    Classify the target

    Report browser_ready, composition_required, or api_only without manufacturing a browser surface.
  3. 3

    Review dependency strategies

    Inspect the evidence, proposed substitutes, negative cases, confidence, and unresolved gaps.
  4. 4

    Accept the file plan

    Implementation begins only after an evidence-backed proposal exists and local repository edits are in scope.

What the proposal contains

  • An evidence-backed inventory of every material runtime dependency
  • An attempt-local service topology and acyclic readiness chain
  • A per-file create, no-op, additive, or conflict decision
  • Synthetic data and explicitly authorized, classified fixture handling
  • Deterministic positive and negative browser scenario candidates
  • Separate static, isolated execution, and cleanup validation stages
  • Fidelity limits, unsupported workflows, confidence, and unresolved items

Choose runtime-compatible substitutes

Monkeypatches are evidence of expected calls and outcomes, not mocks for an application running in another process or container. Unknown stub operations fail closed.

existing_fake
Select an existing runtime-compatible fake through non-secret QA configuration.
local_http_stub
Serve only evidenced request and response shapes from an attempt-local service.
stateful_stub
Model polling, callbacks, retries, replay, and idempotency with bounded local state.
local_infrastructure
Run disposable databases, caches, queues, mail sinks, or compatible emulators.
not_mocked_with_reason
Keep an integration outside the proposed workflows and record why.
unresolved
Block affected workflows when evidence or a safe configuration seam is missing.

See how to replace external services.

Understand readiness

The setup report uses analysis_complete, proposal_generated, and static_valid as classifications. The hosted preview separately records its operation status and a static_valid or blocked result. environment_ready and ready require isolated execution with executor attestation. Without an attested executor, local setup stops at static_valid. mocks_incomplete, drifted, and failed require action.

api_only and composition_required cannot run Agentic QA today because the current runner requires a real browser surface. Do not manufacture a browser UI to change that result.

Safety boundaries

Host enforcement is required

Plain local Docker Compose cannot prove no-egress isolation. Execution requires user authorization plus an attested disposable, credential-free executor with host-enforced runtime egress limited to attempt-local services.
  • Require a deterministic broker or sandbox outside the model to enforce the readable source corpus
  • Do not read credential files or environment-example values
  • Do not contact external providers or provider sandbox endpoints
  • Do not copy production data, sessions, personal information, or real credentials
  • Inspect fixture contents only after explicit authorization, then classify or quarantine them
  • Do not execute repository code during analysis or file generation
  • Do not push, open a merge request, or change SaaS settings without explicit approval

A model-authored exclusion list is not an isolation boundary. If the local runtime cannot enforce a readable-file allowlist before repository content reaches the assistant, analysis stops with analysis_blocked: unenforced_read_boundary.

Continue to the environment

Define the QA environment, replace External services, then add Browser scenarios.