vouchfx .e2e.yaml — a guide for agents¶

You are writing a vouchfx end-to-end suite. Start with this guide for common v1 suites, then consult get_schema for the exact contract.

Rules first, then examples. Every YAML block below is a complete, schema-valid document — copy one and edit it. Do not copy fragments from memory of another testing tool; this language is its own.

The shape of a file¶

The v1 examples below use four common top-level blocks; only steps is required in v1. A fifth block, listeners, declares host-owned listeners. DSL v2 requires both metadata and steps, with metadata.schemaVersion: "v2"; request get_schema for version v2 before authoring a v2 suite. These v1 examples are not a complete reference for either version.

• metadata — name, owner, tags, description. Write it. tags is what CI selects on.
• environment — services (the system under test) and dependencies (databases, brokers, caches the engine provisions). Map keys are LOGICAL NAMES; every step's target refers to one.
• variables — constants pre-loaded into the shared context, readable as {placeholder}.
• steps — an ordered list. Every step needs id and type. type is family.provider (http.rest, db-assert.postgres). A bare family is rejected.

metadata:
  name: orders-api-smoke
  owner: team-orders
  tags: [smoke]
  description: The service starts and serves its orders collection.

environment:
  services:
    orders-api:
      image: myco/orders-service:latest
      httpPort: 8080

variables:
  expectedStatus: "open"

steps:
  - id: check-health
    type: http.rest
    description: The service answers its health probe.
    target: orders-api
    method: GET
    path: /health
    expect:
      status: 200

  - id: list-open-orders
    type: http.rest
    target: orders-api
    method: GET
    path: /orders?status={expectedStatus}
    expect:
      status: 200

Never write a host or a port in a step. target names a service; the engine resolves the address. path must be a rooted relative path (/orders); an absolute URL is rejected as an SSRF guard.

Never set, under a dependency's env:, a variable the engine already sets for that dependency type (elasticsearch, minio, s3/RustFS and azureservicebus). validate_suite at semantic or full level reports these overrides as VFX-D-1213; the schema level checks syntax only. The engine refuses them before any container starts. For full control of the backend's environment, declare it as a service with image: instead.

External environment references¶

Use get_schema as the authority for the pinned language. The MCP semantic pass recognises an environment.source reference only to report VFX-D-1212: external driver configuration and its logical service/dependency bindings remain unresolved. It reads no driver files or environment values, and this advice establishes neither schema acceptance nor runtime readiness. Empty service/dependency summaries do not prove absence.

When using a compatible engine, validate the original suite file with normal vouchfx validate so suite-relative configuration retains its base path. Inline YAML has no filesystem base; save it beside its configuration before engine validation. Normalization preserves the relative configuration reference; it does not validate or rewrite the external file.

State threading: capture + {placeholder}¶

A step that produces an id must hand it to the next step. Use capture to pull the value out, and {name} to put it back in.

• capture maps a variable name to a JSONPath over this step's result: orderId: "$.id".
• For XML, use the mapping form: orderId: { xpath: "//id" }.
• A captured value is visible to every LATER step.
• In SQL, bind it as a parameter — never concatenate it into the query text.

Never hard-code an id. A literal order-123 passes once and fails on the next run against fresh state. If you are tempted to write one, you need a capture.

A capture whose path matches nothing makes that step Inconclusive — never Fail, never Pass — even when the step's own expect held (measured at engine v1.0.0-rc.5). Later steps still run and {placeholder} still substitutes — a later step that consumes the unmet placeholder can still Pass — but the run's own verdict is Inconclusive too (the highest-precedence verdict of any step), so a suite with a silently-missed capture never reports Pass; the engine's term is "upstream capture unmet". - A capture is not a shape assertion: a miss surfaces as Inconclusive, not Fail. Use http.rest.expect.bodyContains for a text substring or expect.json for JSONPath assertions supported by the engine. Read get_schema section: step:http.rest for their exact shapes and operators; the served schema remains the authority for the installed MCP engine pin. - The vendored recipes' capture section says such a step "fails with a clear error". That is wrong at this pin: the measurement above is authoritative, and the recipes' own exit-code table agrees ("Inconclusive (timeout, unmet captures…)").

metadata:
  name: order-placement-threads-its-id
  tags: [integration]

environment:
  services:
    orders-api:
      image: myco/orders-service:latest
      httpPort: 8080
  dependencies:
    orders-db:
      type: postgres

steps:
  - id: place-order
    type: http.rest
    target: orders-api
    method: POST
    path: /orders
    body:
      sku: WIDGET-1
      quantity: 3
    expect:
      status: 201
    capture:
      orderId: "$.id"

  - id: read-back-order
    type: http.rest
    target: orders-api
    method: GET
    path: /orders/{orderId}
    expect:
      status: 200

RETRY: how you say "eventually"¶

Anything that asserts on an effect ANOTHER process produces is asynchronous. A worker's database row, a broker's message, a projection's cache entry.

• Set verifyMode: RETRY. The engine polls with bounded exponential backoff.
• Always pair it with an explicit timeout (30s, 60s). This is the window you are willing to call "eventually": too short is flaky, too long delays a real regression.
• The default is IMMEDIATE. An IMMEDIATE step that exceeds its timeout is Inconclusive, never Fail — a slow machine is not a product defect.

Never sleep. Never loop by hand. Never widen an assertion to make a slow answer arrive.

metadata:
  name: settlement-is-asynchronous
  tags: [integration, async]

environment:
  services:
    orders-api:
      image: myco/orders-service:latest
      httpPort: 8080
  dependencies:
    orders-db:
      type: postgres

steps:
  - id: place-order
    type: http.rest
    target: orders-api
    method: POST
    path: /orders
    body:
      sku: WIDGET-1
    expect:
      status: 201
    capture:
      orderId: "$.id"

  - id: await-settlement-row
    type: db-assert.postgres
    target: orders-db
    verifyMode: RETRY
    timeout: 60s
    query: >-
      SELECT status FROM orders WHERE id = @order_id
    parameters:
      order_id: "{orderId}"
    expect:
      rowCount: 1
      row:
        status: settled

Secrets: references only¶

Write ${secret:source/path}. Never a literal credential.

• ${secret:env/ORDERS_API_TOKEN} reads that environment variable at STEP-EXECUTION time.
• The engine redacts it from logs, reports and the event stream. Only the reference path appears.
• A literal password is checked into source control, printed in CI, and flagged as VFX-D-1207.

metadata:
  name: authenticated-call
  tags: [integration, secrets]

environment:
  services:
    orders-api:
      image: myco/orders-service:latest
      httpPort: 8080

steps:
  - id: submit-with-token
    type: http.rest
    target: orders-api
    method: POST
    path: /orders
    headers:
      Authorization: "Bearer ${secret:env/ORDERS_API_TOKEN}"
    body:
      sku: WIDGET-1
    expect:
      status: 201

The four outcomes¶

A run ends in exactly one of these. They are not severities; they mean different things and demand different actions.

Use these four words. The engine's raw event stream spells them differently; that spelling is for machines reading events, not for you.

Do / don't¶

Do

• Start from vouchfx://examples/http-smoke or a scaffold_suite skeleton, never a blank file.
• Call describe_step_type and copy parameter names from the registry output.
• Give every step an id that says what it checks (await-settlement-row, not step3).
• Assert something that can fail. An expect that any result satisfies proves nothing.
• Use verifyMode: RETRY with an explicit timeout for every asynchronous assertion.
• Thread ids with capture + {placeholder}.
• Write ${secret:...} references.
• Run validate_suite at level: full and fix both channels before running anything.

Don't

• Don't hard-code ids, hosts, ports or URLs.
• Don't write a literal credential.
• Don't sleep, poll by hand, or retry in your own code.
• Don't assert only the happy path — prove the system rejects what it should.
• Don't change an assertion to turn a Fail into a Pass. That deletes the finding, not the bug.
• Don't invent step types or fields. If it is not in list_step_types / describe_step_type, it does not exist.

Where to look next¶

• vouchfx://schema/latest — the composed JSON Schema, the exact contract you are validated against.
• vouchfx-docs:///language-reference — every step type's required and optional fields.
• vouchfx-docs:///recipes — worked patterns for queues, caches, mail, seeding and CI.
• vouchfx://examples/{name} — three complete, commented suites.
• explain_diagnostic — what any VFX-D-#### or VFX-E-#### code means and how to fix it.