Skip to main content

Automate Governed Pipeline Delivery

Pipeline Authoring exposes a version-bound delivery lifecycle rather than a single fire-and-forget generation call. Integrations should preserve the same review, validation, idempotency, and evidence boundaries as the Qarion UI.

  1. Create or select a workspace and definition.
  2. Save repository-backed file or graph changes as a new definition version.
  3. Read readiness for the intended operation.
  4. Complete required review interactions and run authoritative validation.
  5. Start a durable publication operation with an Idempotency-Key header.
  6. Poll the operation resource until it reaches a terminal state.
  7. For Airflow, record and deploy the returned deployment-artifact metadata.

Do not cache readiness across definition versions. A change to the saved version, target, validation fingerprint, or governing policy can make prior evidence stale.

Durable Operations​

Prefer publication-operations and resource-sync-runs over coupling client success to one HTTP connection. These resources survive request loss, worker restarts, duplicate delivery, and browser reloads. Their read models expose checkpoints, progress, results, and the recovery actions currently allowed.

Use a stable Idempotency-Key for retries of the same canonical create request. Reusing the key with the same payload returns the original operation. Do not reuse a key for a materially different publication request.

Cancellation and retry are explicit operations:

  • Publication can be cancelled only before the first Git mutation.
  • Resource sync stops between items and retains completed-item evidence.
  • Retry creates an audited child attempt and targets retryable work rather than rewriting the original record.

Treat Readiness as an Authorization Input​

The strict publication gate requires compiler, static-code, support-test, dependency, runtime-resource, and sandbox sections to be exactly passed. Warnings and skipped requested checks block publication. Clients should surface the failing section and its next action instead of reducing the response to a generic validation error.

Agent-authored definitions add further version-bound requirements: the current version must have a complete execution-slice record, resolved interactions, an accepted result, and fresh validation fingerprints. Partial progress is recoverable but never publishable.

Airflow Artifact Handoff​

Successful Airflow publication returns deployment-artifact metadata and makes the artifact available through the definition-scoped list and download routes. Persist at least the following fields in deployment evidence:

  • definition version ID;
  • package mode;
  • manifest path and dependency-lock identity;
  • filename, SHA-256, and size;
  • artifact status.

Download requires the caller to remain authorized for the definition. Verify the checksum before runtime installation. In extracted mode, unpack the archive into the Airflow DAG runtime. In zipimport mode, place the archive in DAGS_FOLDER; this mode is limited to ZIP-import-compatible pure-Python content.

Failure Handling​

  • Treat stale readiness as a request to validate the current version, not as permission to publish the previously validated version.
  • Preserve operation IDs and parent/child retry relationships in logs.
  • Show checkpoint and per-resource evidence when a durable operation fails.
  • Do not bypass blocked sandbox or dependency checks with the diagnostic fast policy in production flows.
  • If artifact packaging fails, no Git mutation has occurred; fix the dependency or runtime compatibility issue and create a new audited attempt.

For endpoint tables, payloads, and response fields, see the Pipeline Authoring API. For the operator workflow, see Publish a Governed Pipeline.