Skip to main content

Publish a Governed Pipeline

Use this guide when a Pipeline Authoring draft is ready to move from a reviewed definition into a repository and, for Airflow, a deployable package. Publication is version-bound: Qarion publishes the exact saved definition version that passed the required checks.

Before You Start

Confirm that the workspace has a repository and target branch, that the draft targets Airflow or Dagster, and that you have manage access. Save or revert dirty editor changes before validating. Unresolved editor conflicts prevent Qarion from proving that the reviewed code matches the code it will publish.

Dagster is compile-and-validate focused in the current release. The publication and packaged-deployment steps in this guide apply to Airflow.

1. Review the Current Version

Open the definition and check the following surfaces:

  • Plan review shows the approved tasks, dependencies, write scopes, acceptance criteria, and required evidence.
  • Files shows the repository-backed draft and any additions, edits, or deletions.
  • Graph shows nodes, dependencies, and the DAG README.
  • Audit history shows the agent runs, decisions, validation attempts, and version changes that produced the current draft.

If the draft has advanced since a review or validation run, use the current version rather than relying on evidence from the older version.

2. Run Authoritative Validation

Run validation from the authoring cockpit. With the strict sandbox policy, publication requires compiler, static-code, support-test, dependency, runtime-resource, and sandbox sections to be exactly Passed. A warning or a requested check reported as Skipped does not count as passed.

The fast sandbox policy permits one narrow non-production exception. When the publication gate is not in block mode, publication can continue only if a skipped sandbox section is the sole gate failure. Any other failed, warned, missing, or skipped section still blocks publication, and fast cannot bypass a block-mode gate.

These sections are necessary but not sufficient. Publication also requires a passing trusted-parent identity for the exact candidate and spec. Executable candidate Python additionally requires canonical platform-controlled behavioral authorization. Candidate-authored exit codes, JUnit, and command output are observations: support-test evidence becomes authoritative only after an independent dependency-backed, sandbox, or other platform-controlled check passes.

When validation finds a problem, use the recovery action shown beside the diagnostic. Qarion can route findings to code repair, plan revision, user input, environment resolution, transient retry, or unsafe-scope review. A repaired candidate is retained only when it improves the result without introducing a new critical, security, or policy blocker.

Repository commands can require a separate approval. Review the exact command, working directory, bounded paths, repository identity, runtime image, and network intent before approving it. The approval is fingerprinted to the exact request and current workspace; changing any of that material invalidates it.

Protected workspace mutations use a different server-generated sensitive artifact approval. CI workflows, CODEOWNERS, hooks, Docker and build files, scripts, and shell files are bound to the repository and ref, target, base definition version, result spec fingerprint, approved-plan hash, path and operation, exact content or deletion-tombstone SHA, expected base SHA, and policy version. Approval-gated Bash nodes are bound to their node, exact command SHA, detected capabilities, repository and ref, target, base version, result spec fingerprint, approved plan, and Bash policy version. Any drift requires a new approval; a stale approval cannot authorize publication.

.git control paths, credential and secrets directories, non-template environment or credential files, and key or certificate material are always blocked and cannot be approved. Compiler-owned generated paths retain a separate structural restriction against direct move or delete; neither approval type overrides it. An authored Bash pipeline task also never grants authoring-time shell execution.

3. Confirm Publication Readiness

The readiness summary should show Ready for publication. Other states need attention:

StatusWhat to do
Not validatedRun validation for the current saved version.
StaleRe-run validation after the draft, target, fingerprint, or policy change.
BlockedResolve the failed, warned, or skipped requirement shown in the inspector.
UnsupportedUse a supported operation for the selected runtime target.

Partial execution slices cannot be published. Complete every required execution unit and resolve pending decisions before continuing.

4. Publish

Open Publish, review the target repository, branch, generated files, and merge-request details, then start publication. Publication revalidates and locks the immutable version before its first Git mutation.

The operation continues if you leave or reload the page. Return to the Publish inspector to see validation, Git, merge-request, and managed-resource checkpoints. Cancellation is available only before the first Git mutation. Retry creates an audited child attempt rather than overwriting the failed run.

If the draft advances while an older version is publishing, Qarion preserves the older result but marks it stale instead of treating the newer draft as published.

5. Use the Airflow Deployment Artifact

A successful Airflow publication creates an immutable ZIP from the validated version. The artifact contains the DAG, its transitively imported local modules, package initializers, explicit runtime files, the selected dependency manifest, hash-verified wheel contents, and .qarion/deployment.json evidence. It is stored separately from the publication branch.

Choose the package mode that matches the Airflow runtime:

  • Extracted is the default. Extract the archive into the DAG runtime. This mode supports pure-Python dependencies and compatible native wheels.
  • ZIP import places the ZIP directly in DAGS_FOLDER. Use it only for pure-Python dependencies; Qarion rejects native extensions in this mode.

The Publish inspector shows the package mode, dependency manifest, lock identity, checksum, size, compatibility findings, and authorized download action. Keep the checksum with deployment evidence so operators can verify that the runtime package matches the version approved in Qarion.

Troubleshooting

  • Manifest requires clarification: set the supported repository-relative requirements manifest explicitly when two candidates exist at the same nearest directory.
  • Native wheel rejected: use extracted mode and confirm the configured Airflow Python and platform target matches the wheel.
  • Sandbox is skipped or disabled: use strict for production and ask an administrator to configure the hardened runner. fast permits only the documented non-production, sandbox-only exception when the gate is not in block mode.
  • Publish button remains blocked: check for dirty files, stale validation, incomplete execution units, unresolved decisions, or a draft that differs from the accepted result.

See Pipeline Authoring for workspace, graph, agent, repair, and reliability concepts. Developers automating this lifecycle can use the Pipeline Authoring API. Review Pipeline Authoring Security for the repository, approval, sandbox, credential, and publication boundaries that remain in force during release.