Skip to main content

Transform Models

Transform Models is the authoring surface for dbt-style projects, SQL models, macros, packages, and previews. It gives data teams a governed place to edit model code and test changes against approved query connectors.

Workspace Setup​

  1. Open Authoring -> Transform Models.
  2. Create or select a workspace.
  3. Choose a default repository and project location.
  4. Set visibility to Space, Private, or Shared.
  5. Select the query connectors allowed for authoring previews.

Workspace sharing follows the same view and manage grant model as notebooks and pipeline authoring.

Project Workflows​

Use Transform Models to:

  • Open and edit project files from the workspace repository.
  • Manage model SQL, macros, seeds, and supporting project files.
  • Configure exact package pins for project dependencies and runtime tooling.
  • Preview model SQL through an approved query connector.
  • Publish reviewed project changes back to the repository.

Execution Profiles​

Execution profiles let a workspace define how previews, macros, and runs select connectors and dbt runtime settings. A profile can set a connector, environment label, resolution mode, scoped schema template, safe dbt profile overrides, and non-secret environment variables.

Use a default profile for routine previews and a production profile when scoped dbt planning needs a state manifest or production relation metadata. One-off actions can pass temporary profile overrides, but Qarion rejects secret-like environment variables and unsafe dbt profile keys. Store credentials on the query connector instead.

Profile overrides are intentionally narrow. Use them for non-secret runtime choices such as schema, database, warehouse, role, location, thread count, or BigQuery cost controls. Do not put passwords, tokens, key files, or full connection strings in profile overrides or environment variables.

Scoped dbt Previews​

Scoped previews compile selected model context in the backend before sending preview SQL to the query connector. Use standard mode for lightweight static validation and direct previews. Standard mode is best when the current draft and selected connector have enough context to compile without production relation metadata.

Use scoped deferred mode when the model needs production or state-manifest context but should run against a temporary scoped schema. Scoped deferred mode keeps preview writes isolated while allowing unchanged parents to resolve from the configured production or state-manifest context.

Scoped deferred runs can use:

  • a selected execution connector for the preview target;
  • a production or state-manifest connector for dependency metadata;
  • a scoped schema template, such as one derived from the user and project;
  • compiled SQL rewrites that point unchanged parents at the state relation;
  • transient file overrides for the current draft only.

Environments that support this workflow need the dbt CLI and the adapter package for each supported query connector family on the backend runtime path.

Toolchains And Packages​

Workspace runner settings can pin dbt-core, SQLMesh, dbt adapters, and extra Python packages by exact version. Package names are normalized and duplicate pins are rejected. Adapter pins require a dbt-core version, and extra packages require either a dbt or SQLMesh runtime.

If package installation needs private packages, link approved Python package repositories to the workspace runtime settings. Public package access depends on the authoring package-fetch policy configured by administrators. Package setup output is sanitized before it is stored.

Use exact versions for runtime tooling. Avoid floating pins such as latest, wildcards, or broad version ranges when a preview or publication workflow needs repeatable results.

If a preview cannot compile, check the workspace project settings, package configuration, and connector selection before changing the model SQL.

Troubleshooting​

dbt is unavailable means the backend runtime or workspace toolchain cannot resolve a usable dbt executable. Check the toolchain pins and backend runtime configuration.

The state manifest is missing or stale means scoped deferred planning cannot compare the draft with production metadata. Refresh the project baseline or configure the production/state connector.

Scoped deferred preview uses the wrong relation means the execution profile may be resolving the wrong production/state connector, environment label, or scoped schema template. Review the selected profile before editing the model.

The connector is unsupported for scoped dbt execution means Qarion cannot generate a safe temporary dbt profile for that connector family. Use a supported query connector or standard validation.

An environment variable was rejected means the name looked like a credential or used an invalid format. Keep credentials on connectors and use environment variables only for non-secret runtime switches.

Package installation is blocked means the workspace package pins, private package repository grants, public-index policy, or backend runtime setup need an administrator review.