Skip to main content

Snowflake Integration

Connect Qarion to your Snowflake data warehouse to sync metadata, run quality checks, and discover lineage.

Prerequisites​

Before connecting, ensure you have a Snowflake account with appropriate access, a service account or user credentials, and that network access and IP whitelisting are configured to allow Qarion to connect.

Creating the Connection​

Step 1: Add Source System​

Navigate to Source Systems, click Add Source System, and select Snowflake as the platform type.

Step 2: Connection Details​

Enter your Snowflake connection information, including the Account Identifier (e.g., xy12345.us-east-1), the Warehouse for running queries (e.g., COMPUTE_WH), the default Database to sync, the optional default Schema, and the Role available to the connection.

Step 3: Authentication​

Choose your authentication method. You can use standard Username/Password, configure Key Pair authentication by generating an RSA key and configuring the user in Snowflake, or usage OAuth through your identity provider.

Step 4: Test Connection​

Click Test Connection to verify network connectivity, authentication validity, and proper access to the specified warehouse.

Create a Dedicated Role​

We recommend creating a QARION_ROLE with usage privileges on your warehouse, database, and all schemas, as well as select access to all current and future tables.

Create a Service User​

Create a dedicated qarion_service user with the QARION_ROLE and your default warehouse, rather than using a personal account.

Syncing Metadata​

Initial Sync​

After connecting, click Sync Now and select the databases or schemas to include. The system will then import metadata for all accessible objects.

What Gets Synced​

Qarion synchronizes Databases and Schemas you have access to. It imports Tables and Views (including row counts), along with Columns (names, types, nullability, comments) and constraints for Primary and Foreign Keys.

Sync Scheduling​

You can configure automatic syncs by setting a Frequency (daily, weekly, custom) and a Time, preferably during off-peak hours. You can also define the Scope as full or incremental.

Quality Checks​

Supported Check Types​

Qarion supports standard checks like Freshness (checking MAX(updated_at)), Row Count, Uniqueness (checking for duplicates), and Custom SQL validations.

Performance Tips​

To optimize performance, use a dedicated warehouse for checks and schedule them during off-peak times. For very large tables, consider using approximate counts or sampling.

Lineage Discovery​

Automatic Lineage​

Qarion automatically discovers lineage by parsing View definitions (DDL) and analyzing Query history (if access is granted).

Enabling Query History Access​

To enable query history analysis, grant IMPORTED PRIVILEGES on the SNOWFLAKE database to your QARION_ROLE.

Troubleshooting​

Connection Errors​

If you see "Login request timed out", checks network connectivity and your account identifier. for "Invalid credentials", verify your username, password, and role assignment. "Warehouse not found" usually means the role lacks proper usage grants or the warehouse is suspended.

Sync Issues​

Missing tables usually result from missing SELECT grants or exclusion from the sync scope. Slow syncs can be improved by using a smaller warehouse for metadata queries, excluding unused schemas, or scheduling during quiet periods.

Best Practices​

Security​

Use Key Pair authentication for production environments and rotate credentials quarterly. Always monitor connection activity for anomalies.

Performance​

Use a dedicated X-Small warehouse with a short auto-suspend setting (e.g., 60 seconds). Schedule heavy sync operations during off-peak hours to avoid contention.

Permissions​

Follow the principle of least privilege: grant only SELECT access (no write permissions), limit the role to specific schemas, and use Future Grants to handle new objects automatically.