dbt Cloud - Download Logs and Artifacts to Workflows

Overview

Download all of the Logs and Artifacts created by a single run on dbt Cloud.

The steps of each run can be variable so we combine the log data from every step into one file.

All of the debug logs will be stored under shipyard-artifacts/dbtcloud-blueprints/logs/dbt.log
All of the console logs will be stored under shipyard-artifacts/dbtcloud-blueprints/logs/dbt_console_output.txt
All of the artifacts will be stored as separate files located under shipyard-artifacts/dbtcloud-blueprints/artifacts/ using their original name.
The responses for this request will always be stored at shipyard-artifacts/dbtcloud-blueprints/responses/run_RUNID_response.json and shipyard-artifacts/dbtcloud-blueprints/responses/step_STEPID_response.json

Variables

Name	Reference	Type	Required	Default	Options	Description
Service Token	DBT_API_KEY	Password	✅	None	-	Your unique service token for dbt Cloud. Instructions for how to get this token can be found in the authorization documentation.
Account ID	DBT_ACCOUNT_ID	Alphanumeric	✅	None	-	Your unique Account ID, found in the URL of dbt Cloud. https://cloud.getdbt.com/#/accounts/ACCOUNT_ID/projects/PROJECT_ID/dashboard/
Run ID	DBT_RUN_ID	Alphanumeric	➖	None	-	The ID of a specific job you want to run, found in the URL of dbt Cloud. https://cloud.getdbt.com/#/accounts/ACCOUNT_ID/projects/PROJECT_ID/runs/RUN_ID/. If left blank, will try to find the run ID from an "Execute Job" Vessel that ran upstream.

YAML

Below is the YAML template

YAML


source:
  template: dbt Cloud - Download Logs and Artifacts to Workflows
  inputs:
    DBT_API_KEY:
    DBT_ACCOUNT_ID:
    DBT_RUN_ID:
  type: TEMPLATE
guardrails:
  retry_count: 1
  retry_wait: 0h0m0s
  runtime_cutoff: 1h0m0s
  exclude_exit_code_ranges:
    - 200
    - 201
    - 211
    - 212
    - 10
    - 14