Skip to main content
Skip table of contents

Databricks SQL Warehouse - Download Query Results to Workflows

Overview

Quickly download the results of Databricks SQL query to Workflows. You can choose to output the results as a CSV or a Parquet file. __Recommended Setup__ Although they are not required in order to connect, it is recommended that you provide the `Catalog` and the `Schema` that you will query. By not doing so, the connection will resort to the defaults and it is possible that the query will fail if the tables don't reside in the default schema and catalog. **NOTE** This will not download the file to your local computer, but to the home directory within Workflows

Variables

NameReferenceTypeRequiredDefaultOptionsDescription
Access TokenDATABRICKS_SQL_ACCESS_TOKENPassword--The access token generated in Databricks for programatic access
Databricks Server HostDATABRICKS_SQL_SERVER_HOSTAlphanumeric--The URL address of the SQL warehouse
Warehouse HTTP PathDATABRICKS_SQL_HTTP_PATHAlphanumeric--The extended path for the SQL warehouse
CatalogDATABRICKS_SQL_CATALOGAlphanumeric--The optional catalog to connect to. If none is provided, this will default to Hive Metastore
SchemaDATABRICKS_SQL_SCHEMAAlphanumeric--The optional schema to connect to. If none is provided, the blueprint will connect to the `default` schema
QueryDATABRICKS_SQL_QUERYAlphanumeric--The query to send to Databricks
Workflows Folder NameDATABRICKS_SQL_FOLDER_NAMEAlphanumeric--The optional name of the folder where the file in Workflows is located
Workflows File NameDATABRICKS_SQL_FILE_NAMEAlphanumeric--The name of the outputted file
File TypeDATABRICKS_SQL_FILE_TYPESelectcsvCSV: csv

Parquet: parquet
The file type to load

YAML

Below is the YAML template

YAML

source:
  template: Databricks SQL Warehouse - Download Query Results to Workflows
  inputs:
    DATABRICKS_SQL_ACCESS_TOKEN:
    DATABRICKS_SQL_SERVER_HOST:
    DATABRICKS_SQL_HTTP_PATH:
    DATABRICKS_SQL_CATALOG:
    DATABRICKS_SQL_SCHEMA:
    DATABRICKS_SQL_QUERY:
    DATABRICKS_SQL_FOLDER_NAME:
    DATABRICKS_SQL_FILE_NAME:
    DATABRICKS_SQL_FILE_TYPE: csv
  type: TEMPLATE
guardrails:
  retry_count: 1
  retry_wait: 0h0m0s
  runtime_cutoff: 1h0m0s
  exclude_exit_code_ranges:
    - 200
    - 202
    - 203
    - 204
    - 205
    - 206
    - 207
    - 208
    - 209
    - 210
    - 211
    - 249

      
JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.