Skip to main content
Skip table of contents

Databricks - Download Files from DBFS to Workflows

Overview

Quickly export one or more files from your Databricks File System (DBFS). The [match type](https://www.shipyardapp.com/docs/reference/blueprint-library/match-type/) selected greatly affects how this Blueprint works.

Variables

NameReferenceTypeRequiredDefaultOptionsDescription
Databricks Folder NameDATABRICKS_SOURCE_FOLDER_NAMEAlphanumericNone-Name of the folder where the file is stored in the Databricks File System (DBFS). If left blank, looks in the /FileStore/.
Databricks File Name Match TypeDATABRICKS_SOURCE_FILE_NAME_MATCH_TYPESelectexact_matchExact Match: exact_match

Regex Match: regex_match
Determines if the text in "Databricks File Name" will look for one file with exact match, or multiple files using regex.
Databricks File NameDATABRICKS_SOURCE_FILE_NAMEAlphanumericNone-Name of the target file in the Databricks File System (DBFS). Can be regex if "Match Type" is set accordingly.
Workflows Folder NameDATABRICKS_DESTINATION_FOLDER_NAMEAlphanumericNone-Folder where the file(s) should be downloaded on Workflows. Leaving blank will place the file in the home directory.
Workflows File NameDATABRICKS_DESTINATION_FILE_NAMEAlphanumericNone-What to name the file(s) being downloaded on Workflows. If left blank, defaults to the original file name(s).
Workspace Instance URLDATABRICKS_INSTANCE_URLAlphanumericNone-The subdomain, domain, and top-level domain (TLD) of your Databricks Workspace URL.
Access TokenDATABRICKS_ACCESS_TOKENPasswordNone-The personal access token associated with the provided Workspace Instance.

YAML

Below is the YAML template

YAML

source:
  template: Databricks - Download Files from DBFS to Workflows
  inputs:
    DATABRICKS_SOURCE_FOLDER_NAME:
    DATABRICKS_SOURCE_FILE_NAME_MATCH_TYPE: exact_match
    DATABRICKS_SOURCE_FILE_NAME:
    DATABRICKS_DESTINATION_FOLDER_NAME:
    DATABRICKS_DESTINATION_FILE_NAME:
    DATABRICKS_INSTANCE_URL:
    DATABRICKS_ACCESS_TOKEN:
  type: TEMPLATE
guardrails:
  retry_count: 1
  retry_wait: 0h0m0s
  runtime_cutoff: 1h0m0s
  exclude_exit_code_ranges:
    - 200
    - 201
    - 202
    - 203
    - 212
    - 214

      
JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.