Databricks - Move or Rename Files on DBFS
Overview
Easily import one or more files from Workflows to your Databricks File System (DBFS). The [match type](https://www.shipyardapp.com/docs/reference/blueprint-library/match-type/) selected greatly affects how this Blueprint works.
Variables
Name | Reference | Type | Required | Default | Options | Description |
---|---|---|---|---|---|---|
Original Databricks Folder Name | DATABRICKS_SOURCE_FOLDER_NAME | Alphanumeric | ➖ | None | - | Name of the Databricks File System folder where the file you want to move lives. If left blank, selects from /FileStore/. |
Original Databricks File Name Match Type | DATABRICKS_SOURCE_FILE_NAME_MATCH_TYPE | Select | ✅ | exact_match | Exact Match: exact_match Regex Match: regex_match | Determines if the text in "Original Databricks File Name" will look for one file with exact match, or multiple files using regex. |
Original Databricks File Name | DATABRICKS_SOURCE_FILE_NAME | Alphanumeric | ✅ | None | - | Name of the target file on the Databricks File System (DBFS). Can be regex if "Match Type" is set accordingly. |
New Databricks Folder Name | DATABRICKS_DESTINATION_FOLDER_NAME | Alphanumeric | ➖ | None | - | Name of the folder where you want to move the Databricks files to in the Databricks File System (DBFS). If left blank, selects from /FileStore/. |
New Databricks File Name | DATABRICKS_DESTINATION_FILE_NAME | Alphanumeric | ➖ | None | - | What to name the file(s) being moved or renamed. If left blank, defaults to the original file name(s). |
Workspace Instance URL | DATABRICKS_INSTANCE_URL | Alphanumeric | ✅ | None | - | The subdomain, domain, and top-level domain (TLD) of your Databricks Workspace URL. |
Access Token | DATABRICKS_ACCESS_TOKEN | Password | ✅ | None | - | The personal access token associated with the provided Workspace Instance. |
YAML
Below is the YAML template
source:
template: Databricks - Move or Rename Files on DBFS
inputs:
DATABRICKS_SOURCE_FOLDER_NAME:
DATABRICKS_SOURCE_FILE_NAME_MATCH_TYPE: exact_match
DATABRICKS_SOURCE_FILE_NAME:
DATABRICKS_DESTINATION_FOLDER_NAME:
DATABRICKS_DESTINATION_FILE_NAME:
DATABRICKS_INSTANCE_URL:
DATABRICKS_ACCESS_TOKEN:
type: TEMPLATE
guardrails:
retry_count: 1
retry_wait: 0h0m0s
runtime_cutoff: 1h0m0s
exclude_exit_code_ranges:
- 200
- 201
- 202
- 203
- 212
- 214