Databricks - Delete Files from DBFS
Overview
Delete one or more files from your Databricks File System (DBFS). The [match type](https://www.shipyardapp.com/docs/reference/blueprint-library/match-type/) selected greatly affects how this Blueprint works.
Variables
Name | Reference | Type | Required | Default | Options | Description |
---|---|---|---|---|---|---|
Databricks Folder Name | DATABRICKS_SOURCE_FOLDER_NAME | Alphanumeric | ➖ | None | - | Name of the folder where you want to upload the local file(s) in the Databricks File System (DBFS). If left blank, uploads to /FileStore/. |
Databricks File Name Match Type | DATABRICKS_SOURCE_FILE_NAME_MATCH_TYPE | Select | ✅ | exact_match | Exact Match: exact_match Regex Match: regex_match | Determines if the text in "Databricks File Name" will look for one file with exact match, or multiple files using regex. |
Databricks File Name | DATABRICKS_SOURCE_FILE_NAME | Alphanumeric | ➖ | None | - | What to name the file(s) being uploaded. If left blank, defaults to the original file name(s). |
Workspace Instance URL | DATABRICKS_INSTANCE_URL | Alphanumeric | ✅ | None | - | The subdomain, domain, and top-level domain (TLD) of your Databricks Workspace URL. |
Access Token | DATABRICKS_ACCESS_TOKEN | Password | ✅ | None | - | The personal access token associated with the provided Workspace Instance. |
YAML
Below is the YAML template
source:
template: Databricks - Delete Files from DBFS
inputs:
DATABRICKS_SOURCE_FOLDER_NAME:
DATABRICKS_SOURCE_FILE_NAME_MATCH_TYPE: exact_match
DATABRICKS_SOURCE_FILE_NAME:
DATABRICKS_INSTANCE_URL:
DATABRICKS_ACCESS_TOKEN:
type: TEMPLATE
guardrails:
retry_count: 1
retry_wait: 0h0m0s
runtime_cutoff: 1h0m0s
exclude_exit_code_ranges:
- 200
- 201
- 202
- 203
- 212
- 214