MotherDuck - Upload File to Table
Overview
Quickly upload a CSV or parquet file to a a database table in MotherDuck. **Recommended Setup** This blueprint should be used after downloading or fetching data from another source. ### File Types This blueprint can be used to upload either CSV or Parquet files. For larger datasets it is recommended to use Parquet files when possible. ### Insert Type This blueprint can be used to overwrite data (with the `Create or Replace` option) or append to existing rows (with the `Append` option). In either case, if the table does not exist it will be created. ### Match Type This blueprint supports uploading multiple files using a Glob pattern or a regular expression. For single file uploads, use the `Exact Match` option and provide the exact file name to upload.
Variables
Name | Reference | Type | Required | Default | Options | Description |
---|---|---|---|---|---|---|
Access Token | MOTHERDUCK_TOKEN | Password | ✅ | - | - | The API token for programmatic access to MotherDuck |
Database | MOTHERDUCK_DATABASE | Alphanumeric | ➖ | - | - | The optional database to connect to. If omitted, the blueprint will resort to the default. Additionally the database will be created if provided and it doesn't exist |
Table Name | MOTHERDUCK_TABLE | Alphanumeric | ✅ | - | - | The name of the target table in MotherDuck to upload to |
Source File Name | MOTHERDUCK_FILE_NAME | Alphanumeric | ✅ | - | - | The name of the file to upload to MotherDuck |
Source Folder Name | MOTHERDUCK_FOLDER | Alphanumeric | ➖ | - | - | The optional folder location of where the source file is located |
Insert Method | MOTHERDUCK_INSERT_METHOD | Select | ✅ | append | Append: append Create or Replace: replace | The insert method to use upon upload |
Source File Match Type | MOTHERDUCK_MATCH_TYPE | Select | ✅ | exact_match | Exact Match: exact_match Glob Match: glob_match Regex Match: regex_match | The match type dictates whether a single file will be uploaded, or multiple that either match a glob or regex pattern |
YAML
Below is the YAML template
source:
template: MotherDuck - Upload File to Table
inputs:
MOTHERDUCK_TOKEN:
MOTHERDUCK_DATABASE:
MOTHERDUCK_TABLE:
MOTHERDUCK_FILE_NAME:
MOTHERDUCK_FOLDER:
MOTHERDUCK_INSERT_METHOD: append
MOTHERDUCK_MATCH_TYPE: exact_match
type: TEMPLATE
guardrails:
retry_count: 1
retry_wait: 0h0m0s
runtime_cutoff: 1h0m0s
exclude_exit_code_ranges:
- 101
- 102
- 103
- 200
- 210
- 220
- 249