Input · Folders and loops

File

Reads the file whose path sits in a column of its input: Parquet, CSV, JSON, Excel or a map file, on disk or in S3. Wire a loop's Current item into it to read a different file each time.

Also known as: Dynamic input, read by path

How it works

The node takes the path or URI in Path column (default path) from the first row of its input and reads that file by its extension: Parquet, CSV, JSON, Excel or a geo format, on disk or s3://. Unreadable file type says what happens when the extension is none of those: Fail the iteration (default) or Skip it with a warning, which lists the skipped items on the loop's badge; a missing or corrupt file always fails. Sheet / layer names the workbook sheet or geo layer to read, blank reading the first with a warning naming the rest. Read all CSV columns as text turns sniffing off so every file types the same way.

Inputs and outputs

Settings

{
  pathColumn: string
  onUnsupported: "error" | "skip"
  allVarchar?: boolean
  sheetOrLayer?: string
}

Example

Sweep a mixed folder: a Directory feeds a Loop, the Current item is wired into this node with Path column path, and Unreadable file type is set to Skip it with a warning so the odd .pdf does not stop the run.

More in Folders and loops

• Directory — Lists the files in a folder, one row per file, instead of reading their rows. Wire it into a loop to process every file in turn.
• S3 Directory — Lists the objects in an S3 location that match a pattern, one row per object, so a loop can process each one.
• DB Directory — Lists the tables in a database connection, one row per table, so a loop can run once per table. You can filter the list by name.
• Sheets — Lists the sheets of an Excel workbook, one row per sheet, so a loop can read each sheet in turn.
• Current item — Sits inside every loop and gives you the item the loop is on right now, such as the current file read as a table. It is added for you and can't be removed.