Input · Databases and cloud

S3 Reader

Reads a file stored in an Amazon S3 bucket, or another S3-compatible store, into a table. A pattern such as s3://bucket/exports/*.csv reads every matching object into one table.

Also known as: AWS, bucket, MinIO

How it works

Object URI names the object to read, s3://bucket/key.csv, and Connection picks a saved S3 credential. A key ending in .csv is read as CSV and any other key as Parquet; Add data source → S3 also has a Format choice (auto, parquet or csv) to set it yourself. Object URI is the field you can bind to a column of the Loop's Current item, so a loop over an S3 Directory reads one object per iteration. Adding the source through Add data source → S3 also lets you set an endpoint for an S3-compatible store such as MinIO or R2, and a region.

Inputs and outputs

Settings

Example

Daily exports land in s3://data-lake/exports/. Point an S3 Directory at s3://data-lake/exports/**/*.parquet, loop over it, and inside the loop bind the S3 Reader's Object URI to the item's uri column.

More in Databases and cloud

• DB Reader — Reads a table from a PostgreSQL or MySQL database through a saved connection. You pick the schema and table.
• Snowflake Reader — Reads a table from Snowflake through a saved connection, or the result of a SQL query you write. A table is read whole unless you set a row limit.
• Salesforce Reader — Reads the records of a Salesforce object through a saved connection, or the result of a SOQL query you write. An object is read whole unless you set a row limit.
• Strava Reader — Reads your Strava activities, athlete profile or activity streams through your own connected Strava app.