Shape · Rows

Distinct

Keeps one row per unique combination of key columns. You choose whether the first or last copy survives; with no keys it drops exact duplicate rows.

Also known as: Remove duplicates, unique

How it works

Key columns are the columns that define a duplicate; leave them empty and the node drops rows that are identical in every column. When duplicates exist keeps the first or the last copy of each key, counted in the order the rows arrived, and the surviving rows keep that order. Case-sensitive keys is on by default; untick it and text keys such as A and a count as the same.

Inputs and outputs

Settings

{
  keyColumns: string[]
  keep: "first"|"last"
  caseSensitive: boolean
}

Example

One row per customer, keeping their latest address: Sort by updated_at ascending first, then Distinct with Key columns customer_id and When duplicates exist set to Keep Last.

More in Rows

• Filter — Keeps only the rows that meet your conditions, combined with AND or OR. For example, keep orders where the amount is over 100.
• Switch — Sends each row down one of several outputs: it goes to the first case whose condition is true, and rows that match none go to Default.
• Sort — Puts rows in order by one or more columns, ascending or descending, with blanks first or last.
• Row Limit — Keeps or drops the first or last N rows, in their current order. For example, skip three header lines or take 100 rows for a quick check.
• Top / Bottom N — Keeps the N rows with the largest or smallest values in one column, such as the 10 best-selling products. You choose what happens to ties at the cut-off.
• Random Sample — Picks a random set of rows, either a fixed number or a share such as 10%. You can sample with replacement, or stratify by a column so each group keeps its share. Set a seed to get the same rows every time.
• Random Split — Divides rows into named groups by percentage, each on its own output, such as 80/20 train and test. Repeatable, and can keep each group's share of a column.
• Add Row Number — Numbers the rows 1, 2, 3… in a new column. You can sort first and restart the count in each group.
• Rank — Ranks rows by one or more columns, optionally within groups, so the top seller per region gets 1. You choose how ties are ranked; percentile rank is available too, and Tile splits the rows into N equal-sized buckets.
• Densify — Adds the rows that are missing so every key combination appears, taken from the data or from a reference table you wire in, and fills the gaps with values you choose. For example, every product for every month, with 0 sales where none were recorded.
• Generate Rows — Repeats each row once per value of a range, such as 1 to a count column or a start date to an end date, and adds the generated value as a column. For a sequence with no input, use Generate Series.