149
ETL and ELT
Extract, then transform, then load — or load raw and transform in the warehouse. Do not transform in a notebook you cannot rerun.
What is ETL and ELT?
ETL transforms before the data lands. ELT lands raw data and transforms it where it sits. Both need to be rerunnable. A pipeline that only works if yesterday’s run succeeded is a script, not a pipeline.
Why does ETL and ELT matter when vibe coding?
Models write a one-shot script that appends forever and double-counts when rerun. Name idempotent load and a key.
How do you do ETL and ELT?
Partition by day or id. Reloading a partition replaces it. Keep the raw extract. Transformations are code in the repo, not cells.
How do you ask a model for ETL and ELT?
Build an (ETL or ELT) job for (source). It must be safe to rerun: replace the partition, do not append duplicates. Keep raw input. Transformations live in the repo, not a notebook.
What goes wrong with ETL and ELT?
Transforming in the extract step so you can never reprocess when the rule changes. Land raw if you might reinterpret it.