OA
All projects Data & AI · Python

Retail Data Pipeline

Three source systems become numbers you can trust, every day.

What it delivers

06:00Fresh numbers every morning
3 → 1Three systems, one dashboard
578errors caught before the report
13of 46Real alerts, holidays filtered

Try it yourself

The 06:00 run: a year of data from three systems in, errors out, finished dashboard. Feel free to disturb the connection.

Try it: follow the golden hints.

Time05:59Nordhafen Living, one year of data
  1. 1Fetch data
  2. 2Set errors aside
  3. 3Calculate
  4. 4Overview

Where the data comes from

Set aside 0

Faulty rows are not deleted but kept with a reason for review.

Calculated tables

    Important this weekReorder 15 items

    White waffle towel 277 units · Lighthouse table lamp 287 units · Sage linen cushion cover 295 units

    Revenue13 days without explanation

    46 unusual days, 33 of them explained: Black Friday, holiday season, sale.

    Data gapShipping file missing

    versand_2026-08-14.csv never arrived. The system reports it instead of silently calculating wrong.

    All numbers come from the real warehouse after a run over a year of test data. The retailer is made up, the errors are built in on purpose, and the tests check that exactly these errors are set aside.

    How it works

    Dagster · daily 06:00SourcesShop · Warehouse · CSVLoadincrementalCheckQuarantinedbtReportsDashboardStreamlit
    omar.dev@web.de