Rush Hour
The shop survives the Black Friday night and I can prove it.
How the shop handles the rush
The proof
The problem
Everyone arrives at once. Plus a risky update and a slow payment service.
- Banks on payday
- Ticket sales at a concert drop
- Logistics before Christmas
Try it yourself
You are the sales night. Send the rush, push a faulty update, take a server down. The shop holds.
Try it: follow the golden hints.
0 orders this night
Rebuilt in the browser with the values from the real test run: update taken back after 45 s, payment disrupted for 65 s, peak of 696 orders per minute. The real code with Kubernetes, Argo Rollouts and Grafana is on GitHub.
When something goes wrong
- Emergency brake for payments: payment hangs, the order waits instead of failing
- Refund: orders paid too late get their money back
- Clean up: remove stuck orders, release the stock again
Game Day
6 minutes of sales night, simulated. With a deliberately broken update and a payment outage in the middle.
- Midnight: rush on the shop
- Faulty update is pushed out
- The system takes it back automatically
- Payment slow, orders wait for 65 seconds
- All waiting orders paid afterwards
The checkout missed its availability target (98.1 % instead of 99.5 %). That is exactly what the error budget is for. The catalog stayed at 100 %.
- Python
- FastAPI
- PostgreSQL
- Redis
- Kubernetes
- Helm
- Prometheus
- Grafana
- GitHub Actions
- Argo Rollouts
- Terraform
- k6