Overview — what is this system?

Outside companies send us customer records (an ID, a name, an email). This system receives those records, saves them into our customer table, and gives operators screens to watch the work happen and fix anything that gets stuck.

Post-office flow

  1. A sender drops a letter in the mailbox — a third party calls our API.
  2. The clerk stamps it received — the API stores an ingest record and answers accepted within a fraction of a second. The sender never waits for delivery.
  3. A courier delivers it immediately — the fast-lane worker writes the customer into the customer table, usually in under a second.
  4. Every 2 minutes a supervisor checks for letters nobody picked up — the sweeper re-queues anything the courier missed.
  5. Once a night an auditor counts everything — aggregate-stats records totals for the Overview page.

Guarantees

Words used in this guide

WordMeaning
CustomerThe current truth: one row per externalId with name and email.
Ingest (inbox row)One accepted request, with a status: PENDING, PROCESSING, DONE, FAILED.
QueueA waiting line of jobs. customer-fast-lane = deliver now, customer-cron = scheduled work.
JobOne unit of work in a queue (for example: process one ingest row).
CronWork on a schedule: the 2-minute sweeper and the nightly stats.
Job run (log)A history entry saying a job finished or failed, with how long it took.

Running locally

pnpm dev starts everything. Fixed ports: dashboard on 3000, API on 3001, this docs app on 3002.

Health screen

Open the real Overview first whenever something feels wrong. If PENDING or FAILED is growing, go to Ingests. If failed jobs are growing, go to Queues. If a card says unavailable (Redis down?), the queue system is unreachable — the inbox cards still work.

Example view — not live data

Fast lane

waiting: 2 active: 0 delayed: 0 completed: 120 failed: 1

Cron

waiting: 0 active: 0 delayed: 2 completed: 48 failed: 0

Inbox

PENDING: 2 PROCESSING: 0 DONE: 118 FAILED: 1

Latest stats

{
  "customers": 300000,
  "pending": 2,
  "failed": 1
}