Campaign Dispatch Engine
Pause. Resume.
Never dispatch twice.
A campaign delivery pipeline that survives pause, resume, retries and races — built on NestJS, Temporal and Pulsar.
NestJS
Temporal
Pulsar
MongoDB · Redis
01 — The Problem
Pausing a campaign of
a million messages is hard.
Duplicates
Resume re-sends messages that were already in flight.
Resume re-sends messages that were already in flight.
Races
Consumers and the pause signal fight over the same rows.
Consumers and the pause signal fight over the same rows.
Orphans
Rows stuck IN_PROGRESS forever when a worker dies mid-send.
Rows stuck IN_PROGRESS forever when a worker dies mid-send.
02 — Architecture
Five moving parts, one truth source.
NestJS APIREST + SSE
→
Temporalworkflows · activities
→
Pulsarshared subscription
→
Consumerepoch fence + CAS
MongoDB rows are the source of truth; Redis holds the pause flag; every message carries its dispatch epoch.
03 — Dispatch Flow
From one API call
to a drained backlog.
- POST /campaigns/:id/dispatch starts the parent Temporal workflow.
- Targeting activity materialises PENDING delivery rows (idempotent).
- Dispatcher workflow claims batches PENDING → IN_PROGRESS, publishes to Pulsar.
- Consumers re-read the row, fence the epoch, CAS to SENDING, then finalize.
// every message carries its generation
{ deliveryId, campaignId, epoch }
04 — The Epoch Fence
One number makes
resume safe.
msg · epoch 1
campaign
epoch = 2
epoch = 2
msg · epoch 2
Resume bumps the campaign epoch atomically. In-flight messages from the old round carry the old number — the consumer compares and ack-skips them without touching the row.
if (msg.epoch < currentEpoch) return; // stale → REJECTED, row untouched
05 — Pause & Resume
Rewind, bump, relaunch.
- Pause sets a Redis flag — the dispatcher stops claiming new batches.
- Resume clears the flag, atomically bumps the epoch (fences old messages).
- Rewind flips IN_PROGRESS rows back to PENDING — exactly the fenced ones.
- Relaunch starts a fresh workflow (epoch-scoped id, no collisions) and drains to COMPLETED.
06 — Reliability
Assume everything crashes.
CAS transitions
Every status change is a compare-and-set; concurrent losers ack-skip instead of double-sending.
Every status change is a compare-and-set; concurrent losers ack-skip instead of double-sending.
Idempotent targeting
Re-running the parent workflow never duplicates the audience — rows are created once.
Re-running the parent workflow never duplicates the audience — rows are created once.
Reconciliation cron
A sweeper returns stuck rows to PENDING and completes campaigns whose backlog has drained.
A sweeper returns stuck rows to PENDING and completes campaigns whose backlog has drained.
07 — Run It Yourself
Three commands to a live console.
git clone <repo-url> && cd nestjs-temporal-pulsar-demo
docker compose up -d
cd backend && npm i && npm run build && node dist/main.js
cd frontend && npm i && npm run dev # console on :5173
Create a campaign, dispatch it, pause mid-flight, resume — and watch stale messages bounce off the fence in the live event stream.