Cron dead-man's switch: heartbeat per tick, alert on the lower bound

Emit one analytics event per effective cron tick under a stable synthetic identity, then alert when the hourly count drops below the expected floor.

Observability

· Chapter

4

·

2

min read

The answer. Error alerts can't tell you about a cron that never runs. Put a heartbeat at the top of scheduled() — one event per tick to whatever you already alert on (we use PostHog's capture API) — and configure a lower-bound alert on its hourly count. A */15 cron should produce 4 per hour; alert when the count falls below that. Emit the heartbeat after your fail-closed gate, not before: an environment whose mutations are gated is supposed to go quiet, and the alert should say so. And give the event a stable synthetic identity. PostHog's capture endpoint requires distinct_id; a heartbeat has no acting user or org, and a helper that resolves identity from those will drop it without a sound.

The pattern.

async scheduled(_c: ScheduledController, env: Env, ctx: ExecutionContext) {
  const gate = mutationGate(env);
  if (!gate.enabled) { console.warn("cron gated", gate.reason); return; }

  // Dead-man's switch: one event per EFFECTIVE tick. Best-effort, never throws.
  ctx.waitUntil(
    capture(env, {
      event: "worker_cron_tick",
      distinct_id: "system:cron",               // required; no user or org here
      properties: { environment: env.APP_ENV ?? "unknown" },
    }).catch(() => {}),
  );
  // ...the real legs follow, each in its own waitUntil
}
// Alert (in PostHog): hourly count of worker_cron_tick below 4  ->  page.

The receipt. We added this to HarperFlow's API Worker on 2026-07-31, after a July stretch when the prod cron was dead for eleven days (07-10 to 07-21) and nothing said a word. The first tick after deploy was silently swallowed: our capture helper derives distinct_id from the acting user or org and returns early when it finds neither, so the heartbeat never reached PostHog until it got the system:cron identity. The event now doubles as a deploy check: after every prod deploy the runbook step is "confirm the next 15-minute tick arrived". The alert threshold lives in PostHog, not in the repo; 4 per hour is the floor a */15 schedule implies.

Watch out.

  • Alert on a floor, not an error rate. Zero ticks is zero errors.
  • A heartbeat placed before the gate lies: it reports "alive" while every real leg is blocked.
  • Don't await the heartbeat inline or let it throw; waitUntil plus .catch(() => {}) keeps it off the critical path.

Related: cron-one-tick-many-isolated-legs · cron-gate-outage-kill-switch-returned-early