What Celery Beat Is, and What It Is Not
Celery is the de facto background task system for Python. The beat scheduler runs tasks on a recurring schedule: every hour, at midnight, every fifteen minutes. If your Django app sends a weekly summary email, prunes expired sessions, or syncs data from an external API on a timer, beat is almost certainly what fires those tasks.
Beat is not a daemon with resilience built in. It is a Python process, usually invoked with celery -A myapp beat, that reads a schedule and queues tasks into your broker (Redis, RabbitMQ, or similar). No replica, no health probe, no checkpoint. When it stops, your periodic tasks stop with it. Nothing tells you.
The Single-Process Constraint
The Celery documentation is explicit: run exactly one beat instance at a time. Two beat processes against the same schedule produce duplicate tasks on every interval, which is usually worse than running none.
To get beat back up after a crash, you need a supervisor: systemd, Docker with a restart policy, or a Kubernetes Deployment set to one replica. That gives you recovery in under thirty seconds. It gives you nothing else.
Consider what happens when beat crashes at 11:58 PM and the nightly report runs at midnight. The restart lands at 12:01 AM. The midnight task was scheduled while beat was down and never got queued. The supervisor log shows a clean restart. Monitoring shows a healthy process. The report was never sent.
What Happens to Missed Tasks
By default, Celery Beat does not reschedule tasks it missed while down. When it restarts, it picks up from the current time and queues what is next. Anything scheduled during the downtime is skipped.
Beat goes down for five minutes across a scheduled interval. That task does not run. No error gets logged, no alert fires. You find out when someone notices the report never arrived or the data has a gap.
There is a beat_max_loop_interval setting that reduces how late the next scheduled task can be. A shorter interval does not recover missed executions. It only limits drift going forward.
The In-Memory Scheduler Problem
By default, beat stores its schedule in a file called celerybeat-schedule in the current working directory. The file tracks when each task last ran so beat can calculate what fires next.
# Default location: wherever you launched beat from
./celerybeat-schedule
On a container restart, if the working directory is ephemeral (it usually is), that file is gone. Beat starts fresh, treating every task as if it has never run. Depending on the schedule type, tasks can fire immediately on restart regardless of when they were actually due, producing duplicate work or confused downstream state.
Pinning the file to persistent storage helps, but only if your infrastructure actually persists it across restarts. Teams tend to discover this problem when a deployment clears the volume and every periodic task fires at 3 AM in a pile.
django-celery-beat: Better, but Still Silent
The django-celery-beat package stores the schedule in your application database. You can manage tasks from the Django admin, add schedules at runtime, and the schedule survives a container restart because it lives in Postgres.
What it does not add is alerting. If the beat process dies between scheduled runs, the scheduler table still shows the task with a future last_run_at. Nothing flags the gap. From the outside, a missed task and a successful one look identical.
Four Ways Beat Silently Stops Working in Production
The broker going down is the first. Beat logs that it queued tasks, but Redis or RabbitMQ is unreachable and the messages are lost. Beat gets no acknowledgement, so it has no way to know.
Workers going away is another. Beat queues the task, but no worker picks it up. The task sits in the queue until it expires against a task_time_limit or message TTL. Beat scheduled it and moved on.
A crash mid-cycle is harder to notice. A task using crontab(minute=0, hour='*/4') with a timezone offset requires beat to correctly recalculate the next run time on restart. A clock skew between the beat host and the broker, or a stale schedule file, can push the next execution out by hours with no visible error.
OOM kills are the subtlest. A long-running beat process that imports your full Django application is not immune to memory leaks. On some setups it gets killed by the OS after several days. The supervisor restart leaves no record of what was missed.
What Monitoring Beat Actually Requires
The dead man's switch pattern catches all of those failure modes. The periodic task checks in at the end of a successful run. If the check-in does not arrive within the expected window, something went wrong.
import requests
from celery import shared_task
@shared_task
def nightly_report():
# Do the actual work first
generate_and_send_report()
# Check in only if the work completed
requests.get(
"https://cronguard.app/api/ping/your-monitor-id",
timeout=5,
)
This skips the question of whether beat is running. It monitors the outcome. Any of these failure modes produces the same result. No check-in arrives, and the same alert fires.
For centralised instrumentation, Celery's task_postrun and task_success signals let you send the check-in from one handler rather than touching each task individually.
Frequently asked questions about Celery Beat reliability
Can I run more than one Celery Beat instance for redundancy? No. Running two beat instances against the same broker produces duplicate task executions on every scheduled interval. The Celery documentation requires exactly one beat instance at a time. High availability comes from a supervisor that restarts the single process on failure, not from running multiple copies in parallel.
What happens to periodic tasks when Beat restarts after a crash? Tasks that were scheduled to run while beat was down are not recovered by default. Beat calculates the next run time from the current moment when it starts and ignores any windows it missed. Whether a task fires immediately on restart or waits for the next scheduled interval depends on the schedule type and how long the downtime lasted.
Why did my periodic task suddenly stop running without any error? The most common causes are: the beat process crashed and the supervisor did not restart it in time, the broker became temporarily unreachable, all workers were busy or unavailable, or the celerybeat-schedule file was lost on a container restart and beat miscalculated the next run time.
How can I tell whether a Celery periodic task actually executed, not just whether Beat is running? You cannot tell from the beat process state alone. A running beat process can be failing to reach the broker, or workers can be unavailable to consume tasks. The only reliable signal is whether the task itself completed successfully, which requires the task to check in at the end of a successful run through an external monitor.
What is django-celery-beat and does it fix these reliability problems? django-celery-beat stores the periodic task schedule in your application database instead of a local file, which means the schedule survives container restarts and lets you manage tasks from the Django admin without restarting beat. It does not add redundancy, does not recover missed tasks, and does not alert you when a task fails to run. It solves schedule persistence, not task-level reliability.
Further reading
- Why Cron Jobs Fail Silently (And How to Catch Them)
- Dead Man's Switch Monitoring: The Only Reliable Way to Watch Cron Jobs
- Durable Job Queues Are Replacing Cron: New Failure Modes
Conclusion: Celery Beat is not a monitoring system, and it was never designed to be one. It queues tasks on a schedule and moves on. Whether those tasks ran, produced correct results, or ran at all is not something beat tracks. The only way to close that gap is to instrument the tasks themselves — a check-in at the end of a successful run, watched by a dead man's switch monitor, turns a silent absence into an actionable alert before the next business day.
Sources: Celery Periodic Tasks documentation, django-celery-beat on GitHub.
