The Job Scheduler

A job scheduler is the part of a system responsible for deciding when work runs. Rather than have a person sit and launch a program at the right moment, the scheduler holds a description of the work and the timing, and starts it on its own. The three common timing models are: run once at a fixed future time, run repeatedly on a calendar pattern, and run when a queue is ready and the machine has capacity to spare. Unix supplied a small, sharp tool for each.

The repeating case belongs to cron. Its daemon, described in cron(8) as one that “examines all stored crontabs and checks each job to see if it needs to be run in the current minute,” wakes up each minute and fires any command whose calendar pattern matches the current time. This covers everything that should happen on a schedule: nightly backups, hourly cache clears, weekly reports. The five-field crontab line is the canonical way to express “this command, on this recurring pattern.”

The run-once case belongs to at. The at(1p) manual page describes a utility that “shall read commands from standard input and group them together as an at-job, to be executed at a later time.” Where cron repeats forever, at fires a job exactly once, at a moment you name, and then forgets it. Its sibling batch handles the third model: rather than tying work to the clock, it queues jobs and runs them when the system load is low enough, so heavy work waits for a quiet machine. The POSIX page notes that batch uses a distinct queue from the default at queue.

These models long predate Unix. On the mainframes of the 1950s and 1960s, “the job” was the unit of computing: a deck of cards or a tape submitted to an operator, queued, and run in turn, often overnight, with the term batch processing describing exactly the queue-and-run pattern. The operating system’s job scheduler decided which job got the machine next. Unix’s cron, at, and batch carried that idea onto interactive, multi-user systems, where most of the time there was a logged-in user but the routine work still needed to happen on its own.

At larger scale the same concept reappears as dedicated enterprise scheduling systems and, later, workflow and orchestration engines that manage thousands of interdependent jobs across many machines, with retries, dependencies, calendars, and monitoring. The vocabulary grows, but the core question is the one cron answered for a single Unix box: given a description of work and a description of when, make it run, reliably, without a human in the loop.

Understood broadly, the job scheduler is one of the quiet foundations of automation. Whenever a system does something “automatically” on a timetable or a queue rather than in response to a direct command, some scheduler, from a one-line crontab entry to a distributed workflow engine, is the thing deciding that now is the time.