DesignFlix 17: Accepted Is Not Finished in a Background Job

You click "Export report" and the page responds immediately. The report is still being generated. What exactly did the application promise?

Background jobs let slow work continue after the request ends. That can make the interface feel much better, provided we give the user a way to find the result and don't confuse "accepted" with "finished."

Let's design a report export around that distinction, from the first click to a downloadable file.

Accept work you can still find after a restart

Suppose an admin asks for a large sales report. Generating it inside the request may exceed the server's timeout, even though the work itself is perfectly reasonable.

Instead, validate the request, check access, create a durable job and arrange for a worker to process it. Only then respond that the work was accepted. Starting an untracked promise in the web process doesn't give us that durability; a restart can lose it.

A database-backed job or a queue can hold the work. If creating the job and notifying a broker are separate writes, use a recovery mechanism for the gap between them, such as the transactional outbox pattern.

Return a job reference, not a fake success

An illustrative response could be:

HTTP/1.1 202 Accepted
Location: /jobs/report-17
Content-Type: application/json

{"job_id":"report-17","status":"queued"}

The job reference lets the client check progress later. HTTP 202 means the request was accepted for processing, not that processing succeeded. The HTTP standard makes that distinction explicit.

Our status endpoint can return queued, running, completed or failed. When completed, it can expose a permitted download link. When failed, it should explain what the user can do next without exposing internal errors or another customer's data.

Let the worker do the slow part

A worker takes the job, generates the report, saves the file and updates the job state. The web server no longer has to keep this request open for the entire export.

The client can poll the status endpoint at sensible intervals or receive a supported completion notification. Avoid having every browser check ten times a second; moving work out of one request shouldn't create a flood of new ones.

Access checks still apply to job status and downloads. An unguessable job ID is useful, but it isn't a replacement for verifying that the caller may see the report.

I declare... QUEUED!

Decide what data the report represents

If generation takes several minutes, sales may change while the worker is reading them. Does the export mean "sales when I clicked" or "sales when the worker ran"?

Choose a snapshot or cutoff policy and show it in the report. Where necessary, use database features that provide a consistent read of the intended data. Recording a requested date range alone doesn't make every later query see the same database moment.

This question belongs to the report's correctness, even though the queue has no opinion about it. A fast acknowledgement is only useful if the final file means what the user expects.

Handle a job that comes back

A worker can save the file and crash before acknowledging the queue message. The job may then be delivered again.

Use the stable job ID to make repeated processing safe. Check the recorded state and save output under a policy that prevents conflicting results. If the job also sends a completion email, think about that repeat separately; regenerating a file and sending another email aren't the same effect.

If a worker disappears halfway through, who can tell whether the job is unfinished or merely missing its acknowledgement?

A lease can give a worker a limited period to own a job before it becomes available again. Design that period, timeouts and recovery around the queue's behaviour. A status stuck at running needs a way back to processing or to a visible failure state.

Separate queues when waiting time matters

Bulk exports can occupy workers for a long time. Password-reset emails or other urgent jobs shouldn't necessarily wait behind them.

Separate queues and worker capacity can protect different latency needs. Monitor both, and avoid starving ordinary work indefinitely in favour of the urgent queue. Priority is a policy, not an excuse to let one backlog grow forever.

Adding workers can improve throughput, but only until another limit takes over. Twenty report workers won't help if they all overload the same database. Measure job duration, queue age, failure rate and downstream pressure before increasing concurrency.

Make waiting a supported part of the feature

The user doesn't need to know which broker we chose. They need to know that the export was accepted, where to check it, and whether the file is ready.

A useful background-job feature therefore includes durable acceptance, an honest status, safe recovery, protected results and a visible failure path. The short web request is the beginning of that work. The downloadable report is the end.