Step 7 · Accept and poll

Step 7 of 8Accept and poll

Objective

Fix step 6 by never running the agent inside the request — and get a live progress UI as a side effect.

Why this step exists

This is the workshop. Everything before it was setup; everything after it is deployment. If you remember one thing:

Never run the agent inside the HTTP request.

The change

app/main.py

@app.post("/runs", status_code=202)
async def create_run(body: RunRequest, background: BackgroundTasks):
    run = await db.create_run(body.query, body.thread_id)

    # Queue the slow work to run AFTER this response has been sent.
    background.add_task(research.run_agent, run["id"], body.query, run["thread_id"])

    return {
        "run_id": run["id"],
        "thread_id": run["thread_id"],   # send it back to continue the conversation
        "status": "queued",
        "poll": f"/runs/{run['id']}",
    }

• 202 Accepted is the honest status code: “I have taken your request, I have not finished it.” Not 200 — nothing is done yet.

Compare with step 6. One await became one add_task. That is the entire fix.

The response comes back in ~200 ms whether the agent takes 5 seconds or 5 minutes, because the response no longer waits for it.

Then the endpoint the client polls:

app/main.py

@app.get("/runs/{run_id}")
async def read_run(run_id: str):
    run = await db.get_run(run_id)
    if run is None:
        raise HTTPException(status_code=404, detail="Run not found")
    return {"run": run, "steps": await db.get_steps(run_id)}

The client side

client/src/useConversation.ts

// 1. Hand over the job. Comes back immediately with an id.
const created = await createRun(baseUrl, question, threadId, signal);
rememberThread(created.thread_id);          // ← how the next question continues

// 2. Ask "done yet?" until it settles.
for (;;) {
  await sleep(POLL_MS, signal);
  const { run, steps } = await readRun(baseUrl, created.run_id, signal);

  patch({ status: run.status, steps });     // ← the live progress

  if (run.status === "done" || run.status === "error") {
    finish(run);
    return;
  }
}

Twenty lines. It works through every proxy, survives a phone switching networks, and you can debug it with curl.

Two details in there are worth more than they look:

• await sleep(...) inside the loop, not setInterval. A timer fires on schedule whether or not the last request came back. Cold-start your backend and the first poll takes 40 seconds — with setInterval you’ve now got 25 requests in flight at once, all for the same run. Awaiting means the next poll starts when the last one finished.
• The signal. Every call takes an AbortSignal, so the loop stops when the user asks a second question or leaves the screen. Skip it and you get a poll loop that outlives the component and keeps calling your API until the tab closes — which on a free tier is how you meet your rate limit.

This is the part that doesn't look broken

Both mistakes above work perfectly on localhost against a warm server. They only show up deployed, on a cold container, on someone else’s network.

The bonus nobody expects

You fixed a timeout problem. You also got a progress UI for free.

Because the agent writes each step to the database, every poll returns the steps so far. The client doesn’t just learn whether it’s finished — it learns what the agent is doing right now:

✓ Planning the research
✓ Searching Wikipedia — "photovoltaic effect"
✓ Searching Wikipedia — "solar cell efficiency"
✓ Reading a source — Solar cell
⋯ Reading a source — Photovoltaics

The same mechanism that solved the deployment problem solved the UX problem. That’s rare enough to be worth noticing.

Try it

From app/:

uv run fastapi dev main.py

Then start the client, from client/:

npm run dev

Open http://localhost:5173, untick Naive mode, and ask the same question as step 6.

Then run both back to back with the checkbox to feel the difference. That toggle is the most persuasive thing in this whole project.

Expected result

The POST returns immediately:

{"run_id": "3f2a...", "thread_id": "9c71...", "status": "queued",
 "poll": "/runs/3f2a..."}

Steps appear one at a time over the next 30–60 seconds, then the answer.

Check it from the command line too:

curl -X POST http://localhost:8000/runs \
  -H 'Content-Type: application/json' \
  -d '{"query":"how do solar panels actually work?"}'
# {"run_id":"3f2a...","thread_id":"9c71...","status":"queued","poll":"/runs/3f2a..."}

curl http://localhost:8000/runs/3f2a...

# Ask a follow-up in the same conversation. "why is that?" only means something
# because thread_id tells the server which history to replay to the model.
curl -X POST http://localhost:8000/runs \
  -H 'Content-Type: application/json' \
  -d '{"query":"why is that?","thread_id":"9c71..."}'

Then open the Supabase table editor and look at steps. The rows are right there. The UI is just rendering database rows — that’s usually the moment this clicks for people.

The catch, stated honestly

BackgroundTasks runs inside your server process. If the process restarts mid-run — a deploy, a crash, a free instance being recycled — the job vanishes and the row sits at running forever.

Fine for a demo. Know it’s a real limit rather than discovering it later. The upgrade path is in Learn · Why deployment is hard, and it doesn’t change the architecture — just how the background work is scheduled.

A test that protects this

From app/:

uv run --extra dev pytest tests/test_api.py -v

test_post_runs_answers_before_the_agent_has_finished fails if anyone ever moves the agent back inside the request. Worth keeping.

What usually goes wrong

Steps never appear, status stays queued

The background task isn’t running or is failing instantly. Check the server terminal — an exception in run_agent is caught and written to the run’s error column, so also check the row in Supabase.

??? failure “"Failed to fetch" in the browser”

Almost always CORS. Locally, set ALLOWED_ORIGINS=* in .env. Also check you’re not calling an http:// backend from an https:// page — browsers block that.

The run sits at running forever

Either the agent is genuinely still working (check steps for the latest row), or the process restarted and took the task with it. That’s the BackgroundTasks limitation above.

Polling hammers the server

1.5 seconds is a reasonable floor. Don’t go below 1s, and stop polling when the tab is hidden if you’re being careful.

What’s next

It works locally. Now put it on the internet.

← Step 6 · The wrong wayStep 8 · Deploy →