Step 4 · The agent loop, by hand

Step 4 of 8The loop by hand

Objective

Make the model actually do research — with no framework, so you can see every moving part.

Why this step exists

A framework hides the loop, and the loop is the thing you need to see once. In this step you write it explicitly. In step 5 you throw it away and let a framework do it, and the comparison is the point.

What an “AI API call” actually is

Strip away the SDKs and it’s this: you POST some JSON to a URL with your API key in a header, and you get JSON back.

app/llm.py

body = {
    "contents": [{"role": "user", "parts": [{"text": prompt}]}],
    "generationConfig": {"temperature": 0.4},
}
if system:
    body["systemInstruction"] = {"parts": [{"text": system}]}

url = f"{BASE_URL}/models/{config.LLM_MODEL}:generateContent"
response = await _get_client().post(url, json=body)

That’s the whole thing. Everything else in llm.py is retry logic and pulling the text out of the response envelope.

Notice that Gemini’s URL shape, its contents/parts body, and its x-goog-api-key header are all written out here by hand. That’s why this file is Gemini-only while the framework version swaps provider with an environment variable — pointing this one elsewhere would mean rewriting the request and the response parsing per vendor. See Model providers.

Retries are not optional on a free tier

429 Too Many Requests is normal, not exceptional. llm.py retries with exponential backoff (2s, 4s, 8s) on 429 and 5xx, and gives up immediately on 400/403/404 — because those mean you got something wrong and retrying won’t help.

An agent without retry logic looks broken when it is merely impatient.

The loop

Open app/agent/manual_loop.py. Four explicit phases:

app/agent/manual_loop.py

async def run(query: str, log=print) -> str:
    # 1. Plan — ask the model what to search for
    log("Planning the research")
    plan = await llm.generate_json(f"Question: {query}", system=PLANNER_SYSTEM)
    searches = [str(s) for s in plan.get("searches", [])][:3] or [query]

    # 2. Search — run each query
    titles = []
    for search_query in searches:
        log(f'Searching for: "{search_query}"')
        for title in await wikipedia.search(search_query, limit=2):
            if title not in titles:
                titles.append(title)

    # 3. Read — fetch summaries
    log(f"Reading {len(titles)} sources")
    sources = [p for t in titles[:5] if (p := await wikipedia.read_page(t))]

    # 4. Write — hand everything to the model
    log(f"Writing the answer from {len(sources)} sources")
    context = "\n\n".join(f"### {s['title']}\n{s['extract']}" for s in sources)
    return await llm.generate(f"Question: {query}\n\nSources:\n{context}",
                              system=WRITER_SYSTEM)

Notice the JSON trick in phase 1 — asking Gemini for responseMimeType: application/json means you get parseable JSON instead of writing a fragile “find the JSON inside this markdown” parser.

Run it

From app/:

uv run python -m agent.manual_loop "how do solar panels actually work?"

Expected result

Thirty to sixty seconds of output, then an answer:

  Question: how do solar panels actually work?

  · Planning the research
  · Searching for: "photovoltaic effect"
  · Searching for: "solar cell efficiency"
  · Reading 4 sources
  · Writing the answer from 4 sources

Solar panels work by converting sunlight directly into electricity using the
photovoltaic effect...

Sources: Solar cell, Photovoltaics, ...

Time it. That number — 30, 45, 60 seconds — is the number that breaks everything in step 6.

The limitation, and it’s the important bit

This works. It also cannot adapt.

We decided the order: plan, then search, then read, then write. Exactly once each. If the first search returns nothing useful, the model has no way to try again — we never gave it the chance.

That’s a pipeline, not an agent. The distinction:

	This file	Step 5
Who decides what to search?	the model	the model
Who decides how many times?	you, in advance	the model
Can it search again after reading?	no	yes
Can it stop early?	no	yes

Who owns the control flow is the whole difference between “a script that calls an LLM” and “an agent”.

What usually goes wrong

404 from Gemini

Model names get retired. Check what’s current at https://aistudio.google.com and set LLM_MODEL in .env. The default is gemini-flash-latest, an alias that shouldn’t 404 — if you pinned a specific version, that’s the likely cause.

429 Too Many Requests even with retries

Free-tier limits are per key, per minute. One run is 2–4 model calls. If you’re still hitting it, you’re sharing a key with someone.

Model did not return valid JSON

The planner returned prose instead of JSON. Usually a safety filter or a model that ignored the format request. The code already falls back to using your raw question as the search query — so this degrades rather than fails.

It takes 90+ seconds

Normal on a cold free tier. Note the number; you’ll want it for step 6.

What’s next

Now hand the same job to a framework and let the model drive.

← Step 3 · Tools over MCPStep 5 · With a framework →