About me

Dileepa Bandara

Associate AI Engineer at Random Software Ltd  ·  Sri Lanka

I'm an AI Engineer passionate about building intelligent solutions that make a difference.

What you'll walk out with

The next 90 minutes

What we're building

A research agent. You ask a question. It plans, searches, reads sources, writes an answer, and saves it. Then you ask a follow-up and it remembers.

It takes 30–60 seconds. That's deliberate — it's slow enough that the naive version visibly breaks.

Gemini · FastAPI on Render · Postgres on Supabase · React client on GitHub Pages. Total cost: $0.

Generative vs Agentic

So what actually changes?

The Timeout Trap

Your agent runs beautifully on localhost, because nothing between you and the server is impatient.

Deployed, there's a proxy, a load balancer, a gateway, a CDN — and every one of them has an opinion about how long a request may take.

One of them gives up before your agent does.

"Just raise the timeout"

You can, a bit. It doesn't fix anything.

• You're renting a request for 5 minutes to do 5 minutes of work — one of the most expensive ways to buy compute.
• You don't control the whole chain. A proxy, a corporate firewall, or a phone switching from Wi-Fi to mobile data still cuts it.
• The user still gets nothing until the end — no progress, no partial results, no way to resume.
• A crash loses everything. No record of how far it got.

The 100% free stack

Speed vs volume

Free tiers limit you two different ways, and picking the wrong one for an agent hurts.

Exact numbers move every few months — check both providers on the session morning. docs/free-tier-notes.md lists where to look; the shape of the trade-off above outlasts the digits.

The honest slide

Free tier isn't a smaller paid tier.
It's a different set of trade-offs.

Accept and poll

The whole backend, honestly

@app.post("/runs", status_code=202)
async def create_run(body: RunRequest, background: BackgroundTasks):
    run_id = await db.create_run(body.query)   # fast: one insert
    background.add_task(agent.run_agent, run_id, body.query)
    return {"run_id": run_id, "status": "queued"}

@app.get("/runs/{run_id}")
async def read_run(run_id: str):
    return {"run":   await db.get_run(run_id),
            "steps": await db.get_steps(run_id)}

Two highlighted lines are the entire pattern. add_task queues the work to run after the response has already been sent.

The whole client, honestly

// 1. hand over the job — comes back immediately
const { run_id, thread_id } = await createRun(baseUrl, query, threadId, signal);
rememberThread(thread_id);              // ← the entire memory feature

// 2. ask "done yet?" until it settles
for (;;) {
  await sleep(1500, signal);
  const { run, steps } = await readRun(baseUrl, run_id, signal);

  setSteps(steps);                        // live progress UI
  if (run.status !== "running") return finish(run);
}

Ten lines. Identical in Vue or Flutter — it's just fetch in a loop.

The catch nobody mentions

BackgroundTasks lives inside your server process.

The catch: BackgroundTasks lives in your process

A deploy, a crash, or a free-tier recycle takes the job with it. The row sits at running forever.

Cheap fix: mark runs stale after 10 minutes, so the UI stops spinning and says something honest.
Real fix: a task queue with a separate worker process — the jobs stop living inside the web service.

Polling vs streaming

Where secrets live

# main.py
API_KEY = "AIzaSy…"  ← in the code

# .env  (gitignored)
LLM_API_KEY=AIzaSy…

# .env.example  (committed)
LLM_API_KEY=

Two kinds of memory

Two tables, that's it

create table runs (
  id         uuid primary key default gen_random_uuid(),
  thread_id  uuid not null default gen_random_uuid(), -- the conversation
  query      text not null,
  status     text not null default 'queued',  -- queued|running|done|error
  result     text,
  messages   jsonb,                            -- memory, for the model
  created_at timestamptz default now()
);

create table steps (            -- the agent's thought process
  id      bigserial primary key,
  run_id  uuid references runs(id) on delete cascade,
  seq     int  not null,           -- display order
  label   text not null,          -- "Agent is searching..."
  detail  text
);

The whole of “it remembers”

# load the conversation
history = await db.get_thread_messages(thread_id)

result = await agent.run(query,
    message_history=history)

# save it back, for next time
await db.update_run(run_id,
    messages=to_jsonable_python(result.all_messages()))

Logging the thought process

await steps.log(
  "Searching the web",
  f'Looking for: "{query}"')

One insert before every phase — before, not after. The user wants to know what's happening now.

This is also your production debugger. You can't attach a breakpoint to a deployed agent, but you can read exactly how far it got.

pgvector in three commands

-- 1. turn it on (already available on Supabase, free)
create extension if not exists vector;

-- 2. a table with an embedding column
create table memories (
  id        bigserial primary key,
  content   text,
  embedding vector(768)
);

-- 3. find things that MEAN something similar  (<=> is cosine distance)
select content from memories order by embedding <=> $1 limit 5;

Keyword search finds the word "car". Vector search finds "automobile", "vehicle", and "my old Toyota".

The dimension gotcha

• Different models output different sizes. Some let you configure it.
• Get it wrong and you either fail at insert time, or build your whole table on the wrong number and have to migrate it later.
• Check your model's output dimensions first, then write the migration. Not the other way round.

The single most common reason a student's pgvector project stops working.

An agent is an unpredictable user
with database credentials

Give it a door, not a bulldozer

execute_sql(query: str)

save_note(text: str)
search_notes(query: str)

MCP goes both ways

A tool is a function. MCP is that function, described so a client that has never seen your code can call it.

app.mount("/mcp", mcp_app)

MCPToolset("https://mcp.tavily.com/mcp")
  .filtered(lambda ctx, t:
      t.name == "tavily_search")

This is two deployments, not one

1 · The agent, on Render — five steps

1. Run schema.sql in the Supabase SQL editor → you now have runs and steps
2. Push to GitHub. Check that .env is not in the repo
3. Render → New Web Service → connect the repo → root app, build pip install uv && uv sync --frozen, start uv run fastapi run main.py --port $PORT
4. Add your three environment variables in Render's Environment panel — not in the code
5. Open /health. Green? You're live.

Same agent, second host: FastAPI Cloud

Built by the FastAPI team. No config file, no build command, no start command — it reads pyproject.toml, uv.lock and .python-version and works the rest out.

Environment variables go in the dashboard with a Secret toggle, then Save and Redeploy. Prefer the terminal? uv run fastapi deploy from app/.

2 · The client, on Vercel — two fields

1. vercel.com/new → import the repo
2. Root Directory → client — the Vite project, not the repo root
3. VITE_API_URL = your backend URL → Deploy

client/vercel.json is already written, so there is nothing else to configure. GitHub Pages needs even less — it's already in the Actions workflow, published at /demo/.

What will go wrong

Before you submit — photograph this

• .env in .gitignore before your first commit Once a key is pushed it's in history forever — rotate it, don't just delete it
• No API key anywhere the browser or the app can read it
• Long tasks return an ID immediately — never block the request
• Every agent step written to the DB, so the UI can show progress
• Errors saved with status = 'error' A silently stuck run looks exactly like a slow one
• The agent gets narrow tools, never raw SQL — and a hard step limit
• Ping your database weekly — free projects pause when idle
• Warm your backend before any demo or judging — cold start is ~a minute
• Test your live URL in an incognito window, and on a phone