LLM Engineering·Lesson 44 of 67·advanced
How an LLM API call actually works
The request, the response and the statelessness — what you send, what comes back, and why the model has no memory between calls.
In this lesson · 18 min
- →Describe the shape of a Messages API request and response
- →Explain why the API is stateless, and what that costs
- →Read stop_reason and handle each value
- →Read the usage block and turn it into money
- →Build a multi-turn conversation correctly
The request
One HTTPS POST to /v1/messages, carrying JSON. That is the whole API — tools, streaming,
caching and structured output are all fields on this one request.
Four things to notice:
modelnames the exact model. It is not a family or an alias you can leave vague.max_tokensis a hard ceiling on the output. Hit it and the reply stops mid-sentence.systemis its own top-level field, not a message. It carries operator authority.messagesalternates user and assistant turns, and must begin withuser.
On your own machine that request is:
import anthropic
client = anthropic.Anthropic() # reads ANTHROPIC_API_KEY from the environment
response = client.messages.create(
model="claude-opus-5",
max_tokens=16000,
system="You are a terse assistant. Answer in one sentence.",
messages=[{"role": "user", "content": "What is a token?"}],
)
The response
content is a list of blocks, not a string. A plain answer is one text block; a reply that
calls a tool has tool_use blocks; a reply with thinking enabled has thinking blocks first. So
always iterate and check block.type rather than reaching for content[0].text:
stop_reason
Always check it before trusting the content:
The API is stateless
The server keeps nothing between calls. Whatever the model appears to “remember” is in the
messages list you resent:
The second request carried three turns. The model did not recall the name — it read it, again.
Usage, and what it costs
Output tokens cost about five times what input tokens cost, on every model. A prompt that asks for a short answer is cheaper twice over — fewer output tokens now, and a shorter history to resend later.
A conversation, properly
Try it yourself
Write extract_text(response): join the text of every text block, ignoring all other block types, and return an empty string when there are none.
Write call_cost(model, usage) returning the dollar cost of one call, and monthly(model, usage, calls) for that many calls a month.
Check yourself
Why does the input token count grow with every turn of a conversation?
Check yourself
A reply comes back with stop_reason 'max_tokens'. What happened, and what should your code do?
Recap
- Everything is one POST to
/v1/messages:model,max_tokens,systemandmessages. contentis a list of typed blocks — iterate and checkblock.typerather than indexing.- Check
stop_reasonbefore trusting the content;stop_detailsexists only for"refusal". - The API is stateless, so a conversation resends its whole history and input tokens grow quadratically with its length.
- Output tokens cost about five times input tokens.
- Append
response.contentto the history, not the extracted string.