Skip to content

LLM Engineering·Lesson 44 of 67·advanced

How an LLM API call actually works

The request, the response and the statelessness — what you send, what comes back, and why the model has no memory between calls.


In this lesson · 18 min

  • →Describe the shape of a Messages API request and response
  • →Explain why the API is stateless, and what that costs
  • →Read stop_reason and handle each value
  • →Read the usage block and turn it into money
  • →Build a multi-turn conversation correctly

The request

One HTTPS POST to /v1/messages, carrying JSON. That is the whole API — tools, streaming, caching and structured output are all fields on this one request.

PythonPython not started

Four things to notice:

  • model names the exact model. It is not a family or an alias you can leave vague.
  • max_tokens is a hard ceiling on the output. Hit it and the reply stops mid-sentence.
  • system is its own top-level field, not a message. It carries operator authority.
  • messages alternates user and assistant turns, and must begin with user.

On your own machine that request is:

import anthropic

client = anthropic.Anthropic()          # reads ANTHROPIC_API_KEY from the environment

response = client.messages.create(
    model="claude-opus-5",
    max_tokens=16000,
    system="You are a terse assistant. Answer in one sentence.",
    messages=[{"role": "user", "content": "What is a token?"}],
)

The response

PythonPython not started

content is a list of blocks, not a string. A plain answer is one text block; a reply that calls a tool has tool_use blocks; a reply with thinking enabled has thinking blocks first. So always iterate and check block.type rather than reaching for content[0].text:

PythonPython not started

stop_reason

Always check it before trusting the content:

PythonPython not started
PythonPython not started

The API is stateless

The server keeps nothing between calls. Whatever the model appears to “remember” is in the messages list you resent:

PythonPython not started

The second request carried three turns. The model did not recall the name — it read it, again.

PythonPython not started

Usage, and what it costs

PythonPython not started

Output tokens cost about five times what input tokens cost, on every model. A prompt that asks for a short answer is cheaper twice over — fewer output tokens now, and a shorter history to resend later.

A conversation, properly

PythonPython not started

Try it yourself

Exercise 1

Write extract_text(response): join the text of every text block, ignoring all other block types, and return an empty string when there are none.

Your turnPython not started
Exercise 2

Write call_cost(model, usage) returning the dollar cost of one call, and monthly(model, usage, calls) for that many calls a month.

Your turnPython not started

Check yourself

Why does the input token count grow with every turn of a conversation?

Check yourself

A reply comes back with stop_reason 'max_tokens'. What happened, and what should your code do?

Recap

  • Everything is one POST to /v1/messages: model, max_tokens, system and messages.
  • content is a list of typed blocks — iterate and check block.type rather than indexing.
  • Check stop_reason before trusting the content; stop_details exists only for "refusal".
  • The API is stateless, so a conversation resends its whole history and input tokens grow quadratically with its length.
  • Output tokens cost about five times input tokens.
  • Append response.content to the history, not the extracted string.