Calling LLM APIs — SDKs, Messages and Streaming
Make your first model call from Python, structure system and user messages, stream tokens to the user, and handle errors and retries.
An LLM API Is Just an HTTPS Request
Every AI feature you have used — a support chatbot on a bank website, a resume reviewer, a code assistant — sends text to a model over HTTPS and gets text back. There is no magic in the transport. Your code builds a request containing instructions, the conversation so far, and some settings; the provider runs the model; you receive generated text plus usage information you will be billed for.
The three providers you will meet most often are OpenAI, Anthropic, and Google (Gemini). Their request shapes differ slightly, but the ideas are identical. Learn the ideas once and switching providers becomes a small refactor rather than a rewrite.
| Concept | What it means | Why you care |
|---|---|---|
| Model | The specific model version you call | Decides quality, speed, and price per token |
| System / instructions | Rules the model should follow for the whole conversation | Sets tone, scope, and output format |
| Messages | The ordered list of user and assistant turns | Gives the model context; the API itself is stateless |
| Max output tokens | Upper bound on the reply length | Caps cost and latency |
| Usage | Input and output token counts returned with each reply | This is what you pay for |
Install an SDK and Keep Your Key Out of Your Code
Official SDKs wrap the HTTP calls, add types, and handle retries. Create a virtual environment and install the SDK for the provider you have a key for. The examples use OpenAI and Anthropic; the pattern is the same for Gemini.
python -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
pip install openai anthropic python-dotenvAPI keys are passwords that spend money. Put them in an environment variable or a .env file that is listed in .gitignore — never in source code, screenshots, or a notebook you share.
OPENAI_API_KEY=sk-...your-key...
ANTHROPIC_API_KEY=sk-ant-...your-key...
LLM_MODEL=the-model-name-from-your-provider-docsLLM_MODEL above) instead of hard-coding it, so upgrading a model is a config change, not a code change.Your First Request in Python
Here is the smallest useful call with each SDK. Both send an instruction and one user message, then print the reply and token usage.
import os
from dotenv import load_dotenv
from openai import OpenAI
load_dotenv()
client = OpenAI() # reads OPENAI_API_KEY from the environment
response = client.responses.create(
model=os.environ["LLM_MODEL"],
instructions="You are a concise assistant for Indian college students.",
input="Explain what an API is in two sentences.",
max_output_tokens=200,
)
print(response.output_text)
print(response.usage) # input_tokens, output_tokens, total_tokensimport os
from dotenv import load_dotenv
import anthropic
load_dotenv()
client = anthropic.Anthropic() # reads ANTHROPIC_API_KEY
message = client.messages.create(
model=os.environ["LLM_MODEL"],
max_tokens=200,
system="You are a concise assistant for Indian college students.",
messages=[{"role": "user", "content": "Explain what an API is in two sentences."}],
)
print(message.content[0].text)
print(message.usage) # input_tokens, output_tokensNotice what is the same: a model name, an instruction, user content, a length limit, and usage in the response. Those five things are the whole contract.
Messages, Roles and Keeping History
To hold a conversation, keep a list of messages and append each turn. Roles tell the model who said what: user for the person, assistant for earlier model replies, and a system or instructions field for standing rules.
import os
import anthropic
client = anthropic.Anthropic()
SYSTEM = "You are a friendly tutor. Answer in at most 4 sentences."
history: list[dict] = []
while True:
user_text = input("You: ").strip()
if user_text.lower() in {"quit", "exit"}:
break
history.append({"role": "user", "content": user_text})
reply = client.messages.create(
model=os.environ["LLM_MODEL"],
max_tokens=400,
system=SYSTEM,
messages=history, # the WHOLE conversation, every time
)
answer = reply.content[0].text
history.append({"role": "assistant", "content": answer})
print("Tutor:", answer)Because you resend history on every turn, input tokens grow with each message. Real apps trim or summarise old turns once the conversation gets long — you will learn budgeting in the next lesson.
Stream Tokens So Users Are Not Staring at a Spinner
A long answer can take several seconds to finish. Streaming sends tokens as they are generated, so the first words appear almost immediately. Perceived speed matters more to users than total time.
import os
from openai import OpenAI
client = OpenAI()
stream = client.responses.create(
model=os.environ["LLM_MODEL"],
input="Give me 5 tips to prepare for a data engineering interview.",
stream=True,
)
for event in stream:
if event.type == "response.output_text.delta":
print(event.delta, end="", flush=True)
print()import os
import anthropic
client = anthropic.Anthropic()
with client.messages.stream(
model=os.environ["LLM_MODEL"],
max_tokens=600,
messages=[{"role": "user", "content": "Give me 5 tips to prepare for a data engineering interview."}],
) as stream:
for text in stream.text_stream:
print(text, end="", flush=True)
print()In a web app, your server forwards these chunks to the browser using server-sent events or a streaming HTTP response. You will build that in the final lesson of this course.
Errors, Timeouts and Retries
Production calls fail for ordinary reasons. Handle each kind deliberately rather than wrapping everything in one bare except.
| Failure | Typical status | What to do |
|---|---|---|
| Rate limited | 429 | Retry with exponential backoff and jitter; reduce concurrency |
| Provider overloaded or server error | 500, 503, 529 | Retry a few times, then fall back or show a friendly error |
| Timeout | — | Set an explicit timeout; retry once; stream long answers |
| Bad request | 400 | Do not retry — fix the request (often too many tokens or a bad parameter) |
| Authentication | 401, 403 | Do not retry — check the key and permissions |
Both SDKs already retry some errors. Configure the limits explicitly so behaviour is predictable:
import os
import time
import random
from openai import OpenAI, APIStatusError, APITimeoutError, RateLimitError
client = OpenAI(timeout=30.0, max_retries=2) # SDK-level retries for transient errors
def ask(prompt: str, attempts: int = 3) -> str:
for attempt in range(1, attempts + 1):
try:
response = client.responses.create(
model=os.environ["LLM_MODEL"], input=prompt, max_output_tokens=300,
)
return response.output_text
except (RateLimitError, APITimeoutError):
if attempt == attempts:
raise
time.sleep(2 ** attempt + random.random()) # backoff with jitter
except APIStatusError as error:
if error.status_code >= 500 and attempt < attempts:
time.sleep(2 ** attempt)
continue
raise # 4xx errors are bugs in our request: surface them
raise RuntimeError("unreachable")Keep Provider Code Behind One Function
Wrap provider calls in a tiny module of your own. The rest of your app calls generate() and never imports an SDK directly. Swapping providers, adding logging, or recording cost then happens in one place.
import os
from dataclasses import dataclass
import anthropic
_client = anthropic.Anthropic(timeout=30.0, max_retries=2)
@dataclass
class LLMResult:
text: str
input_tokens: int
output_tokens: int
def generate(system: str, messages: list[dict], max_tokens: int = 500) -> LLMResult:
reply = _client.messages.create(
model=os.environ["LLM_MODEL"], system=system, messages=messages, max_tokens=max_tokens,
)
return LLMResult(
text=reply.content[0].text,
input_tokens=reply.usage.input_tokens,
output_tokens=reply.usage.output_tokens,
)- Read the model name and limits from configuration.
- Return your own result type, not the SDK object.
- Record token usage on every call — you will need it for cost tracking.
- Add tracing and evaluation hooks here later (lessons 7 and 9).
Common Mistakes
Hard-coding the API key in a script
Why: It is quick while experimenting, then gets committed.
Fix: Load keys from environment variables and keep .env in .gitignore from day one.
Not setting a max output limit
Why: Defaults can allow very long replies.
Fix: Always set max tokens for the use case; a classifier needs 10 tokens, not 4,000.
Retrying every error
Why: A loop that retries 400 errors burns money and never succeeds.
Fix: Retry only rate limits, timeouts, and 5xx errors, with backoff.
Calling the provider from the browser
Why: Front-end code exposes the key to every visitor.
Fix: Call LLM APIs only from your server, and let the browser talk to your server.
Build a Command-Line Study Buddy
Practice
A streaming CLI tutor with history and safe retries
- Create llm.py with a generate() function like Part 7, reading the model from LLM_MODEL.
- Write a chat loop that keeps history and streams each answer to the terminal.
- Print input and output tokens after every turn, and a running total at the end.
- Keep only the last 6 messages in history and observe how input tokens stop growing.
- Turn off your network mid-conversation and confirm the app shows a friendly error instead of a stack trace.
Hint: Streaming and usage both exist in the SDKs: Anthropic exposes the final message (with usage) via stream.get_final_message().
🎯 Key Takeaways
- ✓An LLM API call is an HTTPS request with a model, instructions, messages, and limits; the response includes text and token usage.
- ✓The API is stateless — conversations work because your code resends history, so input cost grows every turn.
- ✓Keep API keys in environment variables and call providers only from server-side code.
- ✓Stream long answers so users see output immediately.
- ✓Retry rate limits, timeouts, and server errors with backoff; never retry 4xx request errors.
- ✓Hide provider SDKs behind one function so switching models, logging, and cost tracking happen in one place.
- Official references: the OpenAI, Anthropic, and Google Gemini API documentation. Check them for current model names and prices before you ship.
Next
What comes next
You can call a model. Next, learn what a token is, how to estimate the monthly bill for a feature before you build it, and how to stay inside rate limits.
Lesson 02 · Tokens, Pricing and Rate Limits →HELP US IMPROVE
How was this lesson?
Your note is shared with the Chaduvuko team for review. Please don’t include personal or sensitive information.
Spotted a mistake or something unclear? Suggest a fix on GitHub or email us. Select the text first and it is quoted for you.
Discussion
0Have a better approach? Found something outdated? Share it — your knowledge helps everyone learning here.
