Adding an AI Feature — Streaming Chat in a Web App
Call an LLM from a server route, stream the answer into a React UI, keep API keys server-side, and handle limits and failures.
Give AI a Specific Job in the Study Planner
An AI feature is useful when it helps a learner make progress on a concrete task. Our planner can offer a study coach that turns a student's own topic and available time into a short revision plan. It should not pretend to know the student's exam syllabus or guarantee a grade. Start with a clearly bounded task and a visible way for the learner to review the suggestion.
Before adding a model call, decide what information the feature needs, what it must not receive, how the learner will evaluate the result, and what happens when the provider is unavailable. A generated answer is not verified educational content. Keep the existing planner useful without AI and never block essential study scheduling on model availability.
| Design question | Study-coach decision |
|---|---|
| User value | Suggest a time-boxed revision outline for one topic |
| Input | Topic and available minutes; no unnecessary personal data |
| Output | A draft the student can edit, not an authoritative syllabus |
| Failure | Show an understandable error and preserve the typed request |
| Cost boundary | Limit input size, output size, and request frequency |
Keep the Provider Call on the Server
The browser should call your application route, not the model provider directly. The server route authenticates the student, validates and bounds input, enforces limits, calls the provider, and streams a response to the browser. The provider key and selected model name stay in server environment variables. A key placed in client code or a public-prefixed variable is visible to users and can be abused.
OPENAI_API_KEY=replace-with-a-secret-from-your-provider
LLM_MODEL=choose-a-supported-model-in-your-provider-accountUse a secret manager for deployed environments and add local secret files to the repository's ignore rules. Do not copy real keys into lessons, screenshots, logs, or support requests. Read the model name from configuration; model availability and naming change, so the application should not assume one hard-coded name.
Define a Narrow Request Contract
Accept only the fields needed for this feature. Enforce reasonable length and range limits before calling the provider, and reject unexpected shapes. Authentication and per-user rate limiting should happen before spending provider tokens. Consider storing only the final study plan if the product needs history; avoid retaining raw prompts by default when they may contain personal information.
import { z } from "zod"
const requestSchema = z.object({
topic: z.string().trim().min(2).max(300),
availableMinutes: z.number().int().min(10).max(240),
})
export async function POST(request: Request) {
const user = await requireSignedInStudent()
await enforceCoachRateLimit(user.id)
const body: unknown = await request.json()
const parsed = requestSchema.safeParse(body)
if (!parsed.success) {
return Response.json({ error: "Enter a topic and available study time." }, { status: 400 })
}
return streamStudyPlan(parsed.data)
}The authentication and rate-limit helpers represent application infrastructure that must be implemented and tested. A hidden button is not a rate limit; enforce limits on the server and consider both per-account and global provider budgets. Also handle malformed JSON and provider failures without returning internal details.
Convert Model Events into a Web Response
Streaming can show the first text before the entire response has been generated. With the OpenAI JavaScript SDK, create a streamed response using the configured model and process text-delta events. Bridge those events into a Web ReadableStream that the browser can consume. Keep output limits and a timeout appropriate to the feature.
import OpenAI from "openai"
const apiKey = process.env.OPENAI_API_KEY
const model = process.env.LLM_MODEL
if (!apiKey || !model) throw new Error("Configure the server-side model credentials")
const client = new OpenAI({ apiKey })
type CoachInput = { topic: string; availableMinutes: number }
export async function streamStudyPlan(input: CoachInput): Promise<Response> {
const events = await client.responses.create({
model,
instructions: "You are a careful study coach for Indian students. Give a practical draft, avoid claiming knowledge of an unseen syllabus, and encourage the learner to verify course requirements.",
input: "Make a revision outline for " + input.topic + " in " + input.availableMinutes + " minutes. Include checkpoints and one self-test.",
max_output_tokens: 500,
stream: true,
})
const encoder = new TextEncoder()
const body = new ReadableStream<Uint8Array>({
async start(controller) {
try {
for await (const event of events) {
if (event.type === "response.output_text.delta") {
controller.enqueue(encoder.encode(event.delta))
}
}
controller.close()
} catch (error) {
console.error("Study coach stream failed", error)
controller.error(new Error("The study coach could not finish this request"))
}
},
cancel() {
// A client disconnect should stop forwarding work where the SDK permits it.
},
})
return new Response(body, {
headers: {
"Content-Type": "text/plain; charset=utf-8",
"Cache-Control": "no-cache, no-transform",
"X-Content-Type-Options": "nosniff",
},
})
}The example focuses on the event-to-response bridge. Production code should connect request cancellation to provider cancellation where supported, set request timeouts, handle provider errors before streaming begins, log a correlation ID and safe usage metadata, and decide how to communicate failures after headers have been sent. Do not log full prompts by default.
Read the Response Incrementally in React
The client can POST the topic to the route and read the response body as a stream. Append each decoded chunk to the current answer. Handle a missing body, a non-success status, aborts, and network errors. The response is untrusted text: render it as text unless you have a safe, explicit Markdown rendering pipeline.
"use client"
import { FormEvent, useRef, useState } from "react"
export function StudyCoach() {
const [topic, setTopic] = useState("")
const [answer, setAnswer] = useState("")
const [busy, setBusy] = useState(false)
const [error, setError] = useState<string | null>(null)
const controllerRef = useRef<AbortController | null>(null)
async function submit(event: FormEvent<HTMLFormElement>) {
event.preventDefault()
controllerRef.current?.abort()
const controller = new AbortController()
controllerRef.current = controller
setBusy(true)
setAnswer("")
setError(null)
try {
const response = await fetch("/api/study-coach", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ topic, availableMinutes: 45 }),
signal: controller.signal,
})
if (!response.ok || !response.body) throw new Error("Request failed")
const reader = response.body.getReader()
const decoder = new TextDecoder()
while (true) {
const result = await reader.read()
if (result.done) break
setAnswer((current) => current + decoder.decode(result.value, { stream: true }))
}
setAnswer((current) => current + decoder.decode())
} catch (cause) {
if (!(cause instanceof Error && cause.name === "AbortError")) {
setError("The study coach is unavailable. Please try again.")
}
} finally {
setBusy(false)
}
}
return (
<section>
<form onSubmit={submit}>
<label>What topic are you studying?
<input value={topic} maxLength={300}
onChange={(event) => setTopic(event.target.value)} />
</label>
<button disabled={busy || topic.trim().length < 2}>
{busy ? "Preparing plan…" : "Suggest a plan"}
</button>
</form>
{error && <p role="alert">{error}</p>}
<pre aria-live="polite">{answer}</pre>
</section>
)
}For a polished application, show a cancel action, preserve the user's request after failure, and announce progress accessibly. Consider buffering text updates to avoid re-rendering on every tiny chunk. A streaming UI should not imply that partial output is verified or complete.
Set Limits, Protect Privacy and Review Output
AI features can produce inaccurate, biased, or unsuitable suggestions. State the feature's limitations near the result and give users control to edit or discard it. Avoid sending account identifiers, passwords, private notes, or unrelated planner history to a provider. If you later add retrieval from syllabus files, check access permissions before retrieval and make source context visible to the learner.
- Authenticate the user and confirm feature access before calling the provider.
- Validate and cap input length, output length, time, concurrency, and per-user usage.
- Keep the API key and provider request on the server only.
- Avoid collecting or logging more personal data than the feature needs.
- Display generated content as a draft and provide a safe retry or fallback.
- Measure errors, latency, usage, and user feedback without logging secrets or raw private prompts.
Do not let model output execute SQL, arbitrary code, or privileged tools. If a later feature uses tools, give each tool a narrow schema and permission check, and require human confirmation for meaningful external actions. For this lesson, the model only suggests text and the student remains in control.
Common Mistakes
Calling the provider from a browser component
Why: The credential is exposed and can be used outside your product.
Fix: Proxy the call through an authenticated server route with a secret key.
Allowing unlimited prompts and output
Why: A public endpoint can create unexpected spend and resource exhaustion.
Fix: Validate, rate-limit, cap output, and monitor usage.
Treating generated text as verified facts
Why: Models can produce plausible but inaccurate study advice.
Fix: Label output as a draft, let learners edit it, and verify important claims.
Ignoring disconnects and provider errors
Why: Work continues after the user leaves, or failures appear as a blank screen.
Fix: Support cancellation, clear error states, timeouts, and safe server logs.
Build a Streaming Study Coach
Practice
A bounded AI feature in the planner
- Create a server POST route that authenticates a student and validates topic and study-time input.
- Read the provider key and model from server environment variables and confirm neither appears in the browser bundle.
- Stream text deltas into a client component with loading, cancellation, and error states.
- Add server-side rate and output limits; record safe latency and usage metadata.
- Try inaccurate or irrelevant requests and improve the UI copy so suggestions are clearly drafts to review.
Hint: Keep provider calls behind a server module. Never use a public-prefixed environment variable for a secret.
🎯 Key Takeaways
- ✓Begin with a bounded user problem, not an AI call looking for a use case.
- ✓Keep provider keys and calls on the server; read the model name from configuration.
- ✓Authenticate, validate, rate-limit, and cap requests before spending model tokens.
- ✓Streaming improves perceived responsiveness but still needs cancellation and failure handling.
- ✓Treat generated study advice as a draft that may be inaccurate.
- ✓Minimise personal data in prompts and logs; keep users in control of actions.
- Check the provider's current API documentation for streaming event shapes, supported models, limits, and usage reporting; these details can change.
Next
What comes next
Push live updates to users with server-sent events and WebSockets, choose between them, and scale real-time connections.
Lesson 12 · Real-Time Features — WebSockets and Server-Sent Events →HELP US IMPROVE
How was this lesson?
Your note is shared with the Chaduvuko team for review. Please don’t include personal or sensitive information.
Spotted a mistake or something unclear? Suggest a fix on GitHub or email us. Select the text first and it is quoted for you.
Discussion
0Have a better approach? Found something outdated? Share it — your knowledge helps everyone learning here.
