LLM Frameworks — LangChain and LlamaIndex: When to Use Them
What LangChain and LlamaIndex give you, what they cost in complexity, and how to decide between a framework and plain SDK code.
Frameworks Package Repeated LLM Application Patterns
A provider SDK helps you call a model. An application framework can connect that call to prompts, structured outputs, retrieval, tools, memory, or orchestration. LangChain offers a broad set of components and orchestration patterns; LlamaIndex focuses on connecting data to LLM applications, including ingestion and retrieval workflows. Their exact features evolve, so compare current documentation before committing to an API.
A framework is not automatically more reliable or more intelligent. It is an abstraction layer your team must understand, operate, test, and upgrade. For a single model call, the plain SDK may be clearer. For a pipeline with many reusable integrations, a framework can save implementation effort—if you need the features it provides.
| Approach | Often a good fit when | Main trade-off |
|---|---|---|
| Plain provider SDK | One or a few calls and a small explicit workflow | You implement integrations and composition yourself |
| LangChain | You need its current integrations or composable orchestration patterns | More abstractions and version-sensitive behavior |
| LlamaIndex | Data ingestion, indexing, and retrieval are central to the feature | Data framework concepts and configuration to learn |
| Mixed / narrow use | One framework component solves a measured gap | Keep ownership boundaries and conversions explicit |
Start with the Workflow, Not the Library Name
Describe the feature first: What data must be retrieved? Which steps call a model? Do you need branching, state, retries, evaluation hooks, or provider portability? Which component owns authorization and validation? Once these needs are clear, compare framework capabilities with the actual requirements and inspect whether the abstraction hides behavior you need to observe or control.
- Write down the user workflow and data boundaries in plain language.
- Identify repeated plumbing you would otherwise implement: retrieval connectors, parsing, or orchestration.
- Check framework coverage, documentation, maintenance, and current provider support for your required path.
- Prototype the smallest representative slice and measure implementation effort and runtime behavior.
- Keep a thin application-owned interface around model, retrieval, and domain logic.
For a small FAQ assistant, a direct SDK call plus a vector-store query may remain understandable. If the product grows into multiple document sources, transformations, and retrieval evaluation, a data-oriented framework may be useful. If the workflow becomes a multi-step orchestration with several model and tool integrations, a broader framework may reduce glue code. Those are starting hypotheses to test, not rules.
Keep a Direct Call When the Workflow Is Small
A provider SDK makes the network boundary visible. You can own request construction, retry policy, context selection, and error mapping without learning a second abstraction. A small function is easy to unit test when you keep the provider client behind a boundary and return an application-level result rather than leaking SDK types everywhere.
from dataclasses import dataclass
from typing import Protocol
@dataclass(frozen=True)
class Generation:
text: str
input_tokens: int
output_tokens: int
class TextModel(Protocol):
def generate(self, system: str, user: str) -> Generation: ...
class StudyHelper:
def __init__(self, model: TextModel) -> None:
self.model = model
def explain(self, topic: str) -> str:
result = self.model.generate(
"Explain accurately for a beginner. If unsure, say so.",
topic,
)
return result.text.strip()The protocol is application-owned, so a provider adapter or a fake test implementation can satisfy it. This example is not a provider SDK implementation; it shows the boundary to preserve. Read model IDs and credentials from runtime configuration, and keep timeout and retry behavior visible at the adapter.
Use a Broad Framework for Integrations You Actually Need
LangChain can provide reusable integrations and abstractions for models, prompts, tools, retrieval, and orchestration. That can be useful when the application composes multiple parts or needs an integration already maintained by its ecosystem. Start with one path end-to-end and make sure you can identify the underlying request, inputs, outputs, and failures when debugging.
The cost is a larger dependency surface and additional concepts between your code and the provider. APIs and package layouts can change; avoid copying an old code sample without checking the version you have installed. Keep business logic and permissions outside a chain so they remain ordinary testable application rules.
from dataclasses import dataclass
from typing import Protocol
@dataclass(frozen=True)
class Answer:
text: str
source_ids: tuple[str, ...]
class AnswerPipeline(Protocol):
def run(self, question: str, user_id: str) -> Answer: ...
class HelpService:
def __init__(self, pipeline: AnswerPipeline) -> None:
self.pipeline = pipeline
def respond(self, question: str, user_id: str) -> Answer:
# Authorization happens before private retrieval inside the application.
if not user_id:
raise PermissionError("Sign in required")
return self.pipeline.run(question, user_id)The interface lets the service depend on a capability rather than a specific framework class. The framework adapter can handle composition, while your app keeps control over identity, policy, output validation, and error mapping.
Use a Data-Oriented Framework When Retrieval Is the Hard Part
LlamaIndex is designed around connecting data to LLM applications. Its ecosystem can help structure ingestion, indexing, retrieval, and query workflows across supported sources. If document processing and retrieval are your main complexity, its abstractions may be a closer fit than assembling every connector yourself.
Retrieval frameworks do not remove the need to understand chunking, metadata, access filters, embedding behavior, and relevance evaluation. Confirm that retrieved records respect user permissions and that you can inspect the evidence behind an answer. Keep an escape hatch for provider-specific options you need to control.
| Question | Plain SDK path | Framework path |
|---|---|---|
| How much retrieval plumbing? | Own and test each integration | Reuse supported ingestion and retrieval components |
| How visible is behavior? | Direct request and response path | May pass through several abstractions |
| How are changes upgraded? | SDK and your adapter | Framework plus integration dependencies |
| Where is it strongest? | Small explicit workflows | Repeated integrations or data orchestration |
Keep the Important Contract Independent of the Framework
Regardless of library, preserve application-owned types for inputs and results. Test authorization, prompt inputs, expected output schemas, tool permissions, and failure handling separately. Add contract tests at framework adapters and end-to-end evaluation cases for retrieval quality. Instrument model and retrieval spans at stable boundaries so a framework upgrade does not erase operational visibility.
from dataclasses import dataclass
@dataclass
class FakeModel:
received_system: str = ""
received_user: str = ""
def generate(self, system: str, user: str):
self.received_system = system
self.received_user = user
return type("Result", (), {"text": "A clear explanation.",
"input_tokens": 12, "output_tokens": 5})()
def test_service_passes_question_through_boundary() -> None:
fake = FakeModel()
service = StudyHelper(fake)
answer = service.explain("What is a database index?")
assert answer == "A clear explanation."
assert fake.received_user == "What is a database index?"This small test verifies the domain boundary without a network call. Add separate tests for adapter behavior and use a controlled golden evaluation set to assess model output. A framework’s own example tests cannot guarantee your product-level behavior.
Treat a Framework as a Reversible Decision
Adopt a framework for a concrete reason: it removes repeated maintenance, provides a needed integration, or makes a complex workflow clearer. Define the boundary so the rest of the application does not depend on framework-specific objects. Record the version and configuration, and upgrade in a small step with regression tests and trace comparison.
If a framework becomes difficult to reason about, trace the feature that causes pain. A narrow adapter can call the provider SDK directly for one specialized operation while leaving other framework integrations intact. If most of the framework is unused, compare the cost of migration with the ongoing dependency and debugging cost; do not rewrite solely for stylistic preference.
Common Framework Mistakes
Choosing a framework before describing the product workflow
Why: A tutorial or trend can lead to abstractions the feature does not need.
Fix: List requirements first and prototype the smallest representative path.
Passing framework objects across the whole application
Why: Business logic becomes coupled to a library and difficult to replace.
Fix: Return application-owned types from a narrow adapter.
Assuming a retrieval abstraction guarantees relevant evidence
Why: Indexing configuration and data quality still determine results.
Fix: Inspect retrieved chunks and evaluate recall, relevance, and access filtering.
Upgrading a framework without a regression check
Why: Transitive integrations and behavior may change across versions.
Fix: Pin dependencies appropriately, read current release notes, and replay tests and traces.
Compare Two Implementations of One Feature
Practice
Choose based on maintenance and measurable fit
- Sketch a small assistant that retrieves a learner’s course notes and answers with cited passages.
- Implement the model boundary and orchestration with a plain SDK adapter or a fake interface before adding a framework.
- Prototype the same representative flow with either LangChain or LlamaIndex after checking current official documentation.
- Compare dependencies, code ownership, trace visibility, authorization placement, test effort, and retrieval controls.
- Write a short decision note naming the chosen approach, what would cause reconsideration, and which behaviors your tests guarantee.
Hint: Do not compare only line count. Keep the same workflow, provider configuration, and evaluation cases for both prototypes.
🎯 Key Takeaways
- ✓A plain provider SDK is often clearest for a small, explicit workflow.
- ✓LangChain offers a broad set of integrations and orchestration abstractions; LlamaIndex emphasizes data and retrieval workflows.
- ✓Framework features and APIs evolve, so verify current documentation for the installed version.
- ✓Evaluate dependency surface, debuggability, testability, observability, maintenance, and authorization boundaries.
- ✓Keep business rules and application-owned types outside framework-specific objects.
- ✓Choose an abstraction for a concrete need and make it reversible with a narrow adapter and regression tests.
- Consult the current official LangChain and LlamaIndex documentation, especially integration support, version-specific APIs, and migration guidance.
Next
What comes next
Run open-weight models locally with Ollama, serve them with vLLM, pick a model from Hugging Face, and compare cost, latency and quality with hosted APIs.
Lesson 13 · Running Open Models — Ollama, vLLM and Hugging Face →HELP US IMPROVE
How was this lesson?
Your note is shared with the Chaduvuko team for review. Please don’t include personal or sensitive information.
Spotted a mistake or something unclear? Suggest a fix on GitHub or email us. Select the text first and it is quoted for you.
Discussion
0Have a better approach? Found something outdated? Share it — your knowledge helps everyone learning here.
