LLM Frameworks — LangChain and LlamaIndex: When to Use Them

What LangChain and LlamaIndex give you, what they cost in complexity, and how to decide between a framework and plain SDK code.

Est. 25 min Updated Suggest a fix
// Part 01 — Why frameworks

Frameworks Package Repeated LLM Application Patterns

A provider SDK helps you call a model. An application framework can connect that call to prompts, structured outputs, retrieval, tools, memory, or orchestration. LangChain offers a broad set of components and orchestration patterns; LlamaIndex focuses on connecting data to LLM applications, including ingestion and retrieval workflows. Their exact features evolve, so compare current documentation before committing to an API.

A framework is not automatically more reliable or more intelligent. It is an abstraction layer your team must understand, operate, test, and upgrade. For a single model call, the plain SDK may be clearer. For a pipeline with many reusable integrations, a framework can save implementation effort—if you need the features it provides.

ApproachOften a good fit whenMain trade-off
Plain provider SDKOne or a few calls and a small explicit workflowYou implement integrations and composition yourself
LangChainYou need its current integrations or composable orchestration patternsMore abstractions and version-sensitive behavior
LlamaIndexData ingestion, indexing, and retrieval are central to the featureData framework concepts and configuration to learn
Mixed / narrow useOne framework component solves a measured gapKeep ownership boundaries and conversions explicit
💡 Note
Choose the smallest abstraction that makes the important behavior explicit and testable. A familiar framework is not a requirement, and a plain SDK is not a sign of an immature design.
// Part 02 — Decision criteria

Start with the Workflow, Not the Library Name

Describe the feature first: What data must be retrieved? Which steps call a model? Do you need branching, state, retries, evaluation hooks, or provider portability? Which component owns authorization and validation? Once these needs are clear, compare framework capabilities with the actual requirements and inspect whether the abstraction hides behavior you need to observe or control.

  1. Write down the user workflow and data boundaries in plain language.
  2. Identify repeated plumbing you would otherwise implement: retrieval connectors, parsing, or orchestration.
  3. Check framework coverage, documentation, maintenance, and current provider support for your required path.
  4. Prototype the smallest representative slice and measure implementation effort and runtime behavior.
  5. Keep a thin application-owned interface around model, retrieval, and domain logic.

For a small FAQ assistant, a direct SDK call plus a vector-store query may remain understandable. If the product grows into multiple document sources, transformations, and retrieval evaluation, a data-oriented framework may be useful. If the workflow becomes a multi-step orchestration with several model and tool integrations, a broader framework may reduce glue code. Those are starting hypotheses to test, not rules.

🎯 Pro Tip
Compare the implementation that would be maintained next year, not only the number of lines in a tutorial. Consider observability, testing, dependency upgrades, and how easily a new engineer can trace one request.
// Part 03 — Plain SDK

Keep a Direct Call When the Workflow Is Small

A provider SDK makes the network boundary visible. You can own request construction, retry policy, context selection, and error mapping without learning a second abstraction. A small function is easy to unit test when you keep the provider client behind a boundary and return an application-level result rather than leaking SDK types everywhere.

llm_boundary.py
python
from dataclasses import dataclass
from typing import Protocol

@dataclass(frozen=True)
class Generation:
    text: str
    input_tokens: int
    output_tokens: int

class TextModel(Protocol):
    def generate(self, system: str, user: str) -> Generation: ...

class StudyHelper:
    def __init__(self, model: TextModel) -> None:
        self.model = model

    def explain(self, topic: str) -> str:
        result = self.model.generate(
            "Explain accurately for a beginner. If unsure, say so.",
            topic,
        )
        return result.text.strip()

The protocol is application-owned, so a provider adapter or a fake test implementation can satisfy it. This example is not a provider SDK implementation; it shows the boundary to preserve. Read model IDs and credentials from runtime configuration, and keep timeout and retry behavior visible at the adapter.

// Part 04 — LangChain

Use a Broad Framework for Integrations You Actually Need

LangChain can provide reusable integrations and abstractions for models, prompts, tools, retrieval, and orchestration. That can be useful when the application composes multiple parts or needs an integration already maintained by its ecosystem. Start with one path end-to-end and make sure you can identify the underlying request, inputs, outputs, and failures when debugging.

The cost is a larger dependency surface and additional concepts between your code and the provider. APIs and package layouts can change; avoid copying an old code sample without checking the version you have installed. Keep business logic and permissions outside a chain so they remain ordinary testable application rules.

framework_boundary.py
python
from dataclasses import dataclass
from typing import Protocol

@dataclass(frozen=True)
class Answer:
    text: str
    source_ids: tuple[str, ...]

class AnswerPipeline(Protocol):
    def run(self, question: str, user_id: str) -> Answer: ...

class HelpService:
    def __init__(self, pipeline: AnswerPipeline) -> None:
        self.pipeline = pipeline

    def respond(self, question: str, user_id: str) -> Answer:
        # Authorization happens before private retrieval inside the application.
        if not user_id:
            raise PermissionError("Sign in required")
        return self.pipeline.run(question, user_id)

The interface lets the service depend on a capability rather than a specific framework class. The framework adapter can handle composition, while your app keeps control over identity, policy, output validation, and error mapping.

// Part 05 — LlamaIndex

Use a Data-Oriented Framework When Retrieval Is the Hard Part

LlamaIndex is designed around connecting data to LLM applications. Its ecosystem can help structure ingestion, indexing, retrieval, and query workflows across supported sources. If document processing and retrieval are your main complexity, its abstractions may be a closer fit than assembling every connector yourself.

Retrieval frameworks do not remove the need to understand chunking, metadata, access filters, embedding behavior, and relevance evaluation. Confirm that retrieved records respect user permissions and that you can inspect the evidence behind an answer. Keep an escape hatch for provider-specific options you need to control.

QuestionPlain SDK pathFramework path
How much retrieval plumbing?Own and test each integrationReuse supported ingestion and retrieval components
How visible is behavior?Direct request and response pathMay pass through several abstractions
How are changes upgraded?SDK and your adapterFramework plus integration dependencies
Where is it strongest?Small explicit workflowsRepeated integrations or data orchestration
📌 Real World Example
A study-material search feature that imports documents, splits them, indexes metadata, and retrieves passages may benefit from a data framework. A static list of a few trusted FAQ entries may not need one.
// Part 06 — Testability and operations

Keep the Important Contract Independent of the Framework

Regardless of library, preserve application-owned types for inputs and results. Test authorization, prompt inputs, expected output schemas, tool permissions, and failure handling separately. Add contract tests at framework adapters and end-to-end evaluation cases for retrieval quality. Instrument model and retrieval spans at stable boundaries so a framework upgrade does not erase operational visibility.

test_service.py
python
from dataclasses import dataclass

@dataclass
class FakeModel:
    received_system: str = ""
    received_user: str = ""

    def generate(self, system: str, user: str):
        self.received_system = system
        self.received_user = user
        return type("Result", (), {"text": "A clear explanation.",
                                    "input_tokens": 12, "output_tokens": 5})()

def test_service_passes_question_through_boundary() -> None:
    fake = FakeModel()
    service = StudyHelper(fake)
    answer = service.explain("What is a database index?")
    assert answer == "A clear explanation."
    assert fake.received_user == "What is a database index?"

This small test verifies the domain boundary without a network call. Add separate tests for adapter behavior and use a controlled golden evaluation set to assess model output. A framework’s own example tests cannot guarantee your product-level behavior.

// Part 07 — Adopt or remove

Treat a Framework as a Reversible Decision

Adopt a framework for a concrete reason: it removes repeated maintenance, provides a needed integration, or makes a complex workflow clearer. Define the boundary so the rest of the application does not depend on framework-specific objects. Record the version and configuration, and upgrade in a small step with regression tests and trace comparison.

If a framework becomes difficult to reason about, trace the feature that causes pain. A narrow adapter can call the provider SDK directly for one specialized operation while leaving other framework integrations intact. If most of the framework is unused, compare the cost of migration with the ongoing dependency and debugging cost; do not rewrite solely for stylistic preference.

⚠️ Important
Avoid stacking multiple orchestration frameworks without a clear boundary. Each layer adds concepts, failure modes, and upgrades that your team must own.
// Part 08 — Pitfalls

Common Framework Mistakes

Choosing a framework before describing the product workflow

Why: A tutorial or trend can lead to abstractions the feature does not need.

Fix: List requirements first and prototype the smallest representative path.

Passing framework objects across the whole application

Why: Business logic becomes coupled to a library and difficult to replace.

Fix: Return application-owned types from a narrow adapter.

Assuming a retrieval abstraction guarantees relevant evidence

Why: Indexing configuration and data quality still determine results.

Fix: Inspect retrieved chunks and evaluate recall, relevance, and access filtering.

Upgrading a framework without a regression check

Why: Transitive integrations and behavior may change across versions.

Fix: Pin dependencies appropriately, read current release notes, and replay tests and traces.

// Part 09 — Practice

Compare Two Implementations of One Feature

Practice

Choose based on maintenance and measurable fit

  1. Sketch a small assistant that retrieves a learner’s course notes and answers with cited passages.
  2. Implement the model boundary and orchestration with a plain SDK adapter or a fake interface before adding a framework.
  3. Prototype the same representative flow with either LangChain or LlamaIndex after checking current official documentation.
  4. Compare dependencies, code ownership, trace visibility, authorization placement, test effort, and retrieval controls.
  5. Write a short decision note naming the chosen approach, what would cause reconsideration, and which behaviors your tests guarantee.

Hint: Do not compare only line count. Keep the same workflow, provider configuration, and evaluation cases for both prototypes.

🎯 Key Takeaways

  • ✓A plain provider SDK is often clearest for a small, explicit workflow.
  • ✓LangChain offers a broad set of integrations and orchestration abstractions; LlamaIndex emphasizes data and retrieval workflows.
  • ✓Framework features and APIs evolve, so verify current documentation for the installed version.
  • ✓Evaluate dependency surface, debuggability, testability, observability, maintenance, and authorization boundaries.
  • ✓Keep business rules and application-owned types outside framework-specific objects.
  • ✓Choose an abstraction for a concrete need and make it reversible with a narrow adapter and regression tests.
  • Consult the current official LangChain and LlamaIndex documentation, especially integration support, version-specific APIs, and migration guidance.

Next

What comes next

Run open-weight models locally with Ollama, serve them with vLLM, pick a model from Hugging Face, and compare cost, latency and quality with hosted APIs.

Lesson 13 · Running Open Models — Ollama, vLLM and Hugging Face →

HELP US IMPROVE

How was this lesson?

Your note is shared with the Chaduvuko team for review. Please don’t include personal or sensitive information.

Choose your feedback

Spotted a mistake or something unclear? Suggest a fix on GitHub or email us. Select the text first and it is quoted for you.

Share

Discussion

0

Have a better approach? Found something outdated? Share it — your knowledge helps everyone learning here.

Continue with GitHub
Loading...