Avi Santoso
Open main menu
AI engineering

Should You Use LangChain When building a New AI chatbot or Just Write From scratch?

Compare LangChain with provider SDKs for new AI chatbots, including when framework overhead becomes worthwhile.

By Avi Santoso4 min read

LangChain versus provider SDKs for a new AI chatbot

Some developers reach for LangChain the moment they decide to build something with an LLM.

It's popular, documented, and looks like it solves the hard parts for you.

My opinion is that this is only true if you've got a complex app, and most chatbot MVPs won't need it.

For simpler systems, LangChain is often just overhead instead of saving you time.

For those who haven't used it before, LangChain sits between your application and the model provider APIs (OpenAI, Anthropic, Gemini) and smooths out the differences between them.

Each provider has its own message schema, its own way of handling tool calls, and its own structured output format.

LangChain wraps all of that into a single interface: HumanMessage, AIMessage, SystemMessage.

You write your conversation logic once, and swapping models is easy.

On top of that, it handles streaming, retries, batching, and output validation through LCEL (the LangChain Expression Language).

However, in 2026, provider SDKs (OpenAI, Anthropic, etc) now handle streaming, tool calling, and structured outputs by default.

If you're building on a single provider, which most MVPs do, LangChain is ironing out differences that don't exist.

You don't really get any benefits from the overhead.

LangChain queries consume more tokens than direct API calls, with at least one person reporting a 2.7x cost increase in their RAG system. 1

That gap comes from verbose default prompt templates the framework injects automatically.

At scale, that overhead adds up fast.

Also, when a raw API call fails, the error points directly at the problem: a bad request, a network timeout, a wrong schema.

When a LangChain RunnableSequence fails, the traceback goes 20 layers deep through internal framework code you didn't write.

The framework assembles the real prompt at runtime from templates, history placeholders, tool schemas, and intermediate outputs.

You can't inspect what went over the wire unless you pull in LangSmith, LangChain's own tracing tool. 2

That's not a good trade-off when you're running a complex multi-step agent and need replay debugging.

It's an even worse trade-off when you're just building a chat interface with one tool call.

LangChain makes sense in three scenarios:

  1. Cyclic agent loops
  2. Complex RAG pipelines with multi-turn query rewriting
  3. Workflows that fan out across multiple models in parallel.

If your system needs to plan, call a tool, observe the result, decide whether to call another tool, and loop until it has enough to answer, that's where LangGraph becomes worth the dependency.

It gives you state persistence across steps, fault tolerance if the server crashes mid-run, and time-travel debugging where you can inspect what the agent was thinking at any checkpoint. 3

You can build that yourself, but you'll end up with your own internal LangChain anyway, except now you've got to document and maintain it.

Plus, you don't get any acceleration from the large and active community working with LangChain.

The rule of thumb is roughly three or more coordinated steps per user message.

Below that, the raw SDK is faster to ship, cheaper to run, and easier to debug.

Above that, you start saving time and money using LangChain.

Here are some alternatives if you're starting from scratch.

If you're writing TypeScript in a Next.js app, use the Vercel AI SDK. 4 (Or even in a non-Next.js app, it's still one of my favourite libraries for this.)

It's built around hooks, not chains, and a streaming chat UI takes about 20 lines instead of 100.

If you're in Python and need to switch providers without extra overhead, use the OpenAI SDK and point it at a LiteLLM proxy you self-host.

You get access to every major model with no framework overhead.

If the model needs to return a specific JSON schema, use Instructor reliably. 5

Define a Pydantic class; it handles the prompting, validation, and retries, then gets out of the way.

None of these choices locks you out of LangChain later if you need it.

My final suggestion:

Start directly on the provider SDK (or Vercel AI SDK).

Set up a DB-backed conversation store, write your RAG adapter manually, and implement your tool loop on the SDK.

It takes one to two weeks, but you'll understand every line.

In week two, if a specific part is getting painful (parallel fan-out, cyclic agent state, cross-provider normalisation), migrate only that part to LCEL or LangGraph.

References

  1. Himanjan (2024). The Hidden Cost of LangChain: Why My Simple RAG System Cost 2.7x More Than Expected. DEV Community. https://dev.to/himanjan/the-hidden-cost-of-langchain-why-my-simple-rag-system-cost-27x-more-than-expected-4hk9
  2. LangChain (2025). LangSmith: AI Agent & LLM Observability Platform. https://www.langchain.com/langsmith/observability
  3. LangChain (2025). Persistence — LangGraph Documentation. https://docs.langchain.com/oss/python/langgraph/persistence
  4. Vercel (2025). AI SDK — Introduction. https://ai-sdk.dev/docs/introduction
  5. Instructor (2025). Structured LLM Outputs with Pydantic. https://python.useinstructor.com/