Stack Guide
What to build an AI app with in 2026
An AI product is a normal web application with three unusual problems bolted on: a vendor whose prices and models change every quarter, a latency profile measured in seconds, and a cost line that scales with usage instead of headcount. This guide picks a stack that isolates all three.
About 6 min read. Recommendations verified October 2026.
The recommended AI app stack
The organizing principle: never let a model provider's SDK reach further into your codebase than one adapter module. Models get deprecated. Your retrieval layer and your product should not care.
| Layer | Pick | Why |
|---|---|---|
| App framework | Next.js 16 | Streaming responses from a server route to a React client is the core interaction of every AI product, and this is the shortest path to it. |
| Model layer | AI SDK v7 | One interface across OpenAI, Anthropic, Google, and open models, with streaming, tool calls, and structured output. Swapping providers is a one-line change. |
| Default model | A mid-tier frontier model | Claude Sonnet 5.5 sits at $2 in and $10 out per million tokens - the current sweet spot for general reasoning without the top-tier price. |
| Cheap path | A small model plus batching | Route classification, extraction, and tagging to a small model. Anything that is not user-facing goes through the Batch API at a flat 50% discount. |
| Vector store | pgvector on Postgres 18 | 8 to 25 ms p95 under 10M vectors for roughly $30 a month, with embeddings, documents, and permissions joinable in one SQL query. |
| Long-running work | A durable job queue | Agent runs outlive HTTP timeouts. Persist the run, stream progress from it, and make every step idempotent so retries are safe. |
| Caching | Provider prompt caching | Cache the stable system prompt and retrieved context at the provider, and cache full responses for identical inputs in your own store. |
| Evaluation | A fixed eval set in CI | Fifty real cases with expected outcomes catches the regression when a provider silently updates a model. Nothing else will. |
| Observability | Trace every call | Log prompt, model, token counts, latency, and cost per request. Without this you cannot debug quality or explain the bill. |
| Guardrails | Per-user token budgets | A hard daily cap per account, enforced server side. One scripted user can otherwise spend a month of runway in an afternoon. |
The verdict
Next.js 16 with the AI SDK v7 in front of a mid-tier frontier model, retrieval from pgvector on your existing Postgres, a durable queue for anything slow, and per-user token caps from the first commit.
Why this stack survives the next model release
The single most expensive mistake in AI applications is coupling. A codebase that calls one vendor's SDK from fifty places cannot switch when that vendor deprecates a model, raises prices, or has a bad week of uptime. The AI SDK gives you one interface over every major provider with streaming, tool calling, and structured output normalized, so switching models is a configuration change rather than a project.
On retrieval, start with pgvector and do not apologize for it. For workloads under 10 million vectors it delivers 8 to 25 ms p95 latency for around $30 a month, and with pgvectorscale it stays competitive well past 50 million. The decisive advantage is not speed, it is that your embeddings live in the same transaction as your documents and your permission rows, so "only search what this user is allowed to see" is a WHERE clause instead of a distributed consistency problem.
Cost discipline is architecture, not an afterthought. Three levers cover most of it: route non-reasoning work to a small model, push anything that does not need to be interactive through the Batch API for a flat 50% discount, and cache the system prompt and retrieved context at the provider so you stop paying full price to resend the same 4,000 tokens on every turn. Then add a hard per-user daily token budget enforced on the server, because the alternative is finding out about it on an invoice.
Finally, build the eval set before you tune a single prompt. Fifty real inputs with expected outputs, run in CI, is the only thing standing between you and a silent quality regression the next time a provider updates a model behind the same name. Prompt engineering without evals is guessing with extra steps.
Credible alternatives and when they win
Python with FastAPI and LangGraph
Wins when the AI work is the product rather than a feature: multi-step agent graphs, custom evaluation harnesses, fine-tuning pipelines, or anything adjacent to data science. The research ecosystem is Python-first and always will be. Pair it with a TypeScript frontend over a plain HTTP API.
Qdrant or a dedicated vector database
Wins past roughly 50 to 100 million vectors, where HNSW index rebuild times in Postgres become a real operational constraint, or when you need first-class hybrid search and heavy metadata filtering. Qdrant Cloud has a free 1 GB cluster and bills paid clusters by the hour on RAM, CPU and disk, so price your cluster in its calculator, and expect to keep Postgres alongside it.
Pinecone
Wins when you want zero operational involvement in the retrieval layer and are happy to pay for it - the Standard plan starts at a $50 monthly minimum, with storage, reads and writes billed on top. A reasonable trade for a small team with no database expertise and no appetite for index tuning.
Self-hosted open-weight models
Wins on high, steady volume with a narrow task, or when data residency rules forbid sending text to a third party. You trade an API bill for GPU capacity planning, so it only pays off when utilization is high and the workload is predictable.
Decision factors that change the answer
- Feature or product. An AI feature inside an existing app should reuse that app's stack. An AI-native product with agent orchestration at its core probably belongs in Python.
- Corpus size. Under 10 million vectors, pgvector with no second system. Past 50 million, start pricing a dedicated engine and budget for index rebuild windows.
- Data residency. If customer text cannot leave your infrastructure, that constraint decides the model layer before anything else does, and it usually means open weights on your own hardware.
- Interactivity. A user waiting on a stream needs a fast model and prompt caching. A nightly enrichment job should be batched at half price and nobody will notice the latency.
- Accuracy stakes. Summarizing a meeting tolerates a wrong answer. Approving an invoice does not. High stakes means retrieval with citations, a human in the loop, and a much larger eval set.
AI app stack questions, answered
Do I need a vector database, or is pgvector enough?
pgvector is enough for the overwhelming majority of retrieval workloads. It handles up to about 50 million vectors comfortably, costs roughly $30 a month, and keeps embeddings in the same database as the documents and permissions they belong to. Move to Qdrant or Pinecone when index rebuild times or hybrid search requirements become the actual bottleneck, not before.
Which model should I default to in 2026?
A mid-tier frontier model such as Claude Sonnet 5.5 at $2 input and $10 output per million tokens. Top-tier models cost two to five times more and are worth it only for genuinely hard reasoning. Route classification, extraction, and routing decisions to a small cheap model, and keep the choice behind one adapter so you can re-benchmark next quarter.
How do I keep LLM costs from blowing up?
Four things, in order of impact: enforce a hard per-user daily token budget on the server, send non-interactive work through the Batch API for a flat 50% discount, enable provider prompt caching so you stop resending the same context at full price, and log token counts and cost per request so you can see which feature is expensive.
Should I build my AI app in Python or TypeScript?
TypeScript if the AI is a feature in a web product, because the AI SDK covers streaming, tool calls, and structured output and you avoid running two languages. Python if orchestration, evaluation harnesses, or model training are core to the product, since the research ecosystem is Python-first. Splitting them across an HTTP boundary is a fine answer too.
Where to go next
Survey the model and framework landscape in AI tools, price the retrieval layer in databases, and set up tracing from monitoring. If the AI work is going to sit behind its own service, the backend API guide covers that boundary, and Vercel vs Cloudflare covers where to run it.