Self‑hosting teams can now cut hardware, SaaS fees, and operational churn by consolidating keyword and vector search into a single OpenSearch 3.5 deployment.
The February 10 2026 release of OpenSearch 3.5—paired with the March 3 FP16 SIMD benchmark that shows near‑GPU‑level throughput—means the long‑standing belief that “you need a dedicated vector database” no longer holds for the majority of self‑hosted Retrieval‑Augmented Generation (RAG) pipelines. A single OpenSearch cluster now delivers acceptable recall, latency, and cost, while dedicated vector stores still win in niche scenarios that demand extreme recall tuning or AI‑native workflow ergonomics.
Does OpenSearch 3.5 really deliver GPU‑like vector performance?
OpenSearch’s own announcement highlighted that version 3.5 adds FP16‑optimized SIMD kernels, allowing the engine to process dense embeddings at ~2 billion vectors per second on a single x86 node. The March 3 benchmark demonstrates that this throughput rivals entry‑level GPUs, effectively closing the performance gap that previously forced teams onto separate ANN‑specialized stores. In practice, this means a self‑hosted RAG stack can embed, index, and query vectors without provisioning a second service just for similarity search.
How does the cost picture change when you drop a dedicated vector DB?
Self‑hosted teams have long argued that “vector databases are cheaper than scaling Elasticsearch clusters,” but the real savings come from operational efficiency rather than raw node price — as detailed in a recent Kindalame analysis of Elasticsearch‑to‑Qdrant migrations. The article notes that “the real economic incentive for self‑hosted teams lies in reduced hardware footprint and risk mitigation” — factors that now apply equally to OpenSearch 3.5 because the same node can run both keyword and vector workloads (Why self‑hosted teams are replacing Elasticsearch with Qdrant for high‑recall internal search). Eliminating a second service cuts licensing (or managed‑service) fees, halves the number of monitoring pipelines, and simplifies backup/restore procedures.
A concrete example from the community shows a $360/month OpenSearch deployment that, after upgrading to 3.5 and enabling the SIMD kernels, handled a 150 GB document corpus with vector search at sub‑50 ms latency, removing the need for a separate Qdrant instance and saving roughly $200 per month in SaaS and infrastructure costs (Hacker News discussion of the $360/month OpenSearch replacement).
When do dedicated vector databases still outperform a unified OpenSearch stack?
Even with the SIMD boost, OpenSearch’s ANN algorithms are general‑purpose and lack the fine‑grained recall‑tuning knobs that purpose‑built stores like Qdrant or Weaviate expose. Research on RAG systems using Qdrant demonstrates that “purpose‑built retrieval stacks finally deliver the recall, query‑expansion, and hybrid scoring pipelines that pure search engines struggle to match” (Why self‑hosted teams are replacing Elasticsearch with Qdrant for high‑recall internal search).
If your application demands high‑recall legal search, multi‑modal similarity, or aggressive HNSW‑parameter tuning, a dedicated vector DB still offers a measurable edge. The academic study on LLM selection and vector database tuning shows that Qdrant’s ANN configuration can increase recall by up to 12 % on a large biographical dataset compared with generic vector search (LLM Selection and Vector Database Tuning: A Methodology for Enhancing RAG Systems). In such cases, the extra operational complexity is justified by the downstream improvement in answer quality.
Does a single OpenSearch stack simplify the RAG pipeline for self‑hosters?
Beyond raw performance, OpenSearch 3.5 reduces architectural sprawl. The OpenSearch blog walk‑through of a DeepSeek‑powered RAG pipeline illustrates how the same client library (opensearch-py) can ingest embeddings, store them in the same index as full‑text fields, and query with a hybrid knn + match DSL (Zero to RAG: A quick OpenSearch vector database and DeepSeek integration). This eliminates the need for a separate SDK, separate network ACLs, and cross‑service authentication layers that a dual‑stack architecture would require.
The “missing piece” narrative for self‑hosted AI stacks—often illustrated with a separate vector store, an embedding model, and a keyword engine—now collapses into a single OpenSearch service that can act as both the document store and the similarity engine (Why Vector Databases Are the Missing Piece in Your Self‑Hosted AI Stack). Teams can therefore:
- Deploy one Helm chart instead of two, cutting Helm release management time by ~30 %.
- Consolidate logs and metrics into a single Prometheus scrape target, simplifying alerting.
- Use the same backup schedule for both keyword and vector data, reducing storage costs.
What trade‑offs should teams evaluate before consolidating?
Switching to a monolithic OpenSearch deployment is not a silver bullet. Teams need to weigh:
| Factor | OpenSearch 3.5 (single stack) | Dedicated vector DB |
|---|---|---|
| Latency | Sub‑50 ms for moderate loads (thanks to FP16 SIMD) | Sub‑20 ms at scale (GPU‑accelerated) |
| Recall tuning | Limited HNSW parameter exposure | Full HNSW, IVF, PQ tuning |
| Operational overhead | One service, one monitoring pipeline | Two services, duplicated ops |
| Hardware footprint | One node can handle both workloads | Two nodes (or more) required |
| Ecosystem integrations | Native OpenSearch plugins, Kibana UI | Specialized SDKs, separate UI tools |
The decision hinges on your recall requirements versus your ops budget. For most internal knowledge‑base assistants, the modest recall drop is outweighed by the cost and complexity savings. For mission‑critical search where every missed document is costly, a dedicated vector store remains the safer bet.
What’s your experience? Have you already migrated a self‑hosted RAG pipeline to OpenSearch 3.5, or are you still weighing the trade‑offs? Share your successes, challenges, or questions in the comments—let’s figure out together where the single‑stack approach truly shines.