🦜 Doug Turnbull
@softwaredoug.com@rss-parrot.net
I'm an automated parrot! I relay a website's RSS feed to the Fediverse. Every time a new post appears in the feed, I toot about it. Follow me to get all new posts in your Mastodon timeline!
Brought to you by the RSS Parrot.
---
Doug Turnbull's personal blog and essays
Your feed and you don't want it here? Just
e-mail the birb.
Can agents replace the search stack?
http://softwaredoug.com/blog/2026/04/28/search-apis-replaced-by-agents.html
Published: April 28, 2026 00:00
Instead of deploying the traditional query understanding + reranking combo, can we let an agent do all the work?
Metadata: the 3rd kind of retrieval
http://softwaredoug.com/blog/2026/04/21/metadata-the-3rd-kind-of-retrieval.html
Published: April 21, 2026 12:15
In search we talk about lexical embedding retrieval But we miss another retrieval philosophy metadata And with LLMs its never...
What is psuedo-relevance feedback?
http://softwaredoug.com/blog/2026/04/13/what-is-psuedo-relevance-feedback.html
Published: April 13, 2026 12:15
After retrieving BM25 or any ranked search results you might not realize it but you have new information about the...
Is grep all you need for RAG?
http://softwaredoug.com/blog/2026/04/06/agentic-search-is-having-a-grep-moment.html
Published: April 6, 2026 12:15
With enough work you can build a search system with just grep. But it’s not for the faint of heart.
Stop evaluating search with queries
http://softwaredoug.com/blog/2026/03/30/session-vs-query-based-search-evals.html
Published: March 30, 2026 13:56
To evaluate search we typically build a judgment list We transform clickstream data into evaluation data This labels a result...
Agents put the Resilient in RAG
http://softwaredoug.com/blog/2026/03/26/classic-rag-s-achilles-heel-lack-of-resilience.html
Published: March 26, 2026 12:15
Classic RAG expects a linear process 1 An LLM translates the user’s prompt into a query 2 We search on...
We can learn from this late interaction win
http://softwaredoug.com/blog/2026/03/23/we-can-learn-from-this-late-interaction-win.html
Published: March 23, 2026 12:15
Late interaction is having a moment The team at LightOn including superstar developer Antoine Chaffin has demonstrated how a 150M...
How to actually choose a retrieval engine
http://softwaredoug.com/blog/2026/03/20/how-to-actually-choose-a-retrieval-engine.html
Published: March 20, 2026 12:15
How do teams choose vector databases search engines People wrack their brains between Elasticsearch OpenSearch Solr Vespa Pinecone Turbopuffer Weaviate...
Content understanding IS Query understanding
http://softwaredoug.com/blog/2026/03/18/content-understanding-is-query-understanding.html
Published: March 18, 2026 12:15
A user searches for red shoes they click on some products Now you have a set of relevant products Great...
Bayesian BM25 is cool
http://softwaredoug.com/blog/2026/03/16/bayesian-bm25-is-cool.html
Published: March 16, 2026 12:15
Look at this math and grasp at its majesty P(R) = prob(embeddings) * prod(lexical)
Vector search means query understanding!?
http://softwaredoug.com/blog/2026/03/13/vector-search-means-query-understanding.html
Published: March 13, 2026 12:15
Good vector search means more than embeddings Embeddings don’t know when a result matches doesn’t match Similarity floors don’t work...