RAG 2.0?
This blog post on “RAG 2.0” by Contextual AI got me thinking. Not sure, if it makes sense for anyone to be “announcing” (or even defining) RAG 2.0 but there are a few tidbits in this post hinting towards a potentially more powerful, general approach to RAG they are working on. The article is light on technical details and heavy on claimed “state-of-the-art” results on various benchmarks.
However, I very much agree that a) defining evaluation datasets first, and b) then end-to-end optimizing RAG performance is the right approach to improve (pre-)production systems.
What’s next for RAG?
That’s an excellent question. For one, I think it’s important to remember that RAG is more of a concept than a single approach, let alone a singular product. That means that, from my point of view, it is very hard to build RAG systems that make sense in general and across domains, even if the underlying models are finetuned to said domain.
For example, I am unsure whether a RAG-based chatbot over a somewhat well-organized website (which is a VERY popular thing to do these days) brings any value that a simple keyword search in combination with the user simply clicking around the menu items does not already provide.
Where there is massive room for a “RAG 2.0” is in the user experience when interacting with the product. For example, building systems that capture previous user interactions and use these to improve the next interaction. If I read all kinds of things about concept X in the past, I want to interact with a system that remembers this and does not start from scratch every time I ask a question, resurfacing basic articles on concept X.
Furthermore, users often have a vague idea of what they are looking for and where. Why not give users the ability to steer answers in the right direction? Such approaches are much more difficult to benchmark (and generalize) but at the end of the day hopefully more useful to the user.
