Name: From Rag to Riches
Uploaded: 2025-12-09
Duration: 16 min 23 s
Description: Want generative AI that actually delivers results? It's harder than it looks. Two leading AI startups that specialize in corporate readiness explore the models and techniques tech leaders must consider to deploy effective AI systems. Open source or closed? Fine-tune your LLM, or retrieval-augmented generation (RAG)? And just how good must your organization's data be? We dig in.

Want generative AI that actually delivers results? It's harder than it looks. Two leading AI startups that specialize in corporate readiness explore the models and techniques tech leaders must consider to deploy effective AI systems. Open source or closed? Fine-tune your LLM, or retrieval-augmented generation (RAG)? And just how good must your organization's data be? We dig in.

From Rag to Riches

Conferences

Great guys, so we're going to talk a little bit in this session about you know how enterprises can really get AI to work at scale and deployments, and a lot of that has to do the title of the session is from Rag to Riches and rag here is retrieval augmented Generation, which some of you. Mayo is a technique for helping AI get the data it needs and the context it needs to give good answers, but we know, you may not be familiar with my 22 panelists. So Dawa contextual AI, just 30 seconds. What is it? What, what do you guys do as a company? Yeah, so you mentioned rag. Reg is really about making Gen AI work on your data, right? So I think one of the The interesting thing about where we are right now is we have these amazing language models and amazing G AI, but the question is how do you make that work on your data, and the problem to solve there is the context problem. How do you give it the right context? So that's what we're focused on enterprise startup, uh, solving hard problems with context engineering, essentially. Got it. And, and Arvind, similarly, Glen, for those who don't know Glen, what's Glean do? Yeah, well, I mean, think of GLEA like GPD but inside your company. It's a place where your employees go and ask any questions or give it work, and GLEN will use all of the world's knowledge as well as all of the internal company's context to, to actually answer those questions or complete those tasks. And so in many ways, you know, I think, you know, the underlying technology, the concept that we use is similar, which is understand your enterprise, understand your people, your data, your knowledge. And bring that as useful context, ah, combine that with the power of language models to actually solve, you know, real tasks and answer real questions. And you guys have seen like tremendous adoption, right? You've just hit 200 million in, in annual revenue run rate, is that right? Yeah, that's right. So we just a couple of months back, you know, we, you know, announced that milestone, I guess we did today, but the, uh, but that's exciting, you know, business is growing for us, uh, rapidly. Great, um. So Dawa, you sometimes talk about sort of RAG 2.0, that there's this kind of new way of doing retrieval augmented generation. What is that really and how is that? How are people doing rag today that might be different than how people thought about doing it, uh, maybe right after chat GPT came out in, in 2023? Yeah, or even before that, right? So RAg is older than, than chat GPT. So, um, so my claim to fame originates is, is I'm an AI researcher, right? So I was the project lead on the original RAg project at Facebook AI Research. And at the time, the way we did things, vector databases didn't exist. The model we gave the data to was not called a language model. It was just a sort of generative model, but it wasn't a language model. So if you look at where we are now compared to then, it's just a very different idea even. So, so the core framework is still the same. You're still doing retrieval to give context to the generative models so that it can deliver value. Uh, but the way you do that is very different. Um, and that's why when we started contextual, we were really obsessed with this context problem. I still think that's the most exciting, most interesting problem right now. So we have these amazing language models, even if you freeze all technology, and you just say, OK, this is, you know, whatever GPT 4 5.2 or Gemini 3, we just keep that. There's still a decade of innovation that we can do just from focusing on the context. Um, and so the, the way you really want to do that is not just a vector database, right? There's a lot more going on. We're, we're doing agentic rag these days. We're using retrieval as a tool, and there are many more tools that you can use. So that's why we, we were looking for a new. We called it RAG 2.0. I don't know if that was the right name really, but I think the core observation was that what you're building there is a system. So it's not a model, right? The language model is maybe 10, 20%, and so it's the same for something like Glean, right? That's an extremely complicated system. And the language model is just a very small part of that, and I think that's why we need to have something like a rag, right? But then if you want to do that the right way, then you have to start from the systems assumption, and that's what, that's what we've been doing. And if you do that the right way, then you get much, much higher accuracy. So that's really what we're seeing kind of across the board is. Enterprise AI adoption is all about high value use cases and if you really want to get there, then you need high accuracy and that's what we do much better than anybody else. Interesting. And, and Arvin, for getting this to work well, Enterprise search is key. How, how do you do that successfully? And when you, uh, you know, there's a lot of data out there that a lot of enterprise, uh, uses of, of AI are not really scaling. They're not getting full deployment. What do you think the pitfalls are there? Why, why, where are companies going wrong? Yeah, I mean, see, first of all, enterprise environments are very, very complex. Um, there is lots of information, lots of data across hundreds of thousands of different systems. Data is permission, it's governed, like not everybody in the company can access our data. So when you sort of start naively, and you say that like, look, I'm gonna actually make AI work in my enterprise. I'm gonna connect AI with all of my systems, that's when you start to sort of run into. all these challenges. Why AI is so hard. Like, first of all, just connecting all of the enterprise information, like bringing that context into AI is hard because that means you need to build integrations to all these different enterprise systems that you have, understand the data models, understand the APIs, you know, to connect with that data. Then the second challenge you run into is that, well, oh, you know, I, I launched AI and I connected it with my Um, with my documents, but you didn't actually take care of security. And you have now AI having access to really sensitive information inside your company, and everybody in the company can then go and use it. So, so like when you, like, you know, like this, this, you know, we're all in the very early stages of making AI work. There was like this tremendous pressure from the executives, you know, uh, team, you know, within the company. Um, we had to do something, make AI work, and we didn't really, they properly like sort of prepare. For it and how to, how to make it happen correctly for us like at Glean like you know we've been in business for now almost 7 years, so we're just like, you know, like you know you're one of one of the original sort of enterprise AI focused companies. The, the first application that we deliver to our customers is, is search, which is, you know, we go in and connect Glean with all of our enterprise systems and we sort of understand more robustly, first of all, you know, what is all this. Information that you have inside the company, what information is up to date, uh, is high quality, what information has become obsolete at this point, because, you know, when you think about context again, you have to make sure that you're using the right context and making AI work on it as opposed to making AI work on scale. How much of that sort of data cleaning can be automated and how much of that is still kind of a manual process? Someone has to look at a human has to look at it, say, yeah, this is out of date. We're not actually going to put that in the database that we allow the AI system to access. Yeah, I mean, I think like when all these projects failed, like, you know, like if somebody needs to take the blame, so who took the blame, you know, the, uh, the startups, you know, and the product companies will tell you that, hey, your data sucks, you know, it is bad and it's your fault, right? So go and fix it. But the reality is that humans cannot fix, you know, the data. You cannot suddenly change our, our habits. You certainly can't like get people to. You know, actively maintain, manage information, keep it up to date, remove the steal information out. That's not how we work. Like, you know, it's, you know, we're going to work the same way as we work today. And so the, the right solution is for AI itself to figure it out. AI has human capabilities, and if you have the right AI, it will actually you know what information is good and what is not, and it will actually make careful use of, like, you know, you know, the information the right way and again, like, you know, the right security. Do, do you agree with that? Is that you can do that. Yeah, and, I, I think one of the most sort of misunderstood sayings I often hear from people is garbage in, garbage out, which is total BS. I think like what, what you want is garbage in information out, and that's why you want to use AI in the first place, right? So I think that is a very common misconception. So maybe we're not quite there yet, but In the long run, what you want to do is just apply AI to your data and all data in an enterprise almost by necessity is going to be garbage. So the question is how do you get the useful information out of that? Absolutely. I'm going to try to get one question from the audience in a second. So think of a question and I'll hopefully get. At least 11 in the audience, um, there's a lot of talk out Arvin, uh, uh, about sort of context engineering as like the next frontier of how to get Enterprise A out of work. What's your view on that and, and how are you, how is Glean working with things like MCP, you know, and scripts and all of these new ideas about giving more context to models? Yeah, I mean, I, I think that that's exactly like the new frontier of innovation is like for any given task, like how do you. build the right context and context is dynamic. Like it's not that like, hey, I have this task, and you suddenly just like you go into all these systems and pull some right pieces of information and you're done. It's actually iterative process where, you know, models and agents are going to be solving complex tasks and Every step of that execution, you have to sort of like figure out like what's the best way to bring the best information to these agents so that they can do their work properly. Um, so, so that's, that's, that's absolutely the key. Like, you know, what we do at GLEAN is, we are, I mean, we think of ourselves as we're trying to build, you know, your enterprise brain. Um, the idea behind GLEAN is you connect Glean with all of your different systems inside the company. So now we understand. You know who your different people are, um, what are the different business processes, projects, teams, uh, product lines that you offer, so sort of building a deeper understanding of your enterprise and also work like, you know, what are the different business processes and how do individuals go and actually execute those business processes today without AI, like, you know, what are humans doing? So sort of if you observe, if you actually connect with all the systems and you observe all the human activity, you're able to sort of then bring, you know, that collective human intelligence. of enterprise in one place and you do that and this is basically the repository of context and now from here, like when you actually ask AI to do some, you know, some piece of work you've got to sort of now, you know, from that repository pick, you know, the right context, you know, and that right context really is like, you know, well how did humans in the past did similar things, you know, in this company so that AI can sort of use that as the as the baseline to start and then do do its work and that's how it sort of really improve accuracy and. And and build these agents that are uh that are capable of actually extending, you know, that human intelligence and, and, and, and automate work. Alright, I wanna get, I wanna get one question for the audience. Who's got a question? And if no one does, I've got, oh, right over here is a guy, uh, with a question. Please state your name and where you're from. Rick Chavie with VX, uh, enterprise scaling platform. One question we, we are getting with our customers is around. If they have a lot of M&A and they've got multiple ERPs, e-commerce, CRMs, whatever, is coming up with that consistent framework unified to be able to apply the agents or any other technology for that matter to give that sort of single view of, of revenue execution or or whatever application you're looking at. So I'm, I'm wondering how. With your models, you know, how you sort through all that confusion, or in some cases systems may not even be really automated. You have to, yeah, and it's, do you need a single source of truth, or can you work across multiple systems? You should work across multiple systems again, so I, I think in a, in the complexity of an enterprise you are not going to have everything in one place. Right, so that's why we have things like MCP, some of the information you might want to index so that you can do faster retrieval over it, but, but in the end I think you just have to accept that you are dealing with a sort of messy reality of an enterprise and now you need to have an AI system that can deal with that. So part of that is understanding what you need to look for and so you know, doing the right tool calls, hooking up into the right ERP systems. A lot of the complexity that that that. that you solve with something like a glean or also with our platform is really about hooking up all this data. Now you have this, so you have this kind of data layer then you have The intelligence layer and then the missing part is the context layer. Like how do you get all of the right little bits of context from that in an iterative way so that intelligence can be useful. Arvin, I, I want to play something you said. You talked about having the AI learn the existing context and the existing workflows to sort of replicate those, which is probably a good place to start. But there's this tension there because a lot of people want what they really want to do with AI is reinvent workflows and change workflows. Can you do both at the same time? How, how does that work? Yeah, in fact, actually what I was saying was, see, there is that human intelligence, the human way of doing things today in your enterprise, and that's the, that's the starting point. So when you, when you, when you ask AI to actually solve a new problem, like either, you know, you let it loose, you know, with no sort of, uh, baggage, you know, to deal with, so to speak, like, you know, like let AI actually go and figure out. From, from scratch, like how to solve that task, or you say that, well, look, you know, like, you know, there's, there's some value in you knowing, like, you know, how we've been doing this work in the past. And you actually bring that context in and then, but you don't like, you know, you don't ask AI to sort of replicate that exactly what the, you know, what, how you were doing that business process before that actually just serves as the inspiration, the starting point, and AI can actually make its own decisions, you know, on top of that. So, that's how it works. So like, you know. Most of the use cases like as people are building these agents today, what we're seeing is that AI converges faster and is able to add value and you know, sort of like, you know, conforming to like you know how the business really wants to work if you bring in that, you know, context. Interesting, um, I wanna do a couple quick rapid fire rounds because we're, we're running a little low on time. Um, if what do you think is the most overhyped sort of, uh, AI technology at the at the moment? uh, what, what do you have to say? Agents, agents, Arvin. I, I, I, I, I would say that any technology, you know, any cool demo, like, you know, it seems to make a complex business process somehow like, you know, magical and automated right away because none of them is gonna work. Interesting, but I think on agents, so I think agents. Is most overhyped but also probably most underhyped. It depends on the time scale you're looking at. Interesting. I was, well, that's my next question is, you know, in, in the next year, what do you think's gonna come along? What, what, what, what's on the roadmap that you think's gonna make the biggest difference, to kind of enterprise implementation, Ivan, what do you think? Well, I mean, I, I think the models like are delivering amazing capabilities every year. Ah, this year was a big one for, for us in terms of the, you know, the, the Like the level of agents, the complexity of tasks we can handle through the thinking models. So I actually don't know what's coming next year, but again, like, you know, to, to what you said before, I think we're very happy with the underlying model technology. And, and now it's time for us to sort of really build robust integrations into enterprise systems and building a deeper understanding of. Um, your enterprise processes and automate them, but for me, like what, what, what's most exciting is that next year AI is going to become proactive. Today, like most of the AI use inside the enterprise is reactive in the sense that I, as an individual, have to go to AI, make it, ask it to do something for me, and it does it, and very few people do that. People don't have time to actually make it happen. So next year I think we're gonna see the, see this flip and AI. OK. Do, what do you, what do you see coming? Yeah, I, I think test time compute to your point is, is really, I think a lot of people are underestimating what this means. So this is really going to change everything. We are just getting started, um, and so when you hook up all your data and then you have these smart reasoning models, uh, that can do so much more, we are going to unlock all of these very complex use cases that really seemed like science. Fiction and that's already happening everywhere now. I mean, like even with things like coding, right, but how good we are at that now with AI. I, I don't think I would have expected that even a year ago. So, uh, it's going to come very quickly. I, I think next year is really going to be the year of, uh, of ROIs and through complex. Well, we're, we're out of time. We'll have to invite you guys back next year to see if your predictions are right. Thank you very much for, uh, for joining us. Thank you. Thank you.