Cloud Next 25 Google Cloud databases and LlamaIndex Section 3 Jerry

Summary

In this enlightening section from Cloud Next 25, Jerry, the founder and CEO of Lavindex, dives into the company's role in AI and knowledge management. Lavindex serves as a developer platform designed to build knowledge agents that help automate enterprise data processes. The talk outlines the two main components of this framework: a robust knowledge management layer that connects and processes data, and an agent framework that facilitates the deployment of complex AI applications. These components enable use cases such as document research, automated workflows, and report generation, aimed at saving time and improving accuracy in data handling. The discussion includes real-world applications and customer success stories that highlight Lavindex’s impact. Key technologies like the Llama Cloud and agent orchestration are explained, focusing on integration within the Google ecosystem.

Highlights

• Jerry from Lavindex explains their role as a developer platform for AI-based knowledge agents and document processing ✨.
• Key components include a knowledge management layer for data connectivity and an agent framework for AI deployment 🔗.
• Llama Cloud helps structure complex documents across formats like PDFs, Google Docs, and Sheets 📚.
• Customer success stories from Rakuten and IM Digital demonstrate Lavindex's powerful applications 🌐.
• Integration of cutting-edge tech like Gemini 2.0 and advanced parsing improves document processing accuracy 📈.

Key Takeaways

• Lavindex is a platform for building knowledge agents, automating data processing tasks in enterprises 🌟.
• The platform features a knowledge management layer and an agent framework for deploying AI applications 🚀.
• Use cases include document research, chatbots, and automated workflows, improving efficiency and accuracy 📊.
• Success stories from companies like Rakuten and IM Digital showcase Lavindex's effectiveness in real-world scenarios 🎉.
• The integration with advanced technologies like Gemini 2.0 enhances the platform's document processing capabilities 🔧.

Overview

In a captivating presentation from Cloud Next 25, Jerry, founder of Lavindex, discusses how their platform supports developers in creating knowledge agents. Lavindex specializes in managing and centralizing unstructured data to streamline AI agent processes.

The session highlighted Lavindex's dual-layer approach: a knowledge management layer for data integration and an agent framework enabling AI deployment. This setup fosters automated knowledge work, ranging from document research to full-scale business process automation.

Through various real-world examples, such as projects with Rakuten and IM Digital, Jerry illustrated the practical impacts of Lavindex's technology. The introduction of advanced technologies like Llama Cloud and Gemini 2.0 was a focal point, underscoring their commitment to precision and innovation.

Chapters

• 00:00 - 00:30: Introduction and Overview of LavaIndex The chapter provides an introduction and overview of the company LavaIndex, presented by its founder and CEO. LavaIndex is described as an agent framework with additional functionalities in knowledge management and document processing. The goal of the chapter is to give a general sense of the company's operations. Additionally, there is a mention of a demo by Avery, demonstrating the integration of Lava Index within the Google ecosystem to build report generation agents.
• 00:30 - 01:00: Developer Platform for Knowledge Agents The chapter discusses a developer platform focused on building 'knowledge agents' that automate various types of knowledge work across different enterprise data sources. This platform is considered one of the leading developer platforms, recognized for its significant community of followers and downloads, and is utilized by a range of companies from Fortune 500 to startups. The chapter also touches on the essential stack needed to build these knowledge agents.
• 01:00 - 01:30: Components of Agent Building The chapter discusses the essential components required to build an agent capable of reasoning over data. It outlines the necessity of a knowledge management layer, which is pivotal for connecting, processing, and centralizing unstructured data for AI agents. This layer ensures data is in the appropriate format, facilitating the use of both structured and unstructured data sources as tools for AI agents to reference or act upon.
• 01:30 - 02:00: Use Cases for Multi-Agent Systems The chapter titled 'Use Cases for Multi-Agent Systems' discusses the application of multi-agent systems through an orchestration or agent framework layer that aids in building complex multi-agent applications on existing data. Examples of use cases include document research, chatbots, and assistant interfaces. These systems allow for human-in-the-loop interactions, providing useful insights to users. In addition, the chapter touches on more automated, multi-step applications of multi-agent systems.
• 02:00 - 02:30: Llama Cloud and Google Ecosystem Integration This chapter discusses the integration of Llama Cloud with the Google Ecosystem, focusing on the transformation of business processes through automation. The highlight is on creating entire automation workflows rather than just simple assistive tasks. This includes capabilities in deep research and large-scale content generation. Additionally, it covers the ability of agents to enhance knowledge work by generating reports or artifacts. The chapter also introduces Law Index, emphasizing its dual functionality in knowledge management and data handling.
• 02:30 - 03:00: Knowledge Management and Document Processing This chapter explores the integration of knowledge management systems with document processing, focusing on how the Llama index framework can be used within the Google ecosystem to build various types of agents. It discusses how Llama cloud serves as a platform to centralize, process, and organize complex documents, including PDFs, Google Sheets, and Google Docs, enabling efficient formatting and utilization of these documents.
• 03:00 - 03:30: Local Cloud Components and Parsing The chapter discusses the combination of local cloud components and parsing technologies to enhance document-related tasks such as research, automated extraction, and report generation. By creating multi-step agents, these technologies aim to automate knowledge work, leading to significant time savings. An example provided is the reduction of time spent reading documents and entering data into spreadsheets.
• 03:30 - 04:00: Indexing and Knowledge Base Management Indexing and Knowledge Base Management discusses the importance of having accessible and relevant data for making accurate decisions. It introduces Local cloud as a knowledge management layer for AI agents, featuring data connectors and file sources such as Google Drive, S3, and SharePoint, providing an overall ecosystem for knowledge management.
• 04:00 - 04:30: Use Cases Highlight: Financial Analysis and Back Office Operations The chapter discusses the integration of a document parsing engine which utilizes best-in-class language models and parsing techniques. This engine is capable of processing complex documents such as PDFs and PowerPoints. A key point emphasized is the importance of having data in the correct format to avoid inaccurate outcomes from analysis agents. In addition to parsing, the engine also offers data extraction capabilities, enhancing its utility in financial analysis and back office operations.
• 04:30 - 05:00: Customer Case Studies: Racketin and IM Digital This chapter discusses customer case studies featuring Racketin and IM Digital. The transcript emphasizes the process of structuring data, including the use of structured JSON and ETL (Extract, Transform, Load) for efficient data management. Additionally, it highlights the indexing of knowledge bases using advanced hybrid search ranking techniques, positioning them as building blocks for managing unstructured data. Furthermore, the chapter points out various cloud use cases relevant to the discussed strategies.
• 05:00 - 05:30: Technology Integrations: OCR and Latest Models This chapter focuses on the integration of technology, specifically Optical Character Recognition (OCR) and the latest models, to streamline processes such as financial analysis. It highlights the capability to handle unstructured documents by extracting numbers for tasks like equity research or due diligence. The idea is to enhance decision-making by accurately modeling financial information derived from a vast array of public and private reports.
• 05:30 - 06:00: Agent Orchestration and Workflow Building The chapter titled 'Agent Orchestration and Workflow Building' discusses the use of document intelligence and agentic decision making in automating processes in administrative operations. It highlights how this approach can manage large volumes of documents for tasks like invoice processing, contract review, and HR onboarding, leading to an end-to-end automated workflow. Additionally, the chapter hints at customer case studies as practical examples, although they are not elaborated in the provided text.

Cloud Next 25 Google Cloud databases and LlamaIndex Section 3 Jerry Transcription

• 00:00 - 00:30 founder and CEO of Lavindex. Thanks Hansa for the great introduction. Um and basically my goal is to first just give you an overview of the company. Uh we are an agent framework. Uh we also have some other stuff around knowledge management and document processing. And so part of the goal is to basically just give you a sense of what we do. Uh and then afterwards, you know, I think Avery has a great demo on how you actually integrate Lava Index within the Google ecosystem to build uh what we call like report generation agents. uh and it falls in this category of agents that we
• 00:30 - 01:00 really care about which is you know what we call knowledge agents over your data. So fundamentally we are a developer platform uh for helping any developer build knowledge agents that help automate different types of knowledge work over their enterprise data sources. Uh we're one of the leading developer platforms and communities for geni you know we have uh you know followers downloads etc uh used by Fortune 500 companies to startups. So if we think about I mean the stack that you need to actually build an agent um especially the type of
• 01:00 - 01:30 agent that can reason over your data um you really need two main components. The first is a knowledge management layer that helps you connect process and centralize your unstructured data for your AI agents. This uh allows you to actually connect to your data sources, make sure that it's in the right format whether it's starting from an unstructured data source or a structured data source so that you can actually serve it as a tool to an AI agent that can you know reference that data as context or take actions over it. The
• 01:30 - 02:00 second is this you know agent framework layer or orchestration that helps you build and deploy complex multi- aent applications on top of your data. Some of the example use cases that we're excited about and I'm sure many of you are probably building include like document research like chat bots assistant type uh interfaces where a user can interact with uh an AI in a human in the loop fashion to get back insights that they want. Uh there's also more kind of like automated multi-step
• 02:00 - 02:30 extraction and automation workflows where it basically automates an entire business process as opposed to just kind of creating an assistant contractor. And then there's also for those of you who are familiar with like you know deep research or basically anything that generates like a giant piece of content uh agents are increasingly able to automate knowledge work uh with like kind of report generation or artifact generation uh interfaces. So law index basically provides two main components um one on the knowledge management or data
• 02:30 - 03:00 processing side and then the second on the uh agent framework. Um, I'll kind of talk a little bit about both. I think for the purposes of this talk, it's probably primarily uh figuring out how like the Lama index framework uh can integrate with the Google ecosystem to help you build these different types of agents. Um, but basically Llama cloud is that layer to help you centralize, process and structure even the most complex like PDFs, documents, Google Sheets, you know, Google Docs, etc. um so that you can actually format it in a
• 03:00 - 03:30 way that again lines can understand. Um and the combination of these two things uh helps to deliver these use cases again that we care about like document research, automated extraction and report generation. What is the benefit of this? Uh one, you know, if we're able to build these multi-step agents that actually help automate knowledge work, one, you save a bunch of time. Uh imagine the amount of time you spend like reading a document, figuring out what to do with it, you know, entering stuff into an Excel spreadsheet. The second is you probably are able to make
• 03:30 - 04:00 more accurate decisions because you just have much easier access to the data that's actually relevant uh at the specific point in time. So I'll give a quick overview of Lacloud. Um this is a little bit less related to this specific demo but maybe just more of a picture of like the overall ecosystem. Local cloud is you know the knowledge management layer for your AI AI agents. Um it contains a few core components. There's data connectors, the file sources like Google Drive, you know, S3, SharePoint, etc. Um, it has a really, really good
• 04:00 - 04:30 document parsing engine under it. Uh, for those of you who might be familiar with LM parse, we combine, you know, the best-in-class VL models with, uh, parsing techniques to process even the most complex documents, PDFs, PowerPoints, um, and and others. And the idea is that if you actually don't have your data in the right format, it doesn't really matter how good your agent is, uh you're going to get hallucinated results downward. Besides parsing, we also have extraction. So besides converting your doc in a
• 04:30 - 05:00 markdown, you can also structure it into kind of like a structured JSON that you can then ETL into whatever downstream system. We can also index uh in a multival fashion an entire knowledge base of documents um and allow you to basically retrieve from it with the most advanced you know hybrid search ranking etc techniques. And the combination of this is really a set of building blocks like data building blocks to help you manage and structure your unstructured data. Um just to highlight some kind of like long cloud use cases uh you know
• 05:00 - 05:30 maybe I mean I'm not going to go through every cell in this but maybe just to highlight two. There's one like financial analysis, which is more in the assistant style. Imagine ingesting a ton of like public private financial reports about a company. Whether you're doing equity research or you're doing due diligence, being able to actually extract out all the numbers from these highly variable unstructured documents in a highly accurate manner so that you can use it for agent decision-m downstream financial modeling generating
• 05:30 - 06:00 a memo for instance. Another category is like back office or administrative operations like invoice processing, contract review, being able to get insights for HR onboarding um and and like compliance all these things. Imagine the volume of documents that again you need to feed through some sort of like document intelligence layer and then leverage a the agentic decision making on top of that to then like solve a lot of this in an end to end automated fashion. Just to highlight two customer case studies. Uh one of our customers is is
• 06:00 - 06:30 Racketin. You know, one of the biggest technology companies in Japan. Um they've used Llama cloud to basically power some of the core rag infrastructure behind uh kind of some main use cases within the company. Uh one is an internal agent platform for every knowledge worker with 10,000 plus daily active users. Another is just like providing core rag developer SDKs for various business apps. A second is uh IM digital a digital commerce agency where they use llama cloud to basically ingest and index brand documentation from
• 06:30 - 07:00 Google drive confluence and more to help create downstream brand assistance that can answer very detailed uh questions about specific products for their clients. Um a really quick note on this I mean part of the secret sauce here again is really just combining lens with her sourcing techniques. Um, if you think about traditional OCR, you know, there's a lot of document processing stuff out there. Um, a lot of what we do is integrate with the latest models like Gemini 2.0 Flash, make it a gentic, so
• 07:00 - 07:30 add a bunch of runtime compute. Um, and then add in like a decade plus of experience and parsing techniques to basically drive up accuracy. And again, the end result is you get really really nicely formatted data that you can then use for your devs. Um this is just another you know benchmark that showcases uh how good Gemini Gemini 2.0 Flash is pretty good. Um it's you know obviously cheaper than Sonnet 3.7 um and you know like OpenAI
• 07:30 - 08:00 but we basically just built an agent around it that could reason and do validation and reflection. And then again when you combine it with a lot of kind of like the core document understanding technology that we have it's really really good at reading tables, charts um and images. So just want to highlight that in case you want to give us our stuff to try. Now let's go to the agent orchestration layer. So you got your document structured in the right format and now how do you actually build an ache workflow on top of it? The
• 08:00 - 08:30 framework is also called lava index. might be a little confusing, but I'm just going