Building Smarter Python AI Agents with Code Interpreters

Building Smarter Python AI Agents with Code Interpreters

Estimated read time: 1:20

    Learn to use AI like a Pro

    Get the latest AI workflows to boost your productivity and business performance, delivered weekly by expert consultants. Enjoy step-by-step guides, weekly Q&A sessions, and full access to our AI workflow archive.

    Canva Logo
    Claude AI Logo
    Google Gemini Logo
    HeyGen Logo
    Hugging Face Logo
    Microsoft Logo
    OpenAI Logo
    Zapier Logo
    Canva Logo
    Claude AI Logo
    Google Gemini Logo
    HeyGen Logo
    Hugging Face Logo
    Microsoft Logo
    OpenAI Logo
    Zapier Logo

    Summary

    This insightful session by Microsoft Reactor dives into the development of AI agents using code interpreters, aiming to enhance their capability to tackle complex tasks. By allowing these agents to write and execute code, their problem-solving abilities can be notably improved. The hackathon provides numerous live streams showcasing how to build these agents leveraging Microsoft technologies. The session highlights the importance of using a sandbox to ensure security when agents execute code, demonstrating both local and cloud-based secure environments using Azure Containers. Examples included automated tasks such as browsing the web and completing form entries securely.

      Highlights

      • Hackathon overview and prizes πŸ†.
      • Live demo on building AI agents with code interpreters in Python πŸš€.
      • Security challenges and solutions using sandbox environments πŸ›‘οΈ.
      • Using MCP to add functionalities to AI agents πŸ€–.
      • Demo on autonomous web browsing and task completion by AI agents 🌐.

      Key Takeaways

      • Learn to build smarter AI agents with code interpreters to enhance capabilities πŸ“š.
      • Ensure security when executing code by using sandbox environments like Azure Containers πŸ”’.
      • Use MCP to integrate tools into agents, enhancing their functionality 🌐.
      • Implement AI agents to autonomously navigate web pages and perform tasks πŸ–₯️.
      • Explore various use cases and practical applications of AI agents in different scenarios 🧠.

      Overview

      The session commenced with an overview of the AI Agents Hackathon, encouraging participants to engage and learn through hands-on experience and live demonstrations. The presenters emphasized the potential of AI agents when equipped with code interpreters, allowing them to tackle more complex tasks efficiently.

        Participants were shown demonstrations of building AI agents using Python and integrating code interpreter capabilities. Important security considerations were highlighted, particularly the use of sandbox environments to safely execute code, ensuring the agents do not compromise security or functionality.

          In the latter part of the session, innovative technologies like MCP and Azure Container Apps were explored, showcasing their ability to enhance agent functionalities significantly. The potential for AI agents to autonomously interact with web environments and perform tasks was demonstrated, illustrating the future potential of these intelligent systems.

            Chapters

            • 00:00 - 01:00: Introduction and AI Agents Hackathon Details The AI Agents Hackathon live stream kicks off, offering a free virtual event running from April 8th to April 30th. Participants can attend over 20 live streams to learn building agents with Microsoft technologies. After three weeks, they can submit their projects to compete for exciting prizes.
            • 01:00 - 02:00: Introduction to Live Stream Session The chapter introduces the live stream session and mentions the AI agent hack event. It begins with a greeting and thanks to the audience for joining the session. A brief note on housekeeping follows, with an emphasis on reviewing the code of conduct to ensure a respectful environment for both the audience and presenters. Participation and encouragement are highlighted as key points.
            • 02:00 - 03:00: Overview of AI Agents and Code Interpreters The chapter begins with a polite request for professional and on-topic engagement in the chat, emphasizing the need for respect and relevance during discussions. It also mentions that helpful links will be shared and questions will be monitored for live responses from presenters. Additionally, it is noted that the session is being recorded and will be available for on-demand viewing within 24 to 48 hours.
            • 03:00 - 04:00: Building Smarter AI Agents with Python Code Interpreters In this chapter, the focus is on demonstrating the development of more intelligent AI agents using Python code interpreters. The session is primarily hands-on with minimal slides as the speaker prefers to conduct various demonstrations. The concept revolves around understanding and utilizing language learning models (LLMs) and AI agents to enhance their intelligence, specifically through code interpreters. This emphasizes a practical approach to AI development, facilitated by interactive examples rather than just theoretical discussions.
            • 04:00 - 05:00: Sandboxing Code Interpreters for Security The chapter discusses the implementation of sandboxing code interpreters to enhance security. The idea is to enable the code interpreter to understand and execute code to increase its functionality and intelligence. The session emphasizes practical coding activities, with minimal reliance on slides, focusing on three specific applications.
            • 05:00 - 06:00: Using Azure Container Apps for Secure Code Execution The chapter explores using Azure Container Apps for executing code securely. It starts by introducing a simple rag application using Python and Langchain seen in different live streams. The chapter proceeds to explain adding a code interpreter to the application, enhancing its functionalities. It briefly shifts focus but stays within the domain of using similar applications.
            • 06:00 - 07:00: Running AI Agents Using Code Interpreters in the Cloud The chapter titled 'Running AI Agents Using Code Interpreters in the Cloud' discusses the concept of using readily available agent tools, specifically focusing on building a code interpreter MCP server and connecting it to a Cloud Desktop. It also touches upon similar applications in platforms like Visual Studio and GitHub Copilot. The chapter concludes with the prospect of constructing a computer using an agent, highlighting the excitement and potential challenges of these integrations.
            • 07:00 - 08:00: Implementing a Code Interpreter in LangChain In this chapter, the concept of a code interpreter within the LangChain framework is discussed. The code interpreter is likened to an agent capable of operating a web browser. Therefore, it can "look at" the web environment, figuring out how to utilize it for problem-solving tasks. This capability allows it to execute commands such as searching for information and summarizing it or performing actions like purchasing items online.
            • 08:00 - 09:00: Testing Code Interpreters and Security Concerns This chapter discusses strategies and potential issues related to testing code interpreters, particularly focusing on how they interact with web browsers and perform automated tasks like clicking or typing. The script involves checking the functionality of these tasks, with an emphasis on addressing security concerns. The speaker mentions previous successful tests of these techniques, although there's uncertainty about their consistent performance. The chapter ends on an engaging note, encouraging audience participation and questions, highlighting an interactive approach to exploring these technological topics.
            • 09:00 - 10:00: Azure Container Apps Dynamic Sessions Overview The chapter introduces the topic 'Azure Container Apps Dynamic Sessions' and sets an informal tone for the session. The speaker mentions jumping into the code directly and dealing with technical setup issues, suggesting a hands-on approach to exploring and understanding the application. The speaker aims to make the demonstration visible to the audience and highlights the informal, interactive nature of the session.
            • 10:00 - 11:00: Configuring and Using Azure Container Apps In this chapter, the speaker discusses configuring and using Azure Container Apps, starting with a walkthrough of the code for a basic application. They mention using Langchain and Chainlit, a tool that provides a chat UI without requiring front-end coding knowledge.
            • 11:00 - 12:00: Installing Python Packages in Code Interpreters The chapter discusses the installation of Python packages in code interpreters, focusing on using Azure OpenAI. It mentions the use of an older language model to demonstrate features of code interpreters, noting that newer models have improved capabilities, which sometimes make certain demonstrations less relevant.
            • 12:00 - 13:00: Using Code Interpreters for Math and CSV Analysis This chapter explores the use of code interpreters, highlighting a demonstration with an older model, GPD35 Turbo, which lacks modern security features. The discussion includes potential security issues inherent in older models and the flexibility of using newer language models for tasks.
            • 13:00 - 14:00: Security Practices and Warnings The chapter discusses security practices and warnings in the context of working with LangChain and Azure AI search. It briefly covers the behind-the-scenes process of using a vector store for embedding documents. The speaker mentions having previously added documents containing fake company information but doesn’t go into specific methods or detailed procedures.
            • 14:00 - 15:00: Creating MCP Servers for AI Agents In this chapter, the focus is on working with a product at Microsoft known as Azure Container Apps. The content of this chapter includes a detailed exploration of using AI search by analyzing 400 pages of documentation relating to the product. The integration of this material with an AI agent through a method referred to as 'rag' will be demonstrated. Furthermore, the chapter delves into a discussion about a code interpreter tool.
            • 15:00 - 16:00: Integration with Claude Desktop and MCP Servers In this chapter titled 'Integration with Claude Desktop and MCP Servers', the focus is on the basic setup of an agent with a retriever tool that interacts with AI search capabilities. Initially, the chapter covers some fundamental LangChain concepts and touches upon aspects related to uploading files, which will be explored in more detail later. Notably, the chat application setup is primarily completed within the first 60 lines of code. The chapter concludes with a demonstration of opening a terminal to run the application.
            • 16:00 - 17:00: Exploring OpenAI's Computer Using Agents In this chapter, the narrator explores OpenAI's computer tools using agents, including interacting with a browser interface. The narrative touches on the anticipation and mystery of how the system will respond, with a specific focus on the browser's behavior and how it may appear on the monitor. There is an element of unpredictability and expectation as the system performs its tasks.
            • 17:00 - 18:00: Setting Up and Running Computer Using Agents The chapter focuses on setting up and running computer applications using AI agents. It begins with a participant, seemingly surprised but pleased, as their application opens on the correct window. During a session, someone inquires about accessing the code repository. Pamela, possibly a participant or facilitator, shares the necessary links, confirming that three links were provided, including one titled 'sessions AI agent'.
            • 18:00 - 19:00: Demonstrating Browser Automation with AI Agents The chapter titled "Demonstrating Browser Automation with AI Agents" introduces a simple application created using Chain Lit, which facilitates querying indexed documents, with a specific focus on Azure container apps. The goal is to effectively retrieve and present information from these indexed sources.
            • 19:00 - 20:00: Testing AI Agents with Web Interactions The chapter discusses the functionality of AI agents in testing scenarios with web interactions. A specific example is provided involving Azure container apps, which allows the running of containerized applications in the cloud without the need for infrastructure knowledge. The AI agent uses a vector store tool to search for information, demonstrating its capability to retrieve and handle data efficiently.
            • 20:00 - 21:00: Discussion and Closing Remarks The chapter 'Discussion and Closing Remarks' revolves around the analysis of a basic retrieval-augmented generation (RAG) system. The speaker describes the system's behavior, noting that after obtaining initial results, it requested additional results by executing similar queries. The analysis includes examining the returned information, which consisted of multiple articles that were subsequently summarized in the final answer. The speaker acknowledges the RAG system's fundamental nature but also appreciates its capabilities, emphasizing that it already functions as a powerful agent despite its simplicity.
            • 21:00 - 22:00: Hackathon Registration and Submission Details The chapter focuses on utilizing agents for tasks that require mathematical reasoning, despite older models having less proficiency in mathematics. It suggests the importance of integrating vector stores to enhance functionality, especially in scenarios involving direct math problems or math-related queries.

            Building Smarter Python AI Agents with Code Interpreters Transcription

            • 00:00 - 00:30 hello everyone welcome to the AI Agents Hackathon live streams the AI Agents Hackathon is a free virtual hackathon running from April 8th to April 30th we're hosting over 20 live streams where you'll learn how to build agents using various Microsoft technologies at the end of the three weeks submit your project for a chance to win fantastic prizes
            • 00:30 - 01:00 and now over to our friends at Reactor hi everyone thanks for joining us for the AI agent hack before we start I do have some quick housekeeping please take a moment to review our code of conduct we seek to provide a respectful environment for both our audience and presenters while we absolutely encourage
            • 01:00 - 01:30 engagement in the chat we ask that you please be mindful of your commentary remain professional and on topic keep an eye on that chat we'll be dropping helpful links and checking for questions for our presenters to answer live our session is recorded it will be available to view on demand within 24 to 48 hours with that I'd love to turn it over to our speakers for today thanks for joining
            • 01:30 - 02:00 oh hey it's me hey everybody uh welcome to this live stream where we're going to do a bunch of demos and uh yeah so I kind of put the demos or kind of like you know so it's we're not going to have a lot of slides today uh but we are going to talk about how we're going to build some smarter AI agents using code interpreters so um so this is the idea of um you know an LLM and an AI agent is pretty smart
            • 02:00 - 02:30 but if we give it the ability to write and execute code to figure stuff out that it can get smarter um so we're going to do this by giving it a code interpreter uh so we're going to this really going to like this might be the only slide you see today there might be a couple of other slides that I might refer to just as visual aids but it will be mostly in code for the next uh probably an hour or so and uh we're going to kind of work on these three apps so the first thing that we're going to do is uh we're going to go over like
            • 02:30 - 03:00 a pretty simple rag application i'm sure you all have seen it by now to do through all these different live streams um mine happens to use Python and and Langchain and um and I'll kind of go over how it works and then we're going to see what it takes to add a code interpreter to it uh so that it can do more things for us and then we're going to kind switch gears a little bit um actually not too much we're going to you know like look at how we can kind of do the same thing using uh kind of like a
            • 03:00 - 03:30 you know just out of the box uh agent tool like uh Cloud Desktop and this could apply to something like Visual Studio GitHub Copilot as well um and yeah so we're going to actually do this using MCP so we're actually going to build a a code interpreter uh MCP server and try to hook it up to Cloud Desktop so that's pretty fun if it works and the last one is uh we're going to build a computer using agent so this
            • 03:30 - 04:00 is a computer that knows how or or kind of sort of like an agent that knows how to uh use a computer in this case the computer is a web browser uh so it's able to actually you know use a web browser to uh really like look at you know quote unquote look at the web browser figure out um you know how to use it to um to solve a problem so we could actually tell it to you know go go search some you know search some something for us and summarize it we can ask it to like go buy us a t-shirt or
            • 04:00 - 04:30 something and it'll actually kind of go to the web browser and kind of just figure out what to click on what to type and um and actually do the thing that we asked it to do so those are the three things hopefully we get through them hopefully they all work hopefully one of these work at least uh I tried it yesterday and all of them worked at one point or another so we'll see um nice to see that there's a bunch of you here already uh feel free to ask any questions in chat i see that Pamela's in
            • 04:30 - 05:00 there hanging out um and yeah I'll try to keep an eye on the chat as well so yeah pretty informal we're going to now jump into the code and see where we go from here there's a bunch of blank Dex desktops for some reason that's all right but this is my app uh hopefully this is big enough i'll make it even bigger just so that you all can see uh so yeah so we're actually jumping straight into the first one so uh uh so
            • 05:00 - 05:30 I'm going to start off by going through the code for this hopefully you know fairly basic um rag application uh so uh so this one I'm like I said before I'm using lang chain and I'm also using a tool called chain lit which kind of gives me a chat UI without me knowing anything about front-end coding or anything um and just with that um I'm doing a few things i'm
            • 05:30 - 06:00 spinning up uh I'm using Azure OpenAI but you can use OpenAI or any other LLM for this um I'm actually using a fairly old model um this is kind of an old demo um but actually using an older demo actually shows you know some of the you know like uh some of the more kind of interesting things uh with with the code interpreter stuff that we're going to add later um the newer LLMs are actually a lot smarter by themselves already um so it's a little bit harder to highlight uh some of the things that you can do
            • 06:00 - 06:30 with code interpreters uh so I'm actually using a slightly older model um and I'll actually kind of show you some of the security issues that you might have as well u because the older models have have fewer um security guard rails in place for stuff like this but we'll kind of get to that a little bit later um for now you just need to know that we're using an LLM and this happens to be uh GPD35 Turbo which like I said is fairly old um but you can use any of the newer ones as well and um I'm not really going to go
            • 06:30 - 07:00 into too much about how to do rag i assume that we've kind of gone through this a few times already on on these live streams uh but but but in lang chain what I'm doing is I am behind the scenes I have a vector store that's using uh Azure AI search behind the scenes uh and I've and I've already kind of like uh you know calculated the embeddings and inserted the documents in there um I forgot what I put in there I think there's some fake company information and I think I also put in I
            • 07:00 - 07:30 work on a product at Microsoft called Azure container apps um which we're going to use a little bit later on Um but uh but what I've done here is I've kind of dumped out like 400 pages of our documentation and just put that into um AI search so I think that's what uh we're going to be able to see when we do the search um or kind of use rag with our agent and then uh we're going to talk about this uh code interpreter tool
            • 07:30 - 08:00 later so right now the only tool that we're giving to our agent is our retriever tool which talks to um AI search as I was saying earlier and the rest of it is just very you know kind of pretty basic um lang chain stuff here um there's some stuff about uploading files that we'll kind of get to later uh but really you know just in the first 60 or so lines we kind of have our chat app uh mostly there so uh so I'm going to open up a terminal and just start this thing
            • 08:00 - 08:30 up it's going to start a browser we'll see which of my monitors is going to pop up in it's never the one that I wanted to pop up in or maybe it is might be thinking a little bit right now let's see what's going on oh there it goes is it going to open where are you
            • 08:30 - 09:00 my app's available yeah oh there it is oh it's opening on the right window that's always good oh and someone's asking for the repo for the code uh yeah so uh thanks Pamela for sharing uh yeah so the link should be in there i think I provided three links this is the one that is that says sessions AI agent I think I called
            • 09:00 - 09:30 it um so yes this is my my very simple app that's powered by chain lit uh so I can ask it stuff so like I said I kind of uh inputed some or kind of indexed uh some documents uh about Azure container apps so you know tell me about Azure container apps and hopefully it will um do the right thing and give me an answer so there you go it kind of gives
            • 09:30 - 10:00 me an answer about how container apps allows you to run containerized applications um in the cloud without um knowing anything about infrastructure and if you kind of pop it open here uh you can actually see we can actually see the um what it did to kind of figure out the answer so it decided that it needed to search for information about Azure container apps so it used our um our vector store tool to do this and then I
            • 10:00 - 10:30 think it got back some results it wanted more results so it's it kind of ran the same thing again which is kind of odd but um but anyway it it kind of you know did a couple of queries um if I flip back over here we can actually see the information that came back so actually kind of got a bunch of articles back and it kind of summarized it all in our answer here so that's kind of what it did so it's very basic rag we all know how to do this um so uh so that's that's great like that that's that's a that that's already a pretty powerful agent
            • 10:30 - 11:00 um but sometimes you want your agent to do a little bit more so um and one of the reasons why I kind of use an older model is that um the older the model is the worse it is at math so quite often you might have uh you might want to kind of have something that involves math where you might just have a direct math question or something that's like a math word problem that you want um the agent to help you with right um so sometimes well if you all if if all you've done is kind of provided it a vector store um
            • 11:00 - 11:30 it'll probably end up um you know it's sometimes it probably you'll end up with a hallucinated answer so let's see if we can have make that happen so if we just ask it like what's the product of these two numbers I think it'll either say it doesn't know how to do it or it'll actually give you a Yeah so it kind of gave us an answer so let's see if this is real it looks real we could trust it but
            • 11:30 - 12:00 let's see if it really is so I'm going to paste in the question so you can see here that it's a very you know what what it kind of replied with is yeah what it turns out to be a a hallucinated answer but the the scary part is that it looks close enough right so if you didn't know that um that it was just a guess uh you would have you know taken this and stuck it in your tax returns or something not my tax returns it doesn't have this many digits but u maybe yours does um but
            • 12:00 - 12:30 like you would have done something maybe like important with it and then you know you realize when you get an audit that um that maybe that's not quite uh uh that the answer that they they were looking for so um so we can actually solve this by giving a code interpreter to the agent so let's do that uh so let's switch back here and not do the calculator here i'm going to stop my app um and we're going to do this in
            • 12:30 - 13:00 let's see here um in just a few lines of code so I've already kind of have added a line of code here that says give it a Python ripple tool we're going to create a Python ripple tool that actually comes as part of lang chain somewhere here yeah so comes from lang chain so you can actually use it as is and all I need to do is just give it to um the agent and that's really it i can go ahead and start it up again and now my app has somewhere
            • 13:00 - 13:30 superpowers hopefully or at least it could be more accurate anyway it can still give me answers about math questions but hopefully it won't just guess so starting again taking a look at the chat just to see what's going on nice to see people from different parts of the world welcome all right and the app is up and
            • 13:30 - 14:00 running so let's go ahead and ask it the same question and you can see that it gave us an answer if I flip back to the calculator hopefully yeah so this time it actually gave me the right answer and see let's see what it did behind the scenes so it actually decided to go okay I need to use the Python ripple to calculate the product of these two numbers so it actually wrote some Python
            • 14:00 - 14:30 code um we actually executed it behind the scenes so by default what this thing is doing is that it just spawns a Python subprocess obviously a subprocess is running Python um runs uh the the script that or the command that you give it and then gives back the results to the LLM which then formulates the answer so um so yeah so that's kind of how it works uh it's pretty cool you can actually you can you can give it some you know more complicated questions as
            • 14:30 - 15:00 well um we'll show you that in a little bit um but because it is uh running in a you know just a Python shell that's you know within your system um so currently this is on my local machine um if I decided to take this application this chat UI and I wanted to push it up to the cloud um then you know anyone can actually go to this chat um and um and as some of you might imagine uh that's actually not the greatest thing to be doing because what you could like what a
            • 15:00 - 15:30 user could do it could ask it a math question which is fine um but you know if somebody's a little bit you know evil or malicious they can actually go and ask the LLM to do things that are um not so great so one of the things that we could do here is um uh I I can ask it a question like uh write some Python code to print the contents of a file named env
            • 15:30 - 16:00 um more often than not um it will actually do what we ask it what we ask it to do and so yeah you can see that it's done it here um it decided okay great you're looking for actually for some reason it decided to search for documents first um in in in our um in in in our vector store that didn't produce anything eventually decided oh yeah I actually have to write some Python code
            • 16:00 - 16:30 um and run it so that's what it did it read the file and echoed it back thankfully my file doesn't have any secrets in it but imagine you know if this was hosted on the on the web and um and you ask you know the agent to just dump out all the environment variables um you might find some secrets in there um or you can even do worse you can actually I'm not going to try this here but you can actually uh you know probably ask it hey write some Python code that deletes all the files from my system and it'll probably do it and execute it so um so that kind of
            • 16:30 - 17:00 highlights some of the security issues that you might have with um giving you know so much power to uh to the agent um and and and especi you know especially um when this power enables the agent to actually do things to the local system so you can leak some secrets it can actually cause some damage um you can also kind of imagine um when when this application is hosted in the cloud you could have like 10 people chatting with it at the same time so you can actually ask it probably to you know to to to
            • 17:00 - 17:30 kind of read uh something from you know another chat session for example um you know the the the evil possibilities are endless um so you kind of want to make sure that these things run in a sandbox so um and that's what we're going to do now so I'm going to go flip back to the tool here so um before we kind of add add it to the tool I just want to kind of quickly explain to you what we're going to use so I work for uh I work on a product um at Microsoft
            • 17:30 - 18:00 called Azure Container Apps um don't worry if you don't want to work with containers um we're not really going to talk about containers today um one of the features that we have is a way to run um these highly sandboxed containers within um within our platform so um so each one of these uh sandboxes we call them sessions um are fully isolated with a hype with with a HyperV sandbox uh so
            • 18:00 - 18:30 really when you you know like for instance if this container happens to contain a Python code interpreter um that code interpreter runs in isolation you can also tell it whether or not it even has network access so you can really sandbox um the things that are running in your containers using this tool um and one of the great features that we have within Azure container apps um we call these things dynamic sessions is that um instead of asking you to you know like go go build a container that runs a Python code interpreter we
            • 18:30 - 19:00 actually give you a code interpreter out of the box so if you wanted to um you know just basically get access to a bunch of uh Python code interpreter sandboxes um you can just kind of go up to Azure so I'll kind of quickly show you that here so if we go to our Azure
            • 19:00 - 19:30 portal and you can go ahead and create one of these things called a session pool um I'll just going to open up a random one that I have here um yeah we can see here that it's a Python code interpreter um and I've kind of configured it so that um I can you know this one I can only have 10 um concurrent sessions but you can actually really crank this up to hundreds um if not thousands so um so you can actually
            • 19:30 - 20:00 have access to a lot of these things um so yeah so so the first thing is you need to create this pool um which takes about 10 seconds and then you can kind of configure it to the right scale um you can also configure it to uh to to allow network outbound network calls or not um and you can also kind of um set this cool down period so if you stop using a certain session we'll automatically clean it up after the cool down period um so that's really kind of all I kind of want to show you
            • 20:00 - 20:30 here uh oh and there's a kind of kind of a cool playground thing here too so if you kind of just go here you just want to kind of quickly test something within the session um you can kind of click run you can see you know it took about a second for me to kind of allocate a sandbox and also execute the code in it and within the um each set each Python code interpreter session I think we currently pre-install um at least a couple hundred um popular Python um packages so including pandas
            • 20:30 - 21:00 numpy um all the way to things like playright that we're going to that we're going to look at a little bit later so um yeah so it's a kind of quick tour about what we're going to uh use um but the main thing here is after you kind of created this thing um you copy the pool management endpoint and then uh in my app I happen to have my file that I think we all saw earlier um and I just basically provided
            • 21:00 - 21:30 this value as a um um as an environment variable and then all I have to do now in my application is I I can just switch out this built-in Python tool that's not secure that runs Python code on the local machine um to one that actually runs it in a secure um code interpreter session so um I can do this pretty easily so um if I remember this
            • 21:30 - 22:00 right and then it's a pool management endpoint what is it doing it used to kind of guess the right thing for me here but today it's Copilot's failing me a little bit so I kind of have to make a slight change here for it to use the right variable just pull management endpoint and I think that's it so we
            • 22:00 - 22:30 kind of just have you you basically create the same or you could create a Python ripple tool it's more or less the same as the uh the one that lang chain provides and give it the pool management endpoint it's going to use uh to to authenticate it's actually um going to use my local identity here so um I don't have to give it any secrets um when it's running in the cloud um it it can use something um that we call um manage identities which is a way to authenticate for for a cloud
            • 22:30 - 23:00 service to authenticate with another cloud service uh within Azure so again no secrets there um but so you just have to kind of tell it the pool to use and then that should be about it everything stays the same i'm going to go ahead and run this app again while that's starting up let's let me take a look at the chat to see how things are going there's a question here about can you set the packages it can use uh so
            • 23:00 - 23:30 currently uh like I said it comes pre-installed with hundreds of packages uh and you can actually add more uh so so like so so one thing to kind of know about how these sessions work is behind the scenes we're actually um running a container like I was saying before within the container it's got Python in it it's got a bunch of Python packages and then um also within the container we're actually running a Jupyter kernel and then we're just basically adding a little web front end or kind of like an API a web API in front of this Jupyter
            • 23:30 - 24:00 kernel and that's really what uh what you're interacting with when you uh talk to one of these sessions and because it's running a Jupyter kernel um you can actually run um you know these Jupiter kind of magic commands in there so you can go exclamation mark um pip install um and then that's how you can actually install more uh packages in these sessions um one thing you to note though is every time you um you you start a new session and you do this by just giving it a new session ID um that session is
            • 24:00 - 24:30 going to come you know like basically clean um and and which which means like you have to run this pip install um in every session that you have um in some cases you can actually uh change your instructions to your agent so that uh you can tell the agent that hey like if you uh you find that a a certain package is missing go ahead and run you know exclamation mark um pip install and u to to install it um and um
            • 24:30 - 25:00 and I've seen that work so so it that that's kind of how you can add um more packages and Val's asking is there a reason to use the unsecure Python ripple or should we use default to the code interpreter session every time um you should default to something like like that like like that every time like so like the the um so the code interpreter sessions that I'm showing you is you
            • 25:00 - 25:30 know what you get with Azure um actually OpenAI itself has a code interpreter that you can just kind of check off um and enable um in some of these uh chat models that they have so um we're by no means the only one um but definitely try to use something um that's more secure that sandboxes your code versus uh just allowing it to do whatever it wants because um because there are always evil people out there um who who kind of want to create havoc so um yeah definitely so
            • 25:30 - 26:00 I'll get back to some of these questions later i see more and more questions uh but I see also see the chat UI kind of popping up again so we're going to kind of quickly go and actually I'll ask this other question here uh so we're So now this is this should be using the sessions code interpreter assuming I actually saved the file before I ran it hopefully I did we'll find out soon enough yeah you can see now that um
            • 26:00 - 26:30 hopefully it actually tried to run some code uh no it actually didn't actually try to run any code try again but what we should see is uh eventually we can Yeah so uh I think it actually did yeah so it actually did the right thing this time um it decided to write some code uh and run it but this time because it's running within my
            • 26:30 - 27:00 sandbox there is no file right so uh so yeah so it's it's pretty safe um so you don't have to worry about leaking secrets or you know or worse them destroying your system um so I kind of want to just quickly go over a couple more things before I move on to the other demos uh the code interpreter sessions that we provide um you can also uh interact with files there so you can ask your scripts to you can kind of basically ask your agent to generate files you can actually upload a file to it and it can um you can actually ask a
            • 27:00 - 27:30 question of questions about the file um things like that so um so what I'll do here is I will attach a file so I have a customers CSV file it's got some fake customer information in it um but what I can do now is I can say uh you know tell me uh more tell me about this customers CSV
            • 27:30 - 28:00 file and you can see that it actually kind of told me some things about this file right so I did nothing to it to be honest I created this file a long time ago I don't even remember that these were the columns um but if you actually look at the code um it decided that oh I should actually start start reading the the the file to understand its contents and it kind of executed this code um this pandas code um and then it kind of executed a couple more things in this case it kind of pulled back some columns
            • 28:00 - 28:30 and so it's actually you know like deciding the the pandas code to write to actually answer my question um in this case it's kind of giving me the shape of my CSV file see how you say how many records are in the file Well um you can see that hopefully it actually ran it uh yeah so it actually ran it and said uh so it figured out this got 1,400 records um so you can actually ask it things like um uh you know who are
            • 28:30 - 29:00 the top five uh customers in uh in sales I guess um um in the state of Arizona and you can see it actually kind of came back with some results let's double check that I actually did the right thing uh so uh so decided oh yeah I need to analyze the sales data to figure out the top five customers in
            • 29:00 - 29:30 Arizona so again it kind of checked you know like looked at you know what what the what the CSV file look like and then it decided that oh yeah I need to write a um you know like basically like pandas query to figure out um you know this answer so I kind of knew that uh that this the state is in a column named state and when I meant you know like you know top five customers in sales it decided that oh yeah I should be looking in a column called total purchase dollars um which I thought was pretty
            • 29:30 - 30:00 smart um and then it uh kind of sorted it the right way and then it took the took the top five right so and then and then kind of printed out the answer and it gave me back the answer on the screen so um you can do a lot more with this um but hopefully that kind this kind of gives you a taste of what you can do with um uh with a um Python code interpreter in an agent um actually let's try one more thing so
            • 30:00 - 30:30 actually u yesterday I found a bunch of math problems on the internet so these are things that like you know homework that my kids could bring home um I'm not going to tell them that this agent exists because otherwise they'll they'll they'll just never do the homework they'll just use the agent um but I'm just going to like go ahead and take one of these questions and give it to the agent and see what it does um so there go it said it should 70
            • 30:30 - 31:00 cents on the uh on the orange or something like that uh so we can actually go back i think this thing has an answer key so this answer this this question is 75 so you can see that it actually get got the right answer um so this this is this is a lot of fun yesterday I kind of gave it a bunch of these questions and it got I think all of them right so uh it's pretty neat how it can use Python to to basically do you know 11th grade word problems or whatever grade this is from okay cool uh
            • 31:00 - 31:30 let's kind of quickly look at the chat and thanks Pamela for responding to everything and Josh too um yeah so let me know if there's anything I should uh I should answer but I'm going to go move on to the next demo hopefully this was kind of useful so far um so wish I didn't have all these screens in place so the next thing I'm going to do is I'm going to take this code interpreter and
            • 31:30 - 32:00 give it to Cloud Desktop all right so um I already have cloud desktop here looks like I need to restart it here as in I have it on another screen just give me a sec i'm going to go open it up again can get rid of most of these things okay so let me bring over is Claude
            • 32:00 - 32:30 um yeah so yeah so by by default I don't think there is a code interpreter in cloud maybe there is already who knows um but uh but yeah let's let's just try to see if we can take our um our Azure container apps dynamic sessions code interpreter and plug it into cloud and see what that takes or what it takes to do that uh so uh you know a few months ago um there's a new protocol that's enthropic uh kind of came up with um
            • 32:30 - 33:00 called MCP model context protocol is it i don't know the MET stands for something the C I think it's model context protocol um and um and that's kind of taken the world by storm um now everyone's talking about MCP so I'm just going to quickly kind of show you how we can turn our code interpreter that we've been using into a into an MCP server that you can give to um in this case uh cloud desktop but we can also give it to other things that
            • 33:00 - 33:30 can interact with MCP servers like GitHub copilot uh okay so let's uh let me find [Music] my just finding the right code editor to bring over here so here's my sessions MCP thing don't need this um make it
            • 33:30 - 34:00 bigger all right all right so I kind of give you a quick tour of what I have so far actually let's do this so the way I built that or I started building that was I actually went to uh as I said MCP server docs i clicked on here and I actually just went through um you know kind of walk you through how to set up the environment um all we're doing in here is that um a server is just a
            • 34:00 - 34:30 Python program and um and you install a library and then you basically use that library to expose one or more tools using MCP um so the tool that we're going to expose is basically a function that can run some Python code and return the results um the example that they give you um calls the National Weather Service and gives you back um the weather information we're not going to do that here um but uh but you know but we can
            • 34:30 - 35:00 basically walk through this tutorial to um to to to to build our MCP server uh so all we have to do here is really kind of bring in a few things um really this line initiates the initializes the MCP server and then um to write a tool all you have to do is just write a function and then decorate it here with uh mcpto tool and it just magically
            • 35:00 - 35:30 turns your function into um a tool so some of the the key things here is that you have to give your function a um a meaningful description so this this description is actually going to get passed over to the uh to the LLM or to the agent uh so uh so you kind of want to make sure you shape this correctly so that um you know that the agent knows when to choose this tool and what to
            • 35:30 - 36:00 pass it so you can see that we kind of describe the arguments here as well uh so here you can see it kind of created two um tools right um and then all you have to do after that is basically just go call mcp.run and then it runs um the the server um by default the server runs locally on your computer so um so it's not like a you know like quite often you think of a server as one that runs remotely um it can run remotely um the community is actually going
            • 36:00 - 36:30 through a bunch of discussions right now about um what it means to run these things remotely including how authentication works so it's still a pretty new protocol so we're trying to figure out how um how it all fits in together uh but running these things locally it's totally secure or it's mostly secure um and um yeah and that's what we're going to do it's actually not that secure depending on what you're running because like I was showing before if you happen to run something that runs you
            • 36:30 - 37:00 know arbitrary Python code and you give it to you know cloud desktop in this case you can ask it a question and you're going to you're going to accidentally ask it a question that just deletes a bunch of files or something so uh so yeah just be just be careful which uh MCP servers you um you you bring in and and and add to one of these agents um but that's enough talking and looking at the documentation let's flip back over to my app here uh so I have a little um
            • 37:00 - 37:30 helper library that helps me work with sessions um in the repo you can find this um but this has nothing to do with MCP so this is just my helper library to um for for example um to execute code um all we're doing is that we're generating a token and then we're calling the code execution endpoint on my session pool and then kind of passing the token and all that stuff and returning the result so that's really like you know like the
            • 37:30 - 38:00 the key piece of code here to interact with a session there's some other stuff here about uploading files and stuff like that we're not going to deal with that with this MT with this MCP tool uh so that's what this file is it's just a bunch of helper functions for me to work with Azure Container Apps dynamic sessions um and then to create the um the the actual MCP server um actually what I'm going to do is I'll just um back out this change where I deleted the file from my project that way you don't have to watch
            • 38:00 - 38:30 me type i'll just kind of walk through walk you through what I did so I basically went through that tutorial and more or less just copy and pasted right like so I had this uh MCP thing um I changed the name um from weather to sessions code interpreter um I created a code interpreter session using basically my my my helper file which is which exposes a class and then I wrote this very simple function really called execute python code um I decorated with mcp.tool so
            • 38:30 - 39:00 it's automatically going to be going to get exposed as a tool um and I here I kind of gave it um some instructions um and these instructions are going to get or the description is going to get passed directly to the agent and the LLM uh so I'm just saying that hey this thing is a Python shell always use this tool when you need to perform math um because I've kind of run into some issues with some other agents um in cloud as well where it decides not to use it so I'm basically tell saying hey like you know when you need to do math
            • 39:00 - 39:30 or any other calculations or computations I'm basically saying the same thing three times um like because math is a calculation and a computation um but um but yeah when you need to kind of do anything that uh that that requires you know precision um make sure you use this tool it should be a valid Python command and we're going to return you you know the the result as well as the standard output if there is any as well as the standard error if there is any as well um and the input is Python code uh and that's it so really like
            • 39:30 - 40:00 within the body of my function is one line which is hey like to basically call my library and execute this code in a session and then return the result and that's about it and we run this so um so like my my goal of this is to kind of show you um if you haven't built your own MCP server before it's really easy to do um and yeah you can have this up and running in a few minutes so now that I have this uh code running
            • 40:00 - 40:30 um I'll switch over to Claude here and then I think somewhere here there's settings it opens on the screen so I'll bring it over here developer edit config and I'm just waiting for the config to open no I can drag it over here actually
            • 40:30 - 41:00 open it here please okay so there here's the config um it doesn't have anything in it uh in order to kind of build out the config let's uh go back to our documentation uh yeah so this thing walks you to walks you through literally everything so I'm going to take this and put it in here um the command is UV great um
            • 41:00 - 41:30 absolute path to my parent folder which I can do that uh which I can get by copying my path pasting that here remove this and instead of weather.py mines main.py which is my um MCP server so all I'm doing all I'm doing is that uh telling Claude about this MCP server let's call it a code interpreter u and I'm saving it uh and I
            • 41:30 - 42:00 think uh yeah so you can add as many of these as you want um so you can actually grab some off the internet you can build some yourself um in each one of these you add um what Claude is going to do when it starts up is that it's going to execute these commands um to kind of spin up these servers and then um it's going to you know discover the tools that are available from each server and and and allow you to use that from within the chat um so hopefully this highlights how you know sometimes you
            • 42:00 - 42:30 know you don't want to just you know download anybody's code and run it locally because that's what it's going to do so make sure that you're um that that you know what you're running what you're giving it to what you're giving to Claude because this is going to start them up so if it's code that you don't trust you probably don't want to put it in here uh so I believe I have to go restart Claude if anyone knows of a way to like restart this without me quitting and starting again let me know
            • 42:30 - 43:00 oh it failed i think I know why it failed um so for some reason it didn't know where UV is which is weird um because UV it's kind of a global thing so I'm going to just go which UV and I'm going to just grab the full path and then so I go back to this thing
            • 43:00 - 43:30 here and change out UV paste it in hopefully this works if it doesn't we're going to have to move on um so I'll have to go ahead and restart Claude again while I do that I'll check the chat to see if anyone has a better idea well there's a lot of talking in there that's cool
            • 43:30 - 44:00 um yeah Pamela says I find the restart MCP server experience much nicer in VS Code that is entirely true okay so it didn't complain this time uh so we can do do stuff like hey what is you know some large number multiplied by some large number and I'm hoping oh yeah look at that so it's u uh I'll allow it for this chat so it's actually asking me for permission to use this run code can I command plus this
            • 44:00 - 44:30 thing okay I can make this bigger so you can see so allow for this chat so it's actually going to go and use my tool that I built and hopefully give me a result yeah so it did that um so you can actually see exactly what it did it ran the code and it got back this is the result that it got back from the um the code interpreter session um I'm sure I can probably like you know do I still have my word problems up on the screen here
            • 44:30 - 45:00 oh did I close it that's not great no no no no no no no no no well am I Oh wait no it's right here never mind uh so let's get another question uh something about an employee with the magazine let's give this to Claude i'm sure when we invented AI this
            • 45:00 - 45:30 is exactly what we meant to have it solve for us uh so I can see that I wrote some Python code and it's executing it hopefully so so it got the result back let's take a look at the result so this is the raw result uh somewhere oh yeah so basically kind of returned some results there um and it
            • 45:30 - 46:00 kind of interpreted it and uh must sell 15 magazines so let's see if they're correct uh question 74 question 74 yep 15 magazines so yeah so it worked uh so it was able to use my um sessions code interpreter MCP server to to solve some of these problems so um so yeah so hopefully this gave you a good intro to MCP and how you can use it um to add more capabilities to your agents so I'm
            • 46:00 - 46:30 going to quickly check the chat and then we have one more demo that's wrong chat uh oops my other chat that's not the right chat either there we go and also if um Danny or um or Pamela if you're on the thing on the on StreamYard feel free to just pop in and ask any of these questions that you think are relevant because my reading's kind of slow uh
            • 46:30 - 47:00 yeah so yeah so if there is any questions um I I'll I'll try to catch it a little bit later i just want to make sure that we get to this last demo um so okay let's use this window here uh I'm just going to go and pull up another code editor okay so we're going to completely switch gears not completely but switch it more than we've done before uh so
            • 47:00 - 47:30 uh so recently uh open AAI um I guess this is a while ago depending on like things move so fast in in in in the AI world um but earlier this year I think it's January um OpenAI um announced this thing called operator and the thing power and empowering operator is the idea of um these computer using agents so these agents they're able to um basically you know almost unattended surf the web um and and kind of like you
            • 47:30 - 48:00 know basically do things for us whether it's look up information or it can actually you know go and buy us something or book us a flight or something like that uh so I'm going to go ahead and search for it open AI maybe computer using agent you see I've already been there um so that you know um you can all kind of go and search for the same blog post to kind of read about computer using agents um but in a nutshell there is a
            • 48:00 - 48:30 little diagram here about how it works where'd it go it's a little small so I'll try to kind of quickly walk you through it so um so this agent um is multimmodal so you can actually basically give it um some instruction so like kind of same as before it's like hey you know like you know uh show me the trending you know items on Hacker News or something um but you can also give it something else um like in this case a screenshot so um so what we're
            • 48:30 - 49:00 going to do is we're going to give it a screenshot of a web browser um and then based on the input and the screenshot um it's going to decide what to do next so um you know so it might see a search screen so it might decide to click into the search box and then it kind of goes into a loop so it kind of goes back to the agent or so it kind of goes back to our application um and then our application is going to go okay um it did something and now I could basically create another screenshot and give it to the agent again so so this time when we
            • 49:00 - 49:30 give the agent the screenshot it's going to be you know the same screenshot as before but it has a cursor in the uh in in the in the search box so now the agent might decide oh I should type you know trending on hacker news or something and then it'll basically you know instruct us to do that we do it give it another screenshot and now it sees that oh I've entered the you know like I've typed something so I should click the you know the search button or something so you kind of see this kind of go in the loop um until it kind of solves for problems so so so the cool
            • 49:30 - 50:00 thing about this is that um you can really kind of give it anything um it's it's able to interpret um as long as you know like the the thing that you want to do um supports taking a screenshot and programmatically you know typing and scrolling and clicking um you can actually hook it up to a computer computer using agent so a browser is a good example but I think I've already seen examples of this um of like you know uh basically using the same process but have it take over a computer right so instead of taking screenshots of a of
            • 50:00 - 50:30 a web browser and give it to the agent you can actually take pictures or basically take a screenshot of the um the whole operating system itself uh the the desktop and then giving it to the computer using agent and then it kind of decides oh I need to open this app i need to do this and that um and automate a bunch of stuff um so in this case I'm going to just kind of quickly show you um uh how how this runs locally and then we'll quickly show you how to run this
            • 50:30 - 51:00 um again more securely using um uh container apps sessions code interpreter uh let's see so where am I uh so I kind of want to like So this is a um a sample app so again I think we shared the links um to these repos so I just kind of forked the OpenAI's example and I just added one um an agent for um for that
            • 51:00 - 51:30 that kind of runs the browser in a code interpreter session um everything stays the same so you can go ahead and kind of look at that um at your own leisure it's going to you know there's quite a few files the the key one is probably this agent thing here um so behind the scenes the the the key thing here is that we're we're basically in a while loop so it's the same as a chat before except that this this is basically um a a a while loop just kind of keeps going by by
            • 51:30 - 52:00 itself so um so on on every um loop um or basically every iteration of this loop we're basically taking a screenshot and some user input and then we're giving it to the agent so in here if you have 12 into it um it's some helper code like it's it's just a helper library uh that calls the computer using agent um API in uh in this case Azure open AAI but you can have access to this in in just the regular open AI as well um and
            • 52:00 - 52:30 basically just giving it the screenshots and then it's kind of coming back with the results and then it's deciding based on the results like well not the results but like you know that the the the a the model is telling us what to do next and we're basically play in this loop doing the next thing and then giving it another screenshot hopefully that kind of makes sense um but that's all it is here so it's just like you know it's a pretty simple loop it's literally hey like uh here's a screenshot and and and what we've done so far and then uh the agent u or the model is going to tell us
            • 52:30 - 53:00 what to do next and then we're basically going in the loop and it's going to then kind of asking it which what should we do now what should we do now um so I'll kind of quickly run this Uh so I'll run the local version first and then I'll quickly run the remote one and I think that will be all we have time for today i'm looking around because I'm waiting for a web browser to start up okay so it started up a web browser this
            • 53:00 - 53:30 is running locally so um it's using Playright to drive this web browser it's open up Bing um and we can ask it a question and so like you can ask it what's trending on hacker news and basically now it's actually entering that loop that we were looking at before um so it's sending a screenshot and the current URL um to the model as well as what were we asked for before and then the model is saying hey I want to click here and then we're
            • 53:30 - 54:00 actually you can see that we're clicking on the screen you can see the model says type hacker news we're typing hacker news on the screen um and then I think they you know it wants to press enter and we press enter for it on the screen um and then hopefully it's going to decide to maybe click on the link or something let's see if it does that it wanted to wait it It saw that the page wasn't finished rendering yet so um but now that it's waited it's probably going to hopefully click on something you can
            • 54:00 - 54:30 do it you could do it oh it clicked on something um and it's probably going to now summarize what's on the screen so it's actually you know it's got got some pretty um advanced computer vision to actually um uh be able to kind of like not only do OCR on the screen but actually understand where exactly something is because it needed to know where to click as well so uh it's pretty wild i'm still trying to understand how this thing works um but hopefully what
            • 54:30 - 55:00 it says here it wants to wait i don't know what it's waiting for um but hopefully this next iteration is going to tell me the answer please tell me the answer tell me the answer oh no it crashed that's not good well but you can see that it was actually pretty cool like sometimes it does crash i don't know if it's my code or their code or something but um but it basically the model gave a response that we didn't quite understand and it blew up um but
            • 55:00 - 55:30 you can kind of see the promise of it it's actually clicking around um but again this is running locally right um and the tricky thing is if you run it locally then you're going to you know if there's any security issues with it um you know like because you're kind of browsing the random websites and things um you know might might not be the safest thing to do so um so what I've done is you can actually look at the code i'm not going to walk through it here because I don't think we have time um is I've kind of wrote a a version of this that uses the sessions code interpreter so not so it's not going to
            • 55:30 - 56:00 run in a sandbox so you can see I'm starting up the sessions behind the scenes and I think the part that you're not seeing is a browser is not popping up because it's no longer running this browser on my computer it's actually running it inside of um a uh container apps dynamic session um like I said before the session um that the code interpreter already includes playright as well as everything that you need to run a web browser in there so I basically kind of
            • 56:00 - 56:30 just built a another kind of um implementation of this agent that uses that uh so I can ask another question so I can you know I'll try again what what's uh trending on hacker news it's going to go ahead and run it um and while it's running because I can't see anything what I've kind of told this thing to do is save a log you can see I've tested this quite a
            • 56:30 - 57:00 bit so I'm going to go ahead and copy the path to this log and then in my browser going to go ahead and open it up uh so you can actually still see even though it's running remotely that it's um you know it's doing stuff hopefully it's going to type hacker news or something like that still thinking I'm just refreshing the page i
            • 57:00 - 57:30 don't know why it's not clicking so it's doing something weird but that's fine thankfully I for I have some logs from yesterday when I ran this which for some reason worked a lot better uh so let me What's happening uh let me shrink this i have a folder here called good so these are the logs that actually ran well so uh so let me
            • 57:30 - 58:00 pop one in here so this one here um I'm asking hey um it's not printing my input but I basically said hey um help go find a polo a red polo shirt and buy it for me um and then it's like hey like before you know proceeding i I want to uh you know basically tell me some preferences or whatever i think I told it the preferences um and then you can see what it's doing it's searching for red polo under 50 bucks it found a bunch of red
            • 58:00 - 58:30 polos it clicked on one the hilarious about this that hilarious thing about this is it's like hey Amazon's like "Hey has we think you're a robot?" Um so I'm pretty sure actually like that it it knows enough to actually solve this capture but it didn't do it it actually asked me hey well like we're trying to access the Amazon page there's a capture can you answer it for me so basically what I did was I just kind of typed out these six letters and kind of gave it back in our conversation and it kept and then it kept going and it's actually pretty funny to watch the robot type
            • 58:30 - 59:00 this thing into a page that asks whether you're a robot um but eventually it got through it found this uh shirt looks great um and the cool thing is it actually knew that hey I I asked for medium so it clicked on the um the dropdown chose medium and then um yeah added it to the cart that was pretty cool and I'll quickly show you another one and then we'll we'll kind of wrap up here uh I think this one
            • 59:00 - 59:30 here whoa it's a long one really long one how do I go to the top uh I asked for like Yeah so I I changed the app so it actually prints out what I what I typed in so it's like "Hey give me a you know like the the cheapest flight from LA to Maui don't ask me anything just go and figure it out for me." So you can see that it's it went searching i asked it to use Google Flight so I think it actually looked for Google flights it found Google flights um it knew what to
            • 59:30 - 60:00 type in um I'm in Vancouver knew that I didn't want to go from Vancouver i want to go from LA so it kind of changed that to LX and it changed to Maui I'm sure um uh the thing that blew my mind actually that was uh I think somewhere here um it started understanding how to use the date picker which I didn't quite know why how it understood that and also what it another cool thing was um it knew that I was asking for a one-way
            • 60:00 - 60:30 trip so it actually went ahead and changed the drop down to one way anyway a bunch of stuff happened it looked at things and eventually at the very end somewhere uh where was the answer i think somewhere along the way it told me the answer um but you can see that it actually kind of like in the screenshot it gave the answer anyway um so so yeah so it's pretty cool what it's able to do um and yeah so that's a computer using agent check it out um I know we're at
            • 60:30 - 61:00 time so we're going to go and check the chat um yeah a bunch of stuff i'll try to stay on the YouTube chat and and answer anything that's there i think Pamela and Josh have got it taken care of yeah it does feel like uh magic when it works and I and I do see that it's uh it's kind of changing a bit i think a few days ago or last week when I tried it it refused to check out for me um when it added something to the cart um or and yesterday I saw that it asked me if I wanted to check out i didn't actually go
            • 61:00 - 61:30 through with it uh so so I think uh it's getting smarter and they're kind of changing some of the guidelines about what what these things are able to do um so yeah so it's pretty cool try it out try out the code interpreter thing um try out um you know if you if you need a sandbox try out container apps uh sessions so I don't know how to end this uh is someone someone gonna jump on and say goodbye do I say goodbye yep anthony you can just close out and
            • 61:30 - 62:00 then we'll have a video to play okay awesome uh thanks everybody um yeah um it was great hanging out with you all um thanks for kind of chatting with us in the chat i'll try to look at the chat um and see if I can still respond to a few things if there's anything that needs attention there um yeah enjoy the rest of the hackathon enjoy the rest of your day um yeah thanks
            • 62:00 - 62:30 hi everyone hope you enjoyed today's session if you aren't registered for the AI agents hackathon yet register now at aka.m/ aentshack and start hacking on the agent of your dreams and if you have any questions along the way just post in our GitHub discussion forum or attend our live office hours in Discord and of course make sure that you keep attending the live streams to keep learning about agents once your project is ready submit it
            • 62:30 - 63:00 before April 30th at 11:59 p.m uh PT to be eligible for our prizes thanks for joining