Agents and Hackathon Kickoff with Microsoft Reactor

Agents 101 / AI Agents Hackathon Kickoff!

Estimated read time: 1:20

Summary

The "Agents 101/AI Agents Hackathon Kickoff" session hosted by Microsoft Reactor introduces participants to the world of AI agents and the associated hackathon event. This one-hour session covers various aspects of building AI agents, showcasing Microsoft's technologies and tools. Whether you're a beginner or experienced developer, the session aims to equip you with knowledge on creating agents using multiple programming languages and Microsoft's platforms. Participants are encouraged to engage actively and pursue innovative projects to win exciting prizes.

Highlights

Discover the principles of building AI agents and their applications. 🧠
Explore multiple platforms and tools including Azure AI and GitHub models. 🔧
Engage in a virtual hackathon with numerous tracks and exciting prizes. 🎯
Learn how to integrate and use tools effectively within AI agents. 🛠️
Join live sessions to gain knowledge and get ready for the hackathon challenge. 📚

Key Takeaways

AI agents are semi-autonomous software entities using large language models for tasks. 🌟
Participants are encouraged to utilize Microsoft's technologies for building agents. 🛠️
The hackathon offers various language tracks and significant prizes. 🎉
Understanding tools like Azure OpenAI, GitHub models, and Azure's orchestration services is key. 🔑
Engagement through live streams and expert sessions is encouraged. 🤝

Overview

The Microsoft Reactor's "Agents 101" session is a stepping stone into the fascinating world of AI agents. Aimed at both novices and seasoned developers, it highlights the use of large language models to create agents capable of performing tasks autonomously. Participants are introduced to a range of tools like Azure and GitHub models, key in developing robust AI agents.

This kickoff session is more than just an introduction; it's a segue into the upcoming AI Agents Hackathon. The hackathon provides a platform for individuals and teams to flex their creative muscles, leveraging the diversity of programming languages and technologies offered by Microsoft. With tracks available in multiple languages, there's something for everyone.

Participants are urged to immerse themselves in various live streams and sessions led by experts. These interactions serve as a valuable resource, not only for the hackathon but also for broader learning in AI agent technology. The excitement is palpable as teams gear up to innovate and potentially win prizes, making the hackathon a not-to-miss event.

Chapters

00:00 - 01:00: Introduction and Code of Conduct The chapter introduces the event planner, Danny, who welcomes attendees to the New York Reactor session. The emphasis is on upholding a respectful environment through the code of conduct, which includes maintaining professionalism in chat interactions and being mindful of commentary.
01:00 - 02:00: Session Overview The chapter titled 'Session Overview' discusses the logistics and introduction of a session being conducted. It mentions that useful links will be shared in the chat and the session is recorded, becoming available on-demand on the Microsoft Reactor YouTube channel 24 to 48 hours later. The session is scheduled to run for about an hour with interspersed opportunities for questions. The introduction is passed over to the speakers, starting with Josh, a Program Manager at Microsoft, who collaborates closely with Pamela.
02:00 - 03:00: AI Agents Hackathon Announcement Pamela from the Python developer relations team introduces the AI Agents Hackathon, a free virtual event happening from April 8th to 30th, with live streams to teach participants how to build AI agents on Azure.
03:00 - 04:00: Hackathon Prizes and Registration The chapter 'Hackathon Prizes and Registration' discusses the extensive range of sessions available, focusing on building agents using Microsoft technologies. Over 30 sessions are offered, covering various programming languages such as Java, Python, JavaScript, and .NET, along with both high and low code solutions. The primary aim is to impart comprehensive knowledge to participants.
04:00 - 05:00: Team Participation and Submission Guidelines This chapter discusses the guidelines for team participation and project submission in a competitive event. The emphasis is on encouraging participants to work on innovative projects and submit them for prizes. To accommodate a diverse group of participants, the event features streams in languages other than English, including Spanish, Portuguese, and Chinese, each with multiple tracks. This multilingual approach aims to engage a broader audience, allowing them to contribute and compete in their preferred language.
05:00 - 06:00: Introduction to AI Agents The introduction outlines the various prize categories and rules for an AI hackathon. It specifies that prize amounts are per team, not per individual, unless competing solo. Categories include best overall, language-specific categories, best Copilot agent, and best Azure AI agent service usage.
06:00 - 07:00: AI Agents Demonstration The chapter titled 'AI Agents Demonstration' focuses on the process and importance of registering for a hacking competition in order to be eligible for prizes. It emphasizes the need to use the specified link to reach the registration form. The chapter also instructs participants to ensure that their project submissions are complete after hacking.
07:00 - 08:00: What are AI Agents? The chapter provides details on participating in a hackathon related to AI agents. It outlines the participation rules, which include forming a team of up to four members, ensuring that everyone registers for the event, and submitting the project by April 30th, 11:59 PM PST. The submission process includes providing a link to the project repository as specified on the provided URL.
08:00 - 09:00: When to Use AI Agents The chapter begins with a brief mention of a short demo video followed by a discussion on the timeline for a hackathon, including judging and announcement of winners. After this logistical introduction, the chapter transitions to the core topic of AI agents. The speakers introduce their colleagues, Marlene and Corey, who will delve into an introductory session on AI agents, labeled as 'Agents 101'.
09:00 - 10:00: AI Agents Architecture The chapter begins with an introduction to the hackathon, which is filled with numerous workshops, 30 in total. The speakers, Corey and Marlene, are excited to kick off the event and engage with a global audience. There's a sense of anticipation and enthusiasm about starting with AI Agents 101.
10:00 - 11:00: Building AI Agents with Microsoft Tools The chapter provides a walkthrough for building AI agents using Microsoft tools, beneficial for both beginners and those experienced in the field. It includes practical insights particularly useful for hackathon contexts. Many graphics and contents discussed are derived from Microsoft's AI agent resources.
11:00 - 12:00: Making Good AI Agents The chapter discusses the basics of creating functional AI agents, particularly targeting beginners. It encourages those who have taken the introductory course to engage and share their experiences. The speaker refers to a resource link for beginners and mentions their own involvement in its creation. The focus is on understanding the construction and potential applications of AI agents, making it an introductory yet personal guide to AI developments.
12:00 - 13:00: Tool Integration and Agentic RAG The chapter titled 'Tool Integration and Agentic RAG' begins with James addressing the audience regarding the current status of a set of lessons, mentioning plans for an additional lesson. James shares his excitement about an eleventh lesson he intends to demonstrate. Despite identifying as a hacker, he mentions unable to participate in a specific hackathon due to rule constraints, although he has created a tool to assist the audience with their hackathon projects. He engages with Marlene to discuss challenges commonly faced during such hackathons.
13:00 - 14:00: Introduction to Model Context Protocol (MCP) The chapter introduces the difficulties participants often face in developing winning ideas during hackathons, especially given the substantial prizes at stake. The speaker mentions a prize of $20,000, highlighting its significant value in today's economy. To address these challenges, the speaker introduces a tool designed to assist participants in generating ideas. The speaker demonstrates this tool during the presentation, suggesting it could be beneficial for everyone involved.
14:00 - 15:00: Building Trustworthy AI Agents In this chapter, the focus is on building trust in AI agents. The presenter discusses a recommendation project for hackathons, specifically mentioning a GitHub user named Marlene. The goal is to confirm the GitHub username before proceeding with the recommendations. Furthermore, there's an emphasis on interacting with Marlene's GitHub account, suggesting to give her a follow.
15:00 - 16:00: Corey's Advice on Building AI Agents The chapter, titled 'Corey's Advice on Building AI Agents,' explores the processes involved in creating AI agents. It specifically highlights a scenario where an AI agent is in action, as it handles requests related to a GitHub hackathon and events agent. The GitHub agent, for example, is capable of recommending projects and repositories for the user by leveraging the GitHub MCP server to access relevant data. This exemplifies how AI agents can autonomously interact with platforms and deliver tailored suggestions to users based on their needs and preferences.
16:00 - 17:00: AI Agents for Beginners Course In this chapter, we discuss the strategic approach to winning a hackathon using AI agents. Initially, the focus is on setting up and describing your repository, including use cases for the tools you've developed. The chapter emphasizes the importance of not just participating in the hackathon, but also aiming to win. To achieve this, a hackathon-specific AI agent is introduced. This agent is designed to assist in refining participant submissions for better outcomes.
17:00 - 18:00: Building AI Agents with Microsoft Technology The chapter details a hackathon focused on building AI agents using Microsoft technology. It mentions creating an AI agent named 'smart document processor' that utilizes Natural Language Processing (NLP) to manage various document types effectively. Though the naming isn't very creative, the functionality seems promising. The chapter also suggests tools to aid in this development process.
18:00 - 19:00: LLM Options and GitHub Models The chapter discusses the convenient integration of frameworks with GitHub repositories, allowing for easy access to specific locations within the code. It highlights the utility of building systems, such as invoice management and resume analysis, which can be stored and accessed through GitHub repositories. Additionally, the chapter offers architectural recommendations for implementing these systems.
19:00 - 20:00: Azure AI Agent Service The chapter discusses the Azure AI Agent Service and highlights its features, particularly focusing on its capabilities to recommend prize categories. It provides users with various options and emphasizes the learning aspect through its sessions. The service incorporates over 30 sessions into a searchable database using Azure AI search, making it easier for users to find specific information.
20:00 - 21:00: Orchestrators and Data Storage In the chapter titled "Orchestrators and Data Storage," the discussion focuses on optimizing sessions by recommending event names, making it easier for users to RSVP. There is an emphasis on how Python can be effectively used to develop agents, particularly in helping select relevant events. The aim is to facilitate attendance at all possible events, though the full schedule is unknown. This approach aids in filtering and identifying suitable events.
21:00 - 22:00: Copilot and Standalone Agents The chapter titled 'Copilot and Standalone Agents' discusses using the AI agents repository to find relevant events and suggestions for hackathon projects. It mentions an example lesson labeled as MCP that is not fully public yet but includes a code sample that can be run with a user's GitHub name. This allows users to get tailored suggestions for their own projects. The discussion appears to be enthusiastic about the potential for projects using this tool.
22:00 - 23:00: Conclusion and Next Steps The chapter titled 'Conclusion and Next Steps' draws inspiration from a demonstration (demo) that was hopefully clear and motivating for the audience to try on their own. The chapter highlights the importance of this demo as a source of ideas and learning.

Agents 101 / AI Agents Hackathon Kickoff! Transcription

00:00 - 00:30 hello everyone and thank you for joining us for today's session My name is Danny and I'm the event planner for the New York Reactor Before we begin please take a moment to review our code of conduct We seek to provide a respectful environment for both our audience and presenters We encourage engagement in the chat but please be mindful of your commentary remain professional and on
00:30 - 01:00 topic Useful links will be shared in the chat The session is recorded and will be available on demand in 24 to 48 hours in the Microsoft Reactor YouTube channel Which brings us to today's session It will run for approximately 1 hour with times for questions throughout I will now turn it over to our speakers for introductions Hello everyone Nice to meet you My name is Josh I am a PM at Microsoft and I work very closely with Pamela on the
01:00 - 01:30 Python de developer relations team Hello I'm Pamela I'm on the Python advocacy team and I like to show Python developers how to build stuff on Azure Cool Welcome to the AI Agents Hackathon live streams Uh the AI agents hack is a free virtual event running from April 8th to 30th Yeah we are going to have a bunch of live streams during this hackathon in order to show you how to build agents
01:30 - 02:00 using Microsoft technologies We actually have over 30 sessions I wasn't able to fit all of them into you know into this slide because we have so many so many sessions So our goal is to give you all the knowledge that we have about how to build agents in uh you know multiple programming languages Java Python JavaScript.NET net with multiple Microsoft technologies from uh you know high code even to low code and to give you all that knowledge so that then you
02:00 - 02:30 can use it in order to hack on an amazing project and then submit that for prizes and we even have streams in languages other than English So we do have a Spanish stream with eight tracks in it a Portuguese stream with three tracks in it and a Chinese stream with a few tracks as well Uh so we hope that we can bring in those of you who are speaking these other languages in addition
02:30 - 03:00 So Pamela mentioned prizes and we have big ones this time Uh we do have a bunch of different categories just like our last hacks Um but uh some things you want to note Uh each prize amount listed on the right side is per team and it's not per team member unless you sign up to hack as an individual Um so there's a best overall and then there's a bunch of language specific categories along with the best copilot agent category um and a best Azure AI agent service usage
03:00 - 03:30 category Now it's really really important if you want to be eligible for these prizes you must register using the link shown on the screen here aka.m agent hackregister U make sure you go to this page It should look a little bit like that Um and then it there's a simple registration form and you're good to go And then once you have your submission uh complete you have your project all hacked up um just make sure
03:30 - 04:00 that you and any of your team members up to four people total You can uh enter the hack as an individual You can enter the hack with two people three people up to four Um just make sure that everybody registers for that hackathon Um and then before April 30th um at 11:59 PM PST make sure to submit your hack using the process described on that link listed there aka.ms/ aentshack/submit Um you're going to need a link to your project repository and a
04:00 - 04:30 short demo video And then concluding uh following the hack um there will be a couple of weeks where our judges will spend some time scoring your projects and we'll notify everybody once winners have been announced All right So now we can actually talk a bit about agents We're going to bring in our colleagues Marlene and Corey to go into agents 101 Thank you
04:30 - 05:00 Hello everyone Uh we had the honor to kick off this amazing hackathon full of workshops 30 of them but this has to start with one and Marlene and I are that one Welcome Marlene How are how are things things are good Corey how are things with you i'm excited to be here for the hackathon with agents 101 Very excited about that Perfect Well and every everyone I see everyone in the audience coming globally Uh well what
05:00 - 05:30 first thing we're going to start with is uh some content that we're just generally going to walk through what you need to know about building AI agents And even if this is your first time building an AI agent or you're not familiar with the term you get something out of it or this is like your 30th time because we're going to have some advice at the end on at least from my perspective of building AI agents especially for hackathons uh and what that entails Uh a lot of these graphics and the content that we're going to get is actually coming from our a AI agents
05:30 - 06:00 for beginners course If you've taken the course please write in the chat that you have so you can make my day Say say that you've seen it at least the repo If you haven't the link is there right below Aka.ms ai-agents-bginners That was me that made that long one for you Uh but one of the things is um how to actually build an agent or what can you build um so that's who I want to answer for the first time I love James James says he's taken this
06:00 - 06:30 twice so that's great James Um right now it's 10 lessons but we have an 11th lesson and I want to demo what that is going to look like So uh I consider myself a little bit of a a hacker myself Unfortunately I can't compete in this hackathon That would be against the rules So but I wanted to build something for you in the lovely audience uh that could help you with your hackathon project And one of the things uh Marlene what what would you say is like the hardest thing one of the hardest parts at least of a hackathon
06:30 - 07:00 i think the hardest parts of the hackathon is coming up with ideas that can actually win because these are great prizes I feel like $20,000 I can do a lot with that In this economy I can do a lot for sure In this economy for sure So uh I built a little something to help you with that Marlene and help everyone else with that So if you go ahead and put my screen on there we will demo something to you today Basically what I want to do Marlene I'm going to ask I'm going to
07:00 - 07:30 say uh let's see I'm going to recommend Hackathon projects for the GitHub user And Marlene I believe this is your GitHub user name We're going to confirm this later on and and this is uh basically hopefully you can see this pretty good on your side Um it's gonna it's gonna do a couple of things but before we go on let me just confirm So this is you Marlene Give Marlene a follow on GitHub Um exactly and this is what we want to do here And
07:30 - 08:00 I I want to kind of briefly show you uh some of the processes of an AI agent And this first step is what you see is it's processing requests uh with a GitHub a hackathon and an events agent Uh so the first uh thing that this GitHub agent does is yeah here are some recommended uh projects and repositories for the GitHub user So this is actually going in it's actually using the GitHub MCP server uh to get some of your lovely
08:00 - 08:30 repos that are on your repository now Um and it gets a description uh maybe some use cases of what that is So exactly what you've built and then uh you know it's going to it does make some suggestions here but it's not you know we want to we want to win this hackathon right uh we don't want to just you know have participating good but you know winning is also amazing So we actually then use this hackathon agent and what this hackathon agent does is goes in and
08:30 - 09:00 uh it has uh details around this specific hackathon and also encouraging you to make good ideas around building an AI agent So let's see here what it suggests to you So it has this smart document processor is the the name uh which wow not very creative on the naming but okay So build an AI agent It intentively processes various types of document the agent will use NLP and make it easier to manage their documents Okay that seems pretty good I I also then see that here it recommends some tools to use uh
09:00 - 09:30 frameworks And what's really nice about this uh is it also links to the repos that it uses from your GitHub uh on exactly where that is uh taking place So you know maybe you build some I don't know if you remember this like memory lane for you but invoice system you you at least have in your repo and some resume analyzer Okay Wow Yeah Yeah That's some recommendation on architecture uh how you might want to implement this as well And then even like I said you know this is focusing on
09:30 - 10:00 you know getting the prizes It's going to also recommend some of the prize categories based on that suggestion Um wow that's amazing So it gives you a couple of them uh to do So you know you have all the options in there but another thing a huge thing about this is the learning aspect right we have all of these sessions 30 plus sessions it might be hard for you to find all of them and so what this actually does then I I put in uh in a database in a Azure AI search database all of our descriptions of our
10:00 - 10:30 uh sessions and then it should go in and recommend uh the event name so this is like some events coming up the URL so you can just go ahead and RSVP for that uh And I really like doing this with agents also like tell me why you why you think this is going to be a good one And since obviously you use a lot of Python it's saying oh this is going to be great for you to you know develop agents with Python We obviously want everyone to attend all the events if that's humanly possible I don't know I haven't seen the schedule but this is one way uh to to kind of weed out and find out uh what
10:30 - 11:00 are some of the relevant events for you So this is actually on the AI agents repo Now this example it's in a hidden lesson not necessarily hidden but it's an MCP lesson that we're currently building but this code sample you can run You can use your own GitHub name and you can get some great suggestions on hackathon projects Uh so do check that out and you can run that Um and yeah it's a good start Amazing So cool Corey Um we are I'm feeling personally
11:00 - 11:30 inspired by this demo Uh I hope you were all able to see that and also will be able to also try it out on your own I think this is a great way for us to get some ideas So I think let's jump back into the slides and dive in a little bit more about what agents are Let's do it Great Um so Corey showed us a great example of an agent that is being able to generate
11:30 - 12:00 some ideas for us for this this hackathon But what exactly are AI agents there are a lot of definitions out there on the internet and a lot of disagreements about what an AI agent is But one definition that we can use is that an agent in LLM based applications is a semi-autonomous software entity leveraging large language models to perform specific tasks through natural language interaction So that's one way
12:00 - 12:30 we can think about agents is that they're kind of autonomous They're pretty they're able to do things on their own and they're going to leverage LLMs to perform certain tasks um through through interaction with us that can seem a little bit abstract and complicated So another way I like to think about agents or visualize this is through this diagram on the screen here So we can really think of an agent as an
12:30 - 13:00 LLM that can perform actions and that has access to different tools and knowledge and it combines all of these things to be able to achieve a specific goal So some things that I want to call out about this specifically One thing to note is that the concept of AI agents is quite an old concept It existed even before LLMs existed But when LLMs were created one of the advantages that we
13:00 - 13:30 found was that they are able to interact and understand human language and data And so in that way they can interact with the environment get instructions from us and then perform actions from those instructions or get some understanding of what to do And a second thing to note is that outside of AI agent systems LLMs usually don't have a lot of actions they can perform So if you used chat GBT in the past you'll
13:30 - 14:00 probably know that LLM really only can perform the action of taking in some user input and responding back with some information or some content Uh they've improved a little bit now you know they're adding different tools to it but usually they're quite restricted in terms of what they can do or actions they can perform But now when we introduce tools to these LLMs and give them tools either in their immediate environment or you're allowing
14:00 - 14:30 developers to give those build those tools into those AI agent systems the agent the LLM is actually able to perform those tasks that it that we give it And then a third thing that we can focus on here is the knowledge aspect of this in that LLMs in AI agent systems will have some sort of memory So that memory can usually be an interaction with the user just in the conversation which is short-term memory or they can
14:30 - 15:00 have long-term memory where they're accessing information from outside of their environment like with a Victor store using some rag or something to do that And so when you combine all of these things together we actually have a really powerful system that you have LLM being able to take some action So we're really moving away from just chatting to an LLM like we do with copilot or chatbt and actually expecting the LLM to take
15:00 - 15:30 some action on our behalf to achieve a goal The next thing that we can look at is how is when to use AI agents We have some specific scenarios that we would recommend using agents for The first is if you have an open-ended problem So if you have an a problem where it's hard to predict what the next task is going to be it's really going to be difficult in Python for example or whatever programming language you're using to uh use if else statements at
15:30 - 16:00 every turn because you might not know what what you know what's the next step that you need to take And the cool thing with agents uh the LLM is going to be able to make decisions and plans so that it can uh work on make those decisions for you without having to use an if else statement The next thing that we would recommend is to use agents for multi-step processes So if you have a process that's quite complex and it requires a lot of different tools and
16:00 - 16:30 tasks to switch between agents are really great for this because the LLM can really coordinate that switching in between tools And then a final thing is that agents often improve over time So when you have these systems using an AI agent allows uh the user to give it feedback um maybe from user input or just the environment the LLM is in and it'll take that feedback and can potentially self-improve and so you can
16:30 - 17:00 have these systems that get better over time So those are the three particular scenarios that we would recommend using agents for Uh Corey go ahead Yeah very nice Thank you Uh I I feel like I definitely missed a step in my demo and should have made a matchmaking agent as well because everyone in the chat is looking for teams So that will be my my next build option will be a matchmaking uh for Yeah there you go I think about the wrong matchmaking Yeah but team matchmaking not any other
17:00 - 17:30 type please Yeah Um I don't want to be liable for anything So what we what to your point though uh Marlin what I like about this sample here uh just to give you a bit of our architecture of you know building all these sort of applications is that yeah there's some limitations to like just a large language model outside the box but there's obviously some uses and there's also some uses for uh you know a making a large language model into an agent essentially giving it tools and access
17:30 - 18:00 and this is a good example of using both those things as well as multiple agents So in this case right we have this customer coming in and you know support center or call center customer support it's one of the bigger use cases for agents currently and the first thing that this customer is asking about is something around the like smart home thermostat uh they has got some problems connecting the Wi-Fi which is a problem regardless anytime you have one of the new smart home devices it seems like and and then they keep seeing this era E22 so they have no idea what this is so
18:00 - 18:30 they go into a customer support channel or you know chat or whatever And out of the out of the start of this process is a query pro parsing using just a regular large language model not an agent per se and it's just basically getting out the details of that message which is the top like the device or subject area Wi-Fi uh the error that it has and exactly the device Uh this goes to a planning agent uh that actually knows uh in terms of
18:30 - 19:00 the available agents to resolve this issue And in this case we have three different agents One is an video agent that can look at video clips uh that maybe is in our knowledge database or you know we've stored somewhere the transcripts and could find exactly where uh in that video we resolve you know E22 as an error Then we also have some maybe internal uh tips and things like that that we also want to include into this answer And then let's say we're just the
19:00 - 19:30 just the company selling these uh thermostats We don't the ones that are making it Then uh you know we have uh going out to maybe the you know manufacturer of the thermostat to get that information as well So all these agencing their findings or results if you will and having the planner agent say okay with all this responses does this answer the question right uh you know we won't get into specifics of different frameworks and how they work but you know within most of these
19:30 - 20:00 frameworks we have this ability to define where the conversation terminates or when the end result is uh and then we can present an amazing answer uh to the uh to the user or to the customer in that guard So when you think about architecting your uh hackathon projects look at it from this this angle I think it definitely is a good one Great explanation Corey Um I think another thing we are going to talk about as well is how can you actually get
20:00 - 20:30 started building your AI agents So for example we here at Microsoft offer many different tools for you to get started depending on your level of technical skills So for any users that any of you that are here that are watching and you are not as technical there is an option for you So that would be agent builder Um and this really allows you to have a no code option for making custom
20:30 - 21:00 co-pilots or custom agents And then for users that are a bit more technical but maybe you're not a software engineer but you can write a little bit of code propilot studio is a great way to build co-pilots with a minimal amount of code but it allows you more flexibility compared to agent builder And then finally for developers we have copilot studio that integrates with visual studio code and allows you to customize your co-pilots And we also have a large
21:00 - 21:30 range of developer tools that we are going to talk about uh to be able to build things out as well And when I'm talking about developer tools specifically if you are someone that's comfortable with writing code which I think a lot of us hopefully uh will be comfortable with one language or another um the first thing that we're going to recommend is if you are just building a single agent So if you're working with deploying one agent we would recommend
21:30 - 22:00 using Azure AI agent service and this is for a scenario where you are going to create a m it creates a micros service for you to be both to be able to launch and deploy your agent and it'll give your agent access your LM access to all the tools that we talked about earlier and it also comes with thread storage so you can have memory of your conversation both long and and uh and short-term memory So that would be if you're
22:00 - 22:30 building out a single agent And then if you want to build out something a little bit more complex that requires multiple different agents we will have some um some hackathon uh lessons on that Then we would recommend two options for that for making orchestration between your agents easier The first option would be autogen that comes from Microsoft research Autogen is amazing Um it includes some state-of-the-art
22:30 - 23:00 orchestration strategies and just the state-of-the-art the newest and latest technology but because it's coming from Microsoft research there's often some breaking changes and it's a little bit experimental Um if you are looking for something a little bit more production ready and more stable we would then as well recommend semantic kernel which is also an orchestration AI agent framework but it's designed more for production scenarios So those are the main um agent
23:00 - 23:30 tools that we would recommend for you to use during the hackathon this this uh next couple of days Okay So now we're going to talk about how to make good AI agents And I I realize now it might seem that what we just said might be didn't apply to making good a AI agents but we didn't tell you how to make bad AI agents What this means is just uh how does apply different design principles in terms of uh agentic design Uh so what I guess
23:30 - 24:00 there's you know three areas that you want to sort of focus on when you're talking about building an AI agent uh one is space and this is just basically meaning uh that the a the agent that you're designing operates in an environment and what we mean by an environment basically could mean you know whatever systems it's using uh whether it's making tool calls like for example the environment for this hackathon uh agent that I made could be you know the GitHub MCP server in terms of its tools right uh as well and I need to be cognizant or at least recognizing
24:00 - 24:30 that this is an environment that might change like I'm hoping Marlene will continue to make repo in her GitHub I'm pretty sure she will right so if I just did that on one time uh you know call to get her repos and said that was it That wouldn't be a good agentic design right so I need to make sure that uh you know maybe I'm making more calls to this so that I can get updates and then you know update how she's developing so that we can get more relevant hackathon examples maybe for next year's hackathon I don't know Well maybe we'll do this next year I don't know Right Uh and then on time
24:30 - 25:00 side right is that these interactions between user and agent uh can happen within context but it can be overo also over time as well Marina already explained some things around uh improvements of a agents in long-term memory and you as the developer or the curator of the AI agent uh need to sort of manage that relationship of like when it when a user comes back uh do I want for example uh to have the same exact uh suggestions uh when Marlene asks about
25:00 - 25:30 uh you know recommending uh hackathon projects tomorrow as she did today or should I you know do new ones and maybe you know for this application I want to do new ones right I want to get make So I will store those recommendations in a database uh and then make sure that the agent doesn't make the same ones over again because that would get kind of boring And then at the core of this so that's the last principle is that we are still working with large language models Uh and you know to the point of and this is almost accidental but I'm going to go ahead and give myself credit for it is that once uh you know I typed in uh you
25:30 - 26:00 know recommend the hackathon project for that GitHub user for Marlene's GitHub name it said "Oh okay great I'm calling the GitHub agent I'm calling the hackathon agent and I'm calling uh the events agent." So knowing and being uh clear to the user that they are interacting with an agent is one of the core things because sometimes you know in this case like you know it talked about products that maybe aren't relevant anymore or aren't the names because we change names at Microsoft sometimes right so uh you know also being clear with that and the limitations of your AI agent is core to
26:00 - 26:30 working and designing good AI agent experiences Amazing Um so some other things that we are going to look at now are some design patterns or common ways of designing agentic flows to incorporate tools knowledge and access and actions with your AI agents So the first thing we're going to talk about is tool integration in conversational agents So if you are working with an LLM that you're going to
26:30 - 27:00 be using chat completions for for example how do you actually give that agent tools and so you might start from this place here like we can see on the screen So maybe you have a sales agent that you want to build out So you'll start by choosing a model So maybe I want to choose GPT40 to to build out the sales agent as my LLM And I'm going to give that LLM a base prompt So I'm going to say you are an advanced sales
27:00 - 27:30 analysis agent and you're specializing in assisting users with sales inquiries So I'm giving that LLM sort of this base to know this is the agent you are operating as And then after that I can give it access to data So whether that is going to be with Azure AI search as a vector store or files on my machine or files in Azure blob storage for example you can give it access to some data that
27:30 - 28:00 it will be able to use for getting relevant context You can also give it direct access to tools So what do we mean when we talk about tools we mean anything that will allow the LLM to perform an action And that action could be something like calling a file search on Azure AI search or that could be something like running a code interpreter to run some code or it could be searching the internet with Bing search So there these tools are things that we want to be able to give our LLM
28:00 - 28:30 access to And one way to be able to do that so say for example our user sends in a message and asking tell me the total sales by region We know that the LLM has access to maybe we've given the LLM access to some Python code that can access an SQL database What the agent is going to do is it's going to use something called function calling And we have a whole lesson in the beginner AI for beginners course that you can take a
28:30 - 29:00 look at that goes into how function calling works But what basically happens is that the LLM will then read the description of the tools that are available So it'll know that there's a query SQL like database tool and it'll know that the user is asking for some sales data and I can probably get that from this tool and it will select that tool call that tool with a function a function call and then it will return a response to the user with the answer
29:00 - 29:30 once it's done that So what happens then when the user asks a different question So for example the user is saying show a pie chart this information as a pie chart So obviously in this case the LLM can't use the query database function that it's used or that query database tool So in this case it will probably need to call the code interpreter so that it can take that sales information and turn it into a pie chart And so
29:30 - 30:00 again what the agent is going to do it's going to look through the different functions or tools that it has available to it It's going to see which tool is correct corresponding to the user input choose that tool run that tool and then send the message back as a response to the user So this is basically how your LLM is going to act and interact with tools through something called function calling Um so that's the first thing
30:00 - 30:30 when we're talking about tool integration with conversational agents The second thing we also want to talk about is uh tool integration in agentic rag So rag is retrieval augmented generation and this is a get a way of getting an LLM to answer a question about your own data So the LLM will first go ahead and retrieve relevant context from your data and it'll that data will be sent to the LLM so that it
30:30 - 31:00 knows how to answer based off of that data So there are many ways that we can use agentic patterns to improve rag and increase the accuracy of our answers from the LLM One way to do that is what we can see on the screen where a user would input a question and then the LLM would analyze that question and it would do several things So it it might go ahead and decide which tools to call whether that's an API tool or look
31:00 - 31:30 through different documents or a database and just look for information anywhere it can find that And then it's going to collect all of that information into a group of results And then it will pass that information those results either to a model or to an algorithm that will decide whether the quality of those results are good enough and whether it's good enough to send to the user or not And if it's not good enough it's going to repeat the process of analyzing the user's query getting the
31:30 - 32:00 information again and updating the results um if it is good enough then it will generate a response using that information and give it to the user and also finally update the memory of this conference this conversation for future reference So that is pretty much as well how we would use agentic rag Now a final thing I want to talk about in terms of tool use is something called MCP which is quite new is called model context protocol and this is an
32:00 - 32:30 open protocol that standardizes how applications provide context to LLMs So we can think about MCPS at their website actually they give this this illustration and they say you can think about MCP like a USBC port for AI applications So just like USBC provides a standardized way to connect your devices to various accessories or you know tools um MCP provides a standardized way to connect AI models to
32:30 - 33:00 different data sources and tools So Corey gave an example earlier where he was able to use an LLM to communicate with my GitHub repo and the way he was doing that was through the standard protocol called MCP So this is very new So if you want to build an agent on this you're probably going to be experimenting with lots of new things but it's really powerful if you're able to get it right So those are the main things we're going to talk about in terms of tool use Uh so Corey can you
33:00 - 33:30 tell us a little bit more about building trustworthy AI agents i can but I I would say I'm gonna take back my words that maybe hack maybe it's just me but I said that you know hack finding hackathon ideas is hard but I came up with another one from the chat I saw that uh there's a recommendation of making a fox bot Uh Pamela so we're getting a lot of questions around uh can I do this hackathon solo which the answer is yes Uh do I need to use Azure as services or can I use something else
33:30 - 34:00 which also is the answer is yes Uh but if you are using using without you need to be very clear on your read me H so that's just to give some to people in the chat who maybe are not tracking everything but I could totally make a bot for this Uh and no one's yeah no one's gonna stop me because we have a discord server which I am a moderator on So I could definitely at least deploy it there 100% Yeah given some time I mean I'm seeing all the questions This could be good everyone But yeah this is like our third third uh
34:00 - 34:30 agent idea that we already had in this chat Wow it's amazing Agents what a time to be building But don't just um build build trustworthy AI agents And that's what I'm going to talk about now And there's a a few different like uh tools or techniques I would say to really um you know implement what generally this idea of trustworthiness And that's you know one we want to make sure the agent is doing uh what we intended to do or what we designed as agent designers and builders and also making sure uh that we
34:30 - 35:00 can make this in a very repeatable or scalable way because again if we find a good way to make an agent and you making multiple of them you definitely want to just keep using that method So this first slide is kind of messaging this idea of system message framework uh in a system message or instructions or promp system prompts whatever you would like to call it or whatever you're familiar with in that terminology I know there's a few out there We we at Microsoft refer this very clearly a system message is a way that you're kind of giving the meta instructions to an LLM in general Uh so
35:00 - 35:30 if you build any type of genative AI application or LM you probably done this before like a chat completion uh type of interaction But in the case of agents it's even more I say I say it's even more important right because obviously the agents are taking action So if they don't a take that action or take the wrong action it's not as fun Like yeah you know maybe an agent uh doesn't have the right answer in terms of maybe retrieving something like okay if doing the wrong thing as an agent builder is frustrated like oh no I told you the tools right here Why are you not doing
35:30 - 36:00 it right so this idea of system message framework is basically allowing uh you to design one kind of template uh system message uh for generating system messages and in the the course again AI agents for beginners we have the code sample of this so it's I think I'm making it a bit more real to you but basically you could say uh you know here's a system message you're really good at generating AI agents uh system messages and you know they should have these sorts of things uh and you then as a designer can come in and say you know
36:00 - 36:30 you are a GitHub agent and you're just here to recommend uh you know hackathon projects and this if you run it through this sort of framework you actually get a very much more robust and very clear uh prompt that you can give to that GitHub agent rather than me trying to write it by hand and why it's a framework rather than oh yeah you just use lm to generate the prompt is that that you can then now uh scale this over to multiple agents so if you have a good structure and framework uh because a lot of times the goal of most of the agents are the same in terms of like the
36:30 - 37:00 ultimate user goal but the task subtasks are different Um so in the example we have in the in the course AI agents for beginners it's basically uh doing travel and you know each of these agents do something like booking flights or booking hotels something like that and uh you use this framework to define all of those things first and I can just say you're an agent booking flights and I get this back this beautiful uh prompt for the booking flight agent So it's just a way to kind of keep this uh scalable and repeatable which is one of
37:00 - 37:30 the tenants trustworthy but there's a few other things Uh so if we go to the next slide uh another thing that we get a lot of questions on is uh human in the loop Basically this idea of you know maybe I I don't want just the agents I know we kind of define agents as uh semi autonomous right because the agents are still you using in terms of when they're making some of these actions a user is interacting with them They're not just like I'm gonna go ahead and book a flight now because I'm just feeling like Hawaii right they have the user has to like ask about those things right so uh
37:30 - 38:00 there's also this element that then you also want to make sure like let's say you are booking a flight or you're making some sort of action that you want the user involved to you know push the confirm button or you confirm that this is the right details uh before you perform that action It's very important for certain use cases uh maybe not for all of them like you know getting your GitHub repo I don't need you and your express permission to to do that right but this this idea of uh basically making representing a user as a a human
38:00 - 38:30 this is can be done both with uh the Microsoft frameworks out there I know uh autogen has this kind of proc user agent terminology but can also this also can be done and replicated in semantic kernel but essentially like I said when you're doing this managing these multiple conversations between agents uh you can define when that uh conversation terminates or when it continues and you basically are prompting the user acting as an agent essentially in this group chat collaborative chat uh to say oh yeah confirm buy shop whatever uh and
38:30 - 39:00 then the agent then performs action So basically the agent is waiting for a uh response or a prompt and in a very defined way that you design as an AI uh agent designer uh to do that and then it reaches the user goal or it says you know reject I don't want to do that I don't want to go to Hawaii anymore uh never mind or this I don't like the you know window seat I don't know whatever but these are all things that you can kind of think about when you're designing because I know a lot of people when they go and think about their use cases or their hackathons they immediately get into these scenarios of oh well you know yeah the agent probably
39:00 - 39:30 should ask somebody before doing it and this is the shoe in the loop kind of architecture design pattern is where you kind of deal with that Uh and then the other one even most important I think uh you know we're we're doing this for hackathons and hopefully you know I mean a hackathon it would be beautiful that like you you build a hackathon and then it becomes a real product you you apply for Microsoft for startups you get funding and then you you come back to us Marlene and I you find us on LinkedIn or blue sky or whatever you say you know I built this
39:30 - 40:00 beautiful startup just because I attended your oneh hour session on agents 101 maybe that's the dream but maybe you're just going to make a hackathon project just to you know get your feet wet on designing AI agents and that might be it Uh but even if that's the case evaluating your AI agent will make your life complete much better I think in designing these uh AI agents during the hackathon or with outside of the hackathon And really when we talk about evaluations as well is that uh this is uh you you're evaluating every sort of interaction between the user uh
40:00 - 40:30 the agent and basically everything else So uh the user sorry making sure you recognize the intent of the user because again that's when we talk about tool calling especially to pick the right tool you need to know what tool what kind of intent or what goal the user is having so you can kind of evaluate on that the actual calls to the tools themselves So you know again I use the GitHub MCP uh server and then also my Azure AI search index uh and also evaluating the responses of those
40:30 - 41:00 response times uh making sure you actually get those correctly is a super important thing right because uh in cases of when the responses don't come back how does your agent respond right maybe you know again you might lose your your co-host like I just did like how am I going to respond do I just stop doing anything no I'm going to keep going present hunting even though Marlene is gone I don't know where what happened to her Hopefully she's back Right So you want to make sure you when you're evaluating these things the agent uh has
41:00 - 41:30 that built in and you can kind of monitor and observe all those things I could see Marlene's screen So um hopefully she can change it Yeah perfect Uh so my advice on uh building AI agents and this is going to be uh my personal advice again building a lot of agents and also doing quite a few hackathons around the space as well Uh first off it's very fun I know these kids can look a little scary but let rest assured uh they are having fun uh building um AI agents but to be honest and I'll be I
41:30 - 42:00 don't want to be like the Debbie Downer person or ruin the vibe Um but it can actually be really difficult at at times especially when we start adding multiple agents and more complexity around tools So I really like this quote I think I got it from maybe a podcast or something like that But uh it's very interesting because it says if you think we're close to AGI um try building an AI agent and it's just it just shows again because there there'll be moments what you have while you're building these agents and
42:00 - 42:30 you think it's very clear to you you design it but uh you know there's a lot of incre intricacies about designing these instructions making sure these instructions are repeatable and then making sure the agent is very clear that you know it's only going to make this tool call or uh use this database for that So I have a few uh few tips five tips to kind of get you through that Uh all all again based on experience from these these hands are you know vibe coding through a through a agents Uh
42:30 - 43:00 first one is like list of tools and knowledge that you're going to need It's very important Um you know again because you're going to have an idea uh and you might just say okay I I don't know where to find those things or like I have an idea to do like the GitHub thing right obviously GitHub is going to be where I get the repos but what about the the descriptions of the events right okay we have the reactor website So listing these out and kind of architecting which is my another tip is like diagramming the agent flow I know there's actually some tools out there to do that and
43:00 - 43:30 actually generate some code and stuff like that Uh but knowing the flow like okay the GitHub agent gets the repos The repos are then given to the hackathon agent which has this data and it's going to make the creative suggestions and then based on those suggestions it gets to the uh events agent who's going to look at the Azure search database and pull in the relevant things So just architecting that kind of stuff is really important I know a lot of people do that naturally I don't do that but specifically for agents it's very good way good practice um developing each
43:30 - 44:00 agent and tool iteratively iteratively I can't even say it uh largely because I think you know you get this grand idea about oh we're gonna have these three agents do all these things but it's very great uh to just get one agent working on one thing first uh making the call to GitHub Okay great Okay now I'm going to get this hackathon agent to be able to suggest great hackathon projects Okay great I'm going to get this rag plug-in agent that I made uh to get the rag data uh and and then kind of tying those together in instead of uh you know thinking you're going to get all of
44:00 - 44:30 these done immediately in one step Uh because also from a hackathon perspective if let's say you don't finish in the hackathon time you'll still have two working agents right and that could still apply to whatever you're trying to do rather than the third one because you didn't have enough time Um the fourth thing is hard code if needed I know that's kind of crazy to say uh but even in the the the demo and this is secrets we build you can look at the code in the AI agents for beginners Right now it actually hardcodes looking at if you mention anything about hackathons or suggestions it's then going to start this agent workflow I
44:30 - 45:00 should and that's what I'm kind of getting to next is getting a large language model to route that information once that it's in the contents so it's not hardcoded It gives a bit more flare to and more flexibility to the the application itself So that's another thing uh you know especially when you want to test things out and make sure that the flow works at least hard coding is a definitely a good option first and then you can remove that use large language model to be uh more flexible on those sort of things And the last one which I think is the killer for hackathon projects is don't overcommit
45:00 - 45:30 to one tool or pattern uh you we we we talked about a few design patterns now uh but one of the things is uh you know you might get stuck on oh like you know I for this GitHub thing I definitely had to use GitHub but maybe let's say the MCP server wasn't able to give me uh the repos that I wanted or uh double angle to analyze the repos for whatever reason maybe I could use another tool or maybe use directly the API or whatever uh instead of need me kind of being caught up on like I need to show MCP to this
45:30 - 46:00 lovely audience because MC MCP is so great and everybody needs to know about it Uh I could have just said okay you know we're not going to use MCP because it wasn't sufficient here And I think that happens a lot of times when you're hacking uh is that you get caught up You want to use this one tool because you you see all the options it has but maybe just not necessarily getting to that one piece of data or response or things like that So don't overcommit to it Um look look around Don't uh you know fall into the quicksand again of kind of like falling into looking at that one tool So
46:00 - 46:30 that's my uh five tips Then the one big tip if this is all completely new to you and you want to get some applied knowledge this AI agents for beginners again we have 10 uh lessons and an 11th lesson coming soon or actually 12 lessons but the MCP sample the code sample I showed you is actually going to be in that is actually already in the repo under the code samples for MCP So do check that out and this is a great way to get kind of looking at the code samples playing around it breaking it and you'll get your hands on uh knowledge of building these AI agents at
46:30 - 47:00 least in Python Amazing Yeah just a reminder to check out the course and I think that's all we have for now Uh I thank you Corey Thanks everyone for joining us I believe Pamela is gonna join us Um yeah Pam's going to join us and talk I think about some AI agents with Microsoft services specifically Exactly All right Thank you so much
47:00 - 47:30 Corey and Marlene That was awesome As a reminder definitely dig into that AI agents for beginner curriculum if you want to get more into everything that they were talking about And in that in that repo there's also all these code examples too Uh we're not showing a lot of code We're not showing any code today actually because you know we've got Python developers and JavaScript and Java and C# like all you know all of you coming in with different languages and you know we want to be inclusive So we're going to be talking today more
47:30 - 48:00 about what can be done and not how to do it Uh you'll learn how to do that in the streams that come up for your favorite language Uh so now I'm going to talk about more into how we can build an agent using Microsoft technologies and also talk about all the upcoming streams that we have that will teach you how to do this So when we're building an agent what do we need at the minimum we need an LLM right because we
48:00 - 48:30 said that an agent is using an LLM in order to improve our workflows right so we need the LM The LM is going to make decisions and generate responses Now our options for Microsoft is Azure OpenAI and that's what a lot of our samples use That's where we get really powerful models There's also the Azure AI model catalog which has models besides OpenAI models So it's got Llama Mistro Coher Jamba Deepseek It's got a lot of really
48:30 - 49:00 cool models There's GitHub models and GitHub models has basically the most popular ones from that model catalog So once again we've got Mistro Coher Llama and the really cool thing about GitHub models is that you can start using them for free All you need is a GitHub account Let me actually show GitHub models because I really want to plug it because this is the way that you can start developing for free right so you can use any of these models here uh
49:00 - 49:30 including OpenAI opening eye fi coher deep sea glama mistral etc Uh you can you know get started with these models You probably want to start with the you know GBD40 or GBD40 mini And and then you can start uh you can start using them And one of the things I shared in the chat is that uh I have a a repo that um is specifically shows you how to use GitHub models with popular AI agent frameworks right uh so we've got
49:30 - 50:00 Autogen Lingraph Llama Index the OpenAI agents SDK the OpenAI normal SDK Pantic AI Semantic Kernel SM agents So a ton of different AI frameworks They can all use GitHub models And GitHub models are free as long as you have a GitHub account They are rate limited so you may run out of quota on some days Uh I still haven't actually run through my quota which is great Um but uh but you know they're free they're rate limited and it's a great way of getting started and uh and
50:00 - 50:30 you know uh building your agent and so you could build your entire agent using GitHub models for free Um and that way you don't have to uh use any uh as your credits and you don't have to uh you know sign up for anything So that is definitely an option that I want everyone to know about Uh you could also use models locally So for example you can run Lama I've got Olama running There's my little llama up there in the corner and there I've got like the fi model downloaded right so you
50:30 - 51:00 could be um you know running with local models too Uh the important thing about the LLM is that we generally need it to support function calling Uh that's a specific capability of an LLM Not all of the LLMs support it Uh the GPT 40 40 Mini those definitely support it Llama 3.1 3.2 those support it Uh but not all of them support it So generally what you're looking for is a powerful LLM that supports function calling And if you have that then awesome you can build
51:00 - 51:30 an agent The next thing that you often want is an orchestrator that's going to manage calls to the LLMs right and you you know you want like one that has this kind of agenoriented orchestrator So it knows how to call tools uh usually it has some way of storing memory some way of showing you steps along the way some way of doing planning in order to plan you know which agents it's going to call which tools it's going to call Uh so for Microsoft we have semantic kernel which is available in Python and C and then we
51:30 - 52:00 have autogen which is available in Python Uh and then we also have various Azure AI uh SDKs Uh so those are offerings for Microsoft which you could use but you're also welcome to use open source packages uh because they're pretty much all compatible with Azure and compatible with GitHub models Uh you just have to point them you know point them at at your at the right model right so you might use link chain or llama index or crew AI or you know any of the
52:00 - 52:30 ones I just showed in that repo All right Now you but you don't have to use the orchestrator Some people actually just go really low level and you know create an AI agent from scratch But I do think it really helps to start with an orchestrator um to get a lot of the functionality for free Uh my personal favorite is Autogen So far I found that the most intuitive Um but everybody has their own their own framework I should actually change this to say lang graph instead of ling chain If you are specifically building agents most people actually do it with lang
52:30 - 53:00 graph uh but you can also use link chain So they've got two offerings there Okay So now your orchestrating calls LMS and you know usually your LLMs are either going to be retrieving knowledge from data stores or putting knowledge back into data stores or you know both right so where are you going to get that knowledge from where are you going to store it um so if you're doing you know if you're doing search and you need to like search a bunch of documents then you probably want to use Azure AI search
53:00 - 53:30 Uh or maybe you're setting up a SQL database we've got Azure SQL Cosmos DB Postgress right uh if you're just storing files maybe you'd use Azure blob storage right so we've lots of lots of different options for data stores Uh if you are doing everything locally then you know there's ways of doing all this locally too and doing it in memory Uh so you've got options there too There's also emulators for everything right you can run Postgress locally You can use run a Cosmos emulator locally Uh so you've got local options too because once again local meaning low cost right
53:30 - 54:00 and we we want to give you all these lowcost options because when you're learning things you don't want to spend a lot of money You ideally don't want to spend any money Uh so we want to make sure that you know about all the lowcost options available Finally tools One of the main things that makes agents powerful is that we empower them with tools We're like "Oh hey agent you need to do math Here's a calculator You need to browse the web Here's a web browser Right uh you need to extract information from a PDF Here is a tool that can do that
54:00 - 54:30 Right so we do have a bunch of services for that Uh there's Azure document intelligence That one's really good for extracting information from documents We've got the Azure speech APIs really good input output really fluid voices Uh we've got Bing search grounding We've got communication services if you want to do like a WhatsApp integration Uh so lots of different tools there And a lot of these do have a free tier So if you want to experiment with these for free uh just look to see if they have a free tier and see what the limitations of the
54:30 - 55:00 free tier are Uh because once again we want you to be able to learn without having to spend money So for all of these options we are going to have streams that show you how to use them from experts uh within Microsoft and our MVP community Uh so with the LLM options lots and lots of the streams will be talking about Azure OpenAI Uh I've highlighted a few of them that are specifically talking about the the voice model the real-time preview
55:00 - 55:30 model right so if you're interested specifically in the voice model we've got sessions but really so many of the sessions will be using Azure OpenAI because they're very powerful models and it's what we tend to use by default Um so you're going to learn about Azure OpenAI in lots of sessions Now for the Azure AI model catalog we have a talk from uh Fakundo on that team and this it'll be in both English and Spanish about building an agent with an army of models And there he's really going to talk about like how do you decide which model to pick for what right so do you use FI for this step and then llama for
55:30 - 56:00 this step and you know once again all these models are also available on GitHub models So the things he talks about there we can use that with GitHub models And then for GitHub models specifically I am going to do a talk about that on April 15th where I'm going to go through all the demos in that repo and uh you know just just talk through them Uh so do come to that talk on April 15th if you're really interested in GitHub models as your way of getting
56:00 - 56:30 started Now at Azure we do have the Azure AI agent service Now what this basically is it's like a managed agent micros service that has built-in tools and built-in memory Uh and that's like the reason for why you might want to use it is if you want those built-in tools and that built-in memory those would be the reasons to use it And many many people do want their agents to have access to to certain tools and and especially to have easy ability to remember threads
56:30 - 57:00 and and store memory across sessions So we're going to have multiple sessions about that Um starting with one from Marlene who you just met but we'll have ones about using the AI agent service with Python withnet with JavaScript and also with Azure functions Uh and we do have a specific prize for the Azure AI agent service to motivate you to try it out Once again th this one does require you to have an Azure account and to sign
57:00 - 57:30 up and to use some some credits Um but uh we do encourage you to check it out at least go to the sessions learn what it's about learn when you might want to use Azure AI agent service versus uh just an LLM with a framework right that's kind of a decision that you need to make Uh and you can also use in combination So you can actually use like semantic kernel with the AI agent service uh if you if you want to combine them that way Uh orchestrators we do have multiple
57:30 - 58:00 orchestrators right so we have semantic kernel We're going to have talks about how to use semantic kernel in Python how to use it in Java how to combine it with different uh you know different tools like Azure Cosmos DB We also have a talk from the autogen team about how to build apps with autogen in Python So that should be really cool We also have talked about open source orchestrators So we have Lori coming from Llama index Lori uh works for Llama index and is their head of Devril So Lori he'll be telling us how to do use
58:00 - 58:30 llama index uh in Typescript in order to augment your rag And we also have multiple talks that are going to be using lang chain and lang graph Uh so you're definitely encouraged to use open source orchestrators if you know if that you like the way that they um go about things right all these orchestrators have slightly different ways of doing things So I think it's really helpful to see multiple options and see what really works for you and say like yes I like that one I like the way it's doing like that works for my brain and I'm going to
58:30 - 59:00 use that one For data stories we're going to have talks from the Cosmos DB team We're going to have multiple talks from the AI search team about how to do aentic rag and how to do voice rag Uh so that should be really cool for those of you who are interested in rag from the chat We also have talks from our SQL team So we have Davidid from Azure SQL and he always does great talks and we've got two about Postgress one in Python one in C All right Now when you're making your agent you have to decide where it's you
59:00 - 59:30 know what kind of agent is it is it going to be an agent that actually lives inside another tool right so you can actually make agents to extend the Microsoft 365 co-pilot That's a very popular use of agents is to uh make those tools more powerful or you could be doing standalone agents that you're you know just using for your own purposes that aren't necessarily living inside They're not embedded inside an existing co-pilot right so that's two different ways of thinking about where you know where your agent live what is its environment Uh so for co-pilot we're
59:30 - 60:00 going to have a bunch of talks about building for co-pilot if that's what you're interested in And there is a prize specifically for that So if you're into co-pilot go for it and go for that prize And then we also have talked specifically about different Azure clouds Azure functions is a really good fit for agents Really really also really good for MCP We just came out with this Azure functions MCP uh repo Actually a bunch of repos in each language So if you're interested in MCP an authenticated MCP server you should check out Azure Functions And we have a
60:00 - 60:30 few talked about Azure container apps which are really good if you're doing code interpreters Okay now over to Josh for next steps Hi everyone Hope you enjoyed today's session If you aren't already registered for the AI agents hackathon yet uh you can register now at the aka.ms agents hack Um Danny do you mind moving to the next slide oh that's me
60:30 - 61:00 I'm Pamela Um yeah so there's the link that I just uh was just uh listing Um if you have any questions along the way I know there's been a ton of discussion in our uh comments uh just now and we weren't able to answer everything Um frankly we need a we need a fox spot right um you can go to our GitHub forum um linked aka.msentac uh resources and you can go there and take a look at our resources Um you can
61:00 - 61:30 ask questions there and there's also a ton of other things that you can see there Um there's also the aka.ms/agentsacisord um which allows which brings you to our Discord office hours and you can take a look at that if you want you're wanting to ask some of our speakers um questions live Um I also encourage you all to attend more of our live expert sessions and you can find those at aka.msagentshackstreams Um there's a ton of different tracks that you can look into whether you're a Python developer
61:30 - 62:00 Java developer you want to get into some of those copilot agents that Pamelo was kind of just talking about embedded agents Um you can kind of you can go and take a look there and explore and see what you want to learn more about And finally make sure that you submit your project once it's ready uh before April 30th 11:59 p.m PST to win prizes Thanks for joining Bye everyone