Transforming Business Processes with Multi-Agent AI using Semantic Kernel

Summary

The Microsoft Reactor webinar highlights the power of multi-agent AI through Semantic Kernel. It delves into how Semantic Kernel can help in transforming business processes by enabling light-weight, open-source, production-ready orchestration middleware. This framework connects AI services and models, helping build agentic applications across Python, .NET, and Java. The webinar also covers integrating various AI models and deploying them on different platforms. The session emphasizes developing multi-agent systems, shedding light on complex business problem-solving and optimizing resources through collaboration.

Highlights

• Discover the transformation of business processes with Semantic Kernel. 🚀
• Learn about the orchestration middleware for building agentic applications. 🌟
• Semantic Kernel connects AI services with ease for robust productivity. 🔧
• Experience the flexibility of Semantic Kernel across different languages. 🌍
• Understand the importance of having a stable platform for AI applications. 📉
• Dive into multi-agent systems for better resource optimization. 🎯

Key Takeaways

• Embrace Semantic Kernel for building robust, production-ready AI applications. 📈
• Multi-agent systems enhance task-solving capabilities and performance. 🤖
• Connect AI services effortlessly to build versatile agentic applications. 🔗
• Leverage the open-source nature of Semantic Kernel for innovative developments. 🌐
• Integrate various models and platforms effortlessly with Semantic Kernel. 🎛️
• Semantic Kernel supports multi-language compatibility—Python, .NET, and Java. 💻

Overview

The Microsoft Reactor webinar presented an engaging overview of integrating multi-agent AI with Semantic Kernel, focusing on transforming business processes through innovative application building. Sean Henry, showcasing his expertise, introduced Semantic Kernel as a lightweight, open-source middleware that is production-ready, helping developers introduce robust AI applications into business environments.

Throughout the session, significant emphasis was placed on the utility of multi-agent systems in solving complex problems. By leveraging these systems, businesses can optimize their AI capabilities, allowing collaboration across agents to deliver solutions efficiently. The Semantic Kernel framework supports this approach, connecting various AI models and enabling seamless application development.

Integration is a core theme within the session—Semantic Kernel allows straightforward connections to multiple AI services, ensuring that developers can focus on innovating rather than being bogged down by integration challenges. With compatibility spanning Python, .NET, and Java, Semantic Kernel proves its versatility and capacity to transform how businesses approach AI-driven development.

Chapters

• 00:00 - 01:30: Introduction and Housekeeping The chapter 'Introduction and Housekeeping' sets the stage for the AI Agents Hackathon, a free virtual event running from April 8th to April 30th. Over 20 live streams are planned, focusing on building agents with Microsoft technologies. Participants have the opportunity to submit their projects at the end of three weeks for a chance to win great prizes.
• 01:30 - 09:30: Overview of Semantic Kernel and Its Features The chapter introduces the topic of AI agents with a focus on a hackathon event. The presenters begin by acknowledging their association with Reactor and emphasize the importance of adhering to a code of conduct. They encourage participation and ensure a respectful environment for both the audience and presenters. However, details specifically surrounding the Semantic Kernel and its features are not addressed in the provided transcript.
• 09:30 - 16:00: Demo: Building a Basic Agent with Semantic Kernel This chapter serves as an introductory session where participants are instructed to maintain professionalism and stay on topic during chat engagements. The hosts indicate they will be providing useful links and checking the chat for questions that presenters can address live. It is noted the session is recorded and will be available for on-demand viewing within 24 to 48 hours after the live event. The opening remarks are concluded by handing over the session to the speakers.
• 16:00 - 23:00: Demo: Building a Plugin for Semantic Kernel The chapter introduces the topic of building a plugin for Semantic Kernel. The speaker, Sean Henry, who leads the Semantic Kernel team at Microsoft, intends to discuss the use of semantic kernel, particularly focusing on agents and how to build them. The speaker will also cover what semantic kernel is and its applications in building agents. Opportunities for audience interaction via questions are indicated.
• 23:00 - 25:00: Demo: Using UI with Semantic Kernel In this chapter, the speaker opens the floor to questions and mentions that another person, Anna, may also interject with questions during the presentation. The main topic discussed is the 'semantic kernel,' which many participants are already familiar with. The semantic kernel is described as a lightweight, open-source, production-ready tool.
• 25:00 - 30:00: Understanding Multi-Agent Systems This chapter discusses orchestration middleware offered by Microsoft for building agent-based and AI applications across various programming languages, including Python, .NET, and Java. It emphasizes the focus on Python during the discussed session but assures that full support and sample codes are available for .NET and Java as well.
• 30:00 - 38:00: Demo: Creating Multi-Agent Systems with Semantic Kernel The chapter discusses the capabilities of Semantic Kernel, a lightweight, open-source orchestration framework designed for production enterprise developers. It emphasizes its robustness and suitability for building AI applications.
• 38:00 - 45:00: Advanced Multi-Agent Systems Concepts This chapter delves into advanced concepts related to multi-agent systems, particularly focusing on stability in software development. It highlights the development cycle, emphasizing the importance of maintaining stable APIs once they achieve general availability. The discussion revolves around how their agent framework, which has recently become generally available, supports this stability, ensuring that APIs remain consistent across updates.
• 45:00 - 56:00: Demo: Audio-Based User Proxy Agents The chapter discusses the development of audio-based user proxy agents, emphasizing the extensive period of six to seven months spent on iterations and previews before reaching a stable version, dubbed version 10. This version is highlighted as a stable platform aimed at providing a consistent API for developers building agent applications. The chapter also underscores the importance of supportability from Microsoft, ensuring users have continued technical support and resources.
• 56:00 - 59:00: Autogen and Magentic One In this chapter, the focus is on three main components involved in the development and support of semantic kernel applications. First, there is a strong emphasis on community engagement through platforms like GitHub and Discord, where the team actively participates in answering questions and providing support. Secondly, the importance of security is highlighted, with Microsoft full-time employees dedicated to verifying codes, ensuring a rigorous software development life cycle. This approach aims to maintain high security standards within Microsoft.
• 59:00 - 62:00: Building Complex Business Processes with Multi-Agent Systems The chapter discusses the integration of secure processes within multi-agent systems, emphasizing adherence to Microsoft's secure foundation initiatives. It highlights the importance of having all necessary features for building AI applications, focusing on an orchestration layer provided by the semantic kernel. This orchestration layer serves to connect various AI services and models necessary for developing both AI and agent applications. This foundational connection to AI services and large language models is central to constructing sophisticated business processes.
• 62:00 - 64:00: Conclusion and Additional Resources The conclusion chapter discusses the flexibility of the semantic kernel to connect to a range of AI models. It highlights the ability to connect not only to models hosted on Azure and the foundry catalog but also to those available through OpenAI and Hugging Face. Furthermore, there is support for Nvidia NIMS, which optimizes models for Nvidia cards both in the cloud and locally. There is also compatibility with DeepSeek, Google Gemini, and AWS Bedrock, demonstrating the extensive connectivity options available.
• 64:00 - 64:30: Q&A and Closing Remarks The chapter discusses the versatility and connectivity provided by semantic kernel, allowing applications to interface seamlessly with different models and AI services. It highlights the ability to maintain code consistency while choosing various models and capabilities. Additionally, the chapter mentions the integration with local models through adapters such as Lama, LM Studio, and Onyx. The session seems to end with Q&A and closing remarks, wrapping up the discussion on maintaining seamless application development across diverse AI platforms.

Transforming Business Processes with Multi-Agent AI using Semantic Kernel Transcription

• 00:00 - 00:30 hello everyone welcome to the AI Agents Hackathon live streams the AI Agents Hackathon is a free virtual hackathon running from April 8th to April 30th we're hosting over 20 live streams where you'll learn how to build agents using various Microsoft technologies at the end of the three weeks submit your project for a chance to win fantastic
• 00:30 - 01:00 prizes and now over to our friends at Reactor hi everyone thanks for joining us for the AI agent hack before we start I do have some quick housekeeping please take a moment to review our code of conduct we seek to provide a respectful environment for both our audience and presenters while we absolutely encourage
• 01:00 - 01:30 engagement in the chat we ask that you please be mindful of your commentary remain professional and on topic keep an eye on that chat we'll be dropping helpful links and checking for questions for our presenters to answer live our session is recorded it will be available to view on demand within 24 to 48 hours with that I'd love to turn it over to our speakers for today thanks for joining
• 01:30 - 02:00 hi everyone hope you're having a good day wherever you are um my name is Sean Henry and we're going to talk about agents and hacking and semantic kernel today um so first of all I already introduced myself uh my name is Sean Henry i run the semantic kernel team here at Microsoft um and we'll talk a lot about using semantic kernel today what it is how you can use it to build agents i'll stop at a couple spots for questions so if you have any questions through throughout this please uh you
• 02:00 - 02:30 know put them in the chat and also leave some time at the end to um you know take questions take take questions as well and and Anna may interrupt me as well and if there's some questions that um that folks would like to um like talk about while I'm going through stuff um I'll start off with a question i saw a lot of people in the chat already are using semantic kernel and and are familiar with it um but for those of you that aren't um I'll introduce it as our lightweight open-source productionready
• 02:30 - 03:00 orchestration middleware for Microsoft so that's a bit of a mouthful but what it really is is is we're a library to help you build agentic and AI applications uh on top of you know your favorite languages either Python ornet or Java i'm going to be focusing in this session mainly on Python um but uh for those of you that that use .NET and Java um we do have full support for those languages and all the samples that I'm showing today on Python are also available in our .NET framework agent framework as
• 03:00 - 03:30 well and you can find those on our GitHub repo and I'll have lots of links to those as well so what does this mean to be a lightweight open source productionready orchestration framework well really uh the thing we wanted to focus on with semantic kernel is we wanted to be ready for production enterprise uh developers there's a lot of orchestration frameworks and libraries that you can use out there to build your AI applications and with semantic kernel we really wanted to focus on that robustness and that need for taking your applications to
• 03:30 - 04:00 production and what does that mean well it means number one we focus on being stable um you know we go through that typical um software development process of of having previews and and and release candidates and and general availability and at that point we declare our our APIs for existent for example version 10 and we we don't break our APIs at that point and in fact many of the APIs I'm talking about today in this um in this session are built on our agent framework which just recently last week went to uh general availability
• 04:00 - 04:30 we've been working on it for about six or seven months um and iterating on it and we we went through previews and release candidates and we finally kind of locked it down got to a place where we liked it um and now we've declared that version 10 and so that's going to be a nice stable platform for folks to build their their agent applications on and not have to worry about the API changing every week as you uh update your libraries and the other one of course is important is supportability um that's something we focus on as well um you know not just through being supported through you know Microsoft
• 04:30 - 05:00 support but also through um GitHub and uh and and and uh the communities on Discord where our team my team is there every day answering questions about semantic kernel and helping you build your semantic kernel applications and the third one that we focused on is security we wanted to have a library that um uh there was all that a Microsoft full-time employee was verifying all the code that went in so we could do our software development life cycle internal internal to Microsoft and verify that um you know
• 05:00 - 05:30 everything is done in a secure way and meets uh Microsoft secure foundation initiatives those are kind of the things that we really wanted to focus on in addition to of course all the features that you need to build your AI applications and so what what are some of those features that you might need so with semantic kernel we're an orchestration layer um so we're about connecting all those AI services that you need in order to build your applications your AI applications and your agent applications and this starts with kind of the core piece which is of course being able to connect the AI services and models your large language
• 05:30 - 06:00 models so semantic kernel we allow you to connect not just to uh the Azure and the foundry models through Azure OpenAI and through the F foundry catalog um but also through models that are not hosted in Azure like OpenAI directly all the models available on Hugging Face we recently announced support for Nvidia NIMS uh which is uh which allows you to use uh models optimized for Nvidia cards both in the cloud and locally um you can also support things like DeepSeek and even Google Gemini and AWS Bedrock
• 06:00 - 06:30 models and anthropic through AWS through connectors that we have so you connect your application to those models and then all your code stays the same um as you as you use it through semantic kernel so you can kind of build your application once and you can kind of pick your model and and and your capabilities that you're using with it in the same way that we connect to all sorts of different AI services we also connect to local models so we have adapters on top of Lama um we have adapters on top of LM Studio we have adapters on top of uh Onyx which is our
• 06:30 - 07:00 our our native runtime for Microsoft for running these models this allows you to run local models like the llama classic models but an interesting one that's that's recently been released is the new uh 54 uh multimodal and mini models uh which are are great for building agents because they now support function calling so you can now have full agent experiences that are run 100% locally are run free um you can develop them locally on your on your machine then you can deploy into the cloud or you can continue to run them
• 07:00 - 07:30 locally so those are kind of the AI services the LLM models we also support connectivity to memory services so if you're doing things like retrieval augmented generation if you're doing search you want to connect your agent to a vector store um again we support not just Azure AI search but also connectors to all the popular thirdparty vector stores like Elastic Search MongoDB Quadrant and others uh in the same way that we've been connecting to you know large language models in the cloud and also to
• 07:30 - 08:00 memory services in the cloud we're also now starting in preview with Samantha kernel to connect to other agent hosting services so first and foremost is is Microsoft's Azure AI agent service as part of Foundry and I think you'll be hearing a lot about that um today i think there's a bunch of sessions on that and that is a great way to build and deploy your agents quickly and easily into the cloud and you can build them through um through uh the the Foundry AI studio you can build your build your agents there and then you connect to them through semantic kernel
• 08:00 - 08:30 you can orchestrate multi- aent systems together through through semantic kernel there in the same way that we do that with um Foundry agents and the Azure AI agent service you can also connect Bedrock agents you connect to Crew AI and you connect to Autogen which is Microsoft's research framework for building multi- aent systems so you can build you can take your agents that you may have built already in something like Autogen and you connect them in through semantic kernel so those are the services that we connect to from third parties you also want to be able to connect it to your own services and your own capabilities
• 08:30 - 09:00 so we support that through a plug plug-in model in semantic kernel this gives you connectivity to APIs and services you may have built natively so I'll show you how you how you do that in code you can just annotate your code and you can have your LLM then call directly into it you can have endpoints that are open API endpoints uh so other services you may have you may have uh enabled and created an open API spec for mcp is very popular now this is model context protocol um this is a format for bringing in tools the LMS can use is
• 09:00 - 09:30 very popular if you're using things like um like Visual Studio Code or Cursor or other um idees that support local code execution and then also all the Microsoft capabilities through things like logic apps um you can also declarative define your agents and prompts and semantic kernel we have our own format um our own format for prompts through through YAML and our semantic kernel format we've now recently merged that if you've used Prompty before which is a Microsoft format for defining prompts we've now merged that into our agent
• 09:30 - 10:00 framework so that's something you can use as well and then finally as you deploy your application one of the most important things is being able to observe your agents so semantic kernel um emits open telemetry uh throughout the whole library so you can connect that to things like Azure Monitor or Aspire especially folks in the .NET space Aspire is a particularly good choice um as well as things like Azure AI content safety through prompt shield so this allows you this the whole package allowing you to build your productionready AI applications it's a library that helps you build that more quickly stable platform that you can
• 10:00 - 10:30 build on top of and then uh uh build your applications and then deploy them anywhere you can deploy them to Azure you can deploy them to AWS you can deploy them locally whatever you like so we're an open source framework and really our goal is to connect connect all these services sean we do have a question in the chat um they're wondering "Does Semantic Kernel have anything in their pipeline development pipeline to support MCP?" Oh spoiler alert yeah we'll get to that we'll get to that i got I got a good demo for them
• 10:30 - 11:00 in a second perfect mkg yeah yeah we'll get to that i'm just scanning through some of the other questions yeah looks like looks like this is great great crowd um I will keep going here so let's talk about agents this is what you're here to build is agents you probably all seen this slide i bet you everyone from Microsoft is showing a slide similar to this what is an agent um so when we think of an agent it's you know something that can reason over a provided business model can reason over some task can get context to complete
• 11:00 - 11:30 that task complete the process and you can perform an action for the end user um so I like to think of concretely like what does this mean what does this actually mean when you're building your application when you're coding your application so in semantic kernel you know we think of these as models your reasoning over a business process that's done by a large language model that um that you use you know you use GPT40 or you use any of the models available available to reason over some particular process that's the brain of your agent
• 11:30 - 12:00 you retrieve context for that process that's a memory that's the memory of uh of your agent that might be either something you just you stuff in the prompt if it's just a small amount of of context or you might have a knowledge base that the agent accesses through something like a vector store it does a semantic search or any other kind of search to retrieve that information might be a regular database search and then brings that information into the context so that it can reason over it and lastly which I think is the most important thing which really differentiates
• 12:00 - 12:30 um you know agents from assistants and kind of traditional large language model usage is that the agent can perform that action and it does that through something called tools and we call them plugins and semantic kernel some people uh some some models call them skills um some models call them tools um all the same thing and what this is is is the ability for the large language model to make to to be given a set of tools it can use to perform an action and then choose which of those tools it wants to use um and this is kind of really the
• 12:30 - 13:00 innovation that unlocks the ability to build to build agents this is what we have extensive support for in uh in semantic kernel okay so semantic kernel has been around for a couple years and we've had this concept of models and context and memory and plugins i showed that on the last slide all the things we're doing with those and when you combine all those together that's where you have an agent so we've been doing agents in semantic kernel for you know since the beginning since semantic kernel was originally created um since before
• 13:00 - 13:30 agents were cool I guess you could say um but we now have an agent framework that builds on top of these that takes these concepts and and ties them together in a single API that you can use in order to build these agents that have models connected to them have context and memory and plugins connected to them then you can interact with them like agents and use these to build your applications whether line of business applications or bisops applications or building new APIs or new agents to unlock unlock new value for
• 13:30 - 14:00 uh you know for for your for your task that you want um that you have at hand i'll show you a quick demo uh by the way all these demos are up on GitHub in ak.mskhacks um you can take a look at all of those there um but I will switch right over and show you building a basic agent with semantic kernel um again uh all of these uh demos are in Python that I'm going to go through we're going to focus on Python here but they're also all available um in C.NET if you go to
• 14:00 - 14:30 our GitHub repo you can find them there as well um so I'll just quickly quickly go through some of the prerequisites and needs for building uh an application with semantic kernel and Python um obviously having Python installed for local development all these samples will use Azure OpenAI you can use regular OpenAI you can use a different model and I'll show you how to do that and then of course you have to install the SDK um within Python very easy to install the SDK um I'm running everything here in a code space which means I don't really worry about virtual environments or
• 14:30 - 15:00 anything um but to install it you can just use pip install one package to install semantic kernel um there's a few different ways you can set your keys you need to go get your keys from the OpenAI service um I like to do those with a M file here's an example of one of those files here where I've set a few of the keys um these aren't real keys don't try them i'm not going to show you my real keys um but you'll need all these keys from uh you know either from Azure from uh from the AI Foundry Studio here um or
• 15:00 - 15:30 if you're using OpenAI you can get them from OpenAI directly that's what that looks like um and then all the samples on the repo you can run directly in Python but I'm going to walk through one of the first ones here so this is this is all it takes 17 lines of code write an agent with semantic kernel um so we'll look at the first relevant part here i'm creating my agent in this case I'm creating a chat completion agent i'll talk about some of the different types of agents there are um later but chat completion is the most common one this is the one where you
• 15:30 - 16:00 connect your own model your own service your own tools um and the most simple of an agent connects to a service in this case I'm connecting to the Azure chat completion uh service i could give it you can see here I could give it my API keys directly but semantic kernel will pull those from yourv file if you have that set up um so here I'm not actually passing any parameters we'll pull that locally from from thev file i give it a name and I give it some instructions here are just very basic instructions um and then to actually call your agent you take your agent and you uh you get a
• 16:00 - 16:30 response in this case you ask for a response to the message in this case I'm asking it to write a haiku about semantic kernel um so we'll see that run here when it comes up so you can see down here that it has given us a response code weaves meanings thread semantic kernel ignites AI knows the way it's very inspiring um so that's the most very basic symbol of agent you may be saying well that's that's no different than what I can do when I talk
• 16:30 - 17:00 to a large language model directly um and in some ways it is um so the next thing we want to do is we want to add a plugin to our agent this is the tools I was talking about how do you have an agent actually um do something useful so in this sample here exactly the same as the sample we showed before um except a couple things that we've added um number one is down here on this plug-in line we've uh we've given it a plugin called menu plugin and I'll show you that in a second and then
• 17:00 - 17:30 we've given it some arguments in this case these arguments are telling it to give our response format in a menu item format so let's look at what those things are so first of all the menu plugin is a set of uh functions we've defined that um in this case an example simulate getting getting information about a uh a menu at a restaurant so we have two functions in this case one that gets the list of specials uh and in this case it returns a string you would probably have it you know in reality talk to your database here at this point
• 17:30 - 18:00 or be able to pull that data in um so it's returning the the soup special the salad special and the drink special and then another function for getting the price of a particular item and then you can see here we've added some annotations to this um again for the C# folks we use we use the annotations from C but very similar um very similar mechanism to what we do in Python so we've given it we have this kernel function annotation we say it has a description this this function provides
• 18:00 - 18:30 a list of specials from the menu and it returns the specials from the menu so we say we we give it the data we that about what the function does and what it returns um and for get item price we say this provides the price of a requested menu item given the name of the menu item it returns the price of the menu item so what's going to happen when we call the LLM is it's going to take all this metadata it's going to bundle it up and it's going to give it to the large language model and the large language model is going to decide which tools
• 18:30 - 19:00 it's going to use and then instead of returning a textbased answer we're going to ask it to return a menu item based answer which is a structure that has a price and a name so instead of saying you know the salad special is Cobb salad and it cost $9.99 it's going to return uh it's going to return an object that has a string for the name and and the float so I'm going to put a breakpoint here and we'll actually see see exactly how this works so if I um I'll run it here and so what's going to happen again uh you know exactly the same as what we
• 19:00 - 19:30 had before in terms of building the agent giving it a name giving it some instructions but we've now added these two extra pieces to it so when this runs what's going to happen is it's going to call into our breakpoint you see what happened is behind the scenes when we called this get response for this message semantic colonel took this message and it added the instructions for the agency that what we gave to the large language model basically in text is we said you're a helpful
• 19:30 - 20:00 assistant you have the following tools to answer any of your to answer any questions and we say you have a tool called get specials that provides a list of specials for the menu and uh returns specials from the menu you have another tool called get item price which provides the price of a requested menu item and given a menu and takes the parameters of the name of the menu item and returns the price of the menu item and always return your answers in the form of a of a menu item with the format
• 20:00 - 20:30 price and name so we give it all that we take that we autogenerate that and put that into the prompt that we give the LLM and then we say user message what is the price of the soup special so that's all the text that that the LLM gets and then it looks at that and says oh okay to get the price of the soup special I need to call into uh these functions so in this case it's calling in so I guess it already called the special one it didn't have a break point in the right spot um but it is now getting the uh
• 20:30 - 21:00 menu item it's now getting the menu item for the We should be able to see here the uh the menu item and then it's returning $9.99 for the price and wait here we go actually that was because I had this break point wrong we'll put this back to here there we go that was wrong we haven't called it yet so we're going to call it here and now it's going up to the LLM and now it's asking for the price of the menu
• 21:00 - 21:30 item and it's returning that the soup special is $9.99 and we can ask it a compound thing so we can ask it maybe two things it needs to you know call into both these functions we could say what is the soup special what is the soup special and its price so in this case the LM is going to
• 21:30 - 22:00 need to make two calls it's going to need to make one call to get the super special and then one call to get its price and so we'll actually do both of these in one turn of the call to the large language model so we see this run okay now it's all set up and we're going to ask for the response what is a super special and its price and so now we'll see it call both of these functions so now it's asking for the the specials from the menu and we're going to return what the three specials are that was back of the
• 22:00 - 22:30 LLM and then it calls back and it says what is the price of the clam chowder so it took the information from the first call and then gave it to the second call to ask for ask for the price and then now it's going to return down here that the price uh is $9.99 and it's the name is clam chowder so that's now in this format so I showed two things that are very powerful kind of the two most powerful concepts in semantic kernel are being able to annotate your existing code and have it be used by an LLM and then to return
• 22:30 - 23:00 structured data in the format that you can now give back to your existing code so semantic kernel is really about connecting your AI code to your existing uh existing code for your application okay so all that was done in the terminal what if we want to see this in UI so I just want to quickly show you guys a nice easy way to build a UI with semantic kernel um I really like using a tool called Chainlit um Chainlit provides integration with semantic kernel if you go to their website and
• 23:00 - 23:30 you look up integrations with semantic kernel and chain lit you can see how to use this um it's very easy uh we've built integration semantic kernel and integration to chain lit to make it uh to make it work really well um you can just copy paste this code and uh it will get you started building building an application with um chain lit and I'll show you what that looks like here um I'm just going to run this application here um I'm passing this W parameter which means it'll automatically reload it's not actually necessarily important
• 23:30 - 24:00 in this case um but what this will give you is um a nice UI for actually uh chatting to your agents um and so here you can see I've created this this this UI with chain lit um and then I can ask it so this agent here is a weather agent so it's got a function we talked about functions where it gets the weather um and so we can ask it uh it only does some cities so we can ask it what is the weather in what
• 24:00 - 24:30 is the weather in Paris and you can see we get this information of what the tool call is so we can actually see in our UI when with the integration with the colonel and chain lit we can see that we provided the input of city Paris and the output the weather in Paris is 20 degrees C so we can actually kind of debug what tools it's using directly from the UI um and then of course it's maintained the context of this chat so I can now ask it um how about London so I haven't
• 24:30 - 25:00 actually asked it about the weather but it knows from the context that I'm asking about the weather um and it's able to call um call the tool directly and again now it's asking for London and we get the weather for you know for London um now again in real life you would connect this to a weather service um and um and use it that way instead of having sample sample data okay so those are some quick demos of how to use semantic kernel um I'm
• 25:00 - 25:30 just going to check if there's any questions um on in the chat it's pretty good so far um the question I I'll get to at the end around I see ADK in here um I'll talk about that at the end okay let's go back to slides um okay did that we don't need my backup slide which is great um okay so what we showed there is we
• 25:30 - 26:00 went from a actually you know what i'm going to do one more demo let's do this demo first i think this will Let's show We want to see MCP someone was asking about MCP let's do an MCP demo because that's basically the same i showed you tools in line you can also connect to tools through OpenAI or you connect to tools through MCP so for those of you that don't know what MCP is it's a fairly new protocol i think from like November December of this year um that Anthropic developed um in particular it's useful or was developed for um uh
• 26:00 - 26:30 use for local tools so used either with these kind of computer use assistance um or with with idees like Visual Studio Code or or Cursor where you can have them actually operate on files on your local file system but it's now being used a lot more even for remote services and there's a lot of traction you can go online you can go to the MCP um website the modelcontextprotocol.io website and you can find all sorts of connectors that you can use in fact let's do that real
• 26:30 - 27:00 quick we can see we go here we can see um if we go to their GitHub you can see on there a lot of servers for if we go to servers right here um here's a bunch of servers that they support um so they have a kind of a catalog of servers you can see these are kind of the reference servers which are well supported um you can see things like being able to do search through
• 27:00 - 27:30 Brave um Git I'll show you that one being able to generic things like being able to fetch content from the web Google Maps um Slack SQLite if you want to have a database connection as well as a ton from third parties that have been added as well and a bunch from Microsoft in here you'll see um so you can you can connect these directly into your LLM uh instead of writing code for these you can use these tools that have already been created and are specially designed for LLMs to interact with so this is a great catalog as you're hacking as
• 27:30 - 28:00 you're building your applications don't try and write this code yourself see if there's an M MCP server available for you to use and integrate with that i'll show you how that works in semantic kernel so we're going to use the uh the GitHub plugin so um in this case we're using the um the MCP uh studio plugin uh the MCP studio plugin from the connectors that we have in semantic kernel so we've built this this S sorry not studio SDIO
• 28:00 - 28:30 plugin um for connecting directly to MCP this is based on and we're taking a reference on the uh official MCP SDK and the same thing on the net side we're using the Microsoft.NET MCP server under the covers and so when you connect to an MT MCP server you give it these parameters um a name and description but the important piece is you give it the commands and arguments uh that are needed and you can find these if you're um if you're over here and you click on one of these um so for example we're doing git you can find the information
• 28:30 - 29:00 here on how all the all the tools it has but then also um the parameters that you need to um to use it so this is how you use it with um with a Docker container and this is how you use it with with with UB um so in this case we're using the MPX version and so what actually happens with these with these model context protocol servers is it actually uses MPX if you guys don't know what MPX is this is the package manager for for Node.js um you can also have it use uh you know
• 29:00 - 29:30 YV for for for Python it's a Python plugin um and it actually then pulls that code down is executing in this this uh server locally that then knows how to talk to GitHub um so it's actually pulling the the the code down and using it in in the local directory so we're giving it the command it needs to do that we're saying use MPX and find uh under model context protocol find the GitHub server um so we've done that and we've created this GitHub plugin now we're just going to create our agent just as we did before we're creating an
• 29:30 - 30:00 agent that can search the semantic kernel GitHub project and we're giving it uh this MCP plugin as plugin so now it's a tool that our LLM can use to answer questions about GitHub so this sample we're going to ask it three questions we're going to ask it what are the latest five issues in the semantic kernel repo are there any untriaged issues and what's the status of this particular issue so we have those those three things we're going to run this we're going to run this sample here and again we're just running our agent exactly as we did before we've connected
• 30:00 - 30:30 this tool and then we're running in a loop for each of these three user inputs that we gave it and what it's going to do is it's going to pull down all that information from the GitHub plugin and it's going to uh it's going to call GitHub and it's going to um answer these questions so what we'll see here it's now running and the first question we asked was what are the latest five issues in the in the semantic kernel repo and it's now returned those five issues look we got someone you guys are
• 30:30 - 31:00 talking about MCP aa is kind of the latest thing that that's been was announced earlier this week we have someone working on on that integration with the agent that's agent to agent protocol a bunch of stuff we're working on internally um are there any untriage issues yes there are 26 oh so we got to do some work triaging our Python issues i guess we didn't get to that this morning so we got to do our triage um and we can look at this particular status of an issue so particularly with the GitHub with the GitHub plugin um you can give it your personal access token from uh from
• 31:00 - 31:30 GitHub and now you can have it actually create projects check in things for you and this is how things if you've looked at um agent mode from uh GitHub Copilot that's how it works uses these plugins to be able to actually suggest changes and make changes to your code create new repos pull pull down repos do all that kind of stuff and of course you can also execute uh you can do things on files locally and you can connect to all all these different servers so that's MCP you know once you once you create your MCP plugin um you use it just like you
• 31:30 - 32:00 would any other plugin and semantic kernel it works just like that um there's also some samples in our repo of if you want to use semantic kernel as an MCP server which is also interesting um if you want to build a semantic kernel application and then have something like Visual Studio Code or cursor be able to actually talk to it and then have it do your own action on your behalf so that's another interesting thing you can do with MCP and semantic kernel okay let me get back to my slides
• 32:00 - 32:30 so we looked at a bunch of agents here we looked at the very basic agent an agent that has no tools and is really just a prompt we still call this an agent and the reason why we like to still use an agent framework here is because later on as you grow your application it makes it much easier to add things like tools down the road to add things like telemetry observability if you start by using an agent framework even if you have a very simple install then we show really kind of the core of the single agent which is an agent that's scoped to a particular task
• 32:30 - 33:00 this is an example here of a rag agent so an agent is doing search the user here asks a question um the agent generates a query might be to a semantic search of vector database for a semantic search might be a regular database generates some responses takes those responses and generates a human readable response for those using the LLM so that's the concept of like a single agent that's a very common example but what we're kind of here to talk about is multi-agent systems um and this is where now when you have more advanced problems
• 33:00 - 33:30 and you want to have agents that you can chain together that can collaborate together to solve uh uh more complex problems so this is a case here this example or maybe you're trying to use a multi- aent system to build a marketing campaign so you start with with an agent that specializes in talking to the user this might be an agent that maybe uses audio instead of text to talk to the user so this is your user proxy agent it then talks to your coordinator agent which builds the plan for this marketing campaign and then the coordinator agent uses all these specialized agents to
• 33:30 - 34:00 solve different parts of the problem it might do a web search it might do a search on a local knowledge base on a local database or it might execute some code the code the coding one is really interesting um you may have noticed that you know large language models are not very good at math but they're very good at writing code to do math um so anytime you have a problem that needs some math it's actually much better to write the code to solve that math problem and then have it execute you know in a container this is something you can do with a multi- aent system you also see here
• 34:00 - 34:30 that there's different models being used you can use in your coordinator agent you might use one of the more advanced reasoning models to build out your plan then you might use some models that are that are cheaper um and and more tuned for for tasks like like searching the web or or talking to a database or you might use a small language model that's maybe running locally and very specialized in things like coding so with a multi-agent system you have all these options of how you want to break apart and build your system so why would you want to do this
• 34:30 - 35:00 so what we see when you build multi-agent systems if you start with um this is some data from uh from agentic one which is a um a pattern that micros the autogen team developed uh uh for for solving general general problems uh with a multi- aent system and when they kind of solved this particular benchmark which is the web arena benchmark when they went after this with a single agent system a simple modelbased agent which is what I showed you a model that doesn't have any tools that is just
• 35:00 - 35:30 talking to an agent that was just talking to the model directly it didn't perform very well and when they used that kind of technique in the middle where you had an agent that had tools and capabilities it worked a little bit better um the agent was actually able to do things but they got a big jump when they started moving up to these multi-agent systems and particularly when you use these multi-agent systems that have the ability to get feedback from a human um so the human can assist it as as it's um uh as it's solving the task and of course uh you know you get
• 35:30 - 36:00 very close to pure human uh levels of success on on these benchmarks and so you're saying well why why is it not as good as the human well yeah it's not quite there one of the advantages of multi- aents with a human in the loop is that you can scale much more you can move much more quickly so you don't have a human spending 100% of their time on a particular task you have the multi- aent system doing most of the task and the human is helping along or improving it at particular steps um so you really get that scale and that speed by combining a
• 36:00 - 36:30 multi- aent system with with a human loop the other thing that you get with a multi-agent system is you get better performance so this is some data from two benchmarks here uh one using uh a single agent GPT305 agent and performed you know 48% and 32% of this particular task um and then the blue bar is is performing the same task with uh GBT4 with a single agent and you can see there's a much
• 36:30 - 37:00 larger improvement but in the middle you can see a GBT35 multi-agent solution performs almost as well as a GPT40 GPT4 solution um and then of course a multi-agent system performs even better and basically breaks the tests at this point so what does that mean well it means with a multi-agent system you have options you can either you know you can use potentially a less expensive model and get the same performance as you would with um with a high performance model and actually this is what you see a lot with these reasoning models is they're actually
• 37:00 - 37:30 emulating multi- aent systems you know within within the service um and so you can use a non-reasoning model in a multi- aent system and get kind of reasoning levels of performance um or you can you can tune your performance using uh using a less expensive model when you potentially don't need as much performance as as the high performant model or you can use the best model in a multi- aent system and get even better performance and is even is possible with a single LLM or with a single
• 37:30 - 38:00 model i just went way too far forward spoilers okay let's look at uh multi-agent systems with semantic kernel so we'll start with a pretty simple multi-agent system um let me go back over here to my code okay multi-agent systems so um one of the most common ways to build multi- aent systems is what I showed you there where you have a single coordinator agent that then talks
• 38:00 - 38:30 to other agents within the system so in this sample here we're building three agents we have a top level agent this is kind of the user proxy agent that agent that talks to the user we're calling it the triage agent here um and then it has two other agents that it uses it has a refund agent and a billing agent so this is like a re this is a um a billing system for an organization that's going to help the user kind of reconcile their bill and so we have one agent that specializes
• 38:30 - 39:00 in specializes in refunds and we have one agent that specializes in billing and then we have a triage agent that decides which of these agents to use and you know in the same way that you know I showed you how to use tools this is how you build multi-agent systems you just pass those agents in as tools so in semantic kernel you can pass in plugins uh as that that I showed you earlier into your plugins but you can also pass in agents agents as plugins agents as tools you can connect those directly so now this triage agent it will perform
• 39:00 - 39:30 its task but it's we have here we have forward requests about charges billing cycles payments fees blah blah blah to the billing agent forward requests about refunds to the refund agent so it's going to use those other agents to perform its task it's going to have those agents help it because those agents are specialists in those particular tasks so in this case like these are fairly simple prompts but you can imagine like what I was showing you in those previous examples you might have agents that specialize accessing a particular type of data agents that specialize in coding keeping your agents as small and focused as possible can
• 39:30 - 40:00 deliver much better performance so let's let's run the sample and we'll see we'll see what happens um so I'm just going to grab here this from the example okay so what we'll see is we'll end up into an interactive chat here when this starts running and it's we're going to ask it a question and we're going to say I was charged twice for my subscription last month can I get um those payments refunded what you'll see is it's now
• 40:00 - 40:30 calling both the billing agent and the refunding agent because I asked it I asked it both about refunds and billing and so it's asking and so it's asking each of those agents to help and now it asked them and it got a response so the refund agent now gives this response um and then uh and then the where's the refund agent gives this response and the billing agent here oh that's not quite formatted yeah it's hard to see this is the billing agent giving its response here so the triage agent asked both the agents for a response and they both gave
• 40:30 - 41:00 a response and now the triage agent is going to take both those responses and it's going to synthesize them into a single response so we had the refund agent here give its response we had the billing agent here give its response and now the triage agent is respon is the triage agent is responding to the user with its response and it says it seems you may be dealing with a billing issue here's how we can proceed how to handle the duplicate charge that came from the billing agent how to handle the refund request that came from the refund agent so you can imagine now building these trees and these these complex nodes of
• 41:00 - 41:30 um uh of agents um you know using using semantic kernel I want to show you another multi-agent system or a case where you may want to use multi- aents and this is you know I talked about earlier using semantic kernel with agents from different systems so in this case you know one of the things if you've attended um any of these um any of the sessions earlier maybe even later around the Azure AI agent service um it offers a bunch of built-in capabilities for your agents it offers um Bing search
• 41:30 - 42:00 grounding so it can search the web it offers file search vector search it offers code execution these are things that are already built into those agents so we're going to build a multi- aent system where we have one agent that is talking to the user and it leverages another agent that is hosted in the Azure AI agent service um to perform some of those tasks let's take a look at what this looks like here so we have two agents we're creating we're creating a search agent which is an Azure AI agent so this is an agent
• 42:00 - 42:30 that's hosted in Foundry so there's an Azure AI agent class that you can use and I had my def you have to do some definition work to pull to pull it in and get the credentials for it i'll show you that in a second and then we have a second agent that is our chat completion agent that helps helps the user and just to distinguish the two I'm asking it to talk like a pirate um and so now what we're going to do we're going to ask it you know which team won the 2025 uh March Madness this is something that none of the models would know on their own they were all trained you know months ago um but we're going to have it
• 42:30 - 43:00 actually search the the web for um you know recent piece of news um so let's have this run but while this is Oh my code space is restarting i think I hit F5 too many times while it's happening this is a great opportunity for me to show you exactly what's going on here um with agents in Foundry so again a lot of sessions on this but you can actually build your agents directly in Azure Foundry and so here I have a couple agents I've created i've created a weather agent and I've created a Foundry agent i can um create these agents here
• 43:00 - 43:30 i can give an instructions so this one I've given it answers questions about searching the web my weather agent does a bunch of stuff around the weather um I can try it there's a playground here that I can use um to talk to my agent uh so this is another chat interface you can use to talk to your agent directly um but I'll show you if I go back here what um what the most important thing is with this is I can connect it to uh grounding with Bing so
• 43:30 - 44:00 you can see down here I've given it some knowledge um we can take a look at this here if I go into here we can see that I've given it a connection to uh Bing grounding search um so now it's going to be able to use Bing to search the web so going back to our code here um hopefully I didn't lose any of my loginins or anything uh we have our first agent that I'm going to talk to and it's going to now and I've given it as a tool like we talked about earlier a plugin i've given it the search agent so the first agent my agent is going to use the search
• 44:00 - 44:30 agent to actually do the search on its behalf let's see what happens when we actually run that and so uh while this is starting I'm going to ask it a question i'm asking a question about March Madness which is something that just happened a couple days ago um that I've asked it that question it's now going to use this search agent as a tool um and that
• 44:30 - 45:00 search agent is then going to go up to uh it's hosted in the cloud it's hosted in Foundry so it's not hosted locally oh it didn't show okay it did not show the answer from there but what we can see actually okay so this is actually great we can show um we asked the question and then it responded with as a pirate and said the Florida Gators won March Madness against the Houston Cougars great talking like a pirate um did it actually talk to our search agent well let's go check so if we go up to Foundry here and we click on our search agent and we can see our threads um and we can
• 45:00 - 45:30 see here a thread um right now I'm on the West Coast at 12:44 we can look at this thread and we can see that it was asked the question who won the 2025 March Madness and it replied uh not like a pirate that was the first agent the user proxy agent that talked like a pirate the search agent talked like a normal agent um and it gave us the response from searching the web and it gave us some sources so the sources aren't showing up here but you can pull those out from the SDK if you you can reconcile these sources and you'll actually get like um uh they'll actually
• 45:30 - 46:00 you'll actually get like um uh a website that you can then go and use as a reference and if you wanted to download the data from that website you could then take the URL you could use one of those MCP servers that I showed you earlier that will actually download that data they'll convert it into a format like markdown that agents like to use plain text instead of all the HTML and then you can pass that back into your region so I see a question about that here so that's how you do that okay I'm going to keep rolling along
• 46:00 - 46:30 um next um I will show you group chat real quick so group chat is a technique where um you have agents help each other uh with uh with a solution so in this case we have two agents we have an art director agent um and a copyriter agent so this is a copyrer agent will generate slogans and the art director is a is a critic so we have this author critic pattern where the critic is giving feedback to the author the author agent so we we take
• 46:30 - 47:00 those we have this copywriter that writes copy write writes humorous slogans and we have an art director that gives it a critique on those slogans so this is a small example you can imagine this like if you're writing a large report you may want to have an agent that generates sections in that report and then have another agent that gives it a critique on those sections generate a better response um and so we can see here when this runs um we put it inside a group chat here so we create this group chat with semantic
• 47:00 - 47:30 kernel we give it the two agents we created the writer agent and the reviewer agent and we give it a termination strategy so our termination strategy is first of all only go for 10 iterations so that means it doesn't go forever and then also uh use the approval termination strategy from the agent reviewer so if we look for this pay approver uh strategy up here it says if the art director says the word approved then um then we can stop and so we can see here when we're running we
• 47:30 - 48:00 start with the copywriter um gives an option says shockingly smooth electrically yours the act director gives it some feedback the copy director says um says uh drive the future fully charged it tries uh it uh and then the art director gives it more critique and actually I see a bug here in my code so what happened in this case this is an interesting thing with LLMs you got to be a little bit careful with the response so we asked we just did a simple check here to say if the word
• 48:00 - 48:30 approved was in the history and guess what the art director says not approved so our code didn't work and so there's actually another sample that falls on this on our repo that actually does an even better thing where instead of looking for the word approved we actually send this text up to an LLM and say does the art director approve true or false so when you add that extra step you wouldn't kind of fall into this trap that we did here with this simplified example where the art director said not approved but we assumed that was
• 48:30 - 49:00 approved and then we we exited and said uh we now have the right response so that's an interesting here live live code you guys see some things um but so what you should do when you actually build this instead of just looking for the text string actually send the response up to an LLM and ask it if it's an approval or not and then it wouldn't get trapped by this not approved um or the other thing you can do is you could ask the art director i showed you structured data before you could ask you could have the art director uh respond in structured data where it has one um
• 49:00 - 49:30 par one um uh uh one part of the strct where it um gives a feedback and another where it explicitly says approved equals true or false that's another way you can do it um in this case okay um those are multi-agent systems and multi- aent techniques in semantic kernel um one thing I want to show you guys quickly before we go couple quick demos um so I showed agent service uh using agents from agent service i I
• 49:30 - 50:00 showed semantic kernel that's our production ready library we also have autogen at semantic kernel which is our our autogen at Microsoft which is our research library um and so the research the the autogen folks built a really nice technique called Magentic One which I talked about a little bit earlier um and what it does it's a technique for multi-agent systems where you have that planner and coordinator agent and it has a set of other agents that it can use specifically for solving a task so there's a file file surfer agent that
• 50:00 - 50:30 can read files there's a web surfer agent that can access the web there's a coder agent there's a computer terminal agent um and so all these agents can be used together like a team of agents working together to solve generalized cases and kind of the innovation that it has on top of these special agents is it has this thing called a task ledger so it builds this plan and then it tries to get these agents to solve to execute the plan and then it detects if the plan is successful or not um and we'll actually look to see if
• 50:30 - 51:00 progress is being made and if it's not it'll try a different plan so it's kind of self-healing in that way um so I'll show you a sample of that we have a sample of that on the semantic kernel repo um and actually what I want to show you I'm going to show you here recording I did the other day with actually the C version we do have a Python version as well um but I want to show you the C# version because something interesting happened when I ran it again um normally I only go for one error in my demos uh this will be the second one uh but this one's on purpose so this one this one
• 51:00 - 51:30 actually doesn't count so I'm running this demo here and there's this coordinator agent that is going to build the plan and it's going to talk to this agent team that has all the all these other agents that it's um using to solve the task um so when I run it when I run it here it's going to bring up a semantic kernel application it's going to create this magentic one team and then I'm going to give it a task so in this case I'm giving it a complicated task calculate the distribution of the top 20 largest cities in the USA and so now it's building a plan for that task
• 51:30 - 52:00 it's deciding which agents it's going to use for it and it's now asking each of those agents to um solve the task so the first thing it's going to do is it's going to ask generate the plan and now it's going to ask uh I think the first thing is the web server agent it's going to ask the web server agent to go look on the web to figure out uh see search and retrieve the latest authority distribution of the top 20 largest cities so it does that it found a website it got the information it's now going to execute in the in the code agent it's going to execute um the find the distribution of
• 52:00 - 52:30 those agents um and so it's going to execute some code to find that distribution uh for uh for the different a the different cities in the category and now it's going to visualize it so the last thing it's going to do is it's going to ask an agent to generate a chart for it to generate an image for it and this is when I was running this demo where I screwed up i didn't actually configure the um the visualizer agent properly and so it didn't have the ability to actually generate the image so you can see here the illustrator agent didn't work it says I'm experiencing technical difficulties and
• 52:30 - 53:00 so what we'll see is is the coordinator agent is going to keep trying it's going to try in a few different ways it's going to say try harder you try try and generate this image um and it's not going to work because I I didn't configure the illustrator image illustrator agent correctly but what happens next after it tries three times is really interesting so we'll see here um it's going to try I think one more time so it's a video so it's not gonna it's
• 53:00 - 53:30 not going to stall yeah okay so now it didn't work it tried three times and now it went back to the um the coordinator agent it evaluated the status and it generated a chart in ASKI because it was it wasn't being successful generating an image so it now generated a textbased chart with the distribution so that's kind of a neat way where these systems can be self-healing and solve solve problems on their own um so that's magentic one that came from autogen um but we were this demo here is running that magentic one sample on top of
• 53:30 - 54:00 semantic kernel okay so um last but not least let's talk about kind of more complicated uh agent systems so now with all these pieces we can pull them together this is how you build something like a deep research multi- aent process you've seen things from deep uh things like there's um there's a deep research in Bing Copilot now there's a deep research in that OpenAI developed there's a deep research that Google developed this is how those things work they're multi- aent systems they go off
• 54:00 - 54:30 and you ask it to research a particular topic maybe like how to build agents um it it it goes back and forth there's a user proxy agent that generates an outline for you searches the web for some basic uh basic searches to figure out you know what what is the report you want to generate and then once it figures that out gets confirmation from the user um by the way we're using kind of our best models at the beginning to try and generate that best outline possible we can then fan out into these different section writers and we can run
• 54:30 - 55:00 those in parallel and within those we can have these subprocesses that are doing things like that writer agent author critic pattern where they're iterating and refining and building each section um talking to the it's querying the web trying to find information about each section for this this research project uh this research uh paper that it's trying to write and then finally have an editor agent at the end that consolidates everything and then generates a report and then maybe uses something like MCP to write it to the file system right some markdown file or a PDF or something to the file system
• 55:00 - 55:30 that's how you can bring all these things together to build um you build a multi- aent system i do have one final demo I want to show you uh that is somewhat agent related but I really like it and actually hopefully it'll work because I had that reset earlier um I will just do this real quick let's see if this see if that's working okay so one of the things that we've done in semantic kernel I mentioned those user agent proxies don't necessarily have to be textbased we can also make them um real-time audio based so we have real-time audio capabilities in semantic
• 55:30 - 56:00 kernel which allow you to um you know talk directly to your agents and to your to your semantic kernel applications instead of talking to them through through text i'm just going to make sure this is running here okay looks good and what we've done on top of that is you can actually use things like Azure communication services which let you um actually connect via things like SMS or calling so what I have here is I've uh we have this sample application that actually connects you can see our semantic kernel functions here for the
• 56:00 - 56:30 weather um and we actually connect it to real-time audio so we handle the real-time audio data and then further down here we actually connect it to um Azure communication services which gives us the ability to actually phone our agent so uh what I'm going to do here I actually have my phone here and I'm going to connect it to my speaker phone and what we should see if you guys
• 56:30 - 57:00 can hear this hello how can I assist you today yeah can you tell me what the weather is in London the weather actually can you tell me what the weather is in Seattle the weather in Seattle is currently would you like to know anything else no that's it thank you bye so a couple things there you know really interesting a I was talking to you on the phone that's magical two I was able to interrupt it halfway through so these
• 57:00 - 57:30 new real-time models you can actually have a real conversation you don't have to wait you don't have that lag it happens very quickly in real time and then I was able to talk directly without any changes to my agent code and semantic kernel um so that was a nice little demo showing you know how we can actually talk to our agents in real time okay um great way to cap it off a couple other things so our journey we talked about single agents we talked about multi- aents we talked about complex business processes um being built using
• 57:30 - 58:00 using semantic kernel more things to try um here's some examples here you can connect to other agent services like bedrock agents crew AAI autogen you connect to copilot studio with semantic kernel you can have copilot studio connect to semantic kernel by using a skill copilot studio is our low code framework for building agents at at Microsoft um you can use Azure AI agents i showed you that an example you can have that execute code do file search search the web um you can use local models um I didn't show you an
• 58:00 - 58:30 example here but all these examples that we that I showed you we could use uh local models with O Lama um or any other connector to run your models locally and then of course um I know for hacking you're probably not going to do this but you know you really need to add logging and telemetry at some point so we have extensive logging and telemetry support in semantic kernel um so that's another thing you can try okay finish things off um our GitHub repo go to GitHub semantic kernel um definitely uh check it out there we have
• 58:30 - 59:00 over 150 samples for Python 150 for for .NET we have all sorts of samples up there um that you can go and learn how to use agents uh we have a learn site tons of documentation there and then of course please join us every Wednesday we have office hours if you have any questions about semantic kernel the entire team is there we're answering questions and we're showing demos of the latest stuff um that is available from uh from semantic kernel great that is it um I have a little bit of time for some questions if
• 59:00 - 59:30 there's any um Anna if you've been paying attention or Pamela any uh particular questions that I can go over we have had a couple questions pamela has been doing a great job in the chat answering them um let's see what we have here just scrolling sorry bear with me every I see one that just came up uh rag with magic internal yes of course we have ext I didn't talk about it too much here but we have extensive support for connecting
• 59:30 - 60:00 to memory services um so I talked about that a little bit at the beginning in that first slide um but yeah you can have uh you know Azure AI search you can use that as your vector store for rag um you can use uh things like elastic search and and and we use a lot um for um for for local search you can use um some of the other some of the other models some of the other vector stores available there as well got another question from Sam here uh do these agents talk to each other also
• 60:00 - 60:30 what if you need the agents to run in certain order of operation how do we orchestrate that yeah great question on on order of operation one of the things I didn't cover here on top of this is we have something called the process framework so if you want them to run in a particular order we have a graph-based approach as well that you can use um that's we're still working on that that's in preview and most of it's been done on the C# side but we do have some of the capabilities in Python and those will be coming over to Python very soon but certainly if you're a C# developer that's all ready to go on that side and I'll be coming to Python soon and that's
• 60:30 - 61:00 a graph-based approach similar to if you've used something like Langraph like that's that's um that's an approach for building uh agent flows that way and that gives you a lot more control if you don't want the agents to kind of come up with their own plan uh you can use a graph-based approach to to give them a deterministic uh deterministic execution order awesome and I think we can wrap with this nice little piece of house question or housekeeping question what time are the office hours on Wednesday oh great question so um every week they are at uh
• 61:00 - 61:30 8:00 a.m pacific um so that's um uh 11:00 a.m eastern um and then also once a month we have APAC office hours which are late in the day Pacific i think it's at 7 or 8 um if you click on this you'll see you'll see links to to when the office hours are um and so if you're if you're elsewhere uh if you're not uh kind of in North America or Europe um you can you can do those once a month office hours as well
• 61:30 - 62:00 great thank you very much thank you for joining me uh please reach out to the team if you have any questions i'm really excited to see what you guys can build with semantic kernel i think it's a really great way to quickly quickly build a gentic applications hi everyone hope you enjoyed today's session if you aren't registered for the AI agents hackathon yet register now at
• 62:00 - 62:30 aka.m/agentshack and start hacking on the agent of your dreams and if you have any questions along the way just post in our GitHub discussion forum or attend our live office hours in Discord and of course make sure that you keep attending the live streams to keep learning about agents once your project is ready submit it before April 30th at 11:59 p.m uh PT to be eligible for our prizes thanks for joining
• 62:30 - 63:00 [Music] hey [Music]