Lecture - 2 Intelligent Agents

Estimated read time: 1:20

Summary

In this lecture, the concept of intelligent agents is explored in depth. The instructor begins by defining what an agent is, followed by a detailed discussion on intelligent agents and rational agents. Key characteristics of the environment in which these agents operate are covered, as well as various agent architectures. The lecture emphasizes the importance of understanding percepts, actions, and performance measures for designing successful agents. The concept of bounded rationality and how it impacts agent behavior is also explained. Examples of agents in different environments, such as human, robot, and software agents, are provided to illustrate these concepts. Finally, the lecturer discusses the significance of autonomy in agents and introduces the subsumption architecture for stimulus-response agents.

Highlights

The lecture starts with an introduction to intelligent agents, emphasizing their role in interacting with and manipulating their environment. 🌟
Various agent types, including reactive, goal-based, and utility-based agents, are discussed, showcasing the diversity in how agents can function. 🤖
The concept of bounded rationality, which refers to an agent's ability to function within its limits, is key to understanding agent design. 💡
Subsumption architecture, a method for building reactive agents by layering simple behaviors, is introduced, highlighting its significance in AI development. 🏗️
The course discusses different environments agents can operate within, such as fully observable, partially observable, deterministic, and stochastic. 🌍
Insight into performance measures and how they define success for intelligent agents is considered essential for evaluating agent effectiveness. 📊
Examples of different agents, including humans, robots, and software, help clarify the discussion, providing concrete insights into agent-based systems. 🤖

Key Takeaways

Understanding Intelligent Agents: The lecture dives into what intelligent agents are and how they operate within environments. Agents perceive, act, and interact with their surroundings, making autonomy and decision-making crucial aspects. 🤖
Types of Agents: From reactive agents that operate based on stimuli to utility-based agents that maximize performance, various agent architectures are explored. Each type has its strengths and use cases. 📚
Bounded Rationality: Introduced by Herbert Simon, bounded rationality explains how agents make the best possible decisions with limited resources and information. This forms a cornerstone for designing rational agents. 💡
Importance of Percepts and Actions: A critical takeaway is how agents rely heavily on percepts (sensory inputs) and actions (responses) to achieve goals and optimize performance. The environment plays a key role in shaping these decisions. 🌍
Subsumption Architecture: Professor Rodney Brooks’ subsumption architecture for reactive agents is highlighted as a pioneering approach to build intelligent agents by layering simple behaviors to achieve complexity. 🏗️

Overview

In this educational lecture, we delve into the fascinating world of intelligent agents, a pivotal concept in the field of artificial intelligence. The instructor kicks things off by laying down the basics—defining agents and explaining what makes them ‘intelligent’. The session is packed with insights into how these agents interact with their environments, the decisions they make, and how they maximize performance.

The lecture covers a variety of agent types, including reactive and utility-based agents, each demonstrating different levels of decision-making complexity. The highlight is the introduction of the subsumption architecture by Professor Rodney Brooks, which outlines a revolutionary way to create complex intelligent behaviors through simple layered components. This architecture is particularly pertinent in the development of robots and autonomous systems.

Furthermore, the concept of bounded rationality is explored, shedding light on how agents can behave optimally based on their knowledge and resource constraints. The educator rounds off the session by encouraging students to think about performance measures and agent effectiveness—important considerations in the creation and evaluation of intelligent systems. Overall, this lecture sets a solid foundation for understanding and designing intelligent agents in real-world applications.

Chapters

00:30 - 01:00: Introduction to Lecture 2 The transcript provided for 'Introduction to Lecture 2' contains only '[Music]' and lacks additional content. No significant summary can be deduced from this text. Further details or script are needed for a comprehensive summary.
01:00 - 02:00: Recap of Previous Lecture The chapter begins with a recap of the previous lecture on Artificial Intelligence. It outlines the introduction to the course and sets up the foundation for the second lecture. More details of the first lecture may follow.
02:00 - 03:00: Overview of Intelligent Agents The chapter begins by recapping previous topics such as the basic definition of artificial intelligence, notable AI systems, and a historical overview of AI development.
03:00 - 04:00: Defining Agents and Intelligent Agents In this chapter titled 'Defining Agents and Intelligent Agents,' the concept of an intelligent agent is introduced. The chapter covers the instructional objectives, which include defining what an agent is, what constitutes an intelligent agent, and understanding the concept of intelligence as it relates to an agent. Additionally, the chapter aims to define a rational agent, providing a foundational understanding of these concepts.
04:00 - 05:00: Introduction to Rational Agents In this chapter, the concept of rational agents is introduced. The discussion covers the definition of what constitutes a rational agent, followed by an explanation of rationality as a concept. The chapter continues to explore the idea of bounded rationality, a critical aspect of understanding rational agents within their operational environments. Lastly, it addresses the various characteristics of the environments in which these agents operate.
05:00 - 06:00: Characteristics of Agent's Environment This chapter aims to help readers understand what an agent is and how it interacts with its environment. The environment is highlighted as a crucial component of agent design, and the lecture will elaborate on these interactions successfully.
06:00 - 07:00: Agent Interaction with Environment This chapter focuses on how an agent interacts with its environment effectively. It outlines the key skills that students should acquire, such as identifying the percepts available to an agent (what the agent can sense) and determining the appropriate actions an agent should take in response to these percepts. The lecture aims to equip students with the ability to analyze problem situations in various environments and devise practical solutions for agent operation.
07:00 - 08:00: Understanding Percepts and Actions The chapter focuses on understanding agents' performance optimization and evaluation. It introduces the concept of performance measures for agents and aims to define and clarify what constitutes a rational agent and successful agent design.
08:00 - 09:00: Performance Measures of Agents The chapter covers the concept of bounded rationality that is essential in understanding agent behavior.
09:00 - 10:00: Autonomous Agents In the 'Autonomous Agents' chapter, the focus is on understanding how to identify the characteristics of an environment in relation to a given problem. Furthermore, it discusses how to recommend an appropriate architecture for an agent in the specified environment. The chapter provides insights into assessing the operational dynamics of the proposed agents within these environments.
10:00 - 11:00: Agent Behavior and Performance The chapter 'Agent Behavior and Performance' discusses how agents operate within environments. It emphasizes that agents receive percepts from the environment and act upon it, potentially changing the environment. The role of an agent's sensory organs is highlighted, indicating that the specific sensors available to an agent influence its interactions with the environment.
11:00 - 12:00: Agent and Environment Characteristics The chapter discusses the capabilities of an agent, including its ability to perceive its environment through tools like cameras or sonar sensors, which allow it to see or hear. It also touches on the internal workings of the agent, particularly the agent program, which makes decisions based on current perceptions or inputs from the environment.
12:00 - 13:00: Human Agents The chapter titled 'Human Agents' describes how agents use previously received sequences to decide on the best actions in the current situation. It explains how agents have actuators or effectors to perform actions that can potentially alter the environment.
13:00 - 14:00: Robot Percepts and Actuators In the chapter titled 'Robot Percepts and Actuators', the focus is on how an agent operates within an environment. The agent uses its sensors to perceive changes in the environment, allowing it to interact with and respond appropriately to the surroundings. The discussion outlines the processes involved in environmental perception and interaction through the agent's sensors and actuators, crucial components in robotics.
14:00 - 15:00: Software Agents The chapter 'Software Agents' explains how agents interact with their environment through sensors and actuators or affectors. Agents have specific goals which dictate their actions, aiming to satisfy these objectives through various interactions and responses to the environment.
15:00 - 16:00: Examples of Robots The chapter discusses the concept of a 'percept' in the context of robotic agents. It explains that a percept encompasses the complete set of inputs that an agent receives at any given time, which could be from a keyboard or various sensors. These inputs, or percepts, could be the immediate percepts or all perceived over time, and they influence the actions (ACT) taken by the agent.
16:00 - 17:00: Fundamental Faculties of Intelligent Agents The chapter discusses the role of intelligent agents in altering their environment through the use of effectors or actuators. An operation involving an actuator is defined as an action, allowing agents to interact with their environment. These actions can be executed via output devices or various actuators possessed by the agent, leading to the grouping of these actions.
17:00 - 18:00: Rationality in Agents This chapter delves into the concept of rationality in agents. It highlights the structure of an agent, which includes sensors for perception, actuators for taking actions, and defined goals. Furthermore, it articulates how the agent's program is designed to create a mapping from sequences of perceptions (percept sequences) to corresponding actions, thereby outlining a foundational framework for understanding agent behavior and decision-making processes.
18:00 - 19:00: Concept of Bounded Rationality The chapter discusses the importance of having a performance measure to evaluate the success of an agent's design. The focus is on assessing how well an autonomous agent operates under the concept of bounded rationality, highlighting the considerations that need to be taken into account to determine its effectiveness.
19:00 - 20:00: Characteristics of Agent Environment This chapter focuses on the characteristics of agent environments, emphasizing the importance of autonomy in artificial intelligence systems. It highlights that an autonomous agent must be capable of making independent decisions about the actions it should take in various situations. This decision-making process is designed to ensure the agent maximizes progress toward its predefined goals.
20:00 - 21:00: Environment Observability The chapter titled 'Environment Observability' discusses the importance of an agent having a clear goal and striving to maximize its performance based on this goal. It is essential for intelligent agents to evaluate their behavior and performance in relation to their predefined objectives. The discussion includes an analysis of agent functions, emphasizing the necessity for an agent to function efficiently and effectively to achieve its desired results.
21:00 - 22:00: Deterministic vs. Stochastic Environments The chapter discusses the ideal mapping that agents should implement in deterministic and stochastic environments to maximize performance measure, emphasizing on how agents decide actions based on perception history.
22:00 - 23:00: Strategic Environments The chapter explores the concept of strategic environments, focusing on how agents can maximize their performance measures. It discusses that the performance measure is subjective and characterizes the success of an agent. The measure can vary, depending on the specific criteria, such as how wealthy the agent becomes due to their behavior.
23:00 - 24:00: Episodic vs. Sequential Environments This chapter discusses the different types of environments an agent may encounter and how they impact problem-solving. Specifically, it focuses on episodic versus sequential environments. An episodic environment consists of independent episodes where each decision does not affect the next, while in a sequential environment, the current decision can influence future situations. The chapter highlights that the problem can be measured in terms of how quickly it can be solved, the precision, and the quality of the solution. Agents also aim to achieve specific objectives, such as minimizing the amount of power consumed, and the overall performance may be assessed based on a combination of these factors.
24:00 - 25:00: Static vs. Dynamic Environments The chapter discusses the concept of an agent, which can range from a human to even a calculator. To define something as an agent, its characteristics such as percepts and actions need to be evaluated.
25:00 - 26:00: Environment Continuity The chapter 'Environment Continuity' explores how environments influence the characteristics of agents. It highlights human agents, discussing sensory organs like eyes, ears, nose, touch, and taste.
26:00 - 27:00: Single vs. Multi-Agent Environments The transcript discusses the concept of five primary input mechanisms humans possess and relates this to designing robots. It identifies actuators such as hands, fingers, legs, and mouths that humans use to take action. The chapter also suggests considering corresponding sensory organs or percepts for robots, likely referring to how robots can be designed to perceive their environment and respond appropriately with actions similar to human motor functions.
27:00 - 28:00: Complex Environments This chapter discusses how robots can be equipped with various sensors, such as cameras, to capture images of their surroundings. These images can then be analyzed by the robot's program to identify key features of the environment. This analysis helps the robot to make decisions on how to act. Additionally, the chapter mentions the use of other sensors like sonar to aid in the robot's interaction with complex environments.
28:00 - 29:00: Agent Architectures Overview The chapter delves into agent architectures, focusing on the types of sensors and actuators that can be employed in robots. It mentions infrared sensors and various actuators like wheels, speakers, lights, and grippers. Additionally, the chapter discusses software agents, noting their growing prevalence and common applications.
29:00 - 30:00: Table-Based Agents The chapter discusses soft bots, which function as sensors and actuators. The focus is on understanding the type of functions these software agents take as input and compute as output.
30:00 - 31:00: Reactive Agents The chapter titled 'Reactive Agents' discusses the development of early robots. It mentions the Xavier robot from CMU (Carnegie Mellon University) and the Cog robot from MIT, developed by Rodney Brooks' group. These examples illustrate the progression and characteristics of reactive agents in robotics.
31:00 - 32:00: Subsumption Architecture The chapter explores the concept of Subsumption Architecture in robotics, focusing on Rodney Brooks' work. It introduces the motivation behind creating the humanoid robot, Cog, which aims to emulate human-like intelligence. The lecture touches on Brooks' various activities and contributions within this domain.
32:00 - 33:00: State-Based Agents In this chapter, the focus is on state-based agents. Professor Rodney Brooks is mentioned in relation to the robot Cog, highlighting the significance of visiting Cog's site for more information. Additionally, the chapter touches on entertainment robots like the IBO robot, sold by Sony, which claims to be autonomous.
33:00 - 34:00: Goal-Based Agents The chapter discusses different types of agents, focusing on goal-based agents which are sensitive to their environment, capable of learning, and adaptable to different stages of life. These agents can be personalized according to their developmental environment. The chapter also mentions other types of agents such as soft bots or software agents, using the example of websites like askjeeves.com as a type of software agent.
34:00 - 35:00: Utility-Based Agents Utility-Based Agents are discussed in this chapter, highlighting different types of software agents and robots.
35:00 - 36:00: Learning Agents The chapter titled 'Learning Agents' focuses on the concept of an intelligent agent in artificial intelligence and robotics. The key points discussed include the faculties necessary for an agent to be considered intelligent. Primarily, it emphasizes that an intelligent agent must be capable of taking actions, perceiving, and understanding its environment. This capability allows it to make decisions on various aspects, such as lighting conditions and air conditioning, highlighting the importance of sensing the surrounding environment to perform tasks effectively.
36:00 - 37:00: Summary of Lecture The chapter discusses the concept of agent-based architectures, emphasizing the importance of sensing and acting in creating intelligent agents. It argues that blind action, or actions conducted without sensing the environment, cannot lead to intelligent behavior. The lecture also introduces robotics as a field focused primarily on the interaction between sensing and acting, with less emphasis on deeper deliberative actions.
37:00 - 38:00: Lecture Questions The chapter discusses the role of understanding in the functioning of intelligent artificial general (AG) agents. It suggests that while appropriate action selection by an agent might not require understanding, sensing activities by the agent necessitate a certain degree of understanding to be effective. Understanding can enhance the ability of an AG agent to choose the proper actions, although it may not always be essential for action selection itself.

Lecture - 2 Intelligent Agents Transcription

00:00 - 00:30 [Music]
00:30 - 01:00 uh today we will start our second lecture of artificial intelligence in the first lect leure we
01:00 - 01:30 introduced uh certain things namely what artificial intelligence is all about the definition of intelligence and we looked at several example AI systems and we also traced the history of AI today uh we will go to the second part of this introduction where we will talk about intelligent agents since uh in a major part of this course we will talk about the various
01:30 - 02:00 aspects of an intelligent agent uh today we will introduce what an intelligent agent is and how we look at intelligent agents so now to describe the instructional objectives of today's lecture uh we like to Define an agent Define an intelligent agent what we mean by intelligence as an agent Define a rational agent
02:00 - 02:30 um in the last class we talked about rationality we will talk about what we mean by a rational agent then we will explain the concept of rationality as we see uh or the concept of bounded rationality which we will deal with we will discuss the different characteristics of the environment in which the agent operates and we will explain different
02:30 - 03:00 agent architectures on completing this lesson you will be able to understand what an agent is and how an agent interacts with the environment as we will see in the course of this lecture the environment is an important uh component of the agent design and the agent should be
03:00 - 03:30 designed so that it can uh work properly in its environment after you have taken today's lecture you should be able to do the following when you are given a problem situation you should be able to identify which are the percepts available to the agent or what the agent can sense and how the agent should act so that
03:30 - 04:00 uh to optimize its performance we should also look at what are the performance measures by which we should evaluate an agent to try to see if the agent design has been successful and we will uh we hope to understand what we mean by the definition of a rational agent what a rational agent is
04:00 - 04:30 we will also look at the concept of bounded rationality which is the rationality that we will deal with so in summary we'll be familiar with the different agent architectures including stimulus response agents state-based agents deliberative or goal directed agents utility based agents as well as learning agents
04:30 - 05:00 and you should also be able to identify given a problem situation the characteristics of the environment and recommend what the architecture of the desired agent should be in this environment so this is our agent the agent operates in an
05:00 - 05:30 environment the agent operates in an environment the agent receives percepts from the environment and the agent acts and its actions can change the environment the agent uses its various sensory organs so depending on the sensors that
05:30 - 06:00 the agent has for example the agent may be able to see if the agent has a camera agent may be able to hear if it has a sonar sensor and so agent can see or hear or accept different inputs from the environment inside the agent there is an agent program which decides on the basis of the current percept or the percept
06:00 - 06:30 sequence that it has received till date to decide what should be the good action to take in the current situation so the agent has actuators or affectors to take actions so these actions can potentially change the environment these actions can change change the environment and the agent can
06:30 - 07:00 use its sensors to sense the changed environment so we see that the agent does the following things the agent operates in an environment the agent perceives its environment M through its
07:00 - 07:30 sensors the agent acts upon the environment through actuators or affectors and also the agent has goals goals are the objectives which the agent has to satisfy and the actions that the agent will take will depend upon the goal that it wants to achieve
07:30 - 08:00 so what is a percept the complete set of input at a given time that agent gets is called its percept the input can be from the keyboard or through its various sensors the sequence of percept maybe the current percept or maybe all the percepts that the agent has perceived so far they can influence the ACT actions of an
08:00 - 08:30 agent the agent can change the environment through effectors or actuators an operation which involves an actuator is called an action so agent can take action in an environment through the output device or through the different actuators that it might be happy these actions can be grouped into
08:30 - 09:00 action sequences so we have already seen the agent has sensors and actuators and has goals and the agent program implements a mapping from percept sequences to actions so the agent program implements a mapping from percept sequences to
09:00 - 09:30 actions we also have to talk about a performance measure by which we will evaluate an agent evaluate how successful the agent design has been and finally we will like to talk about autonomous agents in
09:30 - 10:00 artificial intelligence artificial intelligent agents autonomy is extremely important the agent should be able to decide autonomously which action it should take in the current situation so an autonomous agent decides autonomously what action to take in its current situation in order to maximize imize its progress towards its goals so
10:00 - 10:30 if the agent has a goal the agent should try to maximize its goal to so that its performance measure is its performance is maximum so the behavior and performance of intelligent agents we will look at in terms of the agent function an agent function function is as we have already
10:30 - 11:00 talked about is a mapping from perception history to Action Now what is the mapping that the agent should Implement in order to uh so the obviously the mapping that the agent should Implement is the one which maximizes its performance measure the ideal mapping of an agent specifies which action an agent should take at any point of time we will talk
11:00 - 11:30 presently about uh how the we the agent should achieve this maximization the performance measure is in fact a subjective measure which characterizes how successful an agent is and how the performance measure is characterized can vary the performance measure could be the amount of how rich the agent will become if the agent behaves in a particular way or how
11:30 - 12:00 quickly the problem can be solved or how precise or how good the solution is what is the quality of the solution that the agent has been able to achieve the amount of power it might the agent's objective maybe to minimize the amount of power consumed or the performance measure could be a combination of several of these factors
12:00 - 12:30 now let's look at you see actually an agent could be anything we can look upon a human being as an agent we can look upon a calculator as an agent so in order to characterize something as an agent we have to look at we have to look at the different characteristics in terms of the different the Mapp stage in perform its percepts its actions the
12:30 - 13:00 environment it operates on the characteristics of the environment and certain other things so let us look at some common things that we are familiar with and look at what are the characteristics associated with such agents we are all familiar with human agents so what are our sensory organs we can we can see with our eyes we can hear with our ears we can smell we can touch we can taste these are these are the uh
13:00 - 13:30 five primary input mechanisms that we possess what are the actuators that we possess we have our hands with which we can take some action we have our fingers we have our legs and several other things are mouths with which we can take our actions now if we designed a robot what are the sort percepts what are said different sensory organs that we can
13:30 - 14:00 give to the robot that we can build into a robot we can have a camera with a robot a camera with which it can take pictures of what is it in in front of the robot and the robot function or the robot program can analyze that image and find out Salient characteristics of the environment which will decide the way it should act then the robot can can use other types of sensors like sonar
14:00 - 14:30 sensors infrared sensors Etc and the type of actuators that we can have we can build into a robot are wheels speakers lights grippers any other output device that we may want to and then we can look at an software agent a software agent is becoming increasingly common is so many people
14:30 - 15:00 call them soft BS these soft pods has some functions as sensors they compute a function of the input so they have some functions as sensors and functions as actuators we will later look at some specific software agents to look at the sort of function which they take as their input and the sort of function they compute to give their output
15:00 - 15:30 so this is an example this is a picture of a robot this is a Xavier robot developed at CMU it's one of the earlier robots which were developed at CMU then this is the this is a robot called the Cog robot which was developed by Rodney Brooks group at MIT
15:30 - 16:00 so let's go to the homepage of this robot so we will have occasion to talk more about Rodney Brook's activities in the course of today's lecture so the basic motivation behind creating Cog is the hypothesis that they wanted to build a humanoid robot a robot having intelligence like a human being so this picture may be too small for you to see but this shows Rodney
16:00 - 16:30 Brooks Professor Rodney Brooks is with the robot Cog so you may like to visit the site and get more information about the Cog robot and thirdly we have this uh entertainment robot this IBO robot is sold by Sony so this robot is a to claim theya claimed that it is autonomous it
16:30 - 17:00 is sensitive to its environment it can learn it has different um stages of its life and it can be personalized according to the environment in which it grows up in and then there are other types of Agents like soft Bots or software agents for example there are sites like askjeeves.com which is an example which you can consider it as an example of a
17:00 - 17:30 software agent then there are various expert systems including medical expert systems for example the cardiologist which are also software agents other than software agents we have other types of robots for example autonomous spacecrafts like the Mars rover that we talked about in the last class and then there are other types of robots like intelligent buildings which have intelligence built into them to
17:30 - 18:00 decide the lighting condition air conditioning Etc so an intelligent agent must have certain fundamental faculties it should be able to act an intelligent agent must act the intelligent agent must sense its environment and so an agent must sense its
18:00 - 18:30 environment if the agent acts without sensing that is blind action and blind action cannot be intelligent so there are certain types of architectures where the agents can sense and act and the Agents don't do don't perform any deeper deliberative action in fact robotics is about sensing and and acting only at the outside we only
18:30 - 19:00 require the agent to act appropriately understanding is not necessary understanding may be important for prop choosing proper actions but understanding by itself may not be necessary however we must realize that sensing really needs understanding to be useful so uh an intelligent AG agent which
19:00 - 19:30 possesses complete intelligence must be able to uh do the following it must be able to understand or interpret what it senses it should be able to reason and finally the agent should also be able to learn so that the agent can operate in an unknown environment in fact learning is a pre is a prerequisite for an agent to be autonomous an autonomous agent
19:30 - 20:00 which can adjust to a changing environment must have some learning component and also intelligent agents must be rational in artificial intelligence in this course we will talk about building rational agents and we will talk about
20:00 - 20:30 different aspects that rational agent has and look at these sub felds now what is a rational agent a rational agent is an agent which always does the right thing now what is the right thing so in order to understand that we must understand what are the functionalities or the goals of the agent what are the components of the agent and
20:30 - 21:00 we must look at how we should build these agents perfect rationality assumes that the rational agent knows everything and will take that action which maximizes her utility so perfect rationality is equivalent to demand Deming that the agent is omniscient or all KN if the
21:00 - 21:30 agent knows everything and can and the agent can reason extremely fast and everything is U then the agent is a perfect rational agent however a perfect rationality is something which is not within our scope of not even within the scope of human beings we do not satisfy the definition of perfect rationality we are not omnism there are
21:30 - 22:00 many things which we do not know there are many things which we cannot reason in a reasonable time frame given what we know the concept of bounded rationality was introduced by Herbert Simon of CMU in 1972 in his theory of Economics so bounded rationality says that because of the limitations of the
22:00 - 22:30 human mind humans must use approximate methods to handle many tasks so bounded rationality does not aim at maximizing the absolute utility of the agent because that is something which may not be achievable given a realistic agent given realistic resource bounds so
22:30 - 23:00 instead rationality as we will look at will concentrate on using approximate methods where appropriate so that the agent can take the best action given its resource limitations and given what it already knows rational action is the action that
23:00 - 23:30 maximizes the expected value of the performance measure given the percept sequence to date now when an agent takes an action the action may not always have the same outcome sometimes the outcome of the action may be good sometimes it may be bad so when we talk about rationality we must look at the expected value of the performance measure not always the
23:30 - 24:00 absolute value of the performance measure also we must evaluate an agent not on the basis of how it should behave in a perfect case but how it can behave based on what it can sense so something that the agent cannot sense or foresee we cannot hold the agent respons ible uh for those things
24:00 - 24:30 therefore rational action in the light of bounded rationality talks about maximizing the expected value of the performance measure given the percept sequence which is available to the agent so is rational means is R that's rational action means the best action well the answer is yes but to the best of the agent's knowledge based on
24:30 - 25:00 what percepts the agent already has access to does rational mean optimal yes to the best of the abilities of the agent and subject to the resource conres the agent has may have limitations in the way it can act the agent may have limitations in terms of the resources that it has access to they could be time limitations there could be space limitations so given these
25:00 - 25:30 constraints given its abilities the a agent should take the best action that is expected to maximize its utility that is the rationality that we will talk about uh we must understand that a rational agent need not be omniscient because it does not know the actual outcome of its actions and certain aspects of the environment
25:30 - 26:00 may be unknown to the agent so rationality takes into account the limitations of the agent the percept sequence it has access to the background knowledge the agent has and the actions that the agent can take and we only deal with the expected outcome of actions so as we have already talked about in in 1957 Herbert Simon proposed
26:00 - 26:30 this notion of bounded rationality formally if you define bounded rationality it is that property of an agent that behaves in a manner that is nearly optimal with respect to its goal as its resources will allow as nearly optimal as its resources will allow now we come to another very important important component of an agent the agent actually the design of
26:30 - 27:00 the agent depends a lot on the environment in which the agent operates and the environment the task environment of the agent can be defined in many different ways so we will look at several ways of characterizing the environment and there are two ways of characterizing the environment one in the absolute sense the other from the point of view of the
27:00 - 27:30 agent for example when we want to talk about whether a an environment is deterministic or stochastic we must not look at the deter whether the environment is deterministic from an absolute point of view but rather whether the agent appears whether the environment appears deterministic to the agent based on what it can
27:30 - 28:00 perceive now environments can be divided into two types based on the observability of the environment an environment may be fully observable or it may be partially absorbable if the environment is fully abs observable all of the environment which
28:00 - 28:30 is relevant to the action being considered is observable all the relevant portions of the environment are observable on the other hand in the case of partially observable environments not all the relevant portions of the environment may be observable the relevant features of the environment are only partially
28:30 - 29:00 absorbed for example consider an agent playing chess like the deep blue chess playing program the agent has complete knowledge of the board so everything about the environment is accessible to the agent so the chess environment is a fully observable environment consider again the environment where the agent is playing Focus so the agent cannot see
29:00 - 29:30 the hand of the opponent so the environment in this case is only partially observable to the agent so if the environment is fully observable the agent need not keep track of how the environment is changing and deliberating about the environment so if the environment is partially observable the agent in order to behave successfully in such an
29:30 - 30:00 environment has an added task of keeping track of the environment and reasoning about the properties of the environment then if you look at the aspect of determinism again environments can be divided into two or three types deterministic environments in deterministic environments the next state of the environment is completely
30:00 - 30:30 described by the current state and the agent's action so when an agent we have already seen when we looked at the diagram of the agent environment that the agent's action changes the environment now in a deterministic environment given the agent's action how the environment changes is deterministic there is no other uh
30:30 - 31:00 unknown U thing which comes into the picture to decide how the environment changes in stochastic environments on the other hand there is some element of uncertainty therefore what how the environment changes depends not just on the action that the agent takes now as I was
31:00 - 31:30 saying just now whether an environment is deterministic or stochastic depends on how you look at the environment if you cannot observe the environment fully the environment may appear stochastic to you whereas it is actually deterministic if you have access to the entire environment for example take the example of poers again if the agent did have access to
31:30 - 32:00 both the players hands and all the cards which are there then the environment actually behaves in a deterministic way however when the it is partially observable it appears to be a stochastic environment so this is an environment which appears to be stochastic because of partial observ ability if you look at the game of Ludo it is a stochastic
32:00 - 32:30 environment because the environment how it evolves depends on the the value which the D has rolled so whether the D has rolled one or two or six depending on that the agent can decide its action so this is a truly stochastic environment a strategic environment is an environment state which is wholly determined by the preceding State and
32:30 - 33:00 the actions of multiple agents so a strategic environment in a strategic environment apart from the current agent agent itself there are other agents around and the actions of all these agents influence the environment a strategic environment is one where the environment is only Changed by the actions of the agent itself and other
33:00 - 33:30 agents for example chess a chess playing Agent operates in a strategic environment then if we look at the episodicity of an environment an environment may be episodic or sequential an episodic
33:30 - 34:00 environment is one where subsequent episodes do not depend on what actions occurred in previous episode first of all sometimes certain tasks can be divided into different phases or different episodes in such episodic environments the the one episode may or may not influence the subsequent episode
34:00 - 34:30 so if one episode does not influence the subsequent episode such an environment is called an episodic environment so in such an environment an agent did not plan Beyond episodes in a sequential environment on the other hand the agent engages in a series of connected episodes so subsequent episodes are dependent on what happened in the previous episodes
34:30 - 35:00 so the agent in this case the agent may need to plan ahead or base its action upon what happened in the previous episodes then if we look at another characteristics of the environment based on its dynamism environments May may be static or dynamic a static environment does not
35:00 - 35:30 change by itself so an advantage is that a static environment does not change when the agent is deliberating so the agent does not need to observe the world during its thinking process a dynamic environment changes by itself that is apart from the action that the agent
35:30 - 36:00 takes also another dimension by which we can characterize environments is by its continuity an environment is discrete or continuous an environment is discrete if the number of distinct percepts and actions are limited and the number of states are limited if the number of states are
36:00 - 36:30 discrete then the environment is a discrete environment an environment is a continuous environment if the range of percepts is continuous or range of actions that it can take is many or the number of states is either continuous or too many so that we treat them as continuous and finally we can also characterize for the environment
36:30 - 37:00 according to whether there is one agent which we are talking about in the environment or whether there are other multiple agents in the environment so that is if the environment contains other agents then it is a multi-agent environment in game uh when we look at uh two person games we usually talk about one opponent
37:00 - 37:30 agent there are environments where there is a single agent but uh in so there are many tasks that the agent does usually we talk about a single agent environment but there are many situations which require distributed agents or social and economic systems we deal with multiple agent
37:30 - 38:00 systems so the complexity of the environment includes knowledge reach environments and input Rich environments in knowledge Rich environments there is enormous amount of information that the environment contains knowledge Rich environments contain lots of information and input Rich environment is one where there's an
38:00 - 38:30 enormous amount of input enormous amount of percept that the agent can get and in such environments which are complex which are either knowledge rich or which are percept rich the agent must have a way of managing this complexity so such considerations when the environment is complex the agent must develop its strategies for
38:30 - 39:00 sensing or its strategies of attentional mechanisms the agent must decide selectively what to sense what is more relevant and what it should give its attention to rather than deal with the entire complexity of the full environment so an agent needs to focus its effort in such Rich environment now we will look at the different types
39:00 - 39:30 of agent architectures um table based agents stimulus response agents goal based agents utility based agents and learning agents we will briefly talk about this agent architectures a table based agent is a very simple agent where you see we have said that an agent has to take
39:30 - 40:00 actions given its sense given what its senses given its percept now in a table based agent the mapping from percepts to actions is stored in the form of a table so on the left side we have the perets and on the right side we have we have actions so and the behavior the agent
40:00 - 40:30 program is extremely simple so given the percept the agent looks it up in the table and decides what action to take So based on this mechanism we can U develop what we call reactive agents which can take the action depending on the percept unfortunately the best action for the agent depends not only on the current
40:30 - 41:00 percept but on the percept sequence now the number of possible percept sequence can be very large in fact it can be infinite if the agent acts over many time steps many unbounded time step then it can become infinite so this table can become very large if we have a mapping from percept sequence to action in case of simple and U simple
41:00 - 41:30 tasks where the action only depends on the current percept it may be easy to develop a table based agent but it is infeasible when the action good act correct action or the best action depends on the past and not only what the agent sees or perceives at the current time step
41:30 - 42:00 so as I have already said a table is a very simple way to specify a mapping from percepts to actions but tables may become very large and in a table based agent there is no intelligence of the agent himself all the work is done by the designer in designing the table so design designer makes the table the agent just looks it up to decide what how to
42:00 - 42:30 act so such agents have no autonomy all actions all behaviors are predetermined and there is really no concept of learning now the mapping from percept to action may be done in different ways it may be in terms of an actual table it may be in terms of a produ ction system or rules this mapping can also be implemented by neural networks or one
42:30 - 43:00 can implement the mapping by an algorithm so this mapping can be implemented in a rule based manner using a neural network or using some ALG so such agents where the action depends only on the percepts and there is no deliberation involved they are
43:00 - 43:30 called reactive agents so this term reactive agents means those agent whose information come from sensors and they can take actions through their actuators or effectors and we can additionally assume that the act the actions changes the current state of the
43:30 - 44:00 world and then agent can only take some action without any deliberation such agents are also called stimulus response agents they have no notion of History all their history is encoded in the current state as the sensus see it right now now at this point we will talk about
44:00 - 44:30 Professor Rodney Brooks of MIT and the subsumption architecture that he has proposed which is based on stimulus response agents in 1986 Professor Rodney Brooks gave this notion of the subsumption architecture his argument was that lower animals behave in a large largely reactive manner they have a very little sense of
44:30 - 45:00 deliberation so they have the most of the actions are reactive actions and his argument was that in the time scale of evolution reactive Behavior came much earlier than telora behavior and he has been able to show that simple reactive Behavior a number of such components
45:00 - 45:30 having simple reactive Behavior can achieve a surprising degree of intelligence so Brook's idea has been to follow the evolutionary path how life is evolved and build simple agents for complex worlds a combination of simple agents which can work well in the complex world the features of his subsumption architecture are that there is no
45:30 - 46:00 explicit knowledge representation behavior is not centrally controlled but behavior is distributed among different components the response to stimulus is reflexive and importantly the design is bottom up the complex behaviors are fashioned from combinations of of simple behaviors simple behaviors are put
46:00 - 46:30 together to achieve complex behaviors and each individual agent is simple and inexpensive so in the subsumption architecture is basically a layered architecture uh so as I mentioned according to to Professor Brooks the time scale for evolution has been 5
46:30 - 47:00 billion years it has taken 5 billion years to evolve from single cells and cells evolved to the present day the first humans appeared 2 and half billion years ago only whereas the first cells appeared 5 billion years ago and symbols did not appear until only 5,000 years ago and therefore his um his proposition is
47:00 - 47:30 that we should look at simpler Behavior as the first step towards building intelligent agents so in subsumption architecture there are different layers of behavior and higher layers override lower layers and each activity in each layer consists of a simple finite State machine now this is an example of a simple
47:30 - 48:00 architecture proposed by uh Rod de Brooks in this uh system there are several layers this is a very simple robot in the layer zero there is the avoid obstacles layer layer zero is the avoid obstacles layer in this layer there are several sensory organs the the sonar component generates sonar scan the Collide component sends halt
48:00 - 48:30 message to forward and the Field Force component is the signal sent to run away or turnaround so layer zero of the agent is to avoid obstacles this is the first layer of the agent which enables the agent to navigate in environment without colliding with other agents so very
48:30 - 49:00 simple behavior of a navigating robot without using a lot of deliberation in the second layer or the layer one there is the Wonder Behavior this Wonder Behavior generates a random heading it allows the robot to occasionally wander so in the Wonder layer there are two components one generates a random
49:00 - 49:30 heading and secondly avoid the void component reads repulsive force and generates new heading and feeds that to turn and to forward Layer Two that is the third layer has the exploration Behavior so this has components the vlook component notices Whenever there is idle time it looks for
49:30 - 50:00 an interesting place the path plan component sends new direction to void the integrate component monitors path and sends them to the path plan so as we have seen these percept based agents are efficient because they
50:00 - 50:30 do not require internal representation for reasoning or inference however there is no strategic planning or learning and they are not good for multiple opposing goals in the second uh type of agent the more knowledge Rich agents become to stateb based agent in state based agents information comes from sensors and it changes internally the agent's
50:30 - 51:00 current view of the world based on the state of the world and the knowledge the agent has it triggers action through the effectors and in order to do this the agent does some deliberation So based on the state of the world and knowledge the agent chooses the actions and carries out the action through the
51:00 - 51:30 affectors now in cold based agent the agent's action depends upon its goals and goal formulation is based on the current situation so based on the goal that the agent has to achieve the and the current state of the agent as it perceives the agent goes through some deliberations to decide what should the next action be we will look at uh search and planning
51:30 - 52:00 these are the two fields of AI in which we will look at different U ways of deciding what action to take in order to achieve the agents goals so in goal-based agents the sequence of steps required to solve a problem is not known at prior they're not available in a table and they must determined by a systematic exploration of the alternative actions by
52:00 - 52:30 deliberations then thirdly we have utility based agents utility based agents are Ain to goal- based agent but they are a more General framework if an agent has several goals and it can achieve only some of them these goals may have preference associated with them which goal go s are more preferable so in this case we can talk
52:30 - 53:00 about utility based agents where we have different preferences for different goals a utility F function is a very general function that Maps a state or a sequence of states to a real valued utility so a state is mapped to an utility value and we can say that the goal of the agent is to maximize its utility or the the goal of the agent is to maximize its expected utility when we
53:00 - 53:30 look at decision Theory we can considered agents which are based on utility and finally and very important we have learning agents we have said that learning is an extremely important component of autonomy and autonomous agent should have some learning built into it a learning allows learning allows an agent to operate in unknown environment and the learning element
53:30 - 54:00 modifies the performance element and learning is required for True autonomy we will take up uh learning later in this course when we will look at several types of learning that are carried can be carried on by agents so in summary of today's lecture an agent person receives and acts in an environment it has an architecture and an agent is implemented by an agent
54:00 - 54:30 program an ideal agent chooses the action which maximizes its expected performance an autonomous agent uses its own experience rather than buil-in knowledge of the environment by the designer an agent program maps from percept to action and updates its internal State reflects a agents respond immediately to percepts goal-based agents act in order to achieve their
54:30 - 55:00 goals utility based agents maximize their own utility function and then in order to have gold-based agent the agent must represent its knowledge must represent its history must have some background knowledge this we will study when we look at States spased search so representing knowledge is important for success successful agent design so we will come to now to a set
55:00 - 55:30 of questions of based on today's lecture question one Define an agent question two what is a rational agent question three what is bounded rationality question four what is an autonomous agent question five describe the Salient features of an agent question six find out about the
55:30 - 56:00 bars Rover what are the percepts for this agent characterize the operating environment for this agent what are the actions that this agent can take how can one evaluate the performance of this agent what sort of agent architecture do you think is most suitable for this agent question seven answer the same
56:00 - 56:30 questions above for an internet shopping agent with this we complete today's lecture thank you