Barnard Computer Science Seminar - Alex LaGrassa - Carnegie Mellon University - December 3, 2024

Summary

In an enlightening seminar at Barnard College, Alex LaGrassa, a PhD candidate from the Robotics Institute at Carnegie Mellon University, delves into the interplay between machine learning and robotics. Through the integration of limited data and resources, LaGrassa aims to enhance robot decision-making, focusing particularly on handling deformable objects like plants and cables. With a passion for inclusive STEM education, they lead initiatives to diversify AI fields. During the session, LaGrassa explains how robots can navigate incomplete information to make better decisions and highlights the importance of mentorship in technological advancements.

Highlights

• LaGrassa's research focuses on enhancing robotic decision-making using limited knowledge to handle tasks with greater efficiency. 🤖
• A passionate advocate for inclusive STEM education and diversity in AI research, LaGrassa leads efforts to engage underrepresented communities. 🌟
• Through mentorship programs, they aim to diversify AI by promoting undergraduate research involvement. 📚
• They discuss the concept of model preconditions in robotics, highlighting how robots make decisions with incomplete knowledge. 🧠
• Their innovative methods enable robots to use strategic data collection and task-specific learning to achieve tasks efficiently. 🚀

Key Takeaways

• Robots can learn their limitations to improve decision-making and achieve goals even with limited resources. 🤖
• Combining different resources can expand capabilities and enhance performance. 🔧
• Mentorship is crucial, as it helps nurture future researchers and enables diverse perspectives in STEM fields. 🌟
• Inclusive technology development allows various communities to contribute, reflecting diverse needs and insights. 🌍
• Expanding STEM education access can inspire and empower the next generation of innovators. 🎓

Overview

Alex LaGrassa opens the seminar by sharing their research on expanding robotic capabilities using a blend of machine learning and traditional robotics approaches. They aim to enhance robots' ability to manipulate complex objects like plants, despite having limited data and computational resources. LaGrassa's dedication to inclusive STEM education and mentoring shines through their work on diversifying AI research.

Focusing on decision-making under incomplete information, LaGrassa outlines how robots, much like humans, make choices based on available data and resources. They explain strategies to improve these decision-making processes by understanding model preconditions and utilizing multiple models to extend robots' operational capabilities.

LaGrassa emphasizes the importance of expanding access to STEM education, aiming to inspire and engage diverse communities in technological development. Their initiatives, like mentoring programs and educational courses, are designed to break barriers and encourage underrepresented groups to participate in shaping future technologies.

Chapters

• 00:00 - 01:00: Introduction and Speaker Background This chapter begins with an introduction by Rebecca Wright, chair of computer science at Barnard. She expresses excitement about hosting Alex LaGrassa, a PhD candidate at the Robotics Institute at Carnegie Mellon University. Alex's research focuses on integrating machine learning with classical robotics by enhancing and expanding robot capabilities through algorithm development.
• 01:00 - 06:00: Research Focus: Decision Making with Incomplete Knowledge This chapter discusses the challenges and solutions in teaching robots to manipulate complex and routine deformable objects such as plants, cables, and liquids, especially when there is limited access to data and computational resources. It also highlights the importance of inclusive STEM education and the role of the Paths to AI Research program in diversifying the field of AI by engaging undergraduates from diverse backgrounds in research. The chapter emphasizes the current work being done in the area of robots learning, showcasing efforts to bridge knowledge gaps while fostering community and educational growth.
• 06:00 - 09:00: Mentorship and Education The chapter titled 'Mentorship and Education' features a segment with Alex LaGrassa, who is introduced and acknowledges their advisor, Oliver Kroemer. Alex is a PhD candidate in robotics at Carnegie Mellon University in Pittsburgh. The discussion touches on the importance of recognizing the limits of one's knowledge to enhance decision-making.
• 09:00 - 15:00: Research on Robot Decision Making and Model Preconditions The chapter titled 'Research on Robot Decision Making and Model Preconditions' discusses the intersection of machine learning and classical robotics. The focus leans slightly more towards classical robotics, with a particular interest in plant manipulation. The chapter highlights the challenging problems within agriculture and the significance of working with deformable objects. It also mentions personal motivation for the research, inspired by the desire to automate work in a small vineyard with five grapevines.
• 15:00 - 21:00: Efficient Learning of Model Preconditions The chapter titled 'Efficient Learning of Model Preconditions' addresses the challenge of making decisions when faced with incomplete knowledge. It highlights the significance of undergraduate research in fostering essential skills like critical thinking and problem solving, which are crucial for technological advancement. The narrative emphasizes the human aspect in the context of decision-making processes.
• 21:00 - 30:00: Combining Multiple Models for Better Decision Making The chapter emphasizes the importance of global collaboration in research and the role of mentoring in personal and professional growth. It highlights a personal experience from high school where the speaker first learned how to conduct research as part of their education, showcasing the foundational role of early mentorship in developing research skills.
• 30:00 - 40:00: Model-Free and Model-Based Planning The chapter discusses the speaker's journey to Columbia University at the age of 15 to engage in chemistry research. They focused on understanding the underlying chemistry of a system and how it affects its physical properties. This experience was valuable to the speaker, leading them to want to give back by involving undergraduates in research as a PhD student.
• 40:00 - 52:00: Adapting Models to Complex Environments The chapter discusses a program called 'Paths to AI Research,' which is a mentorship initiative designed to connect undergraduate students interested in AI research with graduate students. This program aims to provide undergraduates with insights into the research process and encourage inclusive participation in technological development. The main emphasis is on fostering involvement and diversity in shaping the future of technology through mentorship and guided engagement in AI research.
• 52:00 - 60:00: Task-Guided Model Generation The chapter discusses the inclusivity of computer science, emphasizing that it is a field open to everyone regardless of their background or identity. It highlights the importance of diverse perspectives in addressing real-world problems. The speaker shares a personal reflection on their own identities, being queer, non-binary, neurodivergent, and disabled, and how these experiences influence their approach to research.
• 60:00 - 70:00: Future Research Directions and Values The chapter titled 'Future Research Directions and Values' focuses on technological development and collaborations. The speaker emphasizes the importance of making the talk accessible and encourages the audience to engage with questions and discussions. However, more in-depth discussions are reserved for the end of the talk. The speaker intends to move around the room to facilitate interaction.
• 70:00 - 88:00: Q&A Session In the chapter titled 'Q&A Session,' the speaker expresses openness to audience engagement during talks, appreciating the use of Zoom for accessibility. They emphasize their non-research work, particularly highlighting an introductory programming course that uses robotics. This ambitious course aims to teach individuals with no prior coding experience to program a robot to move objects within the first two hours, utilizing a web-based platform.

Barnard Computer Science Seminar - Alex LaGrassa - Carnegie Mellon University - December 3, 2024 Transcription

• 00:00 - 00:30 [Rebecca Wright] Welcome everyone! I'm Rebecca Wright chair of  computer science at Barnard and we are really   excited to have with us today Alex LaGrassa. Alex  is a PhD candidate in the Robotics Institute at   Carnegie Mellon University researching methods  that combine machine learning uh with classical   robotics by quantifying and then expanding robot  capabilities. Alex develops algorithms that
• 00:30 - 01:00 equip robots with the intelligence to manipulate  challenging but common deformable objects such as   plants cables and liquid with limited access  to data and computational resources they are   also passionate about inclusive stem education  um so that all communities contribute to shaping   techn technological development and towards this  goal they lead paths to AI research a mentoring   program that aims to diversify AI by introducing  undergraduates from a wide range of backgrounds   to research and we're uh really excited today to  have them talk about their work on robots learning
• 01:00 - 01:30 the limits of their knowledge for better  decision-making thank you. [Alex LaGrassa] Thank you   for the for the introduction. My name is  Alex LaGrassa and I use they them pronouns. This   is my lab and that's me right there and I'm  advised by Oliver Kroemer who's the the person   in green right there and I'm a PhD candidate  in robotics at Carnegie Melon in Pittsburgh.
• 01:30 - 02:00 My research is at the intersection of machine  learning and classical robotics maybe leaning a   little bit more towards the classical robotic  side and what I've been really excited about   recently is plant manipulation because it has  a lot of really challenging problems within   it and agriculture is incredibly important and  deformable objects are ubiquitous in our lives   on the right is another inspiration for this  work which is one of my five grape vins I have a   small vineyard and I would really like to automate  some of that work there uh and my research really
• 02:00 - 02:30 focuses on the problem of decision making with  incomplete knowledge which is a lot of decision making um I'm really passionate about  undergraduate research which is how   I got here today um part of it is that it  exercises some really critical skills for in   for technological velopment such as critical  thinking problem solving and uh humans are
• 02:30 - 03:00 naturally curious and I believe that research is  a very natural extension of that Curiosity um but   most importantly we're not doing research alone  research is really a global effort and as part of   that the relationships we build are really core to  our growth so I think mentoring is one of the most   important parts of research uh I actually got to  experience this for the first time in high school   where a New York City public high School taught us  how to do research in our first year in uh in high
• 03:00 - 03:30 school and that's how I ended up across the street  at Columbia University doing uh chemistry as a   15-year-old and I got to research the underlying  chemistry of a system to understand how it affects   its physical properties and participate in the  research process this was incredibly valuable   for me and as a result I really wanted  to uh to give back and as a PhD student   uh I've been focused on bringing other undergrad  uh bringing undergrads into research so they can
• 03:30 - 04:00 find their mentors through a program called paths  to AI research which is a mentor matching program   for matching undergrads interested in AI research  with grad students who can talk to them about what   the process is like and that way they can engage  in research in a way that includes everybody in   shaping how technology develops um to that I think  the main takeaway I would like for uh people to
• 04:00 - 04:30 get from uh well one big takeaway i' like people  to get from this talk is that computer science is   for everybody there aspects in real world problems  that everyone can contribute to um and this is   this is also a not just abstract for me this is  also quite deeply personal I do come from several   communities that are heavily uh underrepresented  in uh in Tech I am a queer non-binary neurod   Divergent uh and disabled and these identities  have really shaped how I view research
• 04:30 - 05:00 technological development and collaborations  as such it is important for me that this talk   is accessible to to you there will be uh I may  ask a few questions to see where people are at   in following some ideas I think are important and  also you're welcome to interrupt me at any point   for clarifying questions although I do ask for  more discussion back and forth for that to wait   until the end of the talk accordingly I will move  around the room a little bit and you're also very
• 05:00 - 05:30 welcome to do that as well I will not be offended  um in fact I think it's good for engagement much   of the time and relatedly thank you so much for  setting this up on Zoom it's another way of making   these talks more accessible I'd like to highlight  some of the non- resarch work I do towards these   values one that I've been really excited about is  this introduction to programming through robotics   course this was a somewhat ambitious attempt to  bring people that have never done coding before   to programming a robot to move objects in their  first two hours and we do this on a web- based
• 05:30 - 06:00 interface so that as many people as possible can  participate uh and lastly I've had the privilege   of working with the girls of Steel robotics team  which is an all girls and gender minority robotics   team based out of Pittsburgh where they build a  cool robot that does some really uh challenging   tasks for basically a sports competition uh  and I'm really proud of them and got to see   how many of them grow over the years so it's  still um then moving towards my research again
• 06:00 - 06:30 the cruxs of my research is decision- making  with incomplete knowledge we have a lot of   decisions to make in our lives so I'm going to  we think of a difficult decision that youve made recently and much of the time for that decision  you didn't have all the knowledge that you   would like to have this is a sample scenario for  people to think of so suppose you uh one of your
• 06:30 - 07:00 classmates or colleagues is starting a nonprofit  and this is a big investment you're not sure   whether to help out with that or if you'd like to  stay on your current path perhaps in Academia in   order to make this decision you have resources  available to you you can do internet research   uh you can talk to people however making  these decisions takes resources and time   so you're doing this with limited knowledge  and with limited resources very similarly   robots also do this they also have to make  decisions where to go what to do um from
• 07:00 - 07:30 the sometimes even the the low-level decision so  what the motors do and they have some knowledge   as well which is different from how we think so  they might have Maps they do their thinking on   the computer and they also may have some Physics  reasoning but they also have incomplete knowledge   both humans and robots make decisions the best  they can with the knowledge they have available   this brings me to the main two questions of  my research the first is how can robots make
• 07:30 - 08:00 decisions to achieve goals with limited access to  knowledge and resources such as data and compute   and then how can they adapt their decision-making  as they have access to uh different sets of   resources and knowledge sometimes it increases  sometimes it decreases and I also am interested   in thinking about how we do that as humans so  we can also learn from the decisions we make for
• 08:00 - 08:30 robots these are the uh some of the takeaways  I'd like everyone to get from the talk um first   is that the idea that we in robots can learn  what our limitations are in order to better   achieve goals in order to do more the second  idea is that when we have limited resources   we can combine them to be able to achieve more  and to be able to do it so better and lastly is   we do learn and grow throughout our lives  uh and the idea of ways of expanding uh
• 08:30 - 09:00 capabilities in order to achieve tasks and  adapt to new needs I do expect everyone to   uh I do expect everybody to reach these talk  objectives uh however there will be a couple   points that may require some more background  knowledge I'll try to keep these brief and   I'll indicate that with a a sneak peek icon  um but the talk is geared to be accessible
• 09:00 - 09:30 to all audience members right with that  I'm going to go into the context for my research which is what a using models for  decision making a model is a in this context   is it's a function that looks at the state  of the world um which can be for instance a   description of all objects and positions but it  can be something more General and then predicts   given a action that the robot takes where  the robot will end up uh in this illustrative
• 09:30 - 10:00 example it's moving to the right and uh moving  down and then uh okay there's a bug um if it   moves to the right then we expect the robot  to move to the right so this is actually an   example of a an inaccurate model um when you're  planning you're deciding which actions the uh to   take in order to achieve a goal so suppose  a a robot or an agent well in this case it   would be you you're deciding how to get to  a location to get a cake you might choose to
• 10:00 - 10:30 take the six local you may choose to take the six  Express and that will affect how you get to the location um your goal is generally to find a  path to the goal but you may also want to find   the shortest path um in for robots oh yeah so  this is an examp another example of incomplete   knowledge so we have this robot we have the model  in this case what should the robot take to get to
• 10:30 - 11:00 the cake um describe with the arrows or right  left yep just go right now I'm going to so you   predict that the robot will go right using your  intuition and now let's complicate things a little   bit now there's a portal and you don't know where  uh where the portal is going to take you um what   action should the robot take to get to the cake  question will the portal keep you on the grid uh   see you're not guaranteed that information  okay I've been that uh okay so going around
• 11:00 - 11:30 so and with limited information you can still  make decisions to reach the goal and this is   relevant to all of really all of life but  especially in robotics because knowledge of   the real world is incomplete uh this is that  example I'm really excited about robot plant   watering and there how does liquid interact  with plants we don't even know that well let   alone have robots reason about it a way of so now  I'll talk shift into talking about a framework for
• 11:30 - 12:00 making decisions with limited knowledge so this  relies on a idea called the precondition um raise   your hand if you're familiar with a precondition  from any other context um okay so I will describe   here so a precondition is a condition something  that needs to be true before something is used   so if you're going to drink water from a bottle  the cap should be off and there should be water
• 12:00 - 12:30 inside the bottle so these are preconditions  models can also have preconditions these are   the conditions where you can use a model to  make predictions or just to use it in in this   context more specifically it's the states or  the states and actions where a model can be   used to make this more concrete with a portal  example you can trust the model in any space   except for the one with the portal so the portal  place would be outside the model precondition um
• 12:30 - 13:00 and I will give a robotics example next and  then check comprehension here so suppose we   have a model where it's something you can write  pretty easily in Python where if the water is   above the bowl it will end up in the bowl and if  there's water that's outside of the bowl then it   will uh then it will spill it will go outside  of the bowl um so this isn't an accurate World
• 13:00 - 13:30 model um the model let's now let's  add a plant um the model is accurate   for a couple pores so an example here is  the pore where it's under the leaves so   that's in the model precondition where's  an example of a scenario where we expect   the action to and state to not be in the model  precondition where would this model not work
• 13:30 - 14:00 where the leaves are the umbrell yeah so over the  leaves for example can anyone think of any other examples okay um yeah yeah that could be in this scenario  so we're making an assumption about the Cup being   empty um so this is model preconditions this is  kind of the the core of my of my research now   I'm going to get more into to how we can do this  in a less abstract sense so how can we quantify
• 14:00 - 14:30 the limitations of our knowledge in a way that  is informed by data and that can guide decision   making um the first project I'll talk about is um  is how a robot can learn model preconditions with   just a little bit of real world data a little  real world data is especially important for   robots especially with robots with um with water  around it because there can be something dangerous   happening or can just be inconvenient if you have  to mop up a spill uh to make this more concrete
• 14:30 - 15:00 one method that I and some others have used to  get traction is the idea of model deviation so   model deviation is the distance between a the  predicted next state when making a prediction and   what the actual next state is then so in in the  portal example we might have expected the robot   to move on to here and then to address Brian's  question in might end up off grid um and then the
• 15:00 - 15:30 deviation is one way of measuring that is with the  distance we can then Define model preconditions as   in math terms as States in action such that the  distance between the prediction and the actual   state is less than a threshold and this threshold  is uh it's a practical measure of how much error   the system can tolerate so this can be how much  it can correct or how much is Just acceptable   or just some really low number uh and as a result  we have a nice concrete way of defining the model
• 15:30 - 16:00 precondition so the distance is lower than dmax  then it's inside and if it's greater than dmax   then it is outside think let's see all right  questions about model preconditions and model deviation all right and is and Rebecca are you  monitoring chat yes okay thank you so much okay   now this doesn't uh one question you might have  is that this might this isn't doesn't seem useful
• 16:00 - 16:30 because in order to do this we need to know what  the state of the world is and if we already know   that then why do we need the models well we don't  know what the state of the world is so I frame   this problem as learning what the deviation  will be from data the problem is a supervised   learning problem so that means that you have a  input and you have a desired output the input   is the state in the action and the desired output  is is that distance and in your data you do know
• 16:30 - 17:00 what the next state is and you do know the model  prediction then this this uh estimator predicts   what the deviation will be for that state in  action um which is a relatively easy learning   problem in many scenarios um then you define the  model precondition as this is the sneak peek math   way of doing it it is probabilistic and it's  the probability that the deviation is within   the threshold and then you can reason about this  um getting out of the sneak peek another way of
• 17:00 - 17:30 thinking about that is the inside is where the  estimated deviation is probably lower than dmax   then outside is that the deviation might be higher  than dmax not that it necessarily is but that it   might be and then when we're planning we would  want to only use um only use knowledge where   we know that it will be accurate which is inside  that model precondition oh it doesn't show on the
• 17:30 - 18:00 screen it's interesting okay I'll just have to use  my hand now if we're doing this and we're getting   the real world data that makes it expensive  because robot data is expensive so another thing   that we care about is to be able to estimate this  efficiently in a planning problem you have start   States and you have goal States uh goal states  are where you want to go and then you might have   a little bit of data uh this the dash line shows  what the estimate is so right now we can't connect
• 18:00 - 18:30 these these regions the problem cannot be solved  so our goal is to expand this region we want   to do so in a way that's data efficient so not  collecting too many R World samples um a one way   to do that is to ensure that it's relative that  it's relevant to the task so we can do just paths   to the goal or attempts to get to the paths to the  goal we also want to cover more problems so cover   a wide range of space and we want points that are  inform if that actually bring us closer to being
• 18:30 - 19:00 being able to achieve goals this is a sneak peek  of the intuition for it but hopefully some of it   is accessible to everyone the key idea is that we  want points that are uh low in estimated deviation   and high in uncertainty it's the key principle it  does get a little bit more complicated because a   lot of decisions are sequential so if there's a  really informative point but you can only get part   way and then there's a deviation then you've not  reached that that uh that point uh a the intuition
• 19:00 - 19:30 behind one way of addressing that is to weigh the  importance of those later points lower and then   hopefully it will be able to make good decisions  um the way we evaluated this was to look at the   success rate of achieving a task in this case  it's a plant watering task with uh more data so   x-axis is effectively data y AIS is success and we  find that when we're using our method the the Top
• 19:30 - 20:00 Line we're able to achieve a higher success with  less data than if we don't use uh that reasoning   about the use the U usefulness and if we're not  finding the pass to the goal we also try this on   a real world watering task which is why there's  much more uh uncertainty and we finally we're   able to get 80% success uh success rate on plant  watering when it was rarely able to do that before
• 20:00 - 20:30 with only about that was about four data points  um so that's a that's a very useful thing to have   and here's an example of the robot doing one of  those uh plants where it avoids pouring over the leaves what we've learned from this is that it is  possible to efficiently learn model preconditions   using knowledge of the task and informed by what  the deviation estimates are and we're able to get
• 20:30 - 21:00 that that nice real world result however in  this work we're assuming that we can solve   the problem with just one model um in this work  I'll talk in the next work I'll talk about how   we can expand what the what the robot can do  first by adding more models and then by adding   something different with human demonstrations the  different models have different efficiencies and   how it gets access to knowledge so suppose you're  assembling a an Ikea cabinet you can look at the
• 21:00 - 21:30 manual and that will give you some information  and then that will help you assemble it it might   be faster where did the video go okay well uh  but it might be faster to just watch a YouTube video and similarly when making decisions about  water some models are really slow so simulators   tend to be really slow and you might have some  math models that are really fast it's just an   equation with sometimes you can learn a model  to make it more efficient and it's somewhere
• 21:30 - 22:00 in the in between um so my research is on how  we can combine these models in order to solve   more problems and also do so more efficiently  now when we're dealing with multiple models   then the problem of let's say there's a point  that's outside the model precondition well now   we can add another model and now we can solve  more problems furthermore we can add models of   different types so we can add a simulator which  might be B to solve a large range of problems
• 22:00 - 22:30 however we know that the simulator is slow if  we add a learned model that's maybe trained   on some of that simulator data we might have  something that can solve some of those problems   but much faster for this project I was thinking  about it in a uh a office cleaning up Tiding up   domain where we have two math models very simple  one where it's just pick in place and one that   Recons about the drawer but uh the robot in the  drawer but nothing else for more complex inter
• 22:30 - 23:00 interactions we we could use a simulator the  uh complicated part about this uh about this   domain is that the rod can rotate so the math  model that just looks at Pi in place does not   uh does not reflect that rotation which can lead  to failure so if it uh plans with that assumption   that it's rigidly attached you get interesting  behaviors like this where it grasps the rod   and then it accidentally Clos well uned  closes the drawer with the rod so that's
• 23:00 - 23:30 an example where we would want to avoid model  deviation however when we use our method with   model preconditions we avoid that scenario  so it picks uh scenarios that it picks States   and actions that the model is that uh that it's  accurate for and that also achieve High success um I just noticed that my laptop is not
• 23:30 - 24:00 charging just maybe that a table it's just yeah right thank you for that yep thank you okay all right uh another  thing that we can do is we can reason about   the speed So In classical planning  there's some algorithms that if you   select if you trial actions from a state it  will let you uh compute a plan to the goal and
• 24:00 - 24:30 it find it guarantees that you'll find the plan  if it exists and also that it's the lowest cost   so this gives you a guarantee on the quality  of the plan uh I was able to apply two tricks   to switch off between models the first trick  is quite straightforward which is for each uh   state in action we we're predicting the next  state we just pick the fastest model such that   the model is in the precondition so if this is  the fastest model FN uh and F0 all the way on
• 24:30 - 25:00 the left is the slowest and the checks and  xes indicate which um if it's in the model   precondition or not um raise one finger for use F0  two fingers for use F1 and three fingers for use FN okay yeah so we want to use f uh F1 since it's  the the fastest that the uh that such but where
• 25:00 - 25:30 the model precondition still holds however we can  make it faster by also just prioritizing search   with faster models an example that illustrates  this problem is we're trying all actions and   let's say one of them is really slow and one of  them is really fast to get these guarantees much   of the time we need to wait for the slow model to  finish Computing in order to continue the search   and sometimes that can become a bottleneck so  if we can reason about when we can find Optimal
• 25:30 - 26:00 paths to avoid the goal we can uh to avoid that  to Optimal path to find the goal without using   that slow part then we can speed up search and  the intuition for how I did this was we have   a CU so you can think of it as states that are  waiting to be EV expanded or evaluated we can wait   for those slower points to be evaluated later  when we know that it will improve the quality right so this this was about how we speed up the  searge so let's just make sure that this leads
• 26:00 - 26:30 to high success and also is fast so we do find  that if we use both tricks that we achieve a high   success compared to baselines and we also get the  speed up compared to if we don't use those tricks   so if we use the trick we get three compared to  just the simulator is about 100 seconds which is   about what like a um a little over 30X feed up  and here's another sneak peek with this type of
• 26:30 - 27:00 planning you can get an optimality um you can get  a upper bound on the suboptimality of the plants   and the user can specify how much suboptimality  is allowed while making these computational   trade-offs going back to the big idea this is a  way of combining incomplete knowledge in a way   that gives us more reliable plants more efficiency  and still gives us guarantees but we're still   assuming that we can solve the problem with now  our current models sometimes it's good to we can
• 27:00 - 27:30 try solving a problem with a method we know but to  really grow we need to try learning new skills so   for a while I was a believer in always just using  models for planning since we understand them well   but there was a big Trend in uh robot learning  where a robot can just learn from data to their um so modelbased planning is what we showed before  we have a model we send it to a planner which
• 27:30 - 28:00 gives us the action to take but in model-free  planning we don't use a model instead we have   a machine learning skill that goes directly  from what the status to what the action is   and generally the way one way it can work is by  having a human demonstration where uh given the   this the current state and maybe some image data  a human shows what the robot it should do and   then it learns what action it should take in those  scenarios now there's a bit of a data problem here
• 28:00 - 28:30 where if we want to have something that works more  generally then we need more data points which can   be especially expensive with humans so having a  smaller precondition and less data but still being   useful is really important uh I'm going to show a  demonstration of kind of the the training what the   training looks like and then the testing so during  training time the robot plants tries to execute   its plan and then if there's a problem it will  ask the human for help in order to patch what
• 28:30 - 29:00 the deviation is with that skill at test time then  it either plans to the goal or to the be able to   do the skill then executes the plan until it can't  and then it executes the skill knowing it can to   ground this in a real world in a a real example  this is for a shape insertion task the left shows   the robot planning um so kind of thinking about  what going to do the robot tries to fit the block
• 29:00 - 29:30 into the hole but then it finds that it's not  able to complete it and so at this point the   human demonstrator takes over and completes  the task by sliding it in that's that's the skill I get it eventually um then when we test  out the method instead of trying to go directly   into the hole it goes to the beginning of the  skill which is kind of at the corner and then it   slides it right into the hole so we can get better  performance at test time so we get much higher
• 29:30 - 30:00 reliability compared to just doing pure imitation  learning or planning with very little data we're   able to get a high success rate again with about  five demonstrations here um sometimes if we're   lucky the robot doesn't even need to to learn a  skill and this is something that this framework   also addresses so if a robot was planning with a  simple handle the we kind of we can expect this   plan to work work um but if you switch it out with  a decorative handle sometimes it will still work
• 30:00 - 30:30 uh and our method evaluated how reliable that  was and decided no skill was needed in summary   there's a we can patch modelbased planning to get  the benefits of both with the demonstration and   with efficient use of data um however we're still  assuming that we're we're still using deviation to   indicate whether the model is useful which if you  think about it is not always the case and we're
• 30:30 - 31:00 also not really learning more from new data which  is something that robots can do so finally I'll   talk about some more recent work U which is on  expanding what the the capabilities of robots the   first is in well we can just gather more data this  problem is on if we have a simple Model A simple   learned model in this case how do we adapt it to  a more complex environment let's say we have a   simple uh rope model that we trained in a Sim and  there's no obstacles but we want it to be useful
• 31:00 - 31:30 to hook a rope in an engine bay we want to focus  on the parts of the D dynamics that it already   sort of knows in order to expand it um however we  may not be able to solve planning problems yet now   we may have some goals where it's actually on the  hook but right now it currently can't solve the   problem this approach seeks to expand that model  precond condition in the adapted learn model so
• 31:30 - 32:00 that way it can solve more of those planning  problems in the complex domain the intuition   for our approach is based on an idea from  pedagogy some people call it the zone of proximal   development where when a student is earlier in it  uh in their learning they would might benefit from   more help or lower difficulty tasks but if it's  too if it's too difficult they may not progress   and if it's too easy then they're they may get  bored or may not learn much so the idea is to be
• 32:00 - 32:30 in that that sweet spot the um so we do this by  prioritizing learning on points that are easier   so that's described as low deviation in the Rope  example low deviation means that it's uh usually   in free space so it's not touching any obstacles  but in some points it does have to touch obstacles   we'll start with some parts that are similar which  are easy some that are difficult and we'll start   with the DAT the data in the easy region and  some points may not be useful but we'll run
• 32:30 - 33:00 into them however uh the the darkness indicates  how important the point is to uh to the learning   right now over time by focusing on the easying  easy points it eventually learns more so that   more points become similar and then it iteratively  becomes better at the task uh however it's it's   we're not expecting this to learn over the whole  the whole Space not all transitions there's still   going to be some parts of the data that the robot  is really bad at predicting so maybe Parts where
• 33:00 - 33:30 it snags on the on the engine and some difficult  places this is where the model precondition   becomes really helpful we don't need that to be  able to solve problems we can just avoid the parts   where it's uh snagging on the motor uh we wanted  to make sure that we can achieve High success with   small amounts of data so we tested success rate  over data again here the method is called Focus   because it's focusing on a region of space um  we evaluated with uh mostly comparing using
• 33:30 - 34:00 this focusing versus using all of the data and we  found higher uh higher sample efficiency on a uh   in the engine bay um on the engine bay task when  using our method and in a plant watering example   we also found a similar Trend where it was all  all methods were able to learn in this case but   our method was able to do so more efficiently so  in this work we've shown a way to do um a data
• 34:00 - 34:30 efficient adapting knowledge in a way that's data  efficient and that if we get more data we can get   improved performance by learning strategically the  limitation in this work is that we're expecting   that there's going to be some parts of the task  that are quite similar and we're still using   deviation to describe if the model is useful this  NEX workk is a way of expanding in this it's only   going to be one slide uh so all of this is kind  of a peak this is really exciting work that I'm
• 34:30 - 35:00 doing with uh some students oh I should mention  the last one is also a collaboration with the   University of Michigan um they're they're great  at rope uh rope manipulation uh this is a project   also with the same lab different student and uh he  he has a big very big ambition so um we eventually   settled on this where we're doing task guided  model generation uh we have a state okay the   goal is we want to get a Dynamics model we want  a model to do this prediction and we're making
• 35:00 - 35:30 this model we want the model to be efficient so  not use too much compute but also still compute   high quality plants our input is a description of  the task so we want something that can generate   these models from a description of the task um  the way we describe the task and this relaxes   that con that idea that deviation and success  are the same because sometimes you can have   a mistake and still make good progress towards  your goal there's a lot of problems that are like
• 35:30 - 36:00 that so this this problem seeks to address that  to make it a little bit more concrete this is a   tree and we represent the tree with a graph and  the idea is when it's when it only needs to move   one branch we would hope that it doesn't need to  model the UN unrelated parts of the tree with as   many nodes so this can save us on computation in  practice this is actually really complicated and
• 36:00 - 36:30 it depends on what the model you're using is so  what we do is we have the robot practice it tries   to plan and tries to achieve tasks and it does  this a lot uh and it does it in simulation the   then to do the model generation we use a diffusion  model which is a technology you might have used   to generate images based on a prompt but instead  of generating images we're generating a model in   the form of how we're simplifying this uh the tree  so we specify a way of of simplifying it and then
• 36:30 - 37:00 that is generated in uh using the context where  the context is what the task is so for this robot   the task would be say where the tree starts where  the robot is and where we want the tree to end up   uh and so far we have some good promising we have  promising results on this where it can get similar   cost as the as a full model and we can get a a a  reasonably significant speed up when using this   method um so that that's just a sneak preview of  a project I'm really excited about um then finally
• 37:00 - 37:30 I'll talk about some future research ideas I would  love to uh discuss further with people if we are   not already scheduled you can email me and these  are all still in the idea of decision- making with   limited models so uh starting off this is just  kind of a high level idea of what I'm interested   in which is the interplay between uh knowledge  and decision- making and all of my uh my research
• 37:30 - 38:00 focuses on adapting knowledge to changing needs  and resources some sub areas in that first is   transferring knowledge in a way that's resource  efficient from task to task and I'll go into more   detail for each of these doing so in a way that's  modular so we can add capabilities throughout the   robot's life time or even throughout  a Technology's Lifetime and finally is
• 38:00 - 38:30 in individualizing the capabilities of the robot  in a way that's it keeps the human in the loop   so the robot can do what they want it to do or  at least so they can understand the limitations   that the robot has um really more important to me  is actually the the values that I'd like to have   my research and other work uh reflect so first  is that I think we're I I really would like to   move more towards uh Environmental sustainable  technology and other sorts of technology that
• 38:30 - 39:00 benefits Humanity uh so this is somewhat on  the application side I also would like to   lower barriers to more people participating in uh  in technology so robotics AI machine learning so   I'd like to build in a way that includes more  people and finally uh related to that is to   use platforms so uh technology that everyone  has access to so something that's low cost   an open source so anyone can build so first this  is in uh adapting the robot's knowledge according
• 39:00 - 39:30 to to new needs and resource constraints um these  I'm just going to describe a couple General ideas   that hopefully can start some conversations later  if you're interested so first is we have a model   and we want it for specific use case so maybe  we want to deploy it onto a uh a small device   then we can still make it useful and shrink the  model precondition and the question is how should   we do this so that way it's still useful for the  task and can still be deployed on um on a limited
• 39:30 - 40:00 compute the next direction is in um transferring  from task to task this is something that builds   pretty directly from the last project if you have  a robot that can do a kiwis how well can it work   on grapes and how can we facilitate that transfer  or even if it's something it's very different so   Recycling and there's some uh there's some  technology from uh like the language models   that we can leverage to make progress there  and then finally is this one's more longterm
• 40:00 - 40:30 which is if we have general purpose models how  can we find a smaller version that can achieve   a specific task and create more of a general  framework from that related to this is to add   capabilities in a modular manner so we were always  adding things that the robot can do so an example   is let's say we we want to do a cost benefit  analysis maybe we want to change a component
• 40:30 - 41:00 with an open source an open source component  what how will that change our capabilities um   this this comes from a there's some communities  that are really uh passionate about creating open   source technology that interfaces nicely with  each other on both the on robotic side and also   on the machine learning side um and there's  some people that have started work on doing   this but we're really at the very beginning  of having modular technology for robotics   that leverages machine learning um and the second  research area is in combining different forms of
• 41:00 - 41:30 knowledge so the great can take on different forms  and how can we U share this type of knowledge   so that way we can leverage different forms of  knowledge um this is so I think code for example   is a very nice way code and natural language  of combining say visual knowledge and um and   more low-level uh more lowl tasks um and finally  is how we can work together to add capabilities
• 41:30 - 42:00 to a system and so the the general idea here is  let's say we have people in a robot and people   have an idea of what they want the robot to do and  the robot doesn't really have an idea of what it   can do but I'd like to build something for that  but do it in a way that it can the robot and the   human can communicate about this idea of model  preconditions um there is some work on that of   showing what the visualizing capability  limitations in both Robotics and in broader
• 42:00 - 42:30 broader machine learning that we can build on and  this one is a little bit more out there which is   how we can individualize what the system can do  not only what are the limitations but how can   we move the system more towards the capabilities  that the human cares about so that way it can act   align with their interests and with their values  and yes so this is the summary of some of the um   research areas and the common values uh questions  on uh clarifying questions on some of the future
• 42:30 - 43:00 work all right so finally I would like to thank  my collaborators I've been able to work with a   lot of great people across a number of different  areas and different institutions uh and they've   been really helpful for for mentoring me  and teaching me how robotics ml research   and we're able to make these projects and  many others um possible so I thank them a
• 43:00 - 43:30 lot and hopefully you've also learned something  from this as well and that we can make DEC good   decisions with limited knowledge humans and  robots can do this so hopefully you've gotten   some of these takeaways so ways that robots and  humans can use limited World Knowledge to solve   tasks how combining capabilities can bring  us closer to achieving tasks with limited   resources and finally is finding strategic  ways of growing capabilities to achieve
• 43:30 - 44:00 tasks um yeah so I'd love to keep in touch if  we're not already scheduled to chat today there's   different ways of contacting me if you'd like to  uh discuss any of these uh any of these research   ideas or connect on any sorts of shared values  we have around inclusive undergrad research or   uh robotic inclusive robotics uh education um  I'll also be at the SIGCSE conference later this
• 44:00 - 44:30 week if anyone else will be there that's the end  of my talk and uh happy to take longer questions now than. [Rebecca Wright] We have some time for questions uh if  you're on zoom and would like to unmute and ask   your question you can do that you can raise your  hand first and we'll call on you um uh or uh if   you want to type something to the chat we will  uh read that for you and uh in the room of course   you can use the traditional way of uh raising  your hand go ahead. [Questions from audience] um I that's probably a basic
• 44:30 - 45:00 question what SE of sensors do these have typical  is or or actually what like what we see on the   screen or oh yeah um my robots actually don't use  that much so I usually use cameras I'll put Azure   connect cameras preferably two views the robot  these robots happen to have really good s internal   sensing so they can tell where the arm is and  some amount of information about the forces sorry
• 45:00 - 45:30 oh and the scales there's uh to measure how much  water came out yep it use scales thank you that's   a lot okay sensors AR you mentioned like the tiny  Mount to point on smaller devices uh what type are   you like using like model compression methods or  are you just designing like different like smaller   um Sol I think actually my research would  probably fall more into the once you have
• 45:30 - 46:00 compressed well starting with once you've  compressed a model um describing the model   pre the model limitations around that but  I think the next step is to connect the   capabilities and the model the model compression  process so that graph Network the graph project   is somewhat related to that of what can you  remove that decreases the size of the model   but also doesn't affect task  performance too much Beyond
• 46:00 - 46:30 prediction I was wondering do you have a uh do  you have a a question in mind about yeah I was   yeah I was just wondering if um like in that  kind of goal of trying to I guess prune away   things that uh will allow you to compress  like are you doing like quantization wear   training are you doing like model printing  uh to get that kind of lightweight kind of
• 46:30 - 47:00 uh while maintaining that high ACC yeah I think  it would be important to explore different ways   of bringing it down I think what would be  the most interesting is connecting the uh   what the model uh especially model pruning  I think that one's something that's more   straightforward of what that model affords to  the task um and that way we can we can guide   the simplification process if we can do just the  generation that would be kind of a dream where we
• 47:00 - 47:30 don't have to simplify a model we can just start  from the task and then generate the model and not   have to compress anything I don't know how to  do that that would be really cool hopefully we   as Humanity can move towards that yeah that would  help with your environmental sustainability point   but not massive massive massive yes yes that is  that is huge robot learning is really moved to   moving towards we just need more data more compute  at least my community is very very into that and
• 47:30 - 48:00 it's really even when you go outside when people  think well the environmental impact of training   these is significant at conferences actually  when people ask that that question sometimes gets ignored I question for this the part of we're  talking which I think it's an important problem   in learning in general like learning from  easier data points to generalize for harder   examples can you go back to that side because  you have two different problems and different
• 48:00 - 48:30 and what was the difference in the type yeah  of the easy example what what's the difference   between an easy example and a hard example the  gap between that I think yeah this this is like   a very um very practical way of doing it but  I think it's a good start here we go okay uh   in the results if you can go to the result oh  in the results okay yeah like different yes
• 48:30 - 49:00 yeah these are two different problems that we  tried it on the uh first is the that b manual   rope manipulation a rope manipulation task in the  engine bay so that's just a completely different   task so we learned completely different models  it's a different adaptation process we just   deployed uh we tried the same algorithm in these  two different domains M okay because the nature   seems to be I'm the theor means and I was just  curious in terms of what's the nature of the
• 49:00 - 49:30 examples of the EAS yeah so this actually um this  is a place where it diverges somewhere based on   the problem characteristics so in this problem  it was able to eventually make progress uh so   this was uh do I have no I don't have an image of  that one it's pouring but when you get it wrong   you can still make some progress which is why  all of the algorithms eventually make progress   whereas in that real world domain at least one  of the baselines basically never solved the task
• 49:30 - 50:00 ever and that's because if it gets stuck in  a particular way or if the model precondition   so we're we this is like a under we're assuming  that we do not want deviation at test time so if   it cannot solve planning problems then it's not  going to execute anything so if it never makes   progress during training it will never compute a  plan under the constraints the goal so that does   change the it's a way that the problem character  characteristics changing how the algorithm
• 50:00 - 50:30 behaves do you think there's a way to understand  that impact and like fre guess what kind of   problem space you're in that is oh yeah that that  is the that's also the dream I did start a little   bit of this in my uh in my masters where it was a  question of what are the problem characteristics   that lead best to good performance on De deep  reinforcement learning and the I think one of
• 50:30 - 51:00 the key challenges is in describing what the  task is describing what the constraints are   there's been some progress in describing it  with you feed an image to A system that has   knowledge of the planner and then it can predict  that it can uh solve the task itself but you   asked something that was more um yeah there was  another layer on top can you repeat the question   please well I was just you know it seems like  what was just coming out of this conversation
• 51:00 - 51:30 is that the problem characteristics have like  deep impact on the performance of the approach   right like your approach is is is doing better  always but like just in general like problem   characteristics have an impact on different  approaches and I was just thinking out loud   like is there is there a way or do you have ideas  about a way where you could start to before you   can start understanding like sort of the the  shape of the way the learning is going to go   how does like I give you R problem like are  is this a good one an easy one bad one yeah
• 51:30 - 52:00 it's that that one's really interesting because  there's a lot of factors at play there's some of   it comes down to the optimization problem so if  it a lot of it is if it's just spiky if there's   and there's easy ways to get stuck and never make  progress then it's a it's harder to do and then   if there's something that's more smooth then it's  more likely to make progress um so that's one but   I think broader is really difficult especially  when the yeah so spikes and spaces make problems
• 52:00 - 52:30 harder I think that's something that we've also  we can characterize as problem difficulty there's   some there's ways you can get it at from a  learning theory standpoint um for this for   the spikiness part there's a measure you can use  of how much the function changes over a distance   that you can use in some methods that make guar  guarantees about controllers um what else but I
• 52:30 - 53:00 think for broader ones that's that that's quite  open and I think largely will come down to do we   have ways of specifying constraints and problems  in a way that applies to a lot of problems so   we can combine knowledge about performance of  those systems back to those task specifications   but yeah that is a very very interesting  question all right [Rebecca Wright] Great thank you again Alex!