Barnard Computer Science Seminar - Alex LaGrassa - Carnegie Mellon University - December 3, 2024
Estimated read time: 1:20
Summary
In an enlightening seminar at Barnard College, Alex LaGrassa, a PhD candidate from the Robotics Institute at Carnegie Mellon University, delves into the interplay between machine learning and robotics. Through the integration of limited data and resources, LaGrassa aims to enhance robot decision-making, focusing particularly on handling deformable objects like plants and cables. With a passion for inclusive STEM education, they lead initiatives to diversify AI fields. During the session, LaGrassa explains how robots can navigate incomplete information to make better decisions and highlights the importance of mentorship in technological advancements.
Highlights
LaGrassa's research focuses on enhancing robotic decision-making using limited knowledge to handle tasks with greater efficiency. 🤖
A passionate advocate for inclusive STEM education and diversity in AI research, LaGrassa leads efforts to engage underrepresented communities. 🌟
Through mentorship programs, they aim to diversify AI by promoting undergraduate research involvement. 📚
They discuss the concept of model preconditions in robotics, highlighting how robots make decisions with incomplete knowledge. 🧠
Their innovative methods enable robots to use strategic data collection and task-specific learning to achieve tasks efficiently. 🚀
Key Takeaways
Robots can learn their limitations to improve decision-making and achieve goals even with limited resources. 🤖
Combining different resources can expand capabilities and enhance performance. 🔧
Mentorship is crucial, as it helps nurture future researchers and enables diverse perspectives in STEM fields. 🌟
Inclusive technology development allows various communities to contribute, reflecting diverse needs and insights. 🌍
Expanding STEM education access can inspire and empower the next generation of innovators. 🎓
Overview
Alex LaGrassa opens the seminar by sharing their research on expanding robotic capabilities using a blend of machine learning and traditional robotics approaches. They aim to enhance robots' ability to manipulate complex objects like plants, despite having limited data and computational resources. LaGrassa's dedication to inclusive STEM education and mentoring shines through their work on diversifying AI research.
Focusing on decision-making under incomplete information, LaGrassa outlines how robots, much like humans, make choices based on available data and resources. They explain strategies to improve these decision-making processes by understanding model preconditions and utilizing multiple models to extend robots' operational capabilities.
LaGrassa emphasizes the importance of expanding access to STEM education, aiming to inspire and engage diverse communities in technological development. Their initiatives, like mentoring programs and educational courses, are designed to break barriers and encourage underrepresented groups to participate in shaping future technologies.
Chapters
00:00 - 01:00: Introduction and Speaker Background This chapter begins with an introduction by Rebecca Wright, chair of computer science at Barnard. She expresses excitement about hosting Alex LaGrassa, a PhD candidate at the Robotics Institute at Carnegie Mellon University. Alex's research focuses on integrating machine learning with classical robotics by enhancing and expanding robot capabilities through algorithm development.
01:00 - 06:00: Research Focus: Decision Making with Incomplete Knowledge This chapter discusses the challenges and solutions in teaching robots to manipulate complex and routine deformable objects such as plants, cables, and liquids, especially when there is limited access to data and computational resources. It also highlights the importance of inclusive STEM education and the role of the Paths to AI Research program in diversifying the field of AI by engaging undergraduates from diverse backgrounds in research. The chapter emphasizes the current work being done in the area of robots learning, showcasing efforts to bridge knowledge gaps while fostering community and educational growth.
06:00 - 09:00: Mentorship and Education The chapter titled 'Mentorship and Education' features a segment with Alex LaGrassa, who is introduced and acknowledges their advisor, Oliver Kroemer. Alex is a PhD candidate in robotics at Carnegie Mellon University in Pittsburgh. The discussion touches on the importance of recognizing the limits of one's knowledge to enhance decision-making.
09:00 - 15:00: Research on Robot Decision Making and Model Preconditions The chapter titled 'Research on Robot Decision Making and Model Preconditions' discusses the intersection of machine learning and classical robotics. The focus leans slightly more towards classical robotics, with a particular interest in plant manipulation. The chapter highlights the challenging problems within agriculture and the significance of working with deformable objects. It also mentions personal motivation for the research, inspired by the desire to automate work in a small vineyard with five grapevines.
15:00 - 21:00: Efficient Learning of Model Preconditions The chapter titled 'Efficient Learning of Model Preconditions' addresses the challenge of making decisions when faced with incomplete knowledge. It highlights the significance of undergraduate research in fostering essential skills like critical thinking and problem solving, which are crucial for technological advancement. The narrative emphasizes the human aspect in the context of decision-making processes.
21:00 - 30:00: Combining Multiple Models for Better Decision Making The chapter emphasizes the importance of global collaboration in research and the role of mentoring in personal and professional growth. It highlights a personal experience from high school where the speaker first learned how to conduct research as part of their education, showcasing the foundational role of early mentorship in developing research skills.
30:00 - 40:00: Model-Free and Model-Based Planning The chapter discusses the speaker's journey to Columbia University at the age of 15 to engage in chemistry research. They focused on understanding the underlying chemistry of a system and how it affects its physical properties. This experience was valuable to the speaker, leading them to want to give back by involving undergraduates in research as a PhD student.
40:00 - 52:00: Adapting Models to Complex Environments The chapter discusses a program called 'Paths to AI Research,' which is a mentorship initiative designed to connect undergraduate students interested in AI research with graduate students. This program aims to provide undergraduates with insights into the research process and encourage inclusive participation in technological development. The main emphasis is on fostering involvement and diversity in shaping the future of technology through mentorship and guided engagement in AI research.
52:00 - 60:00: Task-Guided Model Generation The chapter discusses the inclusivity of computer science, emphasizing that it is a field open to everyone regardless of their background or identity. It highlights the importance of diverse perspectives in addressing real-world problems. The speaker shares a personal reflection on their own identities, being queer, non-binary, neurodivergent, and disabled, and how these experiences influence their approach to research.
60:00 - 70:00: Future Research Directions and Values The chapter titled 'Future Research Directions and Values' focuses on technological development and collaborations. The speaker emphasizes the importance of making the talk accessible and encourages the audience to engage with questions and discussions. However, more in-depth discussions are reserved for the end of the talk. The speaker intends to move around the room to facilitate interaction.
70:00 - 88:00: Q&A Session In the chapter titled 'Q&A Session,' the speaker expresses openness to audience engagement during talks, appreciating the use of Zoom for accessibility. They emphasize their non-research work, particularly highlighting an introductory programming course that uses robotics. This ambitious course aims to teach individuals with no prior coding experience to program a robot to move objects within the first two hours, utilizing a web-based platform.
Barnard Computer Science Seminar - Alex LaGrassa - Carnegie Mellon University - December 3, 2024 Transcription
00:00 - 00:30 [Rebecca Wright] Welcome everyone! I'm Rebecca Wright chair of
computer science at Barnard and we are really excited to have with us today Alex LaGrassa. Alex
is a PhD candidate in the Robotics Institute at Carnegie Mellon University researching methods
that combine machine learning uh with classical robotics by quantifying and then expanding robot
capabilities. Alex develops algorithms that
00:30 - 01:00 equip robots with the intelligence to manipulate
challenging but common deformable objects such as plants cables and liquid with limited access
to data and computational resources they are also passionate about inclusive stem education
um so that all communities contribute to shaping techn technological development and towards this
goal they lead paths to AI research a mentoring program that aims to diversify AI by introducing
undergraduates from a wide range of backgrounds to research and we're uh really excited today to
have them talk about their work on robots learning
01:00 - 01:30 the limits of their knowledge for better
decision-making thank you. [Alex LaGrassa] Thank you for the for the introduction. My name is
Alex LaGrassa and I use they them pronouns. This is my lab and that's me right there and I'm
advised by Oliver Kroemer who's the the person in green right there and I'm a PhD candidate
in robotics at Carnegie Melon in Pittsburgh.
01:30 - 02:00 My research is at the intersection of machine
learning and classical robotics maybe leaning a little bit more towards the classical robotic
side and what I've been really excited about recently is plant manipulation because it has
a lot of really challenging problems within it and agriculture is incredibly important and
deformable objects are ubiquitous in our lives on the right is another inspiration for this
work which is one of my five grape vins I have a small vineyard and I would really like to automate
some of that work there uh and my research really
02:00 - 02:30 focuses on the problem of decision making with
incomplete knowledge which is a lot of decision making um I'm really passionate about
undergraduate research which is how I got here today um part of it is that it
exercises some really critical skills for in for technological velopment such as critical
thinking problem solving and uh humans are
02:30 - 03:00 naturally curious and I believe that research is
a very natural extension of that Curiosity um but most importantly we're not doing research alone
research is really a global effort and as part of that the relationships we build are really core to
our growth so I think mentoring is one of the most important parts of research uh I actually got to
experience this for the first time in high school where a New York City public high School taught us
how to do research in our first year in uh in high
03:00 - 03:30 school and that's how I ended up across the street
at Columbia University doing uh chemistry as a 15-year-old and I got to research the underlying
chemistry of a system to understand how it affects its physical properties and participate in the
research process this was incredibly valuable for me and as a result I really wanted
to uh to give back and as a PhD student uh I've been focused on bringing other undergrad
uh bringing undergrads into research so they can
03:30 - 04:00 find their mentors through a program called paths
to AI research which is a mentor matching program for matching undergrads interested in AI research
with grad students who can talk to them about what the process is like and that way they can engage
in research in a way that includes everybody in shaping how technology develops um to that I think
the main takeaway I would like for uh people to
04:00 - 04:30 get from uh well one big takeaway i' like people
to get from this talk is that computer science is for everybody there aspects in real world problems
that everyone can contribute to um and this is this is also a not just abstract for me this is
also quite deeply personal I do come from several communities that are heavily uh underrepresented
in uh in Tech I am a queer non-binary neurod Divergent uh and disabled and these identities
have really shaped how I view research
04:30 - 05:00 technological development and collaborations
as such it is important for me that this talk is accessible to to you there will be uh I may
ask a few questions to see where people are at in following some ideas I think are important and
also you're welcome to interrupt me at any point for clarifying questions although I do ask for
more discussion back and forth for that to wait until the end of the talk accordingly I will move
around the room a little bit and you're also very
05:00 - 05:30 welcome to do that as well I will not be offended
um in fact I think it's good for engagement much of the time and relatedly thank you so much for
setting this up on Zoom it's another way of making these talks more accessible I'd like to highlight
some of the non- resarch work I do towards these values one that I've been really excited about is
this introduction to programming through robotics course this was a somewhat ambitious attempt to
bring people that have never done coding before to programming a robot to move objects in their
first two hours and we do this on a web- based
05:30 - 06:00 interface so that as many people as possible can
participate uh and lastly I've had the privilege of working with the girls of Steel robotics team
which is an all girls and gender minority robotics team based out of Pittsburgh where they build a
cool robot that does some really uh challenging tasks for basically a sports competition uh
and I'm really proud of them and got to see how many of them grow over the years so it's
still um then moving towards my research again
06:00 - 06:30 the cruxs of my research is decision- making
with incomplete knowledge we have a lot of decisions to make in our lives so I'm going to
we think of a difficult decision that youve made recently and much of the time for that decision
you didn't have all the knowledge that you would like to have this is a sample scenario for
people to think of so suppose you uh one of your
06:30 - 07:00 classmates or colleagues is starting a nonprofit
and this is a big investment you're not sure whether to help out with that or if you'd like to
stay on your current path perhaps in Academia in order to make this decision you have resources
available to you you can do internet research uh you can talk to people however making
these decisions takes resources and time so you're doing this with limited knowledge
and with limited resources very similarly robots also do this they also have to make
decisions where to go what to do um from
07:00 - 07:30 the sometimes even the the low-level decision so
what the motors do and they have some knowledge as well which is different from how we think so
they might have Maps they do their thinking on the computer and they also may have some Physics
reasoning but they also have incomplete knowledge both humans and robots make decisions the best
they can with the knowledge they have available this brings me to the main two questions of
my research the first is how can robots make
07:30 - 08:00 decisions to achieve goals with limited access to
knowledge and resources such as data and compute and then how can they adapt their decision-making
as they have access to uh different sets of resources and knowledge sometimes it increases
sometimes it decreases and I also am interested in thinking about how we do that as humans so
we can also learn from the decisions we make for
08:00 - 08:30 robots these are the uh some of the takeaways
I'd like everyone to get from the talk um first is that the idea that we in robots can learn
what our limitations are in order to better achieve goals in order to do more the second
idea is that when we have limited resources we can combine them to be able to achieve more
and to be able to do it so better and lastly is we do learn and grow throughout our lives
uh and the idea of ways of expanding uh
08:30 - 09:00 capabilities in order to achieve tasks and
adapt to new needs I do expect everyone to uh I do expect everybody to reach these talk
objectives uh however there will be a couple points that may require some more background
knowledge I'll try to keep these brief and I'll indicate that with a a sneak peek icon
um but the talk is geared to be accessible
09:00 - 09:30 to all audience members right with that
I'm going to go into the context for my research which is what a using models for
decision making a model is a in this context is it's a function that looks at the state
of the world um which can be for instance a description of all objects and positions but it
can be something more General and then predicts given a action that the robot takes where
the robot will end up uh in this illustrative
09:30 - 10:00 example it's moving to the right and uh moving
down and then uh okay there's a bug um if it moves to the right then we expect the robot
to move to the right so this is actually an example of a an inaccurate model um when you're
planning you're deciding which actions the uh to take in order to achieve a goal so suppose
a a robot or an agent well in this case it would be you you're deciding how to get to
a location to get a cake you might choose to
10:00 - 10:30 take the six local you may choose to take the six
Express and that will affect how you get to the location um your goal is generally to find a
path to the goal but you may also want to find the shortest path um in for robots oh yeah so
this is an examp another example of incomplete knowledge so we have this robot we have the model
in this case what should the robot take to get to
10:30 - 11:00 the cake um describe with the arrows or right
left yep just go right now I'm going to so you predict that the robot will go right using your
intuition and now let's complicate things a little bit now there's a portal and you don't know where
uh where the portal is going to take you um what action should the robot take to get to the cake
question will the portal keep you on the grid uh see you're not guaranteed that information
okay I've been that uh okay so going around
11:00 - 11:30 so and with limited information you can still
make decisions to reach the goal and this is relevant to all of really all of life but
especially in robotics because knowledge of the real world is incomplete uh this is that
example I'm really excited about robot plant watering and there how does liquid interact
with plants we don't even know that well let alone have robots reason about it a way of so now
I'll talk shift into talking about a framework for
11:30 - 12:00 making decisions with limited knowledge so this
relies on a idea called the precondition um raise your hand if you're familiar with a precondition
from any other context um okay so I will describe here so a precondition is a condition something
that needs to be true before something is used so if you're going to drink water from a bottle
the cap should be off and there should be water
12:00 - 12:30 inside the bottle so these are preconditions
models can also have preconditions these are the conditions where you can use a model to
make predictions or just to use it in in this context more specifically it's the states or
the states and actions where a model can be used to make this more concrete with a portal
example you can trust the model in any space except for the one with the portal so the portal
place would be outside the model precondition um
12:30 - 13:00 and I will give a robotics example next and
then check comprehension here so suppose we have a model where it's something you can write
pretty easily in Python where if the water is above the bowl it will end up in the bowl and if
there's water that's outside of the bowl then it will uh then it will spill it will go outside
of the bowl um so this isn't an accurate World
13:00 - 13:30 model um the model let's now let's
add a plant um the model is accurate for a couple pores so an example here is
the pore where it's under the leaves so that's in the model precondition where's
an example of a scenario where we expect the action to and state to not be in the model
precondition where would this model not work
13:30 - 14:00 where the leaves are the umbrell yeah so over the
leaves for example can anyone think of any other examples okay um yeah yeah that could be in this scenario
so we're making an assumption about the Cup being empty um so this is model preconditions this is
kind of the the core of my of my research now I'm going to get more into to how we can do this
in a less abstract sense so how can we quantify
14:00 - 14:30 the limitations of our knowledge in a way that
is informed by data and that can guide decision making um the first project I'll talk about is um
is how a robot can learn model preconditions with just a little bit of real world data a little
real world data is especially important for robots especially with robots with um with water
around it because there can be something dangerous happening or can just be inconvenient if you have
to mop up a spill uh to make this more concrete
14:30 - 15:00 one method that I and some others have used to
get traction is the idea of model deviation so model deviation is the distance between a the
predicted next state when making a prediction and what the actual next state is then so in in the
portal example we might have expected the robot to move on to here and then to address Brian's
question in might end up off grid um and then the
15:00 - 15:30 deviation is one way of measuring that is with the
distance we can then Define model preconditions as in math terms as States in action such that the
distance between the prediction and the actual state is less than a threshold and this threshold
is uh it's a practical measure of how much error the system can tolerate so this can be how much
it can correct or how much is Just acceptable or just some really low number uh and as a result
we have a nice concrete way of defining the model
15:30 - 16:00 precondition so the distance is lower than dmax
then it's inside and if it's greater than dmax then it is outside think let's see all right
questions about model preconditions and model deviation all right and is and Rebecca are you
monitoring chat yes okay thank you so much okay now this doesn't uh one question you might have
is that this might this isn't doesn't seem useful
16:00 - 16:30 because in order to do this we need to know what
the state of the world is and if we already know that then why do we need the models well we don't
know what the state of the world is so I frame this problem as learning what the deviation
will be from data the problem is a supervised learning problem so that means that you have a
input and you have a desired output the input is the state in the action and the desired output
is is that distance and in your data you do know
16:30 - 17:00 what the next state is and you do know the model
prediction then this this uh estimator predicts what the deviation will be for that state in
action um which is a relatively easy learning problem in many scenarios um then you define the
model precondition as this is the sneak peek math way of doing it it is probabilistic and it's
the probability that the deviation is within the threshold and then you can reason about this
um getting out of the sneak peek another way of
17:00 - 17:30 thinking about that is the inside is where the
estimated deviation is probably lower than dmax then outside is that the deviation might be higher
than dmax not that it necessarily is but that it might be and then when we're planning we would
want to only use um only use knowledge where we know that it will be accurate which is inside
that model precondition oh it doesn't show on the
17:30 - 18:00 screen it's interesting okay I'll just have to use
my hand now if we're doing this and we're getting the real world data that makes it expensive
because robot data is expensive so another thing that we care about is to be able to estimate this
efficiently in a planning problem you have start States and you have goal States uh goal states
are where you want to go and then you might have a little bit of data uh this the dash line shows
what the estimate is so right now we can't connect
18:00 - 18:30 these these regions the problem cannot be solved
so our goal is to expand this region we want to do so in a way that's data efficient so not
collecting too many R World samples um a one way to do that is to ensure that it's relative that
it's relevant to the task so we can do just paths to the goal or attempts to get to the paths to the
goal we also want to cover more problems so cover a wide range of space and we want points that are
inform if that actually bring us closer to being
18:30 - 19:00 being able to achieve goals this is a sneak peek
of the intuition for it but hopefully some of it is accessible to everyone the key idea is that we
want points that are uh low in estimated deviation and high in uncertainty it's the key principle it
does get a little bit more complicated because a lot of decisions are sequential so if there's a
really informative point but you can only get part way and then there's a deviation then you've not
reached that that uh that point uh a the intuition
19:00 - 19:30 behind one way of addressing that is to weigh the
importance of those later points lower and then hopefully it will be able to make good decisions
um the way we evaluated this was to look at the success rate of achieving a task in this case
it's a plant watering task with uh more data so x-axis is effectively data y AIS is success and we
find that when we're using our method the the Top
19:30 - 20:00 Line we're able to achieve a higher success with
less data than if we don't use uh that reasoning about the use the U usefulness and if we're not
finding the pass to the goal we also try this on a real world watering task which is why there's
much more uh uncertainty and we finally we're able to get 80% success uh success rate on plant
watering when it was rarely able to do that before
20:00 - 20:30 with only about that was about four data points
um so that's a that's a very useful thing to have and here's an example of the robot doing one of
those uh plants where it avoids pouring over the leaves what we've learned from this is that it is
possible to efficiently learn model preconditions using knowledge of the task and informed by what
the deviation estimates are and we're able to get
20:30 - 21:00 that that nice real world result however in
this work we're assuming that we can solve the problem with just one model um in this work
I'll talk in the next work I'll talk about how we can expand what the what the robot can do
first by adding more models and then by adding something different with human demonstrations the
different models have different efficiencies and how it gets access to knowledge so suppose you're
assembling a an Ikea cabinet you can look at the
21:00 - 21:30 manual and that will give you some information
and then that will help you assemble it it might be faster where did the video go okay well uh
but it might be faster to just watch a YouTube video and similarly when making decisions about
water some models are really slow so simulators tend to be really slow and you might have some
math models that are really fast it's just an equation with sometimes you can learn a model
to make it more efficient and it's somewhere
21:30 - 22:00 in the in between um so my research is on how
we can combine these models in order to solve more problems and also do so more efficiently
now when we're dealing with multiple models then the problem of let's say there's a point
that's outside the model precondition well now we can add another model and now we can solve
more problems furthermore we can add models of different types so we can add a simulator which
might be B to solve a large range of problems
22:00 - 22:30 however we know that the simulator is slow if
we add a learned model that's maybe trained on some of that simulator data we might have
something that can solve some of those problems but much faster for this project I was thinking
about it in a uh a office cleaning up Tiding up domain where we have two math models very simple
one where it's just pick in place and one that Recons about the drawer but uh the robot in the
drawer but nothing else for more complex inter
22:30 - 23:00 interactions we we could use a simulator the
uh complicated part about this uh about this domain is that the rod can rotate so the math
model that just looks at Pi in place does not uh does not reflect that rotation which can lead
to failure so if it uh plans with that assumption that it's rigidly attached you get interesting
behaviors like this where it grasps the rod and then it accidentally Clos well uned
closes the drawer with the rod so that's
23:00 - 23:30 an example where we would want to avoid model
deviation however when we use our method with model preconditions we avoid that scenario
so it picks uh scenarios that it picks States and actions that the model is that uh that it's
accurate for and that also achieve High success um I just noticed that my laptop is not
23:30 - 24:00 charging just maybe that a table it's just yeah right thank you for that yep thank you okay all right uh another
thing that we can do is we can reason about the speed So In classical planning
there's some algorithms that if you select if you trial actions from a state it
will let you uh compute a plan to the goal and
24:00 - 24:30 it find it guarantees that you'll find the plan
if it exists and also that it's the lowest cost so this gives you a guarantee on the quality
of the plan uh I was able to apply two tricks to switch off between models the first trick
is quite straightforward which is for each uh state in action we we're predicting the next
state we just pick the fastest model such that the model is in the precondition so if this is
the fastest model FN uh and F0 all the way on
24:30 - 25:00 the left is the slowest and the checks and
xes indicate which um if it's in the model precondition or not um raise one finger for use F0
two fingers for use F1 and three fingers for use FN okay yeah so we want to use f uh F1 since it's
the the fastest that the uh that such but where
25:00 - 25:30 the model precondition still holds however we can
make it faster by also just prioritizing search with faster models an example that illustrates
this problem is we're trying all actions and let's say one of them is really slow and one of
them is really fast to get these guarantees much of the time we need to wait for the slow model to
finish Computing in order to continue the search and sometimes that can become a bottleneck so
if we can reason about when we can find Optimal
25:30 - 26:00 paths to avoid the goal we can uh to avoid that
to Optimal path to find the goal without using that slow part then we can speed up search and
the intuition for how I did this was we have a CU so you can think of it as states that are
waiting to be EV expanded or evaluated we can wait for those slower points to be evaluated later
when we know that it will improve the quality right so this this was about how we speed up the
searge so let's just make sure that this leads
26:00 - 26:30 to high success and also is fast so we do find
that if we use both tricks that we achieve a high success compared to baselines and we also get the
speed up compared to if we don't use those tricks so if we use the trick we get three compared to
just the simulator is about 100 seconds which is about what like a um a little over 30X feed up
and here's another sneak peek with this type of
26:30 - 27:00 planning you can get an optimality um you can get
a upper bound on the suboptimality of the plants and the user can specify how much suboptimality
is allowed while making these computational trade-offs going back to the big idea this is a
way of combining incomplete knowledge in a way that gives us more reliable plants more efficiency
and still gives us guarantees but we're still assuming that we can solve the problem with now
our current models sometimes it's good to we can
27:00 - 27:30 try solving a problem with a method we know but to
really grow we need to try learning new skills so for a while I was a believer in always just using
models for planning since we understand them well but there was a big Trend in uh robot learning
where a robot can just learn from data to their um so modelbased planning is what we showed before
we have a model we send it to a planner which
27:30 - 28:00 gives us the action to take but in model-free
planning we don't use a model instead we have a machine learning skill that goes directly
from what the status to what the action is and generally the way one way it can work is by
having a human demonstration where uh given the this the current state and maybe some image data
a human shows what the robot it should do and then it learns what action it should take in those
scenarios now there's a bit of a data problem here
28:00 - 28:30 where if we want to have something that works more
generally then we need more data points which can be especially expensive with humans so having a
smaller precondition and less data but still being useful is really important uh I'm going to show a
demonstration of kind of the the training what the training looks like and then the testing so during
training time the robot plants tries to execute its plan and then if there's a problem it will
ask the human for help in order to patch what
28:30 - 29:00 the deviation is with that skill at test time then
it either plans to the goal or to the be able to do the skill then executes the plan until it can't
and then it executes the skill knowing it can to ground this in a real world in a a real example
this is for a shape insertion task the left shows the robot planning um so kind of thinking about
what going to do the robot tries to fit the block
29:00 - 29:30 into the hole but then it finds that it's not
able to complete it and so at this point the human demonstrator takes over and completes
the task by sliding it in that's that's the skill I get it eventually um then when we test
out the method instead of trying to go directly into the hole it goes to the beginning of the
skill which is kind of at the corner and then it slides it right into the hole so we can get better
performance at test time so we get much higher
29:30 - 30:00 reliability compared to just doing pure imitation
learning or planning with very little data we're able to get a high success rate again with about
five demonstrations here um sometimes if we're lucky the robot doesn't even need to to learn a
skill and this is something that this framework also addresses so if a robot was planning with a
simple handle the we kind of we can expect this plan to work work um but if you switch it out with
a decorative handle sometimes it will still work
30:00 - 30:30 uh and our method evaluated how reliable that
was and decided no skill was needed in summary there's a we can patch modelbased planning to get
the benefits of both with the demonstration and with efficient use of data um however we're still
assuming that we're we're still using deviation to indicate whether the model is useful which if you
think about it is not always the case and we're
30:30 - 31:00 also not really learning more from new data which
is something that robots can do so finally I'll talk about some more recent work U which is on
expanding what the the capabilities of robots the first is in well we can just gather more data this
problem is on if we have a simple Model A simple learned model in this case how do we adapt it to
a more complex environment let's say we have a simple uh rope model that we trained in a Sim and
there's no obstacles but we want it to be useful
31:00 - 31:30 to hook a rope in an engine bay we want to focus
on the parts of the D dynamics that it already sort of knows in order to expand it um however we
may not be able to solve planning problems yet now we may have some goals where it's actually on the
hook but right now it currently can't solve the problem this approach seeks to expand that model
precond condition in the adapted learn model so
31:30 - 32:00 that way it can solve more of those planning
problems in the complex domain the intuition for our approach is based on an idea from
pedagogy some people call it the zone of proximal development where when a student is earlier in it
uh in their learning they would might benefit from more help or lower difficulty tasks but if it's
too if it's too difficult they may not progress and if it's too easy then they're they may get
bored or may not learn much so the idea is to be
32:00 - 32:30 in that that sweet spot the um so we do this by
prioritizing learning on points that are easier so that's described as low deviation in the Rope
example low deviation means that it's uh usually in free space so it's not touching any obstacles
but in some points it does have to touch obstacles we'll start with some parts that are similar which
are easy some that are difficult and we'll start with the DAT the data in the easy region and
some points may not be useful but we'll run
32:30 - 33:00 into them however uh the the darkness indicates
how important the point is to uh to the learning right now over time by focusing on the easying
easy points it eventually learns more so that more points become similar and then it iteratively
becomes better at the task uh however it's it's we're not expecting this to learn over the whole
the whole Space not all transitions there's still going to be some parts of the data that the robot
is really bad at predicting so maybe Parts where
33:00 - 33:30 it snags on the on the engine and some difficult
places this is where the model precondition becomes really helpful we don't need that to be
able to solve problems we can just avoid the parts where it's uh snagging on the motor uh we wanted
to make sure that we can achieve High success with small amounts of data so we tested success rate
over data again here the method is called Focus because it's focusing on a region of space um
we evaluated with uh mostly comparing using
33:30 - 34:00 this focusing versus using all of the data and we
found higher uh higher sample efficiency on a uh in the engine bay um on the engine bay task when
using our method and in a plant watering example we also found a similar Trend where it was all
all methods were able to learn in this case but our method was able to do so more efficiently so
in this work we've shown a way to do um a data
34:00 - 34:30 efficient adapting knowledge in a way that's data
efficient and that if we get more data we can get improved performance by learning strategically the
limitation in this work is that we're expecting that there's going to be some parts of the task
that are quite similar and we're still using deviation to describe if the model is useful this
NEX workk is a way of expanding in this it's only going to be one slide uh so all of this is kind
of a peak this is really exciting work that I'm
34:30 - 35:00 doing with uh some students oh I should mention
the last one is also a collaboration with the University of Michigan um they're they're great
at rope uh rope manipulation uh this is a project also with the same lab different student and uh he
he has a big very big ambition so um we eventually settled on this where we're doing task guided
model generation uh we have a state okay the goal is we want to get a Dynamics model we want
a model to do this prediction and we're making
35:00 - 35:30 this model we want the model to be efficient so
not use too much compute but also still compute high quality plants our input is a description of
the task so we want something that can generate these models from a description of the task um
the way we describe the task and this relaxes that con that idea that deviation and success
are the same because sometimes you can have a mistake and still make good progress towards
your goal there's a lot of problems that are like
35:30 - 36:00 that so this this problem seeks to address that
to make it a little bit more concrete this is a tree and we represent the tree with a graph and
the idea is when it's when it only needs to move one branch we would hope that it doesn't need to
model the UN unrelated parts of the tree with as many nodes so this can save us on computation in
practice this is actually really complicated and
36:00 - 36:30 it depends on what the model you're using is so
what we do is we have the robot practice it tries to plan and tries to achieve tasks and it does
this a lot uh and it does it in simulation the then to do the model generation we use a diffusion
model which is a technology you might have used to generate images based on a prompt but instead
of generating images we're generating a model in the form of how we're simplifying this uh the tree
so we specify a way of of simplifying it and then
36:30 - 37:00 that is generated in uh using the context where
the context is what the task is so for this robot the task would be say where the tree starts where
the robot is and where we want the tree to end up uh and so far we have some good promising we have
promising results on this where it can get similar cost as the as a full model and we can get a a a
reasonably significant speed up when using this method um so that that's just a sneak preview of
a project I'm really excited about um then finally
37:00 - 37:30 I'll talk about some future research ideas I would
love to uh discuss further with people if we are not already scheduled you can email me and these
are all still in the idea of decision- making with limited models so uh starting off this is just
kind of a high level idea of what I'm interested in which is the interplay between uh knowledge
and decision- making and all of my uh my research
37:30 - 38:00 focuses on adapting knowledge to changing needs
and resources some sub areas in that first is transferring knowledge in a way that's resource
efficient from task to task and I'll go into more detail for each of these doing so in a way that's
modular so we can add capabilities throughout the robot's life time or even throughout
a Technology's Lifetime and finally is
38:00 - 38:30 in individualizing the capabilities of the robot
in a way that's it keeps the human in the loop so the robot can do what they want it to do or
at least so they can understand the limitations that the robot has um really more important to me
is actually the the values that I'd like to have my research and other work uh reflect so first
is that I think we're I I really would like to move more towards uh Environmental sustainable
technology and other sorts of technology that
38:30 - 39:00 benefits Humanity uh so this is somewhat on
the application side I also would like to lower barriers to more people participating in uh
in technology so robotics AI machine learning so I'd like to build in a way that includes more
people and finally uh related to that is to use platforms so uh technology that everyone
has access to so something that's low cost an open source so anyone can build so first this
is in uh adapting the robot's knowledge according
39:00 - 39:30 to to new needs and resource constraints um these
I'm just going to describe a couple General ideas that hopefully can start some conversations later
if you're interested so first is we have a model and we want it for specific use case so maybe
we want to deploy it onto a uh a small device then we can still make it useful and shrink the
model precondition and the question is how should we do this so that way it's still useful for the
task and can still be deployed on um on a limited
39:30 - 40:00 compute the next direction is in um transferring
from task to task this is something that builds pretty directly from the last project if you have
a robot that can do a kiwis how well can it work on grapes and how can we facilitate that transfer
or even if it's something it's very different so Recycling and there's some uh there's some
technology from uh like the language models that we can leverage to make progress there
and then finally is this one's more longterm
40:00 - 40:30 which is if we have general purpose models how
can we find a smaller version that can achieve a specific task and create more of a general
framework from that related to this is to add capabilities in a modular manner so we were always
adding things that the robot can do so an example is let's say we we want to do a cost benefit
analysis maybe we want to change a component
40:30 - 41:00 with an open source an open source component
what how will that change our capabilities um this this comes from a there's some communities
that are really uh passionate about creating open source technology that interfaces nicely with
each other on both the on robotic side and also on the machine learning side um and there's
some people that have started work on doing this but we're really at the very beginning
of having modular technology for robotics that leverages machine learning um and the second
research area is in combining different forms of
41:00 - 41:30 knowledge so the great can take on different forms
and how can we U share this type of knowledge so that way we can leverage different forms of
knowledge um this is so I think code for example is a very nice way code and natural language
of combining say visual knowledge and um and more low-level uh more lowl tasks um and finally
is how we can work together to add capabilities
41:30 - 42:00 to a system and so the the general idea here is
let's say we have people in a robot and people have an idea of what they want the robot to do and
the robot doesn't really have an idea of what it can do but I'd like to build something for that
but do it in a way that it can the robot and the human can communicate about this idea of model
preconditions um there is some work on that of showing what the visualizing capability
limitations in both Robotics and in broader
42:00 - 42:30 broader machine learning that we can build on and
this one is a little bit more out there which is how we can individualize what the system can do
not only what are the limitations but how can we move the system more towards the capabilities
that the human cares about so that way it can act align with their interests and with their values
and yes so this is the summary of some of the um research areas and the common values uh questions
on uh clarifying questions on some of the future
42:30 - 43:00 work all right so finally I would like to thank
my collaborators I've been able to work with a lot of great people across a number of different
areas and different institutions uh and they've been really helpful for for mentoring me
and teaching me how robotics ml research and we're able to make these projects and
many others um possible so I thank them a
43:00 - 43:30 lot and hopefully you've also learned something
from this as well and that we can make DEC good decisions with limited knowledge humans and
robots can do this so hopefully you've gotten some of these takeaways so ways that robots and
humans can use limited World Knowledge to solve tasks how combining capabilities can bring
us closer to achieving tasks with limited resources and finally is finding strategic
ways of growing capabilities to achieve
43:30 - 44:00 tasks um yeah so I'd love to keep in touch if
we're not already scheduled to chat today there's different ways of contacting me if you'd like to
uh discuss any of these uh any of these research ideas or connect on any sorts of shared values
we have around inclusive undergrad research or uh robotic inclusive robotics uh education um
I'll also be at the SIGCSE conference later this
44:00 - 44:30 week if anyone else will be there that's the end
of my talk and uh happy to take longer questions now than. [Rebecca Wright] We have some time for questions uh if
you're on zoom and would like to unmute and ask your question you can do that you can raise your
hand first and we'll call on you um uh or uh if you want to type something to the chat we will
uh read that for you and uh in the room of course you can use the traditional way of uh raising
your hand go ahead. [Questions from audience] um I that's probably a basic
44:30 - 45:00 question what SE of sensors do these have typical
is or or actually what like what we see on the screen or oh yeah um my robots actually don't use
that much so I usually use cameras I'll put Azure connect cameras preferably two views the robot
these robots happen to have really good s internal sensing so they can tell where the arm is and
some amount of information about the forces sorry
45:00 - 45:30 oh and the scales there's uh to measure how much
water came out yep it use scales thank you that's a lot okay sensors AR you mentioned like the tiny
Mount to point on smaller devices uh what type are you like using like model compression methods or
are you just designing like different like smaller um Sol I think actually my research would
probably fall more into the once you have
45:30 - 46:00 compressed well starting with once you've
compressed a model um describing the model pre the model limitations around that but
I think the next step is to connect the capabilities and the model the model compression
process so that graph Network the graph project is somewhat related to that of what can you
remove that decreases the size of the model but also doesn't affect task
performance too much Beyond
46:00 - 46:30 prediction I was wondering do you have a uh do
you have a a question in mind about yeah I was yeah I was just wondering if um like in that
kind of goal of trying to I guess prune away things that uh will allow you to compress
like are you doing like quantization wear training are you doing like model printing
uh to get that kind of lightweight kind of
46:30 - 47:00 uh while maintaining that high ACC yeah I think
it would be important to explore different ways of bringing it down I think what would be
the most interesting is connecting the uh what the model uh especially model pruning
I think that one's something that's more straightforward of what that model affords to
the task um and that way we can we can guide the simplification process if we can do just the
generation that would be kind of a dream where we
47:00 - 47:30 don't have to simplify a model we can just start
from the task and then generate the model and not have to compress anything I don't know how to
do that that would be really cool hopefully we as Humanity can move towards that yeah that would
help with your environmental sustainability point but not massive massive massive yes yes that is
that is huge robot learning is really moved to moving towards we just need more data more compute
at least my community is very very into that and
47:30 - 48:00 it's really even when you go outside when people
think well the environmental impact of training these is significant at conferences actually
when people ask that that question sometimes gets ignored I question for this the part of we're
talking which I think it's an important problem in learning in general like learning from
easier data points to generalize for harder examples can you go back to that side because
you have two different problems and different
48:00 - 48:30 and what was the difference in the type yeah
of the easy example what what's the difference between an easy example and a hard example the
gap between that I think yeah this this is like a very um very practical way of doing it but
I think it's a good start here we go okay uh in the results if you can go to the result oh
in the results okay yeah like different yes
48:30 - 49:00 yeah these are two different problems that we
tried it on the uh first is the that b manual rope manipulation a rope manipulation task in the
engine bay so that's just a completely different task so we learned completely different models
it's a different adaptation process we just deployed uh we tried the same algorithm in these
two different domains M okay because the nature seems to be I'm the theor means and I was just
curious in terms of what's the nature of the
49:00 - 49:30 examples of the EAS yeah so this actually um this
is a place where it diverges somewhere based on the problem characteristics so in this problem
it was able to eventually make progress uh so this was uh do I have no I don't have an image of
that one it's pouring but when you get it wrong you can still make some progress which is why
all of the algorithms eventually make progress whereas in that real world domain at least one
of the baselines basically never solved the task
49:30 - 50:00 ever and that's because if it gets stuck in
a particular way or if the model precondition so we're we this is like a under we're assuming
that we do not want deviation at test time so if it cannot solve planning problems then it's not
going to execute anything so if it never makes progress during training it will never compute a
plan under the constraints the goal so that does change the it's a way that the problem character
characteristics changing how the algorithm
50:00 - 50:30 behaves do you think there's a way to understand
that impact and like fre guess what kind of problem space you're in that is oh yeah that that
is the that's also the dream I did start a little bit of this in my uh in my masters where it was a
question of what are the problem characteristics that lead best to good performance on De deep
reinforcement learning and the I think one of
50:30 - 51:00 the key challenges is in describing what the
task is describing what the constraints are there's been some progress in describing it
with you feed an image to A system that has knowledge of the planner and then it can predict
that it can uh solve the task itself but you asked something that was more um yeah there was
another layer on top can you repeat the question please well I was just you know it seems like
what was just coming out of this conversation
51:00 - 51:30 is that the problem characteristics have like
deep impact on the performance of the approach right like your approach is is is doing better
always but like just in general like problem characteristics have an impact on different
approaches and I was just thinking out loud like is there is there a way or do you have ideas
about a way where you could start to before you can start understanding like sort of the the
shape of the way the learning is going to go how does like I give you R problem like are
is this a good one an easy one bad one yeah
51:30 - 52:00 it's that that one's really interesting because
there's a lot of factors at play there's some of it comes down to the optimization problem so if
it a lot of it is if it's just spiky if there's and there's easy ways to get stuck and never make
progress then it's a it's harder to do and then if there's something that's more smooth then it's
more likely to make progress um so that's one but I think broader is really difficult especially
when the yeah so spikes and spaces make problems
52:00 - 52:30 harder I think that's something that we've also
we can characterize as problem difficulty there's some there's ways you can get it at from a
learning theory standpoint um for this for the spikiness part there's a measure you can use
of how much the function changes over a distance that you can use in some methods that make guar
guarantees about controllers um what else but I
52:30 - 53:00 think for broader ones that's that that's quite
open and I think largely will come down to do we have ways of specifying constraints and problems
in a way that applies to a lot of problems so we can combine knowledge about performance of
those systems back to those task specifications but yeah that is a very very interesting
question all right [Rebecca Wright] Great thank you again Alex!