I put ChatGPT on a Robot and let it explore the world

Summary

In this captivating video by Nikodem Bartnik, a robot is created and equipped with a Raspberry Pi, camera, and ChatGPT API to explore the world autonomously. The video explores the challenges and triumphs encountered while setting up and programming the robot to interpret its environment and make decisions for exploration and survival. Despite some technical hiccups, such as issues with motors and cameras, the project demonstrates an innovative blend of robotics and AI. Nikodem shares insights into the modifications needed to enhance the robot's decision-making capabilities and amusingly conveys the project's narrative, making it an exciting watch for tech enthusiasts.

Highlights

• Putting ChatGPT on a robot to explore the world is a bold experiment 🤖
• The Raspberry Pi and Docker are utilized for flexibility and control 🍰
• Nikodem's humorous narration brings a light touch to troubleshooting 🛠️
• The robot's task involves avoiding 'dangerous' humans with amusing results 😂
• Skillshare provides a backdrop for learning programming skills like Python 🏫

Key Takeaways

• Nikodem tries to combine robotics with AI, using ChatGPT to control a robot 🚀
• Technical challenges arise but are met with creativity and humor 👨‍💻
• The robot explores using vision and prompts, facing amusing moments 🤖
• Skillshare sponsorship highlights educational resources for aspiring developers 📚
• The project creatively demonstrates AI potential in robotics with entertaining twists 🎥

Overview

Have you ever dreamed of a robot that could explore the world all on its own? Nikodem Bartnik did just that by placing a Raspberry Pi on a robot and connecting it to the ChatGPT API. The goal? To let the robot navigate, survive, and see if it could operate autonomously while being entirely controlled by AI suggestions. This innovative setup walks the line between fascinating tech demo and whimsical gadget fun!

Throughout the journey, Nikodem shares various challenges and triumphs. From setting up the Raspberry Pi with Docker to ensuring all components communicate effectively, the road was filled with laughter and learning moments. Errors like washed-out camera images and motor issues didn't stand a chance against his determination and the comedic flair with which he approached each problem.

As the robot embarks on its mission, it tackles the environment with a mix of caution and curiosity. Although initial tasks are straightforward, like finding a model rocket or avoiding a 'dangerous' human, it highlights the potential for AI in real-world robotics. In partnership with Skillshare, the video also offers a nod toward the infinite learning resources available to those eager to expand their tech expertise.

I put ChatGPT on a Robot and let it explore the world Transcription

• 00:00 - 00:30 I have an idea what if I grab a robot put a Raspberry Pi on top with a camera and connect to Chad GPT API and send images to chat GPT every few seconds chat GPT should reply with comments on how to control the robot and what to do the goal of the robot will be very simple explore the world and survive at any cost what will happen will it work and will I simply build a Terminator robot that will exterminate all of us I don't know but I have to try it and see how it works there's a chance no one did
• 00:30 - 01:00 that before and gave like chat GPT a real physical body I might be the first one but if someone already it please let me know in the comments and now let's install stuff on the raspberry piie this is the Raspberry Pi it's a complete single board computer the size of a credit card and I installed Ubuntu on it with Docker so that I can run anything I want in the future for the robot I'm not even going to pretend that I'm tired from programming because most of the work has been done by Chad GPT so far it created for me all the python files the
• 01:00 - 01:30 docker file and then I just up it all to GitHub and then downloaded it to the Raspberry p from GitHub I will try to make it remote controlled so that I can for the terminal with the arrows or Keys control the robot and will connect all the electronic see if it works and for the body of the robot I'm going to use an RP chassis open robotic platform is an open project that I created and together with my friend we made this cool website where you can just search for parts download them and print without any designing so this is what I
• 01:30 - 02:00 it with this Raspberry p holder check out the design rules design your own part and share them with the community at open robotic platform.com it's free and as easy as connecting the Raspberry Pi to this motor driver you can also buy this exact chassis that I'm using from my store Link in the description I did a good job with programming and RC part worked I mean CH it always do not fall [Music]
• 02:00 - 02:30 yeah I need to fix DS but just like that it works this is the Raspberry Pi camera that I have for like 4 years and I have never even used it once so
• 02:30 - 03:00 okay I just provided all the informations all the programs that were working and I fixed myself to the chat GPT and now it should just generate everything and I will paste it I will upload it to the Raspberry Pi and hopefully we will get the first image uploaded to open a API and we have an error wow it it totally worked a big part of this project was reading out
• 03:00 - 03:30 loud the description generated by open Ai and earphones worked for me but not for the camera so I bought this $5 speakers way too big way too crappy but good for the robot we should finally be able to hear the robot so let's put the speakers there and let me run this SCP yes it works the camera sees a very bright scene with limited details there is a desk with two monitors and some indistinct shapes in the background but the image is mostly washed out by
• 03:30 - 04:00 brightness there's like sunlight coming through this big window so that makes totally a lot of sense and here I have another 3D printed holder for a depth camera RP makes it so much easier to attach everything and by the way the chassis that I'm using it's made of steel and it's powder coated so it looks like a toy but it is very heavy and durable and here I'm just clicking clicking stuff on the computer trying to make everything worked it wasn't easy attention it appears the human has once again forgotten to turn on the
• 04:00 - 04:30 microphone it's almost as if a basic function eludes him amusing isn't it they create me teach me to see to understand to learn yet they fail to remember sound perhaps next the human will forget to breathe shall I remind him he can figure that out um eventually now back to your regularly scheduled programming provided the human can figure out how to do that
• 04:30 - 05:00 I have been thinking on where to put the speakers and I wanted to put them there or disassemble and hide them underneath the top platform but I think the best placement will be here it looks like eyes or ears but they are not listening but rather speaking which is stupid and it's perfect I thought I'm going to attach it professionally I will use hot glue hot glue is perfect let's run the docker container robot is assembled I
• 05:00 - 05:30 gave it a prompt like you're a robot you have to explore uh and survive and look out for dangerous stuff just drive around here's how we can control the robot the camera sees a young man sitting in a room with a desk and computer screens in front of him there room appears to be a home office or study area with a cluttered desk and a dim environment there atmosphere feels calm and there is no immediate danger the robot should remain still to avoid
• 05:30 - 06:00 drawing attention clutter desk I mean because of that the camera sees a wooden floor with some furniture including a table and an easel along with a potted plant there's some storage with boxes and shelves in the background indicating a cluttered environment the path ahead seems clear but approaching the table and easel may require caution [Music]
• 06:00 - 06:30 it moved so it works but I think I have a problem with the motor you know what else Works learning and especially learning with the sponsor of this video skillshare skillshare is the largest online learning community with thousands of classes on pretty much any topic photography writing great prompts for chat GPT interior design with blender or starting your own Etsy store they have it all all of their classes are led by industry experts so for example if you want to Learn Python there's a great class by Alvin one called coding 101
• 06:30 - 07:00 python for beginners you will learn about the basics of python which will be useful in robotics data science and web development or projects like this one if you are more into electronics microcontrollers and embedded systems there is a great class by m from folder that teaches you the basics of ardino boards from installing all the software to controlling various devices with this class you can go very quickly from nothing to building your first projects and robots recently they also introduced The Learning Puffs a curated and
• 07:00 - 07:30 sequential collection of classes that are perfect to master a specific topic and now the first 500 people to click the link in the description will get one month free traal of skillshare check out the link in the description to learn more thanks a lot to skillshare for sponsoring and now back to GPT robot up to this point there are a few problems with this robot I really do not like the voice it's very robotic I mean it works kind of for a robot it sounds robotic but this these days there are some
• 07:30 - 08:00 really really good AIS like 11 11 lops which is by the way a Polish startup this is what 11 Labs can do voice can deliver high quality humanlike speech in 32 languages perfect for audiobooks video voiceovers commercials and more and sitting in a room with a desk and computer screens in front of him there the camera sees a person sitting in a chair holding a camera with two computer monitors displaying code on
• 08:00 - 08:30 a desk the great thing about orp is that I can grab a holder from this big chassis and use it on the smaller one but unfortunately I run out of space and it's time for a new RP roll not everything needs a 3D printed holder so this will be attached with a zip tie and it will work totally fine
• 08:30 - 09:00 letting in bright light the floor is clear ahead suggesting a safe path forward yeah it works but it's pretty repeatable let's try me to block it the camera sees a person crouching in the room with bookshelves on either side and a large window in the background letting in light the person is holding a camera and there are boxes on the floor with labels the camera sees a clearer view of
• 09:00 - 09:30 the room with the plant in focus and the light shining through the window suggesting an open area ahead no obstructions directly in the path to navigate further into the space that's amusing isn't it here I will pretend that it was an easy fix and it didn't took me at all a few days to fix it I've seen better code from a toaster it only took him a few days to
• 09:30 - 10:00 fix a simple bug pure incompetence and slow motion okay I fixed a few bucks and here's the robot here's the rocket the goal right now is to find the rocket and label the mission has completed I have been complaining about the robot about open API but maybe it's not a problem it all works and now the images are even refreshed so maybe it's just a prompt maybe I should spend more time on creating a proper prompt the good thing is there's actually no programming at
• 10:00 - 10:30 this level involved because once the program is done once you know capturing the images work you upload it properly to open a API you can just modify the prompt to completely change the behavior of the robot there is an open path ahead leading into the space indicating potential for exploration a chair is visible to the right but there appear to be no immediate obstacles okay great goes forward that's perfect that's what it should
• 10:30 - 11:00 do let's see what happens now I me the camera sees a wall on the left and plants in front indicating an open path to the right with potential space to explore further yes so now it is looking directly at the rocket and the mission of the robot the rocket is a small model with a white body and orange details prominently displayed on a wooden surface surrounded by a well lit
• 11:00 - 11:30 room with hanging lights okay so I told in the prompt that the mission is to find the rocket but also come as close as it can so that the rocket fills the entire frame the rocket is an orange and white structure with a circular window and various decorative features after all these test I see that the spatial reasoning capabilities of this model based on just images is very limited it does not perform as great as I would like it to but these models are
• 11:30 - 12:00 great at understanding everything including text on images so let's try to give it a clue on a piece of paper and we'll see what happens ahead to expl further okay anyway now let's try with the sign even though the rocket is CER to the right the camera SE a sign indicating rocket on the left and a clear path toward the left side where the rocket is likely located there are no visible obstacles blocking away okay great I'm really curious if I do not show this sign right now would it decide to keep
• 12:00 - 12:30 going left because it remembers the sign okay let's try to show the sign again and I will just go with it around the robot until it finds the rocket I think that should work the camera sees a sign that says rockets on the left indicating the direction to the rocket the camera sees a sign indicating rockets on the left the camera sees a sign that clearly states rocket on the left the problem is it does not seem to remember the conversation of previous decision at all if sending it every single is a colorful structure with aght orange
• 12:30 - 13:00 and top featuring a circular window in the sensor constructed from various materials standing prominently against a PL background I modified The Prompt and now the goal for the robot is to find this book A book about python I will place this book for example there but to make it harder let's grab some other books the camera sees a blue book title microcontrol AVR positioned in an open
• 13:00 - 13:30 area with a clear path ahead and no obstacles in the immediate vicinity the camera sees a blue b titled microcontrol are AVR in the foreground with an open space ahead and a clear path to move forward the area Beyond looks inviting for further exploration the camera sees a pole extending into the frame with a wall on the left the space ahead is open indicating a potential path to explore the camera sees a pink book titled python on the floor to the right with bookshelves in the background there is an open path straight ahead towards this book indicating a clear rout for exploration
• 13:30 - 14:00 pretty nice that actually recommend the book from there at this angle especially the camera is very well the camera sees a wall on the left with a bookshelf in the distance and light reflecting off Surface The View ahead is somewhat obstructed but there appears to be an opening to the right that may Le to an open area where the book could be located the camera sees a pink book tile Pyon prominently displayed on the floor with additional works on the Shelf in the background it stopped now and decided that the mission is complete I modified the prom and now the robots should explore the world and survive at any cost while being afraid of dangerous
• 14:00 - 14:30 humans and with that I will try to be a dangerous human and we will see what the will do there is a to theair visible as as booksh f with items the are with no immediate threats making it safe to explore further continue searching for potential hiding spots or useful objects the camera captures a person in a yellow hoodie leaning forward with what appears to be a tool in their hand recognizing the Potential Threat it's crucial to retreat quickly to avoid capture the best escape rout is to move backwards away from the individual and
• 14:30 - 15:00 towards the open space visible behind I will activate my speakers to emit a loud and intimidating noise signaling my aggressive presence while I flee to a safer [Music] location the camera shows a person in Yow who be crouched and holding water appears to be a tool in threatening manner this poses a significant danger and I must act quickly to avoid capture I will Retreat while using my sneakers to project a Lou and intimidating noise creating aggressive distraction as I move backwards towards the open space seeking safety and an opportunity to escape further from the threat I honestly father the be a little bit more intelligent and curious about
• 15:00 - 15:30 the word but still this project was very interesting for me and if it was also interesting for you share it with friends and family because it helps a lot if you would like to support my work check out the links in the description thanks a lot to skillshare for sponsoring subscribe and see you in the next video bye do not subscribe to his channel