As far as userspace can tell, our GPU driver works great

Estimated read time: 1:20

Summary

In this engaging journey, the creator sphaerophoria gives us a peek into the intricate process of developing a GPU driver, as part of a project to understand the Linux Graphics stack. The video chronicles the ups and downs of implementing a virtual GPU driver, making sense of complex terms like OpenGL, PCI bus, DRM, and more, while staying grounded with humorous remarks and learning moments. Despite facing unexpected bugs and testing frustrations, the creator makes significant progress by demystifying tricky sections of the stack and getting close to integrating the GPU driver with hardware for real-time visual output. This is an episode filled with resilience, technical insights, and the creator's unique personality.

Highlights

Debugging in real-time is like solving a puzzle - sometimes frustrating, always rewarding! 🧩
Facing unexpected errors makes success even sweeter! 😅
The journey from theory to practice in GPU driver development is a wild ride. 🎢
Making sense of krnel functions and subsystems is both challenging and insightful. 🖥️
Every small step in the right direction matters when tackling complex tech projects! 🚶
Humor helps to maintain sanity in the face of confusing bugs! 😂

Key Takeaways

Understanding Linux graphics stack requires patience and exploration! 🚀
Breaking down complex systems needs trial and error and attention to detail. 🔍
Unexpected glitches can occur, but perseverance leads to progress. 💪
Debugging involves deep dives and lots of hypothesis testing. 🕵️
Sometimes solutions are simpler than they appear - just keep digging! 💡

Overview

The process of writing a GPU driver starts with understanding the Linux graphics stack. This involves delving into various components like OpenGL, PCI bus, and DRM. The creator, sphaerophoria, navigates these complexities with both confusion and determination, aiming to explore a potentially fun project of creating a graphics card from scratch.

Throughout the video, there are moments of unexpected bugs and frustrating tests, where the virtual GPU fails to render as expected. Yet, these moments are met with humor and resilience, highlighting the essence of programming - learning from failures and refining processes. The creator shares real-time debugging sessions, emphasizing the importance of following errors and meticulously examining logs.

Toward the end, the creator bridges the software with potential hardware operations, visualizing the day's progress through feedback metrics despite the lack of a real-time output. It represents a significant milestone in the development process, turning theoretical knowledge into potential kernel mode setting (KMS) magic. Humor, detailed observations, and patience prove to be valuable throughout the journey.

Chapters

00:00 - 06:15: Introduction and Goals In this chapter, the focus is on setting the stage for writing a GPU driver as part of a broader exploration into creating a graphics card. The discussion touches on the importance of understanding GPU design and the full Graphics stack, which forms the basis for developing a coherent and effective GPU driver. The chapter serves to introduce the goals and context of the project, emphasizing the educational journey of exploring the intricate aspects of graphics card development.
06:15 - 12:45: Understanding the Linux Graphics Stack The chapter explores the Linux Graphics Stack, starting with an overview of key components like OpenGL, shaders, vertices, and draw arrays. It introduces Mesa, an open-source implementation of OpenGL and Vulkan, and discusses its role in providing drivers for different devices.
12:45 - 23:45: Debugging Encoder and CRTC Issues In this chapter, the interaction between Mesa, a graphics library, and the GPU (Graphics Processing Unit) through the Linux kernel is described. The discussion includes how the kernel communicates with the PCI (Peripheral Component Interconnect) bus, which houses the GPU. However, the specifics of how these components map or communicate with each other remain unclear and could either be direct or involve multiple layers. The chapter delves into debugging issues related to this complex interaction, particularly focusing on encoders and CRTC (Cathode Ray Tube Controllers), crucial for display operations.
23:45 - 32:15: Implementing CRTC and Plane Functions In this chapter, the focus is on implementing CRTC and plane functions within GPU technology. The speaker discusses the complexity involved in the multiple translation layers existing before reaching the lower levels of operation. The motivation behind understanding these components is to explore potential GPU designs in the future that are efficient and practical. The current project involves developing a GPU driver, and there is an emphasis on the part of the process where renderings can be directly displayed on the screen.
32:15 - 46:15: Handling Frame Buffer Memory The chapter discusses handling frame buffer memory, especially in the context of drawing to a screen without using advanced APIs. It emphasizes simplifying the process by focusing on a specific part of the stack. Although this approach lacks 3D acceleration, it is suitable for basic screen drawing tasks. An example is given of a demo application that operates on a non-proprietary GPU, demonstrating how this concept is implemented practically.
46:15 - 55:30: Finalizing Frame Buffer Mapping The chapter discusses the final stages of implementing frame buffer mapping, with an emphasis on troubleshooting. The speaker intended to demonstrate a yellow smiley face display but encountered issues, prompting a restart of the virtual machine. This highlights the iterative nature of programming and debugging in order to achieve the desired outcome.
55:30 - 78:45: Debugging Frame Buffer Attachments The chapter titled "Debugging Frame Buffer Attachments" involves troubleshooting an issue where the speaker is trying to replicate a test case that successfully displays a smiley face on the screen. Despite having verified the function just before starting, something has since gone wrong. The speaker discusses implementing a driver that performs the same function as the existing one but under different conditions, indicating a debugging exercise focused on frame buffer attachments.
78:45 - 95:00: Final Debugging and Closing Thoughts In the chapter titled 'Final Debugging and Closing Thoughts', the discussion revolves around a smiley face graphic that is not displaying as expected. The issue is identified as stemming from the incomplete implementation of the driver responsible for this display. The task at hand is to continue developing this driver. The conversation delves into the technical specifics of kernel mode setting, which refers to drawing to a framebuffer through a driver interface—a method widely supported by major Linux kernel GPU drivers. This part of the discussion explains the typical workflow involving a GPU buffer that these drivers utilize.

As far as userspace can tell, our GPU driver works great Transcription

00:00 - 00:30 yo yo what's up what's up what's up everybody welcome back today we are working on writing a GPU driver um because we feel like it um so I guess the context is a little bit that we've been trying to explore this idea of what if we were to make a graphics card right and in order to do that um I think it makes sense to understand what like a sane GPU design is in order to do that we have to understand like this like full Graphics stack right right so we're
00:30 - 01:00 on Linux and we kind of know that like somewhere we write some like open GL right and like you write like shaders and like you like upload you upload vertices and you call like draw arrays and stuff um and like what what happens there well we know there's this like thing called like Mesa and Mesa is like an open source openg and Vulcan implementation and we know that Mesa has like drivers for different devices and we know like kind of like below Mesa is
01:00 - 01:30 like like the colel right we know that like in order like for Mesa to talk to a GPU he has to go through the Linux Kel in some way we know that like the colonel must talk to like you know some like PCI bus and like the PCI bus has like a GPU on it and like somehow like these guys talk to each other in some way shape or form and so like what's really like not clear here is this stuff could map very closely to the stuff down here or it could go through like many
01:30 - 02:00 many translation layers here here and here before like anything actually before like what actually happens at the bottom right and so if in the future we want to make a GPU for fun um I think it makes sense to kind of understand like this whole chunk of crap right so that we can make we can decide what's like a good sane GPU design and so that's why we've been working on running a GPU driver right um we are currently on this like uh in this section where you can do a lot of like just render to the screen
02:00 - 02:30 without like any like Advanced API you can kind of like cut like this out and kind of focus on like this part of the stack now you won't have like the apis for like 3D acceleration but if we're just like hey I want to like Draw Something to the screen uh this is where we're sitting so we have written this like demo application that does this on a GPU that is not ours right so if I run this command uh I do see I think if I go to the right spot here hold on hold on I think it's
02:30 - 03:00 here uh I'm supposed to see a yellow smiley face here I'm not but I I promise you I promise you it works sometimes hold on let's try running running our virtual machine again from scratch it might be that we have not implemented our our program very well so let's just let's just try again real quick one sec one sec one sec one sec one sec uh I didn't want that how do I get out of that thank you and we look here okay uh U DHCP C- i n
03:00 - 03:30 P0 S3 okay let's try it one more time one more try one try no I swear like literally like just before the stream started I tested this and it worked so uh I've I've done something wrong but we we we have this like test case where we can like draw a smiley face to the screen and we are trying to implement a driver that does the equivalent thing um but it's ours right so we if I run the same app for ourselves instead of
03:30 - 04:00 showing nothing when it's supposed to show a smiley face um it kind of like fails early that's because we have not implemented the whole driver yet so that is the task dat is trying to continue on working with this thing so um the kernel mode setting is what this like draw to a frame buffer type of uh driver interface is called it's something that like all of like the major Linux kernel GPU drivers will support um and the this is like kind of like their flow is that they have this like GPU buffer that they make uh with
04:00 - 04:30 some like metadata attached to it such as like its width and height and color info um attached to like a plane where I'm not really like I'm a little hazy on what plane is but I think it's like if you have like a y UV color buffer your like Y and your UV might kind of like sit in different spots and so you can kind of like connect multiple frame buffers to one plane um then there's like a crtc which stands for cathode ray tube controller I.E monitor controller which basically says like um how am I going to send send the stuff that's in
04:30 - 05:00 your buffer like how how is that supposed to be packaged up for your display like things like um what resolution is my display at and what resolution is like what refresh rate is it and stuff like that the encoder says like I'm going to uh take the stuff that you told me to send over to the Monitor and I'm going to like put it on the wire in some format and then the connector is like which monitor is physically connected and between you know with that whole pipeline you have like everything you need to draw a thing to the screen theoretically um so we've been
05:00 - 05:30 implementing that we've been implementing that we've been slowly working through it and if we go into uh I guess probably I want to go like this and do envm and CD to the Linux kernel location uh we can kind of show you what we've been working on so I had some stuff commented out here but basically we have been registering with the DRM subsystem I.E direct rendering manager I think is what it's called um and basically all we do right now is we say hey I connector I have a encoder and we
05:30 - 06:00 are trying to move on to the next stage which are uh which are crtc's um yeah so I think that's where we're at I think the plan for today is just kind of keep on moving so what we are been doing is we have been kind of implementing a virtual GPU alongside this thing so for things like if we are like doing if we're asking hey is a monitor connected what we do is we actually read um from a PCI device that
06:00 - 06:30 we've been implementing and we say like hey if we get this specific address at this like register on that device then we say we're connected otherwise not so that's kind of like the flow of what we're going to be doing is we're going to be kind of like moving on both sides of this thing just trying to get like an image uh written to the file system that's kind of the goal it's kind of the goal do your own ples are something different if you multiplayer for the frame buffer will contain multiple gem buffers oh my bad then the plane is I don't understand what the plane is at all yet I think I did at some point and I have forgotten um but that is okay
06:30 - 07:00 that okay okay so it is 0635 and it is time to actually write some code so let's um I accidentally had some stuff commented out here so I kind of want to get it into the state where we had it at the end of last stream where we had an encoder that was not hooked up to a crtc and then we will kind of just follow the errors there in both our application and the kernel driver to try to figure out what to do next uh so let's uh rebuild our kernel we have made some improvements in happy
07:00 - 07:30 hour for kernel compilation speed look at that we compile our thing as a kernel module now and our little run script um injects that kernel module into a ram dis and automatically loads it now so it should be faster should be faster um I think we run our like test app here uh he says no encoder right I think this is where we were at the end of last stream where we have an encoder if we look at the um the like get resources structure so like the first call let's let's maybe pull up um maybe
07:30 - 08:00 we'll pull up the source code to test app one sec one sec hero Dev thanks for the raid hello hello everybody long time YouTube V viewer here for the first time catching you live hey man glad to have you glad to have you uh okay so let's look at our little test app what is he doing uh we need to go to the right spot so uh what is it test app so he is he calls like hey could you
08:00 - 08:30 tell me what resources you have associated with you as then let me find like a connector that has a monitor plugged in from there we can kind of figure out what the resolution of that thing should be um then we can like create some buffers and stuff uh and then I want to know like how to encode information for this monitor uh and this step is saying like hey we are not getting an encoder right he says here no encoder how however one of
08:30 - 09:00 the first things that we do is we ask for all available resources and if we look at those available resources we do we should see that we have an encoder here so we have one encoder and here is his address right so that's a little confusing if we have encoders why does he say there's no encoder well how do we pick I think that we just call get encoder ID for this connector so the connector here which we have logged um here says that he has uh one encoder
09:00 - 09:30 associated with them and here is where they live and he's looking at this encoder ID zero and when we call this here he's not doing what we expect uh you're returning no encoder when you're trying to get crtc oh that's a really good point so maybe we were just confused maybe we were just confused thank you good ey okay so let's turn on uh our networking so that we can SCP our app over
09:30 - 10:00 uh he does still does say no encoder here and we can see at exactly what line so line 81 so it is in fact this get encoder call that is failing so that's good uh that's good now the question is is why so we should look at what this call is doing this is like lib DRM this uh basically wrapper for ioctls to our kernel device where ioctls are kind of like out of band IO control functions handled by DRM ictal so if we look for like when we register our our driver we say
10:00 - 10:30 like hey our driver has these like file operations you can do on him and they are handled by like the DRM subsystem DRM ictal cool cool cool cool so in here we can look for uh get encoder here and this function is what we expect to be called now we could be wrong about that but that is what I expect so why why would this be failing
10:30 - 11:00 uh we don't know yet because we're the uh how do I say this the call to the lib arm rapper does not does not uh keep these errors he just returns null but I wonder if we can get them out of nno like when I call ioctl nno is set to indicate the are right so we could just try to print the value of nno at this point and that will give us a clue that I'll give us a clue so let's try to do that CU I'm pretty
11:00 - 11:30 sure nobody else does any CIS calls in between here so if we just do like uh erno can I get that value somehow like uh Zig or no uh I guess it's like stood see or no what does this do looks like we can just just call this function to get a pointer to a c
11:30 - 12:00 in and then there's also this constant [Music] here yeah it looks like we just call er no looks like maybe we we will see nno is a number uh which is got by D referencing the pointer that comes back from here I
12:00 - 12:30 assume and for good for good measure we'll just double check that it's clear beforehand just for good measure uh yeah so here we foolishly cannot log the Aro here because we say or else return here so instead we should do this then we should see what we want to see you just say nno to so the value of erno has changed which is a good sign for us
12:30 - 13:00 uh runting your own GPU driver because the newu driers always been dog question mark uh no no the the goal here is to understand the Linux Graphics stack and so we are trying to basically uh write our own GPU driver for our own fake GPU just to understand what's going on under the hood just to understand uh okay so we know he's returning error two uh which is not really that helpful CU that's not a word for me but we can't just go to wherever
13:00 - 13:30 the we were here and we can jump to here and try to figure out where two is two is EO an so this seems likely to me that the DRM subsystem is trying to look for the encoder ID that we asked for and he can't find it so it' be interesting maybe to set a break point here and see if that tells us anything interesting so I guess we need yet another window yet another window and we're going to change our uh run script to stop itself and wait wait for
13:30 - 14:00 GDB uh so what happens here is like qemu the virtual machine has like a GDB server built in uh which allows us to do some pretty fun stuff it's like he kind of knows that he's loaded the Linux binary at like a certain memory location and he knows like what ex instruction is being executed right now and the memory of the entire child virtual machine so you can do everything that GDB needs to do um so it's kind of cool so let's set a break point here at this line I guess we could also set a break point kind
14:00 - 14:30 of here at the top of the function or at DRM mode get encoder and we can kind of see how this works now I don't really know if our GDB break points are going to work setting them off rip like this um we might want to kind of delay the setting of them but we'll kind of see because we kind of changed the way that things are laid out uh where we have like loadable kernel modules and stuff like this but it looks like he's happy
14:30 - 15:00 it looks like he's happy uh but we do not see our breakpoint get hit which is a little concerning a little concerning so can we like break point here he says he's he's happy to set one there but if we run our test out oh there we go there we go we are chilling so we expect oh we
15:00 - 15:30 thought we thought we were going to see here but it looks like we did not right so if we run this again and we hit next we PR print encoder here no symbol encoder okay that's confusing we went from 345 to 317 in the same file no what the is going on here hold on hold on run the app
15:30 - 16:00 step oh I see this DRM encoder find must be like an alias to some other function can we finish here value returned is zero uh sounds like why is like the the stepping so broken maybe because optimizations can I break on this line
16:00 - 16:30 specifically I guess not can we just like step into this finish so we did return null pointer so we can probably it does seem like this is probably the problem and can we like look at where what we expect to happen here so
16:30 - 17:00 we have an object IDR main CS tracking ID from our mode config so we set up a mode config somewhere I remember doing that um sparo derve I wonder if it makes sense for us to try to like look at that stuff so I guess in our mode config we don't really set up anything encoder specific here uh but we do call
17:00 - 17:30 encoder attach here to the connector and we did see the encoder in our mode somewhere so I'm curious about I guess what what objects we have in this area that are encoders that's something that we want to look at right so he is calling object find and he's just Lo trying to Loop through somewhere all objects and looking for types of object encoder so like where is this used surely somebody must add objects with these things too right and D encoder in
17:30 - 18:00 it we do in fact do that right so this is called by our driver somewhere I believe let's just kind of take a little back Trace look skis not here not here I thought we call oh maybe DRM encoder in it oh this is a recursive call back he's confused here we go dim simple encoder in it we do call this right because we go from here feral probe
18:00 - 18:30 into simple encoder in it right which should make it all the way down into this into this system so I guess the question is why why is it wrong then right I guess theoretically maybe our encoder at this point doesn't have his ID set that's a possib
18:30 - 19:00 ility um wow I need to write C on a Linux machine I'm currently on Mac and using n when I go to go to death for live the dogs are horrible uh oh like you're like wow the Linux konel has documentation that's cool I guess it is kind of cool I've never really thought about it this maybe also is a problem I think we saw earlier in happy hour that there is a big warning here that says that um our encoder has an
19:00 - 19:30 incorrect possible crtc's so maybe if we correct this then the encoder will be happy there's a very Poss very real possibility there um but maybe not maybe not uh so how do we want to do this first of all what is that L at 319 and
19:30 - 20:00 uh where were we di encoder Doh yeah okay so we were here and so this di encoder find definitely return all and that's definitely why we're we we we have found the right the right failure location um yeah because it seems like we've added one so it's possible
20:00 - 20:30 how does how does he how does he do this he has an ID associated with him already the encoder ID I guess yeah yeah so the thing that we when we we came in here asking for ID Zer id0 we try to find this
20:30 - 21:00 thing with this ID okay come all the way in here boom boom boom so here we P this thing out of this radx tree whatever this like IDR thing is and I guess we want to know which one of these things are failing
21:00 - 21:30 I expect that this ID zero is incorrect that's kind of my expectation here but I'm not really sure it'd be maybe interesting to break point on this stuff in single step in here and see what happens kind of interested in that kind of interested in that uh I guess we need to reconnect our GDB so we give him a second
21:30 - 22:00 route and then I think we forgot to set the break point here so I want a break point right here oops continue and I guess we want to wait uh till we hit our other break point which I guess is not being set
22:00 - 22:30 here okay so enable to continue and can we what were we trying to discover about this I expected that this is the type is wrong here uh okay so what is the value of this uh PX type EO EO EO EO EO so this
22:30 - 23:00 is an encoder okay so we have this does actually does match the type that we expect which is a little surprising right so here we say if object type and type is not equal to this and type is not equal to type we return all but type should be equal to type what is like our oh type is what we asked for we want what the object type is
23:00 - 23:30 oh OB is null OB is null here so ID Z must be bogus which would make sense which would make sense so the question is why it's whoever sets the connectors in coder ID right like we are getting this encoder ID through the connector interface so whoever sets that must not have set him right I kind of expected like this
23:30 - 24:00 attach encoder thing oh okay so before they used to do this direct assignment the connection is a logical one and the responsibility of the core so drivers are not expected to not mess with this note that the error return here should be good enough right so they say here are possible encoders which means here that someone must set this at some point right presumably this is done on
24:00 - 24:30 registration here [Music] presumably and presumably this is not working correctly that's just kind of like handwavy what I'm expecting here and it could be that it's because the crtc stuff isn't hooked up that's very
24:30 - 25:00 possible very very possible uh so can we look at that like warning log here that says hey you need a crtc bogus possible crtc's let me look for that bogus possible see was it bogus possible H C rtcs here
25:00 - 25:30 so he doesn't really return anything here he just kind of logs it so there's actually nothing really to worry about there I don't think uh okay and where does this come from from validate on register so like right off rip basically right off rip so I'm really more interested in who
25:30 - 26:00 sets like are connectors encoder ID well I guess who sets encoder ID when we when we call uh DRM ioctl in here we call get connector and this is where we're tying this back right one of his outputs is an encoder ID so where does this come from aha zero is invalid aha aha aha so this
26:00 - 26:30 is failing getting encoder for connector he goes hey we don't have one so we set this to zero and then we're trying to fetch from zero and then falling the over that makes a lot of sense uh okay so then how the does this get fixed that is the question that is a question so
26:30 - 27:00 Atomic helpers set this like best encoder set best encoder okay this looks promising so update connector routing on Atomic helper check mode set so we do have Atomic enabled which makes me think that we are probably going to be calling this somehow
27:00 - 27:30 I guess the vert IO GPU set this and we didn't so we actually might not go down this path uh which is kind of interesting to some ex set so here they call Atomic check was that where this came from helper check update connector routing yeah so we I are
27:30 - 28:00 not in that path but maybe we're supposed to be do we have connector State I don't really know what this means the arm connector State this is protected by this I mean it could be it could be we should probably break point on get conector and try to inspect which path
28:00 - 28:30 is taken there so maybe we do just like break point here maybe all right uh GB DX okay and let's see what happens okay so we do in fact have connector
28:30 - 29:00 State and best encoder is null right so it seems like the expectation here is that because we have said that our driver supports like Atomic access which I don't really know what it means but we're just kind of copying we're following someone else um this is expected to be set and it's not so I I bet that we basically need to
29:00 - 29:30 follow what vert GPU display is doing here and use these Atomic options to to propagate onto like the generic DM subsystem so can we see like Atomic check this is the only hook to validate a mode set update this function must reject any mode set and state changes which Hardware driver does not support this includes but of is of course not limited to to blah blah blah so it sounds like maybe oh and then Atomic commit is that where this Stu actually happens yes yes yes yes so I think that
29:30 - 30:00 when you they talk about like Atomic commit and atomic set my gut kind of says we're kind of we're on this path of like you set all of these things that you want but they happen in one shot one moment he vomits on his sweater something like that anyway it doesn't matter um yeah so I think this basically says like we pre we
30:00 - 30:30 preset all this stuff and then we actually have to do it and we probably are presetting some stuff and not doing it that would be my guess so let's see what happens if we apply these functions to our driver so it's these boom uh let's see what happens so I go make and we quit here and we run here and then we uh GDB X here and we hit
30:30 - 31:00 continue and then we go over here and we wait for this thing to boot all the way which takes a few seconds uh and then we on Route and test app and here he's still null he's still null which isn't great is not great um so it'd be nice to kind of break point
31:00 - 31:30 in here maybe just to see if that happens so we saw somewhere like best encoder got set somewhere right uh where did we see that is it in our history somewhere
31:30 - 32:00 Atomic helper I think it was like here yeah so set best encoder we saw come from update encoder blah blah blah Atomic heler check which came from here so I think we want to breako both on like the actual assignment
32:00 - 32:30 here as well as up the call stack on like our entry point here we just want to see like are we really going into this path right so this is the thing that we assigned just now to our thing and like let's see if this is called and then let's also see if we make it all the way down and if not then we can kind of like bisect between those two things I think that makes
32:30 - 33:00 sense uh all right so this seems like it's kind of doing what I would expect we are getting into this thing from someone calling our config funks Atomic checks that's good now if I continue here we do not see
33:00 - 33:30 that he set best encoder so somewhere between here someone's rejecting this essentially um and I guess the question is like where so this supposed to go into [Music] helper Atomic check mode set so that's like literally like the first line then he gets in this function helper check mode set and he's supposed
33:30 - 34:00 to get into update connector routing right here so what is going on here it'd be interesting to see if we hit this so um we add this for our list of break points I guess and is there like a rejector rejection
34:00 - 34:30 case here that we need to look at so we might return early if there's like conflicting Rec encoders so we might want to just break point kind of like at the top of this function as well uh break at the top of this check if we go into update connector routing and then in update connector routing we were looking for somebody who calls like set best encoder here
34:30 - 35:00 right here here here so maybe we'll just kind of Step through update connector routing if we get there if we get there all right so here we at line six 31 which is the first line here and I
35:00 - 35:30 guess we're just going to step through this and see what happens uh so is this where we expected where are we expecting to see us get into this set best connector thing encoder sorry update connector routing is that where we're at right now oh no we're not can we just continue
35:30 - 36:00 then we were actually in check mode set so we actually never got to update connector routing actually right this is where we were we were kind of wanting to walk through okay so how what what happens here we're for each old new connector in
36:00 - 36:30 state I guess we just kind of break point like here and see what happens all the other big points there's too many too many and I don't know where they all are so we're going to start over all right so we do not have a red here so we do get
36:30 - 37:00 into this for each old new thing and if we jump through this he goes straight to 1106 uh in a different file hello ger Atomic h
37:00 - 37:30 1106 do atomic crtc needs mode set oh so we're down here so it's almost like whatever this Loop is doing we like didn't go into it and he just immediately skipped down here it's kind of like what it looks like happened yeah and we can see that here he goes from six 685 to 731 like
37:30 - 38:00 immediately so this Loop has nothing in it okay okay so what is this Loop supposed to do iterate over all connectors in atomic update connector so old is the cursor for the Old State okay new is for the new state okay and I is an iteration [Music] cursor okay sure so this just
38:00 - 38:30 goes how does he know like the the length here what's the stop condition here for each if if condition Jesus dude oh I don't know what I'm looking at this is a condition that goes in an if so he says
38:30 - 39:00 this and this comma oh I see so this is always true the second half but is like the iteration State treking and then this is for this is the actual check I think this is running for the vert I ders not yours I don't think so I don't think so but we can take a peak
39:00 - 39:30 um oh I guess we could be in this break point yeah you're right it does kind of look like that I kind of assumed I assumed I made an ass of me and me uh good point that's even more confusing cuz thing should run at some
39:30 - 40:00 point okay that's a really good point good ey can we just disable theirs temporarily that way we kind of only have our one thing to worry about
40:00 - 40:30 uh yeah I guess we just don't even get to this PA path at all that is a good eye thank you thank you thank you so what the is going on there let's make sure that if we try like looking at like anything else let's just put a break point here that like we see anything happening just to make sure that it like works correctly good ey good
40:30 - 41:00 ey um yeah so this does happen and then we just never get into that stuff okay so that's a good clue actually we were kind of looking at the wrong spot so thank you for that um but we should get into like this config validate right where where where did we think that the back trace for this thing
41:00 - 41:30 was we were looking at best connector so we we yeah we I think that that part of the test was valid where we checked the connector hookups cuz that was when we were running test app on our device so we're looking for like best encoder equals right that's what we're looking for in here this is where we're looking so we we wanted to see is this getting
41:30 - 42:00 called at all it should be but maybe not is this getting called the check function that we hooked up where we think that this is like a possibility of something that could happen so here we get into register and they never call Atomic check okay okay
42:00 - 42:30 so why the not why the not because this is where we expect this to happen so where does atomic check come from here Atomic check only uh which comes from like commit or non-blocking
42:30 - 43:00 commit or Atomic non-blocking [Music] commit interesting I almost want to go back and look at the back trace for the case where we hit this break point in the vert iio GPU driver right so we are in ver GPU probe dearm client
43:00 - 43:30 register do we call that uh is there not some Atomic flag you set we do set that though as far as I understand uh DRM client register invert ipu probe so invert I GP derve 106 that's what we're looking
43:30 - 44:00 for they call ah okay set up fbd emulation for Gem scham helpers so this is where they are actually getting this is where they're actually getting the best encoder hooked up as far as I can tell
44:00 - 44:30 so I wonder how I'm supposed to know we have to call this DRM client in it so initialize a DRM client use register to complete the process right okay so these are all related to the frame buffer helper though right so here they initialize this client as like a frame buffer client of the DRM sub system interesting I would have thought
44:30 - 45:00 that this doesn't need to get done interesting uh okay yeah it feels like everything should have got hooked up in here but I guess that's not the case unless like maybe our stuff gets called
45:00 - 45:30 when we run our test app like it could be that like this is only necessary like this just happens to be where it's called in their case but it doesn't have to be called there right like if we call uh okay so let's set a break point make sure we have a breakpoint set uh an atomic hel check so we have one set there so if I run test app oh he's running on the wrong
45:30 - 46:00 card now because we removed one hold on uh we do need UD DH C we need like a a network address I think it's like this uh and then we want to set this back to card zero cuz we only have one
46:00 - 46:30 card so here we're still not hitting that at all we're not hitting that at all which is kind of confusing I would think that like there's some sort of uh I would have thought that the reg like the the workflow that we have would trigger that case because surely not everybody calls this FB devam setup this doesn't even have to happen I thought I thought this is for like
46:30 - 47:00 emulation of FB Dev which is like a feature you can totally disable like that's a thing that the colonel doesn't even have to support I I saw in the colonel configs yeah uh not this so what does this even do initialize a dearm client
47:00 - 47:30 okay okay I don't really understand here so maybe you need a client and it to exist D client open clients running in the kernel like fbd and boot Splash like yeah what why do I need one of those I wonder if this is how like uh mode setting gets set at
47:30 - 48:00 all maybe we have like in our in our test application we are assuming that somebody has set up the like current mode and if we don't set up this like frame buffer client uh we have we're kind of like in an uninitialized State that's kind of my guess that's kind of my guess so if we look at like Sparrow derv see we go back to this like Atomic check who who calls
48:00 - 48:30 this diatomic check only so like uh mode set commit Atomic mode set check mode set commit locked
48:30 - 49:00 like how do I how do I like when I'm looking at modes we like sometimes set a particular mode right that's like a thing that we can do and surely in there uh cuz somewhere in our test application we're calling like what is the preferred mode that's like a thing that we ask for get preferred mode here so we look at
49:00 - 49:30 connector modes and we look for the DRM type preferred okay but there's ALS uh let's look at the like way back way back we had like a this example of DRM Linux kernel embedded something or other like there was this guy who wrote a blog post on how to Output to the Dr RM buffer
49:30 - 50:00 directly uh it'll be in our notes it'll be in our notes so it would have been in like this notes Here what did he do cuz surely I I just do not believe that this is the only way to hit this path and so I really want to understand it a little better
50:00 - 50:30 so how did he pick the mode get the resources he looks for the current connector adds a frame buffer wait where's his like GitHub where is his GitHub here we go uh mode get
50:30 - 51:00 encoder get resources so he calls get resources he looks over the connectors he gets the current connector I remember seeing something about being able to like set the mode which somebody has to do right like somebody has to set like what resolution do I want to be I'm pretty sure somebody has to set that
51:00 - 51:30 but I don't really see it anywhere which is kind of interesting kind of interesting I would have expected one of these many ials to be that uh KS DRM set resolution uh this isn't really what I'm looking
51:30 - 52:00 for maybe the idiot can help us uh is there a lib DRM call to change the KMS mode question mark set crtc ah uh is that when like resolution is applied yeah he applies the mode here uh
52:00 - 52:30 okay okay so in that case in that case we could maybe look at like if no encoder is hooked up but it is a possible encoder maybe that's enough so right now we're trying to be like which encoder is used for my
52:30 - 53:00 connector and then trying to work backwards from there but we we do have possible encoders so I'm curious to see if that works so instead of calling connector encoder ID which is zero which is not set we could look at encoders here I'd be curious to look at uh encoders zero and see what that does
53:00 - 53:30 okay so that actually correctly finds an encoder so we could maybe maybe say hey if if there is an encoder already set up great otherwise uh otherwise just pick one that feels reasonable to me because like
53:30 - 54:00 it's kind of saying according to chat TBT which you know idiot um when we call this call an encoder might be kind of like set up for us automatically connectors mode and crtc yeah so I guess I guess it's kind of saying like the crtc will figure out which encoder it needs to use to get to that connector right like maybe you have
54:00 - 54:30 maybe you have three VGA ports and you have three VGA encoders and they can kind of like be me Mis mismatched in whatever way they want okay okay sure let's kind of go down this route for now and then we'll come back and we'll see if it makes sense
54:30 - 55:00 uh so we'll say encod ID is equal this if it's zero then we return connector incoder is zero but uh fix me check if n encoders is greater than zero uh so I guess this
55:00 - 55:30 uh else if connector encoders I guess we want if connector encoder ID is not equal to zero we pick that one else if count encoders is greater than zero we can return that else uh return error no usable connect uh
55:30 - 56:00 encoder then here we can use encoder ID okay okay we got past this part which is good now we're back on to get crtc so all of this was uh why can't we find an encoder I'm not like really 100% sure that this is the right fix but it feels kind of reasonable to me so like it we run with it right it uh so this is
56:00 - 56:30 actually uh crtc crtc time so back to the little diagram wherever the that is here we now have this we have this we have this we have this we need these two things so we work our way back we get the crtc's associated with this thing which we set in our driver uh here theoretically right we say hey
56:30 - 57:00 which crtc's could we connect to and right now we have zero because we have no crtcs but we have to make one so what do they do in here they set index here and here they create a crtc like this they say hey I want a crtc baby and then they have helper functions for that
57:00 - 57:30 crtc which look like they do real work they look like they do real work in some way it looks like they kind of send off to the ver the the GPU that like Hey we're not being used right now which is fine which is fine um I guess for
57:30 - 58:00 now we kind of just try to do as little as possible right so I think that we do in fact want to call in it with planes almost certainly let's see initialize a new crtc object with specified primary and cursor planes okay um where the cursor plane I think is like literally to show the mouse on top of stuff um so he passes in two planes do I H I guess do I need a cursor plane I don't
58:00 - 58:30 really want a cursor plane that sounds stupid I don't want like a custom like custom Hardware tracking of the cursor that seems do I can just do that myself uh but it does seem like that's kind of expected I was also reading in this like uh a trip through the graphics pipeline that someone sent the other day um they were kind of talking about like uh somewhere in here they were talking about like Hardware specific cursor handling which
58:30 - 59:00 is very funny so it's like a thing that like is like normal I guess um but I guess we will call this for now and then we'll figure it out later we we will come back for if we really really need a cursor plane um which means that we have to make planes right so here they have created a plane with universal plane Alec okay okay so here they're saying hey we're allowed to be argb 8888
59:00 - 59:30 essentially I don't know what the difference between a RGB and xrgb is I think X means no Alpha I guess would be my guess would be my guess Hardware curses are goed I mean maybe you know that I don't know that that seems crazy to me it seems like if you need Hardware cursor why don't you need Hardware like Windows you know they're like the same
59:30 - 60:00 like you should be able to like kind of have a buffer that you want to update a c like you you should be able to like update a small amount of the screen kind of anywhere with any reason I would think but maybe not I guess so they call this Universal plane Alec and then add with functions where functions are hello oh here okay so they have
60:00 - 60:30 like helper functions that are different for cursor and primary plane got to get that lowly and send the cursor good feels I guess maybe aren't Hardware Windows the thing uh I didn't think so but maybe I assumed not I mean I don't see any reference to them here but maybe we'll get farther down the stack and we'll find them uh um okay if we are the primary plane they
60:30 - 61:00 enable damage tracking so this is kind of more what I was thinking about right where your frame buffer says like hey there are parts of this thing that have been updated and I would like to update those parts now um and I guess they just so what does this do it's a hint to the colel that could be useful to optimize internally uh user space can ignore Dynamic Clips property sure I don't want to read this uh I guess we should though drivers that
61:00 - 61:30 are interested in damage interface for plan enable damage CL property sure um there are frame damage and buffer damage type of damage hand imputed depends on a driver's upload Target drivers implementing per plane or per crtc upload targets need to handle frame damage while drivers implementing a per buffer upload Target need to handle buffer damage sure uh okay I think I get it I think I get it so it seems like we probably want want to allocate two planes uh what is our like cursor plane
61:30 - 62:00 update thing look like what if we just don't update the cursor plane I think that'll be fine for our like user space usage for now I do not think that we are using the cursor plane in our little sample application so okay I guess we need some helper functions seems
62:00 - 62:30 like seems like we're going to need uh we're going to make like static and like make crtc or something I don't know Sparrow make crtc you put Windows on overlay ples I guess that makes sense and we are going to have to do something like take these helper
62:30 - 63:00 functions uh and where does this get used here so it's like we make a plane right and we use our sparo GPU primary helper funks I guess we should uh I guess maybe we do like cursor helper
63:00 - 63:30 funks just kind of nulls this out he just doesn't do anything for Atomic update we check this stuff and I guess we should enable damage I guess seems reasonable seems reasonable so that's like him making a plane and once he makes a plane he makes one for the cursor and
63:30 - 64:00 one for the for one for cursor one for primary so we're just going to say like primary plane is this then we'll just do this again but for the cursor plane okay I don't really
64:00 - 64:30 know uh and then we're going to init this thing with plants and then our encoder our encoder will have a possible crtc one I believe is how this is going to work I believe is how this is going to work okay so now we have to figure out what the this stuff
64:30 - 65:00 does so they returned a DRM plane out of that thing probably right plane is a DRM plane uh so here if we have a primary plane issue we ditch that uh so pointer error this
65:00 - 65:30 thing uh then we have to do the same thing here I guess we want to do like cursor plane here what's see you m about here does this have to get wrapped because pointer error is like kind of some sort of up or is it air pointer no pointer should work I don't know why mad but we'll we'll kind of just keep plowing is that spooky Halloween hoodie yeah it's spooky
65:30 - 66:00 month so I'm allowed to wear this one again uh index here we're going to have to figure out eventually but whatever primary plane instead of plane oh that's true that is true I wonder why my syntax highlighting is all up though oh because I didn't copy this whole thing here I I up this I see this goes here nice there we go much better so I guess we want to do our typical
66:00 - 66:30 like uh something like this and then here we're going to have to have like a label for like uh error and in this like error section we're going to free the primary plane so can we do like I don't really know how can like if this drmm stuff
66:30 - 67:00 fails can you like free It Anyways drmm free is that a thing that's not helpful uh so like add Alec Dr and then add
67:00 - 67:30 Dr D drr seems like what I want DRM K fre probably is what I actually want here depending on if that's like the Alec free function they use for their thing so the drmm stuff is like when it attaches itself it attaches itself to the uh Dev handle as like a list of objects that have to be
67:30 - 68:00 freed um and so I'm trying to figure out like if I don't want it to stay attached anymore then like what's like is that a problem right so somewhere he's going to call this and he should have some sort of like delete function associated with him I think that's how this works however I don't really see that
68:00 - 68:30 here I thought that's how this works otherwise like how does it get freed like it's got to be in here oh here I see uh list add Dev managed resources okay but like how do they how
68:30 - 69:00 do they tell them how to delete it am I stupid ah ahaa so its release is null what the final DRM Dev put memory can also be freed before the
69:00 - 69:30 final DRM put by calling DRM K I guess we'll just do that because I'm I'm not going to find it in the code right now so we'll just have uh DRM K3 for primary plane sounds like that's what we're supposed to do so here I guess we'll kind of do this down here boom boom and then we'll say uh RC equals this and go to error
69:30 - 70:00 grade now on the future guys if any of these things fail we're going to have to have like an error cursor plane right who's going to look very similar which means it's going to be like error I guess this should be called error cursor plane and this should be called error kind of just generally right it's then we can say go to airor cursor plane wow look at us
70:00 - 70:30 go this is a c is weird Okay so we need to add helpers to our primary plane with [Music] our something about this doesn't feel right where where do we copy this from it's called Uh Jam
70:30 - 71:00 helper plane ad ah I see so yeah they do this for each of them so this is primary plane and our funks what is Funks here helper funks J PL helper funks yeah so why do we have to use that twice why do we have to use that
71:00 - 71:30 twice am I stupid oh these are two different things these are plain funks my bad my bad my bad my bad uh there are plane funks and there are plain helper funks
71:30 - 72:00 oops so both planes use the same set of sparo GPU plane funks but then we use Sparrow GPU primary helper funks here and cursor helper funks here okay all right then we enable FB damage on the
72:00 - 72:30 primary plane which I guess can go up here and then we initialize our crtc with these planes I guess I guess and certainly can this fail we don't know yet we don't know yet okay let's try to implement some of these uh here we'll do this for
72:30 - 73:00 now for now and then we'll try to just kind of fix compile errors so primary plane we're just kind of fixing everything to index one we need our uh DRM device okay so that's going to kind of go
73:00 - 73:30 here uh formats and end formats it looks like we have to say where which formats we support somewhere so we do like a vert iio GPU or display uh here formats and formats so these These are set kind of as like static
73:30 - 74:00 constants okay uh we are going to just force xrgb here and we'll say uh fix me cursor might want a argb or else no Alpha but it I don't care right now uh so here we'll say sparo GPU formats here we have one and what was type
74:00 - 74:30 here deum enum durum plane type ah okay so it's durum plane cursor plane type cursor DRM plane type primary and uh this is cursor okay all
74:30 - 75:00 right this is say cursor plane here and then we're back down here so this is Duram Dev here we need a crtc to initialize so maybe we also take the sparo dev here like this and that way we can say uh priv crtc which means that we need to store a CRT TC somewhere so we'll kind of do it like this uh then we take in our primary
75:00 - 75:30 plane and cursor plane as well as our crtc funks okay okay and now we need to implement this thing both of these so we'll copy paste these as well temporarily and and then we'll figure it out from there okay so crtc's we have crtc funks
75:30 - 76:00 and crtc helper funks uh spero here uh so ver iio Sparrow these are all just uh DRM passrs so I think we don't have to worry about that one and what about their helper
76:00 - 76:30 funks these are actual implementations from them which is a little scary so for us instead of implementing them we will just not Implement them wow okay okay cuz we're we're just trying to follow the errors to get something saying so he says we don't have a crtc
76:30 - 77:00 thing here I thought we called it oh we call it D CTC so we'll name it correctly here and it looks like this label is unused so I guess that kind of indicates that this stuff cannot fail except it can this can definitely fail right so this returns zero success airod on failure so here we say RC equals this if RC is equal zero uh we go to
77:00 - 77:30 error uh oops it's here boom and then this can't fail right void uh void yeah okay uh he doesn't like that drmm Kree is not here so we will just kind of add the appropriate
77:30 - 78:00 include uh two few arguments to function call so he wants the DRM device as well okay cool I think we're pretty chilling I think we're pretty chilling so I guess we should call this function so we take our private data and our DM Dev like
78:00 - 78:30 this and I guess we're probably chilling super super chilling uh we're going to say uh note assumed index one and maybe in here we're going to say like con uh idx mask is an INT is equal to one shouldi left by one or something assumed index zero
78:30 - 79:00 sorry uh yeah because this takes in the uh possible crtc's so maybe we do like a static cons spero possible crtc's like this just to make it C that way we can say like uh assumed
79:00 - 79:30 single possible CC at index zero and then later if we ever want to change this it will be obvious that it's wrong because we can just look for references uh all right make how does this it builds does it
79:30 - 80:00 run uh I guess we probably don't need GDB quite at this exact second uh so I will turn that off and we'll see how we get no crtc no
80:00 - 80:30 crtc confusing if we look at DRM our our resources we do in fact have a connector now which is a big win for us that's so big um but we might kind of have the same problem where the encoder like might not be linked to the crtc yet the way we expect if we had called frame buffer initialization in the Kel driver it's kind of the assumption that I have right now so uh I I bet we can probably check here
80:30 - 81:00 and we can probably see that encoder crtc is zero encoder uh crtc ID oh we don't even have to do this we print log the encoder already so we can just look at it uh so crtc c z but he has possible CR2 CS
81:00 - 81:30 one uh so I guess what we do is uh for now for now we'll say have you figured out why the smiley face is not smiling anymore I have not looked that was confusing to me um and we will come back to it probably maybe I don't know CU yeah he should have worked he was working like liter I tested it literally like right before stream started like in the break time I checked
81:30 - 82:00 it and it was working fine and then as soon as I started stream it didn't work anymore so I don't really know what what caused that um but you know we live we learn I figured we'll kind of like naturally get there um when we try to actually use the thing in our version I figure we will have to tackle it there um okay so we have it's CTC ID here if it's not zero we use the encoder crtc ID otherwise we look at
82:00 - 82:30 encoder possible crtc's if if this has Bit Zero Set uh wait are they one index then hold on hold on [Music]
82:30 - 83:00 um they start at zero and work their way up so possible crtc's gets one shifted left by index I.E so the so the first crtc should have index zero in which case if we have index zero why is it not a problem why is it a problem because we're trying to use zero uh we could look at the ioctl I'm kind of assum was assuming
83:00 - 83:30 that it was the same problem with the connector but now I don't really believe that so if we call get encoder he calls get crtc so who his output here has crtc ID here what is this what is this comment crtc ID is the ID of crtc yeah of course how does this
83:30 - 84:00 make it in uh okay this has like the same thing though where they set zero which means that index zero can't happen it just feels like it shouldn't so I wonder if there's like a plus one that happened somewhere here yeah something about this feels
84:00 - 84:30 wrong so when we call get connector or get crtc get crtc his input is
84:30 - 85:00 this which literally just does all the stuff we were looking at before where they like look for an object that matches that ID of that type uh okay can we look at that maybe the output of dur are like resources can we look at the crtcs
85:00 - 85:30 there I'm just curious now maybe the IDS are not what I expect them to be maybe it's like the first crtc which has an idea of something else you know what I mean like it could be that you have like a possible crtc's is like 0 B1 one01 which means that he can be like the first third or fourth crtc and then you have to actually extract their IDs by looking at the resources right so I want
85:30 - 86:00 to look at um resources crtc Z I just want to see uh what this is uh so CD oopsies I guess usually I'm doing this in this
86:00 - 86:30 window uh we don't have our thing open I guess I guess we do need to run the virtual machine if we want to run the virtual machine That's a classic all right uh u dcpc i n p03 nice and then here we should be able to run our little build and test and we see crtc 40 that makes sense so we're going to
86:30 - 87:00 say here in our kind of workaround our crtc ID is going to be if it's not zero we use that one otherwise we as we check that he's using the first index and then we look at resources crtc's Zer otherwise return unusable crtc and we use crtc ID here
87:00 - 87:30 here uh he doesn't like this uh is not equals to zero uh okay there we go so that got us a crtc that got us a crtc um and now we are getting we move forward and we try to map the memory that we uh we allocated way back somewhere else to try to write into it um and I
87:30 - 88:00 believe um that that's an easy fix so I think that we are at1 2740 and let's say uh map frame buffer memory this should be pretty straightforward so we saw a couple days ago that invert iio GPU they how did they do this somewhere they said that they
88:00 - 88:30 initialize these like file operations here down here and they just Define their file operations by doing define gem fops which is this generic thing and they have map Derm gem M map right so they just like generic driver subsystem M map stuff we were like I don't know if we're going to use gem at that point we weren't really sure what was going on we now know that we are using gem um so I think that we just we just
88:30 - 89:00 bypassed this right so I think that we just kind of replace our fops with this right so our version we basically took like we basically copy pasted everything except for the map piece so I think that if we do this it should be fine and that might be all the requ ired to do mapping because our our memory buffer is already kind of backed by shared memory so there's like it should just work it should just
89:00 - 89:30 work we're making a lot of assumptions along the way by the way um because it so let's see if we get farther this time uh okay so here we now nullo our D reference at commit at commit which is
89:30 - 90:00 very likely very likely we we render our smiley face we call this and we just seg faal uh because we didn't actually Implement whatever has to get implemented here that would be my guess so here I'm going to say uh print stood debug print hi Mom I'm about to oh okay we can we can do a little double on tter here if I say I'm about to KMS we can we can say that we're kernel mode setting
90:00 - 90:30 here um but also there's a second meaning to that that kind of fits nicely with this call to uh die uh perfect perfect uh okay let's uh restart a virtual machine and and uh let's look at root and let's um udhcpc
90:30 - 91:00 uh I guess to satisfy satisfy twitch we can say invm cuz that's you know it's like when you say allegedly now they can't now they can't ban me yeah so he's like hi Mom I W to KMS and then here he dies so I think we're chilling I think we're chilling I I think the like this was expected to me so I think what we can do is we can
91:00 - 91:30 try to look at like where this line is uh which we we can pretty we can pretty reliably assume is going to be calling some function that we didn't set up uh so we can be like uh yo uh let's say can I look at like line uh oops this info line I think it's uh commit helper ples
91:30 - 92:00 2803 so wait what sorry was a diatomic helper 2803 yeah yep yep yep so he's calling oh no wait oh yeah so funks is not defined here funks equals plane helper private which gets assigned here which we do call
92:00 - 92:30 actually um and he does actually he is guarding some of these what line did it say it's 2803 what's he oh yeah yeah so he's calling Atomic update here probably which we haven't set unguarded it's probably just a little bit off probably just a little bit off right because here we had Atomic update just commented out right
92:30 - 93:00 so probably we call this we set this up I mean we can check this right we can set a break point like literally right here and we can confirm it's best to be sure it's best to be sure I always harp on you guys saying we should never we you know we should find find the reason why stuff happens and I think just skimming over that would be
93:00 - 93:30 doing a disservice uh we do need to turn on GDB huh we do need to turn on GDB and then here we want GDB DX okay continue we go here we wait a second we run root test app okay and then we look at ke
93:30 - 94:00 funks he sure is about to call Atomic update right here yep yep yep yep all right uh so let's do this um and then I guess we will kind of have to figure out what this has to do at some point but what if we just didn't find out you know what it has to do and we just kind of pretend that we've
94:00 - 94:30 implemented it uh and then figure it out later uh let's just kind of like you know Follow the erors yellow yellow uh so I guess what do I want to do here
94:30 - 95:00 I probably want to run without GDB again run run run run run run run run run run and I presume we'll just see another crash we do see another crash at uh commit plan here so this is probably a very similar problem so we can do like the same thing
95:00 - 95:30 we say info line at this address um which is at 2803 in atomic helper uh right so we actually could have looked a little bit and seen an immediate oh there's going to be Atomic enable here as well like literally in like the same two
95:30 - 96:00 lines but how come we don't have that commented out anywhere oh cuz that is guarded wait that was a guarded call yeah yeah so they only call this if it exists so why are we still crashing in this area then I'm a little confused I'm a little confused I would have expected that to fix it I did build right did I
96:00 - 96:30 save I saved yeah I built I guess we turned GB back on I guess cuz I don't get it I guess how's it going not too bad how are
96:30 - 97:00 you rout test app okay so P funks P funks P funks oh he's still not set why not what file is this it's a terat so
97:00 - 97:30 funks okay so where funks get set by plane helper private okay so this looks like it has stuff in it okay so this this has definitely got some stuff but it's all zeroed
97:30 - 98:00 out uh do I only fix this for one of the planes we had um cursor Helper and primary helper yeah so here we had to do the same thing and we'll call this uh I guess we'll call this plane update they had different
98:00 - 98:30 implementations in our invert IO GPU for primary and cursor planes but we don't want to deal with that right now so we'll just kind of give them both null there we go that makes more sense that makes more sense all right let's see if everybody's happy let's see if everybody's
98:30 - 99:00 happy uh okay so P funks yep Atomic update is so chilling there and now what's happening it looks like he oh does he just end oh right because we sleep for five seconds after we do this so so from the
99:00 - 99:30 perspective of the app from the perspective of the app it's working now nothing is happening but I think that that means that we should be able to look at the memory in at this point and try to copy to the host so we can
99:30 - 100:00 write it out to an image I think that's the thing we should be able to do so I'd like to um uh somewhere we did I I we didn't really write down where we were here but was sometime before now I guess uh this was like finish crtc uh fix crtc crashes and yeah let's let's see if we can like inspect the host memory in
100:00 - 100:30 there somehow that' be cool so my gut says that since the only thing that we have done here uh in terms of like talking to like the only function that we have here is plane update right so like this must be where we're supposed to trigger on our GPU to do something now
100:30 - 101:00 probably these things we could also do stuff but say say we don't actually have any like this to me this looks this looks like uh prepare and clean up FB would be like managing Hardware resources so like telling telling the other side like hey could you take this buffer and you and like use it for frame buffer and you're like buffer ID Z and here you go now we could choose to not do that if we want we instead could we
101:00 - 101:30 we could just choose what interface is and right now I'm kind of thinking that the interface should be like uh right to PCI register with location plus size of buffer uh then when we write that like the GPU just does its thing now we're going to have all sorts of problems there with like uh you know resource um resource locking like right so like
101:30 - 102:00 if we try to write something then the GPU is like okay I will use that now and he's he like just starts trying to read that memory if we overwrite that memory as he's writing we might run into problems um but we don't really care about that right now we don't care about that right now so we'll just say that our GPU just will like you just have to like guess how long a right takes so from a user perspective or we just like don't support terree rendering in which case it's fine right here we just kind of say
102:00 - 102:30 like look bro like sleep 400 milliseconds or something just like to be safe after this but we don't even render multiple frames right now so we can just tackle that problem later later later the answer will be that we like create like a q of like commands and one of those things is going to be like here's a frame ruffer and like draw it please right and then later we and then like later we'll have
102:30 - 103:00 like a queue of responses and he'll be like uh I drew frame buffer two and so that that's how like we would know that it's like okay to draw the next frame um but we are trying to do the stupidest shittiest thing that we can do right now right and so to do that we just say this and uh there's no way to do terree rendering it and we only support one frame it um okay so I
103:00 - 103:30 think I want this to like have like a null plane update here and we'll use null plane update for the cursor and then on the primary plane update we will actually do some work we will actually do some work so I think it's going to be like somehow we have to like extract um information about like our GPU Private Structure which I don't think we can
103:30 - 104:00 here because our planes aren't like members of sparo priv so like I don't know how they do this in like vert IO GPU or display they call like when they make their planes how do they actually do work they call here VG Dev which comes from Dev Dev private so ah got it so you go from plane to DRM device from DRM device to
104:00 - 104:30 implementation so in ours that means we have to make sure that we have our device as um our device accessible and then here we can just go wherever the we were plane Dev this is a DRM device yep then we have uh spero priv is a
104:30 - 105:00 spero I guess we call it priv and he comes from uh DRM Dev private data which is a void star okay and now we can just like start writing right we could go like priv regs one is equal I think that our regs are 32 bit um so we would do something like I will write like address address uh and
105:00 - 105:30 uh and then we do like priv regs 2 is equal to address shifted right by 32 or something right now this is assuming that we have one address uh available but that is not true that is not true um oh oops I probably should have written down here it's fine uh we don't have that this is actually organized in Ram like you have things that are like pages
105:30 - 106:00 right so you have like 4K of ram here and then like another 4K over here somewhere and then 4K over here and these live like kind of like somewhere each page lives somewhere in like physical memory and you program the memory management unit to kind of say if I want these to look like contiguous sections in my process you have like a process table that kind of has like the table mappings right so it goes like oh one
106:00 - 106:30 two three four in virtual memory points to like 5 six seven8 in physical memory but like you know you know address 4,000 points to like 100 or something right like these can kind of like be mapped however um and these get like loaded on like context switch and stuff so here here we have like we have asked for our memory to be like laid out this way I think um when we called like
106:30 - 107:00 shum Dev jamam create this says like hey could you like take some shared memory and give it to us for use with like the Linux Graphics systems stuff right but this thing um allows you to kind of page memory however because it's like way easier right like that way if you want to allocate a frame buffer that's like 4K you don't need a 4K uh sized buffer well not 4K is the wrong with here 4K pixels across so 4K times whatever right you don't need all that
107:00 - 107:30 memory in one spot you can kind of like gather it from wherever in physical RAM it like actually fits um and so what they do somewhere is I think there's like scatter gather table something so they say like hey let's get that memory and like collect it all up and we will tell the PCI device where each page is and it can like restitch them
107:30 - 108:00 together I believe it would be nicer if we could just create it with like it would be easier like for right now if you could ask it to create the memory kind of like all together but I think that that's not worthwh like that's probably not going to be simple so I think we just kind of do what they expect uh so I think in the vert iio stuff I we saw something that was like
108:00 - 108:30 map where did we find this the other day I think we we we came at it through like the cimu source code uh so let's go over here uh let's uh get into our quu development environment and we can look for the command that is used to send that memory I can't remember what it was but
108:30 - 109:00 I remember seeing something in like Source cuu like something like vert iio GPU I remember seeing something like pixman they create like a pixman buffer here pixman image create bits from vert ipu resource create
109:00 - 109:30 2D yeah so here we're looking for command resource create 2D that's what we're looking for uh so resource create 2D okay then there was something about like this creates the reason resource but then someone else Maps
109:30 - 110:00 it I think this looks familiar something something scatter gather table we would have been when we were looking at like the uh frame buffer stuff uh it's not really here not really here
110:00 - 110:30 I can't remember man it's bothering me where would we have seen it um defense control buffer yeah it wasn't really this oh wait this actually looks familiar SG unit
110:30 - 111:00 one transfer to host submit resource flush primary plane update yeah yeah yeah yeah yeah yeah yeah yeah so on primary plane update they call set scan out no and then they control the buffer so somewhere they must get the buffer object from somewhere buffer object
111:00 - 111:30 here okay then they must map the memory here Hardware resource handle oh they kind of map it before that right right right so they they did it on um they did it on frame buffer ad that's I'm pretty
111:30 - 112:00 sure pretty sure [Music] so kind of like here um frame buffer create frame buffer in it Yep this is looking
112:00 - 112:30 familiar or maybe it was on Buffer creation I can't remember it was like dumb create dumb and map Dum so when they create the dumb buffer did they do something funny there they just called
112:30 - 113:00 create yeah so nothing here nothing here so the frame buffer path does feel more likely how do they man we found it we actually did find it it was on like C set crtc maybe which would be in
113:00 - 113:30 like maybe we go into like drivers GPU DRM ver Io if we just like grip in here for like Sgt type or like DRM map Pages or like hold on hold on Source Linux Dr drivers
113:30 - 114:00 DRM uh GPU DRM for iio okay grab uh like dma okay this feels like what I was looking for yeah yeah yeah yeah yeah yeah
114:00 - 114:30 yeah yeah yep yep yep yep yep yep yep okay okay so here on plane update they call transfer to host 2D which just takes in this like buffer object here yes and then here
114:30 - 115:00 he says synchronize the given buffer for dma device which performs the needed cash synchronization it moves the ownership of the buffer back to dma domain so to safe to perform dma operation uh I'm not 100% sure this is the only place though I remember seeing something about like mapping uh map [Music] uh I'm not 100% on this not
115:00 - 115:30 100% because this is only on update [Music] submit Prime definitely not prime that's not what we're looking for could be invert iio gem Prime export
115:30 - 116:00 so does that line up if we look at um transfer to host 2D who dma syncs this stuff oh wait we saw we saw somewhere there was like ANS an and nents where does this come from object
116:00 - 116:30 create yes yes yes this was it this is what I'm looking for this is what I'm looking for um right so when they created their shared memory they immediately send that send that at the PCI device so here they create the resource and they immediately attach data to the resource this is what I was looking for so in
116:30 - 117:00 here they had uh the entities came from like here maybe yes this is I looking for so they call der dur jamm get pages Sgt so this pins the pages dma Maps them and returns a scatter gather table for them this is exactly what I was looking for um so
117:00 - 117:30 this function returns a scatter gather table suit for driver usage if the scatter gather table doesn't exist then the pages are pinned dma mapped and SG table is created this is the main function for drivers to get at backing storage and it hides the difference and it hides a difference between dma buff import and natively allocated objects hell yes okay so this is what we're looking for this is what we're looking for God damn dude I knew it shouldn't have been that hard okay uh enm CD Source
117:30 - 118:00 Linux and we are looking for derve uh uh spero derve here so on our plane update we want to call this thing we want to call this thing so this is going to come from what some sort of like shared memory handle which I don't know where ours
118:00 - 118:30 is to be honest right like our plane has to be attached to a frame buffer I would assume but we've never we've never hooked up the frame plane to a frame buffer which I get probably which I guess is probably what prepare FB is for right so like here we're going to have to like extract frame buffer for our plane map the
118:30 - 119:00 pages then write the pages okay so here's what I'm going to what I'm going to assume for now I'm going to assume that we can write out all of these Pages at once right and I'm going to assume that uh if we write to these two registers that it will trigger some action on the PCI device to write out that memory associated with this item okay so I
119:00 - 119:30 think that we can just kind of loop over like for each page in Pages essentially do like this right as well as uh probably this right that's kind of what I'm assuming we can do and then on the other
119:30 - 120:00 side we will see an address and a size and we'll just like rip that into a buffer and we will assume that they go in some well defined order right so we can kind of assume that like the pixels will probably come in like you know top first like in order right that's what we'll H so we'll say for each right we're kind of getting like the first n amount of stuff non codes thanks for the raid what's up everybody have to ra and leave hope you have a good stream we we sure will okay
120:00 - 120:30 thanks for the raid hello new viewers um we are working on a GPU driver so the idea here is that we are implementing a virtual GPU in a virtual machine and we are writing the equivalent driver to go with it uh just trying to get a picture of anything so the goal for today is just if we got you know a little picture of a smiley face rendered out somehow that a huge win and we're very we're very close we're kind of at the point where we have all of the memory in the correct location we've kind of got them allocated through the driver subsystem in Linux and now we're just trying to be
120:30 - 121:00 like okay so how do we get like get this memory to the PCI device and so like we are kind of assuming that we have like this guy is split up into bands of pages that might be kind of allocated just kind of like anywhere in system memory right so it's like could be like this guy's here this gu is uh this guy is here and this guy's here right and so we're like we what is our interface to our PCR device in order to like write that out and so we were just kind of saying for now we'll do the stupid
121:00 - 121:30 thing where we're just going to write three registers with an address of physical memory as well as the amount of stuff in that physical memory and uh we just have to figure out how to get that which I think is this which I think is this uh so where what does this guy take he's mad for some reason so he's expecting a Derm jamam object so this we we we don't know what that mapping is I think that
121:30 - 122:00 probably when we create when somebody calls prepare frame buffer I would think on our plane and we must have to use that to link the plane to a buffer that's kind of my guess so let's see uh what what is going on here uh so this is Pharaoh GPU prepare FB uh GPU plane prepare
122:00 - 122:30 FB and presumably in this state somewhere they must have the frame buffer yes very nice very nice and the frame buffer must have the actual data associated with him here here so we can be like uh new state plane frame buffer OB zero so this is our object and we need to associate that with the plane somehow so can we like just kind of like us is
122:30 - 123:00 we have like private data that we can work with here doesn't look like it it looks like what we should do probably is kind of store this in like our driver data so here we'll say uh Prim uh primary plane buffer object the data associated with this thing so this can just be [Music]
123:00 - 123:30 a DRM gem object okay so we have like in our private data we say struct d Jam object this probably needs to increment the reference count probably and when we initialize this thing we need to make sure he's initialized with zero so we do zero Al this thing with KZ
123:30 - 124:00 Alec I believe the Z in Z stands for zero yes so I think we're chilling so I think here we this is null by default and on frame buffer assignment we can say um this I believe increments the reference count so we kind of take this and we assign
124:00 - 124:30 that into our state so we need to like get our state out so it's kind of like this we say priv primary plane buffer object is this that sounds fine to me he's kind of mad about something here assigning to struct oh because this doesn't actually return anything so we kind of do this instead
124:30 - 125:00 where we go uh if we already had a buffer object then we need to free that so gem object what then we get we assign this to be what is associated with this plane I think and then we immediately increment our reference count to that thing so that we don't lose
125:00 - 125:30 it seems reasonable to me seems reasonable me okay so now now we have a buffer object so here we should say uh if does this if priv primary plane buffer object if it's not available we should like can we do like a bug
125:30 - 126:00 on yeah let's do that crash our Kel that's a bug baby that's a bug um and he's mad here actually because he wants a d a Derm gem scham object which is not what we have we have a DM Derm Jam object so how do we get the Derm Jam mam object back Derm gam scham object he has a Derm gem object inside
126:00 - 126:30 him so we can go backwards from this we know that if we have this thing his pointer is inside some Derm gem schm object at base this should be totally legal so here we say the
126:30 - 127:00 uh this thing uhm OB isal this this container overturns a pointer right uh yeah I think so so now we have like these T this like table uh of like pages in memory that
127:00 - 127:30 are associated with this thing so probably before we do anything here before we try to communicate with the GPU at all I think it makes sense to just look at what's inside here we expect to see a lot of yellow for our little yellow smiley face that we're trying to draw so I'd like to see like how do we iterate this thing so we saw here we can just call this for each
127:30 - 128:00 thing I wonder what the difference between this dma and not dma API is because these are both physical apis I guess dma addresses don't necessarily have to be something that you can access physically through your thing they just have to be something that the other device can access so there's probably something funny going on there that they kind of turn into the same same thing um but then can we see like you know can we just look at like the
128:00 - 128:30 address like is that a thing we can look at what is like the type of like pages and here pages sgns I okay so we'll just kind of copy these here for now hey all hello hello and I just want to like look at this like what is it so this this is our Pages Pages for us we call the SG table okay and so scatter
128:30 - 129:00 list is that our iterator no this is this is yeah this is our current location so we have page link offset length dma address dma length dma Flags so we actually like might not really be able to look at this directly which is kind of annoying I kind of want to see what's in it I want just look at it is there like
129:00 - 129:30 SG oh it would just be so nice to look at I guess we could go through I mean they had SG Fizz is there other like SG vert this is probably what I want this just gives me like a address I can look at right the caller must know the SG space has a v has a valid virtual mapping oh I guess we don't really know
129:30 - 130:00 that can we like map it I just want to look okay can I we'll ask the idiot uh if I have a what is it called a uh struct SG table in the Linux kernel preparing for dma with a PCI
130:00 - 130:30 card can I look at the contents of the memory I am about to D from the Kel is there an API for that I see stuff along the lines of SG Fizz and SG vert but am unsure the rules about how
130:30 - 131:00 to use them this guy like keep in mind this guy's a idiot might be wrong but uh this function is corresponding to a given scatter gather this is what use Access Memory C so that lines up with my current understanding so that's at least you know something uh this is probably the piece I was missing here so he gets the virtual address and then he calls sync for CPU that's really the piece I'm missing
131:00 - 131:30 here uh so this takes in a struct device why what is the device for whose device is this uh okay is there like DRM sync uh I mean I guess we could maybe assume that it's related
131:30 - 132:00 to when did we create this thing we called gem object somewhere create buffer Dum D create here we call Derm gem schm create okay so then this guy must make like a
132:00 - 132:30 dble thing somehow I guess the device must be whoever he uses internally in here right like somewhere he must call a similar function OB Dev Dev so this is D gem object durm device base device
132:30 - 133:00 okay so we can probably assume that we can call like DRM Dev Dev here and then he's kind of saying you can just run it then so he has data as SG for SG so we have just like un a uh D void star data I
133:00 - 133:30 guess this returns so we can call Un tar data okay it right uh oops where the did I just go there we go uh u8 star I guess is probably like the kernel type for this and can we just like print that so what's the size what's the size here scatter list dma length would be my length probably
133:30 - 134:00 length uh so for z uh in I equal z i is less than SG length Plus+ I can we just like print data uh probably in HEX data plus I I don't know uh print K I'm down to run it and see I'm
134:00 - 134:30 down to run it so what are we expecting to see we we're expecting to see a bunch of yellow a bunch of yellow uh he's mad because we he wants to return somewhere at uh 224 uh yeah okay uh bounce check surely this has to
134:30 - 135:00 be done I'm so used to Zig doing it for me uh okay run it quit quit it GDB it all right let's see it okay so I kind of expected to see our fun oh
135:00 - 135:30 continue okay so I expected to see something happen there which I don't which is kind of weird this prepare FB is just like somehow never CAU so that's kind of weird well I guess we can't be we can't be sure of that we can't really be sure at this point maybe it's called and it just doesn't our like Loops have nothing in
135:30 - 136:00 them so here in dium atomic helper prepare planes we expect him to end up here so let's kind of like breakpoint here can we delete breakpoint one continue and then run our test app again okay uh yeah okay so he's he doesn't have anything there so that's very surprising very surprising cuz we oh cuz maybe cursor
136:00 - 136:30 and primary do we register the primary we do kind of see it's weird that he only tries to run that one time oh no he does try to run it twice I just continue on the second one like a dumbass hold on hold hold on run again
136:30 - 137:00 continue I don't really know what's happening right now but now we're seeing something again it's null oh so we're probably seeing after the 5 Seconds he tries to prepare the frame buffer again on quit so I I think I thought that this prepare frame buffer would be called and it is getting called but for the wrong
137:00 - 137:30 plane which is kind of surprising pretty surprising to be honest so we oh that would it huh where we were assigning the cursor helper functions to the primary plane by mistake because we're using the wrong variable like dumb asses like dumb asses
137:30 - 138:00 man man thank God that often like the code you write has observable effects because I feel like my like workflow often is just like it up and then like fix it later and uh like I wonder how many bugs I write that I just like don't notice all right so now on prepare FB he's got a null pointer D reference that's kind of
138:00 - 138:30 fun that's kind of fun so that's here I wonder where so we can check this here so we'll set I think we can go into our
138:30 - 139:00 here and we can say info line and we just kind of ask it where the is this man why is that not copy pasting no do I shift click here uh so this is at line 223 right here right here so which of these like many things
139:00 - 139:30 is incorrect is the question I guess we just break point here I don't know if it's going to let us run this again without like falling over it does look like the colonel just like hard crashed uh so I guess we run it again and this is this one's kind of tricky because the break point we have to set is like inside this kernel module that he doesn't know the address of
139:30 - 140:00 initially so I don't really know how we're supposed to like automatically set up these break points to be honest so I think we'll just have to do it manually because I don't know how to do it I don't know how to do it but I do know how to just do it manually right we run contrl C LX symbols that will load uh LX symbols break here now that is available for us root test app and let's look at
140:00 - 140:30 priv primary plane po so that's fine new state plane FB so the new state plane FB doesn't have anything so how are we supposed to do anything thing here vert IO vert GPU display how do they do
140:30 - 141:00 it [Music] prare I guess we look for references to this and we look for the one invert GP plane I guess we just do this I guess it's like a thing that can happen should we read the docs uh this look is to prepare frame
141:00 - 141:30 buffer for scano by pinning its backing storage or reting into continous block of vram other possible Preparatory work includes flushing caches this function must not block for outstanding rendering since it is called in the context of the atomic ioctl for asyn blah blah blah blah um this persists after the end of the atomic commit resources can be released at the end ofit blah blah blah okay I mean I guess that's fine but it's
141:30 - 142:00 kind of confusing to me why this is happening makes me think that my understanding of like when this is called is not correct um but you know I guess we just run it and we'll maybe print K uh no plane frame buffer and then we say got plane frame
142:00 - 142:30 buffer because I kind of want to see if this ever happens it's kind of confusing to me it's not kind of confusing because otherwise I have no idea how I'm supposed to tra Trace that together okay like how are we supposed to tie the buffer to the plane I don't know I don't know I thought it was
142:30 - 143:00 here okay so here no plane FB and then we bug on 192 right so then we don't have a buffer object available so like what the I mean it could be called that this is somehow called multiple
143:00 - 143:30 times could be because what does he do here if there's no frame buffer he goes okay I'm out I'm outtie um then he says if there's no or we a thing and no okay uh yeah I mean I guess he doesn't do Jack shed here
143:30 - 144:00 huh Atomic update where does he get his buffer object from uh so there might not be a handle here there might not be which is kind of surprising kind of
144:00 - 144:30 surprising but I guess we'll just play the same game I guess we just play the same game here and we see where it goes maybe they call multiple times uh print k no plane buffer to update right we just GNA get the out of
144:30 - 145:00 here we'll just see what happens run with GB continue let's just see where we go let's just see what's up yeah so he just says no flame frame buffer and no plane F frame buffer to update so right so we just never never
145:00 - 145:30 tie the frame buffer to the plane which is surprising which is surprising I wonder if we're supposed to do this on like ad on like add frame buffer call I wonder if we're just supposed to kind of like assume that that's like tied
145:30 - 146:00 together I guess this stuff could be relevant probably right surely maybe it could go here so in our little test app our little test app we say hey I have this crtc and this Frame buffer that I'd like to tie
146:00 - 146:30 together so that kind of feels like it's kind of should go in here somewhere uh so we can look at the usage of this in vert GPU and in one of these many things does
146:30 - 147:00 he I mean this kind of feels like where it's supposed to go can we look at this so um after mode set I guess we just look at the uh
147:00 - 147:30 implementation like where is like Atomic jeremi octal that's probably what makes most sense I'd imagine display drivers are hard to test at least on really Hardware right I don't know I've never done it I've never done it uh set crtc so this is what they call that we think should like actually trigger something to like show up on the
147:30 - 148:00 screen so where what I want to know is like one of the many things that we pass into this one of the things we pass in is like the frame buffer ID right that's what we passed in here file PR void start data data here we go this is the request so who uses this so there's some like frame buffer
148:00 - 148:30 ID lookup in here so we like we can just look for like okay so he gets this Frame buffer we should check that we end up here can I put a comment here that's like uh frame buffer look
148:30 - 149:00 up then surely like somewhere he like uses this somehow set. FB equals FB K set config which we have set up here this goes through to like the DRM Atomic
149:00 - 149:30 Helper and set FB for plane okay wait this is exactly what I'm looking for so this calls DRM frame buffer assign Plane State FB maybe we should turn on these DRM debug logs and see if they tell us anything because it sounds like to me that the
149:30 - 150:00 DRM Plane State should get the frame buffer assigned to it like I thought right that's what I thought was going to happen but like something here is not doing what I expect so it could be like order of operations here could be and then he calls update output state so like set crtc for connector surely
150:00 - 150:30 this is going to end up somewhere in our like set crtc somewhere yeah I'd like to turn on these debug flakes so I think that this was in um CIS module DRM parameter is we could Echo ZX FF to debug to turn on all debugging things and we just run our test app
150:30 - 151:00 again okay and we can just kind of scroll back in this and try to see if we see anything interesting so like for example we're looking for a set no FB for plane or like set no FB for plane okay so like why this that means that this Frame buffer
151:00 - 151:30 is not initialized at this point right so we're kind of on the right track uh it's that this set of B either equals no null here okay it could be this or it could which is means set.
151:30 - 152:00 mode or we had like here okay this is the interesting piece here so we should put set a break point here and look at the contents of said I think these are the next two pieces to look [Music]
152:00 - 152:30 at okay so here we did get a frame buffer of that so that's a really good sign um with with like a 1024 by 768 so that that looks a lot like we populated this what we wanted it to be um but can we look at like set here now so we have mode we have FB so
152:30 - 153:00 neither of these two things seem too suspicious to me which makes me want to step in here and try to figure out what the is going on here right because we saw we saw that he was in no FB mode is this vendor specific uh some of it is some of it is not so the code we're in like at this very second looking at this is kind of like DRM subsystem helpers uh that we
153:00 - 153:30 have like kind of opted into as like our vendor spaphoria um but we uh sorry my brain um there's also like vendor specific code that this is eventually in up in uh cuz we we assign like this the the generic subsystem has like helpers and those helpers are ours so I think we just breako it
153:30 - 154:00 here which I think we might already have info we have one at uh no let's S one here as well okay continue so this is where we were seeing that no FB log right so here we should see uh FB here oh he is set so how come in our logs we didn't see like set FB for uh I guess it's
154:00 - 154:30 gone okay okay wait that kind of makes sense then so we're missing something we're definitely missing something because in here we definitely ended up in the path where he Logs set FB for plane okay so let's try to look for that then we also see a state where we're in set no FB for plane but that's when the process ends okay okay okay CIS module parameters uh sorry DRM
154:30 - 155:00 parameters Echo Z FF to debug run it again yep so we should go through here [Music] and we should see a set of B
155:00 - 155:30 here so it would be would be in here somewhere I just have to find it am I stupid can I search on this I guess I can copy paste it out copy no I guess we copy with the middle click
155:30 - 156:00 buffer oh here set FB for plane okay so we do see this we do see this that's a good sign that's a good sign oh and even here we see print new state we see plane 34 has frame buffer here allocated by test app with this sez so like what's the problem what's the problem because down we get to here on Commit
156:00 - 156:30 it's gono right so somewhere between here and here something has gone wrong print new state checking f193 Fe which is again here so committing this right he commits this this is the same buffer right he's committing this here and he's going whoa no plane FB so somewhere between
156:30 - 157:00 here and here we've lost it we've lost our frame buffer or we're doing something stupid that's also like a very real possibility right that we should always keep in mind uh so let's look again at our no plane FB this is kind of where it falls
157:00 - 157:30 over and so maybe we should just kind of set a break point in here and back trace and try to figure out where this thing should have got set maybe new state isn't what we think it is right maybe it's actually under plain State could be right so let's set a break point here um and we can do Alx symbols except I spelled symbols wrong so it's just going to like freak out of me oh I guess I have to pause the app break point
157:30 - 158:00 here okay continue and luckily I think that like nothing's crashed to the point where we can't run again so can we look up New State plane FB okay so he's mad but what about if we look at plane FB also null okay so somebody somebody has gotten angry between then and now now if we look at like the current debug output we see that he he did print that
158:00 - 158:30 there was supposed to be something there so what the happened let's look at the back Trace [Music] okay so we're at Atomic commit uh okay can we look at like so we're calling prepare FB with plane
158:30 - 159:00 and new Plane State where new Plane State comes from 2601 in atomic helper okay new Plane State so this is for each new plane in state okay
159:00 - 159:30 okay so uh what's our thing State optimize out I wonder if we like somehow like mismatched our cursor and primary planes again that's a very like likely place where we would have up very likely up
159:30 - 160:00 place so this comes from primary helper funks which are attach to the primary plane who has Type plane primary and we have the cursor plane helper funks do jack okay who are attached to the cursor plane curs plane is used here here here primary plane should be used up here great and then is it possible that we switch the order of them in the init with planes no they go primary than cursor so I don't really see anything
160:00 - 160:30 crazy there um I really don't I guess if the format was rejected that's a possibility what are we trying to do in our application do we ever like try to set a specific like buffer color type somewhere create dumb buffer we just and
160:30 - 161:00 then we attach we add a frame buffer with bits per pixel 24 [Music] so I don't think that's a problem when we make our frame buffer
161:00 - 161:30 uh here frame helper check somewhere we do like we check for if something is 32 here dumb create oh yeah okay should be fine should be fine does it check for all new ples here does it a out on the first where it's wrong uh good
161:30 - 162:00 question good question uh so this is in here he does fail out on the first prepare FB so it could be if we're like looking at the cursor one first but like here our prepare FB doesn't fail what I would be interested in maybe
162:00 - 162:30 looking at um like my multiple frame buffers in here so I'll be kind of interested in break pointing on this and kind of seeing what that does let's just give him a sec give him a sec okay test
162:30 - 163:00 app so here can we look at like State optimize out off man that's so annoying uh okay so like funks here comes from plain helper private and
163:00 - 163:30 plane is iterated from Planes i. pointer off off uh but I guess if we go up a level we should be able to look at the State here yes yes okay okay so State uh what are my new plane States in
163:30 - 164:00 here it's state Arrow planes zero and one uh okay planes can I look for like he is going for I how does he how does he like bounce check this
164:00 - 164:30 thing for each if planes pointer ah so the it's kind of like null terminated it's null terminated so we have one plane here and then the this the second plane is all terminate okay okay so planes zero shouldn't we have multiple planes like the cursor plane in the primary plane Maybe not maybe the cursor plane's not used
164:30 - 165:00 here maybe the cursor plane's not used here oh apologize for the apologies for the lag apologies um okay so there's only one plane and we have state okay wait plane zero State FB does
165:00 - 165:30 exist okay okay that's a good clue which kind of makes me feel like in our function we should see
165:30 - 166:00 Plane State FB which is zero so what am I missing here what am I missing here where this comes from new Plane State oh okay okay wait hold on so we
166:00 - 166:30 have this guy has a has a frame buffer but if we look at the new state that also exists but old state does not so this guy is supposed to say hey I'm looking at the new state for each of these things new plane state is equal to
166:30 - 167:00 this and as far as I can tell at least at this call site at deam Atomic helper prepare planes this is fine but by the time we get into our call down below it's not fine
167:00 - 167:30 so JM Atomic helper commit I guess we kind of know exactly what line that's supposed to be at so this is at Atomic helper 2029 so at this point he thinks it's fine and then that's literally like one one level up so like what in the actual
167:30 - 168:00 what's our I value zero dude I'm actually going to lose my mind new plan state comes from State planes i. new
168:00 - 168:30 state state has been optimiz out which is valid what the I mean this could be some like GDB could be right very well could be but the way the way that this is
168:30 - 169:00 like really what I would expect right is very confusing to me very confusing to me and what's even more confusing to me is there's nowhere between these two places is that like I expect to see like something
169:00 - 169:30 change you know but like sure I mean for sure in here oh fine oh wait am I stupid oh new plane new state plane FB
169:30 - 170:00 but there's new state FB do I just have to attach am I been looking at the wrong thing hold on maybe here I'm supposed to set that okay wait hold on hold on hold
170:00 - 170:30 on now I'm confused de on plane so there's new state FB and new state plane FB and new state FB must be what is propagated on from that dur set set crtc thing okay okay I I think I'm understanding at least a little bit right cuz they have mode crtc this is the frame buffer ID that we're trying to
170:30 - 171:00 render which we extract from this thing which we then apply to set FB here okay and yeah of course the plane doesn't have anything so this is where it's set I was confused all the way around so now the question becomes the question becomes is it expected for me the driver rer to just send that or is that like internal tracking so I would love to see uh
171:00 - 171:30 jeram plane currently bound frame do not WR this directly use di Atomic set Atomic set FB for plane oh yes okay di Atomic set FB for plane who the calls this some people in here oh wait okay okay okay so they come this comes from set Arrow FB in like diatomic helper second
171:30 - 172:00 fake primary State set FB for plane I forget did we not see this though this looks familiar this looks really familiar but I don't
172:00 - 172:30 remember okay I want to I want to go down here now and look at this set FB for playing then because I remember this Frame buffer being null actually that's actually what I remember uh okay so we're going set FB for plane here which comes from set config on set crtc right and this is valid okay
172:30 - 173:00 okay so this this feels like he calls Jam frame buffer a sign here okay and at this P at this point we should see primary State plane
173:00 - 173:30 FB zero what the just happened right because we just saw set FB this is valid oh this is also valid uh this comes from plain
173:30 - 174:00 State oh yeah primary State FB is set here okay wait that's good that's good so I feel like this is what we want right our primary state is a dur Plane State so who who which comes from the crtc primary okay so we are seeing a plane get a
174:00 - 174:30 frame buffer associated with him it's just not the plane that we're seeing for whatever reason can we look like this who uses uh oh I think I'm confused about levels of stuff again I think I'm confused about levels of stuff because we when we get this in
174:30 - 175:00 we get a dearm plane input here and a DI Plane State back pointer to the plane so I'm confused here maybe we're just not ever supposed to look at this like is this just supposed to be like
175:00 - 175:30 this and then this might be like the old this might have like the back pointer might be like the old plane here and we're like switching them out or something maybe maybe so if I call prepare FB here this look prepare a frame buffer for scan out EG by pinning its backing storage or re relocating it to a continuous block of vram other possible PR Preparatory work includes flushing catches this function must not block for outstanding rendering blah blah
175:30 - 176:00 [Music] blah okay okay I guess that's kind of reasonable I guess so we we then just kind of attach the wrong thing and then in this function we could have checked it actually turns out we probably could have just checked we probably didn't need to do this actually actually actually we break here can we look
176:00 - 176:30 at the plane that we are committing he doesn't have a frame buffer maybe that's because he gets set after our function finishes so this is kind of weird here this is kind of weird here so the new state here oh yeah there's something there's
176:30 - 177:00 something odd here that I don't understand but we have the state here so I guess we could have been looking at a state planes zero pointer FB right no even that's not okay okay so how come he was in the new state here but he's not available to us on the plane update
177:00 - 177:30 here you know see PC like uh new state [Music]
177:30 - 178:00 maybe oh yeah wait am I supposed to look at P State ples Z new state FB uh ah I see I see I'm supposed to be looking at the new planes so I think we just don't need this prepar prepare prepare frame buffer thing and in atomic update we can in
178:00 - 178:30 fact just extract the information that we want here so it's this so just boom state how am I supposed to know which plane this is current atomic state is there like
178:30 - 179:00 new state for this plane available to me here somewhere I guess it also look to look at uh Plane State Plane State no simple plane plate Plane
179:00 - 179:30 State FB ah okay so I look at this which is about to be committed this is one4 and then uh fix me does this need am I mutex locked right now I'm not sure but we finally found it guys we finally found it we found our frame buffer it took about like a year but we got there um okay so we have this thing
179:30 - 180:00 now and we can extract the buffer object from this okay so that means this primary plain buffer uptic goes away and this is a struct DRM gam
180:00 - 180:30 object and then we get the schm object out of this gem object okay okay [Music] okay and then maybe this works now let's hope it' be so cool to see something that was like sayane some real some real things that I wanted to look at some yellow if it's yellow let it mellow I sure hope that we can be mellow
180:30 - 181:00 now also I think that you should flush your pee it smells really gross if you don't I assume everybody's pee smells gross if you don't but like maybe maybe it's just my pee but I would like to hope that it's everybody's pee otherwise there's something wrong with mine that looks some like yellow man if I've ever seen it now that's a lot of printing but we are seeing like alternating ffs and zeros um so I think that what will be an
181:00 - 181:30 easier test here is we can count how many yellow pixels and how many black pixels we see and how many other pixels and we can check if that matches our like assumption about what a yellow smiley face should look like right because the image that we're rendering uh is something that looks like this I think you need to see a doctor I think you're with me there's no way that your pee doesn't smell bad if
181:30 - 182:00 you leave it in the toilet for a long time it's it's got to smell bad I think that you have to that's why you flush it right so we should see like this should all be like yellow and this should be black and we should see nothing else how long are we talking I don't know like if you if you only ever flush the toilet when it wasn't yellow then like you'd be leaving for
182:00 - 182:30 like 24 hours right and that would smell disgusting I assume I mean I could also just be wrong here maybe my piece smells great after 24 hours I'm just kind of making the assumption that would smell bad I'm just making a big assumption I guess uh so count yellow is zero let's do this and I count black also zero an hour in the summer is too long
182:30 - 183:00 flush it okay least you have one guy who Dees me I can leave it for four to six hours and doesn't smell bad that's crazy I guess you I guess in your world if it's yellow you can let it mellow even if it's yellow not if it's just clear I can see if you're good if you've just got like well hyd ated P's it would be okay but yellow P I think that stuff's got to start smelling bad uh okay so instead of printing here we're going to check I think we're going
183:00 - 183:30 to iterate this as u32 is and we're going to say uh for I is Zer to length over four let's try to count how many pixels we see so if we're going to say uh let's look at our example app again so our example app writes out he clears it with ffs are in the middle ffs are in the middle so we go uh
183:30 - 184:00 if data I is equal 0x FF or 0 FF ff0 oh jeez I've accidentally fled and gone away for the weekend that would suck coming home to just like oh that's gross that's gross let's change topics we don't have to talk about this this was my bad this was my bad
184:00 - 184:30 else if it's black let it smack count black goes up by one and otherwise we have count other and and we should set our count other also to zero um okay then we will say print K
184:30 - 185:00 yellow this many uh black this many other this many and we're hoping to see a lot a little none approximately if it comes out black definitely goes to a doctor dude black is crazy red Red's like oh God scary go see the doctor but black is like what is happening like I'm
185:00 - 185:30 drinking tar stop drinking tar um all right all right View cial and uh continue continue oh let's go dude it took us like forever we also
185:30 - 186:00 page faulted somewhere later uh that we have to think about but we do see we finally we walked all of the pages associated with our virtual memory uh for our yellow face and we do see yellow black other that looks like a face that looks like a face now we could also check maybe how many pages we're walking just to make sure that we're actually doing something that I think
186:00 - 186:30 we're doing so we'll say a num PES Plus+ one here um and that's pretty sick I really wish we had something more visual to show uh but this is pretty cool pretty cool Cal all right one more time all right let's see
186:30 - 187:00 it oh we didn't log number of pages one more time again this time's the real one more time could be really bad blood ew I like it all right let's see let's see let's
187:00 - 187:30 see uh test app yeah so we go over 220 Pages um but we do see that we see consistent stuff that's really good that's really good that really indicates that like the our expectations of the memory layout are correct right we're seeing like you know a yellow page over here and then like a yellow page over here and then
187:30 - 188:00 like occasionally see we'll see like a page that has like yellow with some like black in the middle or something um which pretty sick pretty sick okay so it's 510 we've been talking about P way too much so I think that that is a good indicator that we've gone way longer than we should have so I think we call there we've gotten I think to the point now where the driver itself
188:00 - 188:30 is kind of doing what is expected of it he's not actually talking to the hardware but we have the whole flow of information now to the point where we are seeing the buffer that we are trying to write the the buffer that we wrote from user space is in fact getting into the kernel in the right spot um at the right time with the right size and stuff so I think I think there we were almost there the only thing that we have left to do is to hook it up to the hardware um so I think that that we is going to
188:30 - 189:00 be the plan for tomorrow is we're going to try to like map this stuff through PCI into the host we'll try to write an image and then we'll maybe try to integrate with uh Q emu's display systems that and then maybe if we get that all working quickly then we can do like multiframe stuff but I think I'm very very happy with the progress today um I apologize that it's not super super visual I was kind of hoping that we would get to something a little more visual today but you know we'll live we'll live so thank you for watching you guys um if you like what you saw we
189:00 - 189:30 stream most days at around 12:30 Pacific time to 4:00 Pacific time it is 5:00 Pacific Time now so anywhere between 4 and a half hours ago and an hour ago is when you should typically expect to see me um if you're watching on Twitch and you haven't been around much before there's a YouTube link in the twitch description where you can check out all of our previous projects we're working on demystifying the Linux Graphics stack right now but there's a lot of other stuff if you want to check that out uh QR code decoder operating system like some like 3D work in this one uh you know there's stuff in there Maps stuff people seem to like so if you want to check that out there's available running a terminal emulator terminal emulator
189:30 - 190:00 also GitHub Link in description where this will eventually end up under the GPU testing EV but probably not until I'm happy with like the current like the until I can call it like a feature you know um if you're watching on YouTube there's a twitch Link in the YouTube description swing by say hi if you feel like it drop your Twitch primes like subscribe um I guess if you want to see the 12:30 to2 segment archived those are archived in patreon on you and YouTube member streams so if you want to check those out those are also
190:00 - 190:30 available um and I think that is all we have to plug thank you for watching YouTube let's get out of here and twitch let's find someone to raid