Where to start with 3D Gaussian Splatting in 2025

Summary

Highlights

• Learn to recreate scenes in 3D using 3D Gaussian splatting. 🌟
• Utilize smartphones, as an easy entry, for capturing necessary images. 📸
• Iterate the process thousands of times to refine your 3D models. 🔁
• Manage exposure meticulously for best results in scene creation. 📷
• Explore user-friendly apps for Gaussian splatting; perfect for beginners. 📲

Key Takeaways

• 3D Gaussian splatting creates lifelike scenes using millions of ellipsoidal points! 🎨
• Imagine capturing scenes with just a smartphone! Perfect for beginners! 📱
• Get ready to iterate—thousands of times—to achieve hyper-realistic models! 🔄
• Utilizing view dependency creates stunning realistic views from every angle! 👀
• Balance exposure in your shots to avoid those pesky 'floaters' in the scene! ⚖️

Overview

Chapters

• 00:00 - 00:30: Introduction to 3D Gaussian Splatting The chapter introduces beginners to the concept of 3D Gaussian splatting, specifically targeting those looking to create their first scene. It emphasizes the step-by-step process to make an effective Gaussian splatting scene, positioning this as the starting point before delving into advanced techniques. A particular scene is mentioned to illustrate the concept, prompting readers to explore what they see.
• 00:30 - 01:00: Understanding Gaussian Splats The chapter provides an introduction to Gaussian Splats, comparing them to point clouds and explaining their unique shapes and forms. It describes Gaussian Splats as flexible in shape, akin to points that can be football-shaped, orb-like, or elongated into spikes. The purpose is to familiarize the reader with the concept and formation of Gaussian Splats.
• 01:00 - 02:00: Creating Gaussian Splats from Imagery The chapter discusses the concept of creating Gaussian splats, which are ellipsoid shapes with Gaussian distributed density. These splats have a concentration of color in the middle, which fades out at the edges, creating a feathered effect. When many of these splats are overlapped, they form an interesting visual scene. Additionally, these orbs possess a view dependency, altering their appearance based on the viewer's perspective.
• 02:00 - 04:00: Iterative Process of Scene Creation The chapter discusses the iterative process of scene creation, focusing on the concept of view dependency. It highlights how the appearance of points in a scene can change depending on the direction from which they are observed, much like reflections in water. This view-dependent characteristic is crucial for creating lifelike scenes. The chapter emphasizes the complexity involved in managing a dense point cloud to achieve such realistic scenes.
• 04:00 - 06:00: Capturing Techniques and Equipment This chapter discusses the process of creating a scene from imagery, using a telecom tower as an example. The chapter explains that overlapping imagery is collected to see all sides of the object. A point cloud and camera poses are then used to construct the scene, with the note that specialized software handles these tasks automatically, allowing users to focus on capturing the imagery.
• 06:00 - 13:00: Software for 3D Gaussian Splatting The chapter discusses the process of 3D Gaussian Splatting using software. It details how camera poses and shapes in a scene are utilized to form ellipsoids and points in a point cloud. These points are transformed into 'Splats' or ellipsoids to create shapes observed in the scene. The text describes an iterative process where each shape is either cloned or more points are added to enhance the representation of scenes, like forming a tower progressively with these shapes.
• 13:00 - 14:00: Conclusion and Future Content Plans This chapter discusses the transformation and evolution of point clouds into various shapes, specifically in the context of a telecom tower's antenna array. It explains the process of starting with an ellipsoid shape that grows, morphs, and splits into more complex structures, highlighting the complexity and dynamics of such transformations. This serves as a metaphor for how content evolves and adapates over time, hinting at plans for future content development.

Where to start with 3D Gaussian Splatting in 2025 Transcription

• 00:00 - 00:30 are you looking to make your first 3D gin Splat scene and you don't know where to get started well I'm here to show you where to get started with God and splatting in 2025 there's a lot of different things you can do a lot of different ways you can get started but this is the right way to start so you will learn how step by step to make a good gajan splatting scene and then you can move on to the more advanced tips so here is a really interesting scene with gajan splatting and you might be looking at it and thinking what is this is it a
• 00:30 - 01:00 mesh is it a point Cloud you don't know what it is is it a video well first we're going to start off with just talking about what you're looking at and how they're made just so you have an understanding of what we're doing here so a gajan Splat is basically think of it as a point Cloud but each point is really kind of shaped like a football or it could be squashed more like a like a round orb or stretched out real thin like a really spiky Spike but it really
• 01:00 - 01:30 is just an ellipsoid shape and it has what's called a gajin distributed density that also means that if you look at the edges of it it kind of feathers out the density of the coloring and how like how dense it is so in the middle you're going to have lots of color it's going to be really bright and at the edges you're going to barely be able to see it so when you overlap a lot of these you get this very interesting looking scene and lastly they have this view dependency so if I look at one of these orbs and you're going to look at
• 01:30 - 02:00 millions of them at a time and I look at from One Direction one of these might appear to be blue like in water but then if I change the direction which I'm looking at that same point it might be white think of reflections of water it doesn't look the same from each Direction so it's view dependent and that's really important it'll get us these lifelike scenes like this one I'm showing right here so as we think about this it's really just this really dense complex Point Cloud so how is this made
• 02:00 - 02:30 how are these all put together to make your scene well first you take imagery and in this example I'm going to use a telecom Tower because it's a really easy thing to to grasp and I have all this overlapping imagery I'm going to see all sides of this Tower and then what I need is a point cloud and Camera poses and a lot of this information you don't have to actually make the point cloud and Camera poses on your own that's what the software will do for you so you don't have to worry about it but that's what's happening in the background and it's
• 02:30 - 03:00 going to take these camera poses and it's going to take all these shapes in the scene and take these ellipsoids and just take all those points in a point cloud and say each of these points become one of those Splats one of those ellipsoids and start to try to make shapes like you see in the scene so I saw this awesome Tower and of all these shapes and over time it's going to go through what's called iterations and it's going to take each one of these little shapes and it's either going to clone and make more points point to try
• 03:00 - 03:30 to fill up this area or this shape or it's going to basically split like a large one and start to evolve into these shapes and let's see what that looks like that's really hard to grasp when I talk about it but if I move on you what you got here is like here's the antenna array on that telecom Tower and it might start out with a really you know we start with a point Cloud it says all these points start out as an ellipsoid and then it they start to grow they morph they split they become more and
• 03:30 - 04:00 they start to fill that up and this is only 10 iterations so if I take this further after 100 iterations you can see I got this like really large orbs I got these smaller ones and they're trying to recreate the scene as I said that at at the center they're really sharp and they have a lot of color and you get further out they get fuzzier and you can see that fuzziness everywhere but if I move in a little bit further we got a thousand arations so now this sc's starting to make sense because our Splats have doubled now we're making up
• 04:00 - 04:30 the scene with uh 400,000 plus Splats and now I can see okay that's a car we see we see a I see a van I see the Telecom Tower but again it doesn't look great so what we do is the software will Loop through all the images and reference each image one by one through loops and then again try to evolve to create more Splats and so now here we go we're already at 3,000 iterations in this example we're at you know 2. 6
• 04:30 - 05:00 million Splats and you can see that on the right here is all your Splats and on the left things are really coming into Clarity and we got a lot of overlapping images a lot of reference things are looking good on the fringes not so much but you can see that it's starting to learn what the scene looks like and create this hyper realistic model now we're at 7,000 iterations things are looking really good 5.3 million Splats move it along to 15,000 iterations now we're getting into where a lot of the software really
• 05:00 - 05:30 starts to shine and you're getting these fine details like these little cables and uh the grass and everything starting to really pop up and look really really uh realistic but in in the background all is these ellipsoids and when they're overlapped in this gajin density they look like this they look like a photo and so if I take it out to three 30,000 iterations and almost 8 million Splats it pretty much looks a photo at
• 05:30 - 06:00 the edges here where we have very little information things start to look not that great but overall you could almost fool sew so if I show you the ground truth comparison so on the left we have the Splats on the right we have What's called the ground truth which is just the image from where that was taken you can see we've reproduced this whole scene and in some areas it's almost identical and maybe in this background we didn't we didn't get too many images back there so it does its best and then you get this really good scene
• 06:00 - 06:30 so basically what you're looking at is an optical illusion that is basically a fancy Point Cloud if you want to think about it that way and here you go here it is in 3D where it's moving you get all this great information on that telecom Tower something that you really couldn't easily do in meshes you get this Parallax movement in the background so it feels it looks like 3D and this is a really basic explanation of what's really happening here there's a rasterizing to view it there's all these
• 06:30 - 07:00 other things but as far as a beginner all you really need to know is that we're just trying to create the scene as it was visualized in the camera pictures you've taken so if you keep that in mind as I go through this tutorial it's all about getting good pictures and trying to reproduce it okay so now you you're starting on your journey you want to make your first one you got to think about what Hardware I need of course you need a camera because this is a camerab based technology ology that takes images and
• 07:00 - 07:30 makes novel images so basically we're taking a view and we're going to recreate the scene from views you've never seen that's I was able to move around in 3D so you need a camera and there's basically five main camera categories I think about when talking about this but if you're getting started I really want you to focus on the iPhone first Master the smartphone the the Samsung phone whatever you have in your pocket
• 07:30 - 08:00 that is where you should start don't worry about anything else why you already have one also they're really like they're really easy to use you don't have to think about what lens there is to put on them you already have cameras built in um it can support third-party apps which we all talk about uh you so you don't even need Hardware beyond the smartphone you could do this whole thing with just a smartphone uh it's easy to get everything in Focus these are all geared get
• 08:00 - 08:30 hyperfocused images what that means is you can get the person or object in the front really sharp and the background it's all going to be sharp because when you're recreating these scenes if you think about it you want everything to be in Focus because you want the background to be in Focus you want the foreground to be in focus and then you can always add depth the field later using special plugins but if you don't get it in focus with the picture at the start you're not going to be able to reproduce a good repres presentation of those objects in
• 08:30 - 09:00 the background or in the foreground if they're out of focus so start here Master this camera first before you do anything else drones I'd say if if you do have a drone and you know what you're doing and you're like a really skilled drone pilot I would say you can also start here but I would still suggest the iPhone first but you have this gimbal on there so you're going to get this really smooth images um it stay away from any like detachable payloads with a zoom you
• 09:00 - 09:30 basically just want these like I'm going call them I call them prummer or not toy drones but these drones that have a really good camera that don't zoom in and out or anything like that because you don't want to De if you if you zoom you might have some big problems down the road but um these are great and then used to flight modes there's these orbit modes um you have all these different modes to get the right imagery and that makes everything all smooth you don't have to do any like jerking or pain scanning and that's really what you want
• 09:30 - 10:00 because when it comes to Flying a drone it's all about getting that smooth images nothing shaky and so you can get really good photos here or video just um I would suggest if you've never used a drone don't go buy one to learn gaj and splatting either figure out your smartphone first so let's talk about capture so now you have the right Hardware now you got to think about how am I going to capture the images to get these really cool outputs like this one okay so when you're walking around and
• 10:00 - 10:30 you're capturing you want to think about capturing with your feet not with your arms what does that mean that means if you're capturing a room an object you want to be walking around it and getting different viewpoints you do not want to stand in place and move that camera around if you do that everything's going to fall apart very quickly so capture with your feet walk to different viewpoints do not stay still and then along with that for a room if you just want to get started you just walk a
• 10:30 - 11:00 circle and you can get these really great captures like this one in the subway it's amazing what you can get just walking a circle and varying your angles so you want to do multiple Loops so I'm going to walk around this room at three different heights when I say that you want to hold the camera High you want to hold the camera low and as you do that you want to be like up high looking down you want to be in the middle looking straight across and low looking up so you got different angles and perspectives on the room and just walk these Loops this is the easiest thing to master once you've got this
• 11:00 - 11:30 down you can we'll get into more advanced walking paths but I want to make sure you get this figured out before you move along and then again with like objects as you can see with like this is an apple crate you just want to be moving that camera up and walking in circles and you want to be getting different viewpoints so when you want to look at it later if you don't have a Viewpoint anywhere near where you want to look like straight down on this apple uh crate then you won't have good data to look at it you you're gonna have to look down or you want to be able to
• 11:30 - 12:00 move around same with this this Tower I did I I basically did a a spiral around it and I had the camera pointed down so I was able to get the ground around it and get this nice looking capture so again it's all about just doing Loops different heights in varying the perspective that you got um and then start with video that might sound silly cuz you're thinking well video is not the sharpest that's fine I would rather you get a video and
• 12:00 - 12:30 then extract images from a video which is really thousands of images and not worry about like how often to take a photo which I'll get into in a different video but for this if you're not capturing enough images or enough overlap between them everything falls apart so if you capture a video a lot of this software will just optimize your video and say okay I'll grab the right frames and I'll do the best I can with the video and so I always say start with a video and you can take 4K you can work
• 12:30 - 13:00 take 1080p um don't go getting 8K video or raw video you can work with but I at the start to start out with like 4K you might even need to down sample that as well depending on what Hardware you have um and then sharpness is key so as you think about it you want these well lit rooms uh you want anything to be sharp because if it's not sharp it's going to come out fuzzy in the end as we were going through that how to how these things are made it was showing you that it's going to take a reference image and it's going to say okay reproduce what
• 13:00 - 13:30 this image looks like as a gajan Splat well if you give me a not sharp fuzzy image it's going to say that scene is supposed to look funny fuzzy from this Viewpoint and that's not great um so void boka in your blackground we went over that you know you want to make sure everything's all in Focus because if it's it's boka it's not going to work um moving objects again if something's moving through the scene it's just going to be these weird streaks because it's not going to be in every image as you're looking at the scene want to be further
• 13:30 - 14:00 back we want to reproduce a scene unless you're trying to make a spot of just a small object you want to capture some context so don't get super close to what you're kind of you're trying to capture I'd rather you shoot wide and crop in then shoot uh too uh too close and then exposure exposure is like one of the number one reasons I see people get a bad result why is that is because they they they typically we walking around and then all sudden the sky is Overexposed and it's all white and the next one is blue or you might have Parts
• 14:00 - 14:30 blown out and when it's doing that the the technology will say I don't know where the surfaces are I don't know how far back this is I'm going to put a bunch of what they're called floaters like bad Splats all over the scene when it doesn't know where it is so you want to try to keep like an even exposure keep things well exposed and that's easy if you can light up a room well really well um not be putting your camera straight into the sunlight things like that um so the last thing is soft Ware
• 14:30 - 15:00 uh I have it grouped into three different categories we have these kind of like app and website based ones I have here I have four symbols they're not all of them we have scanner secur engine polycam and Luma um I will touch on each of those a little bit in the next slide but those are those are great to start out with I say everyone should start out with scan ofe because you get the result right on the app and that's really really nice because then you can start to learn what works and what doesn't without actually having to upload it anywhere just you get instant
• 15:00 - 15:30 feedback and then you have poot poot is the only kind of commercial executable app out there right now for making these um they run on Windows we'll go into that but again that's more for like a little bit more than just casual having fun I'm a hobbyist or I want to make some cool after effects thing that's that might be where you start again I do advocate for starting with an iPhone or Android based app and learning there
• 15:30 - 16:00 first and then lastly we have these these not not necessar open source but these code based Nerf studio and the original 3dg and splat code base those are great if you're a Creator but also if you're a developer and you want to build something around this you're going to want to start there you that's where you can integrate things into your own products and build from there but again learn the basics using scane or polycam or whatever you want before you start trying to compile code and working with that okay guys at this point I am in the
• 16:00 - 16:30 middle of editing hence why it's different in my room however I realized that I wouldn't make it clear that I wasn't going to tell you exactly how to make one of these using the software but the reason why I didn't is because that would be really hard to do in one guide because every software Works differently I just want to say if you want to use one of these software just go download the app Go download the code follow the instructions post shot it's like drag drop and go they all really good
• 16:30 - 17:00 instructions easy to learn the apps a lot of them are self-guided it shouldn't be that difficult if you follow the tips I talked about so that's where it is I'm going to jump now into just describing the pros and cons of each one of these apps so you know which way to go and from there you should be on your way I also do have a guide on 3dgs which is the inria code that is got over 200,000 views and that is the guide I would follow for that one which is the most difficult of all of these so anyways back to the video and again if you're liking it please subscribe and also yes
• 17:00 - 17:30 that cat is in the same spot every single video talking about each of these little software categories a little bit more we have Scan ofe which is my favorite for just learning because it works on iPhone and it works on Android and it processes your data right on the phone and you're not going to get the highest quality result ever but you will learn what works and what doesn't real fast um some of the pros to it again right on the phone some of these most these have some free tier um and they
• 17:30 - 18:00 have an interactive way to share your your your results so I'm able to like send it to someone they can look at it you can embed them on your website and that's very easy and and a great tool um for doing that and some even have some editing tools I'm mayble like crop the scene down or whatever I want to do with it um but the biggest con to all this is all your data is not yours is all ends up in the cloud somewhere except for a scane which might stay on your phone but if you upload that data now now someone else can have it so if you're trying to
• 18:00 - 18:30 do this for work or something like that think about the fact that you don't actually store all that data in house and that could be an issue later on um and then also they cap the results everything's kind of optimized for mobile make sure everything looks good and it works really fast but that's not necessarily what you want if you want something for let's say you you're a VFX guy and you want to do something for a commercial you want the best quality you're going to want to use a different option uh okay so then there's there's post shot so post shots an interesting software it's in beta so if I go with
• 18:30 - 19:00 cons first it does have bugs from time to time but um that being said it's super easy to install just one caveat there is that you need to have Windows and you have to have um a really nice graphics card so I say Nvidia RTX 2060 they actually have a little bit more requirements on their website I ideally want you to have if you could 10 GB of video RAM and a RTX 3000 Series or newer things take longer on the older cards
• 19:00 - 19:30 and kind of limits how much videos or images you can use so if you don't have a lot of ram you might only get a few hundred images versus if you have a lot of ram you might be able to put a few thousand images into the software and process it um some other big Pros here again just download executable not hard to set up it's got a bunch of controls over your processing so you can start to learn what makes things different I do want want to point out if you don't have
• 19:30 - 20:00 a good capture no software will make your capture look better it's garbage in it's garbage out they also have a plugin for um you for After Effects and I think Unreal Engine and so there's these just different things you can do to edit and display and look at these so this is something if you're a Creator and you want to get really serious and you have the right Hardware as soon as you master the app move on to this the last one is this kind of code based software so Nerf studio and the the 3dgs project which is
• 20:00 - 20:30 by enria um I will link in this description all these different software you can get but I do have a guide on both of these um requires the same Hardware as poot except for you can use Linux which is great if you're in that developer community and it's open source for n studio so you can start to really integrate it in your products and same with the original 3dgs code they will grant you a license if you reach out and I I don't know how much cost but again you can use that to build a product
• 20:30 - 21:00 around and as opposed to the other ones which are more freestanding this software has a really thriving developer community on GitHub uh and they all have a really good Discord Community as well so you can always jump on that and learn a lot so it does have a big barrier of Entry of learning how it works but again there's a community behind it you can learn real quickly and I know you can Master this if you really work at it um and then lastly there's these cool animation tools that they have for these and and they have these viewers you can modify and you get the latest
• 21:00 - 21:30 development so a lot of people who are working on the latest code things like that they're going to throw it in 3D and they're going to throw it in Nerf Studio first for people to learn and they're going to build it on that open- Source software so you always have like the latest if you're using Nerf studio so there you have it if you're new to 3D God and splatting I will be putting out one video per week or more about how to get the best out of this technology I'll be talking about capture tips I'll be talking about why things go wrong I'll
• 21:30 - 22:00 be explaining the technology behind this a little bit more in detail so if you're at this point you got to the end of the video please subscribe to my YouTube channel so you can get all these latest developments and I will be even going into how to compile the code from your studio things like that if you really want to dive into that minua I hope you like this video and learn something new and I hope you set out on the right Journey for 3D Goin spotting I'll see you in the next video