Unlit Fresnel Custom View Mode

Summary

In this video, tharlevfx demonstrates how to create an improved unlit mode, called the Unlit Fresnel Custom View Mode, to enhance detail visibility during set dressing in dark environments. The creator illustrates the process of combining base colors with Fresnel calculations to create ambient shading effects, providing a clearer and more detailed view of the environment without relying on heavy lighting. By customizing buffer visualization and editing the base engine files, users can add this new view mode to their tools, making it a more effective alternative to the default unlit mode. Finally, the video shares tips on adjusting keyboard shortcuts for quick access and highlights the practical application of these techniques in environment design.

Highlights

• Creating an enhanced unlit mode using Fresnel shading techniques 🌟
• Detailed walkthrough of Unlit Fresnel Mode setup in Unreal Engine 🛠️
• Using post-process materials for custom visualization effects 🎥
• Custom view modes streamline environment set dressing 🌆
• Keyboard shortcut customization for quick access 🔄

Key Takeaways

• Custom Unlit Mode enhances visibility in dark scenes 🎨
• Avoids heavy lighting and potential workflow issues 💡
• Combines base color with Fresnel for ambient shading 🌈
• Editable buffer visualization adds flexibility and customization 🔧
• Tips on keyboard shortcut customization for efficiency ⌨️

Overview

In the video 'Unlit Fresnel Custom View Mode', creator tharlevfx takes us through a comprehensive process to improve visibility in unlit modes within Unreal Engine, specifically for set dressing in dimly lit scenes. The journey begins with a problem many encounter: inadequate view mode detail leads to inefficient workflows and potential lighting mishaps. Enter the Unlit Fresnel Custom View Mode—a game changer that combines color base and Fresnel calculations to yield a smoother, more defined ambient shading.

The creator walks us through setting up this mode using post-process materials. By manipulating the buffer visualization settings and refining how the engine processes material data, tharlevfx introduces a stylish view mode that improves on the traditional flat unlit rendering. This custom mode maintains vital details without the need for overhead lighting, ensuring better depth and clarity in environment design.

Towards the end of the tutorial, tharlevfx also covers how to integrate these settings into engine defaults by editing configuration files. This final step allows for seamless usability across projects. Additionally, viewers learn how to customize keyboard shortcuts, allowing for quick toggling between modes, ultimately enhancing workflow efficiency.

Chapters

• 00:00 - 00:30: Introduction and Overview of Unlit Mode The chapter introduces the concept of unlit mode, a technique to enhance visibility and detail recognition in dark and gloomy scenes, especially useful in set dressing. It discusses potential problems with using heavy-handed lighting solutions and aims to offer improvements for better outcomes.
• 00:30 - 01:00: Problems with Using Traditional Lighting The chapter discusses the challenges with traditional lighting in a workflow, such as forgetting to turn off lights, which can lead to issues with lighting artists. It mentions the bothersome workflow of managing multiple light settings. To address these problems, the chapter suggests using the 'unlit' mode in the viewport for a more efficient workflow, explaining that this mode offers different view options to improve lighting management.
• 01:00 - 01:30: Using Unlit Mode Exploration of the 'unlit render' mode which is often useful but presents challenges, particularly in scenes needing depth perception and clarity in intersecting objects. A recommendation to adopt a customized technique for improved visualization is made.
• 01:30 - 02:00: Limitations of Standard Unlit Mode and Introduction of Custom View Mode The chapter discusses the limitations of using a standard unlit mode in rendering, which lacks lighting information. The speaker describes using a basic fel shading technique, calculated from the camera vector, to add ambient shading. This technique highlights the edges and enhances visibility, which is particularly useful for tasks like set placement or set dressing, where distinguishing between similar colors is necessary. The speaker praises this stylized view mode for its practicality in improving visual differentiation in scenes.
• 02:00 - 02:30: Creating Custom View Mode with Fresnel Effect The chapter 'Creating Custom View Mode with Fresnel Effect' discusses the process of enhancing the functionality of a visualization tool by adding a custom view mode. The author suggests an alternative to the default unlit mode, which is functional but could be improved for better usability. It involves modifying buffer visualization by integrating a new mode into the engine, providing instructions on setting up a keyboard shortcut for this new feature. The author indicates that users who prefer not to use the unlit mode can use the new custom view with a personalized shortcut key of their choosing, demonstrating how to assign these shortcuts as per personal preference.
• 03:00 - 03:30: Setting Up the Post-Process Material In this chapter, the focus is on setting up a post-process material within a workflow. The process begins by switching to LIT mode and proceeding to a post-process volume. A post-process material is created and added to this volume, serving as an initial way to manipulate the visuals. It's mentioned that this material will later be integrated into the engine itself. The chapter also touches on buffer visualization, explaining that these buffers are available to the engine for compositing and constructing the final image, using all the resources at its disposal.
• 04:00 - 04:30: Adjustments and Fine-Tuning View Mode The chapter titled 'Adjustments and Fine-Tuning View Mode' guides users on how to access and utilize engine content for materials within a content browser. It instructs on enabling engine content visibility through settings and exploring different materials like metallic in buffer visualization. However, it cautions that the metallic material may not display effectively in non-metallic scenes.
• 05:00 - 05:30: Exploring Buffer Visualization In the chapter 'Exploring Buffer Visualization,' the speaker discusses metallic chains and moves on to a more visually engaging topic, which is roughness. Roughness is highlighted as a key feature, evident everywhere in the environment. The speaker points out how roughness can be assessed by observing puddles on the floor and the texture of various rocks and stones. Additionally, the chapter explains how to view roughness through the buffer visualization in the content browser, which shows it as a material that can be further inspected by double-clicking to reveal its use of a postprocess.
• 06:00 - 06:30: Creating and Editing Post-Process Materials The chapter titled 'Creating and Editing Post-Process Materials' focuses on the process of developing materials in the post-process material domain. It explains the use of a scene texture input and demonstrates simple manipulations to enhance visibility. The chapter also discusses checks necessary for accommodating different shading paths or feature levels to ensure varied outputs. However, the primary emphasis is on guiding the reader to create their own post-process material within a designated folder. The choice of whether to maintain this organization is left to the user.
• 06:30 - 07:00: Implementing Custom View Mode in Engine The chapter focuses on implementing a custom view mode in a graphics engine. It discusses the importance of creating reusable components that can be shared across different projects, emphasizing the need for efficient processes when updating or installing new engine versions. The chapter provides a practical example of creating a new material in the engine, specifically a post-process material using scene textures, highlighting the ease and benefits of making your customizations within the engine.
• 07:30 - 08:00: Adjusting Engine Configuration This chapter discusses the adjustments of engine configurations specifically focusing on postprocess inputs. The speaker highlights roughness as a key input and describes its immediate output when applied directly. To visualize changes, the process involves selecting the postprocess volume and adjusting postprocess materials by adding elements to the material array.
• 08:30 - 09:00: Final Adjustments and Testing Custom View Mode In the chapter titled 'Final Adjustments and Testing Custom View Mode', the focus is on addressing the flickering effect encountered during the rendering process of a custom view mode. The flickering issue is attributed to the order of operations in the rendering pipeline, particularly involving anti-aliasing. A solution involving the 'blendable location' setting is introduced to mitigate this problem. The adjustment aims to stabilize the visual output, ensuring a smoother rendering experience.
• 09:30 - 10:00: Setting Keyboard Shortcuts for View Modes This chapter focuses on setting keyboard shortcuts for different view modes, specifically dealing with issues related to scene color settings after tone mapping. It highlights the problem of flickering occurring when applying changes after tone mapping, and suggests a solution by adjusting the settings, such as applying scene color before depth of field. This adjustment helps manage color ranges better by incorporating the tone mapper application. The chapter explores various settings to reduce flickering effectively.
• 10:00 - 10:30: Conclusion and Additional Resources The chapter discusses the process of adjusting tone mapping for color correction, emphasizing the importance of not replacing the tone mapper completely. It advises selecting appropriate settings depending on the specific needs and situations. The narrative demonstrates how to retrieve original coloring using contrast adjustments. Additionally, it highlights the preference of focusing on scene settings rather than using roughness adjustments.

Unlit Fresnel Custom View Mode Transcription

• 00:00 - 00:30 okay right hello and welcome back to another video So today we're going to be having a look at how we can make an improved unlit mode uh it's going to help us to be a little bit more uh see a little bit more detail when we're doing uh things like set dressing uh so take for example this scene that I've been working on we want to do some set dressing down here uh in this kind of very dark and gloomy area um it's a bit difficult to see it's quite dark we could just create a light that's very uh heavy-handed um but obviously it works problem with that is sometimes people
• 00:30 - 01:00 forget to turn their lights off and then the lighting artist is going to come and Shout at us uh or we just kind of having to sort of make multiples of these things move around it's all just a little bit of a a not very nice workflow um so we're going to get rid of those not do that uh what we're going to do instead is we're going to use unlit mode so if we go up to the top of the viewport here there's a few options one of them is The View mode here we can set this to unlit now there's lots of other ones we can use here as well uh and that's going to give us a completely
• 01:00 - 01:30 unlit render of our scene this is really useful uh and this definitely works uh but personally especially when we're dealing with something like that view um it looks pretty flat there's no depth information here because it's just the unlit um surface base color of our of our rocks especially where we've got multiple rocks intersecting and things it can be quite tricky to see what's going on uh and so instead we're going to be rolling our own uh which is going to be this one so very similar still has
• 01:30 - 02:00 no uh lighting information in it all I've done is taken a kind of a simple Fel calculation based off the camera Vector um and use that to kind of do this uh kind of ambient shading so everything now has this kind of wrapped around Darkness to it um which really helps push out the edges now this is sort of a stylized View mode but actually it's really really helpful I find um for doing this kind of set placement or set dressing where you've got um lots of similar colors on top of each other so if I change that back to
• 02:00 - 02:30 uh to unlit you can see what's going on and it and it it's fine it will work um but I think if we can make something a little bit more useful uh then why not and so we're going to go down here buffer visualization and you can see I've added it to the bottom so uh we're going to go over how to make it how to add it to the engine uh and then how to just quickly create a keyboard shortcut for it so um if you're like me and you never really want to use on lit you can replace the keyboard shortcut like I've done alt three uh or you can just kind of use your own whatever works best for
• 02:30 - 03:00 your workflow cool okay so let's go back to LIT mode uh I'm going to go to a postprocess volume I'm going to create a post-process material and just add it to the postprocess volume um just as a way of kind of working with this and then later one we'll add it to the engine itself so um as you saw buffer visualization uh so these are the buffers that the engine has available to it that it then uses to kind of composite and uh build up the final image um but all of them are just
• 03:00 - 03:30 materials and if we go to the engine content so if you can't see it in your content browser uh show engine content in the settings over here with that enabled you're able to see the actual engine content uh um of your engine installation as well and in engine content buffer visualization you can see we have all of those materials so if we take something like for example metallic this is probably going to be a really poor example because it's not a very metallic scene
• 03:30 - 04:00 well the chains have a bit of metallic to them that's about it let's try something a bit more um visually interesting shall we roughness roughness is a good one there's roughness everywhere so there a really good way of checking what the roughness looks like we can see the puddles on the floor the different roughnesses of the various rocks and stones um making up our scene which is pretty cool uh but if we go to our buffer visualization part of the content browser we can see roughness it's just a material and if we double click it we can see it's just using a postprocess
• 04:00 - 04:30 material so the material domain down here is postprocess we have a scene texture input uh and then they're doing some simple manipulations to it to make it a bit easier to see and a few checks so that if we're using different shading paths or different feature levels um we're going to get different values but we don't really need to worry about those what we need to do is create our own one of these so um so I'm actually going to create this in that folder um up to you if you want to keep this in
• 04:30 - 05:00 your engine uh which will then shared between different projects or when you want to put it in your specific project this I think is something that probably does want to be shared you might need to then remake it every time you install a new engine but it doesn't take long so um so that's fine uh we go back to lit and I'm going to make a new material here and I'm going to call it test post process make it a postprocess material and we're going to use the scene texture
• 05:00 - 05:30 now I'm not going to go through all of these um but if you're doing postprocess um we want to have a look at the various different uh inputs um so we had the roughness and if we just plug roughness straight in we'll get this result now I want to see this in the world or see this applied to my view I should say so I select my post process volume and then I just thought well I found it now here our postprocess materials we search for materials in here we can add an element to our array
• 05:30 - 06:00 asset reference and then it's this one here so test post process now we're going to get this flickering um that's very expected what's happening well where it is applied in the process of rendering to the screen depends whether it has this sort of flickering or not um it's to do with the uh anti-aliasing I believe um but the fix for it um if we type in blend here blendable location
• 06:00 - 06:30 so scene color after tone mapping so by applying this after the tone mappings happened we're uh we're getting this flickering if we go right to the top scene color before depth of the field and apply you can see it's changed the uh the color ranges because we're now using the tone mapper being applied as well um and we can try different settings in here but I believe to get rid of the flicking flickering it needs to be before the tone
• 06:30 - 07:00 mapper I don't want to replace the tone mapper that's not going to work yeah so um any of these ones here at the top depending exactly what you're doing you might need different settings but any of them work for us in this case um and to try and get some of the original coloring back we're going to do that contrast um as we saw with the other one and there we go but we don't want to use roughness what we want is the scene
• 07:00 - 07:30 and so if I go back to my scene texture roughness change that to base color for lighting there we go now this is um obviously quite bright compared to the original unlit and we're unfortunately not able to recreate exactly what's in the unlit shading mode now this one is hardcoded um it's not just something we can recreate out of the materials and so where we've got things like the the particles and the uh the Water surface
• 07:30 - 08:00 we're going to lose that in our final um but we're going to get the surfaces which is what's really the important part so so that's our base color um that looks no that's un lit if I go back to my lit mode that's our post process that's what we're working on so it's very bright but we can obviously just multiply that down um but in fact when we do our forel calculation that's going to U darken everything as well so we'll multiply those two together first and see what happens so if we're using our base color to do our for n calculation we need the
• 08:00 - 08:30 world normal so down here we have World normal and if we just have a look at that we can see the normal Direction uh in World space so anything in the surface down here pointing upwards that's pointing in the blue Direction um remember these are directions that have been encoded as color and give us um the colors that we see here um where we're looking completely in the negative directions we're going to see black because un real can't render negative
• 08:30 - 09:00 colors but you can see the normals of the uh of the surface being displayed here in World space now if we take that and we take our camera Vector again camera Vector WS WS standing for World space so we're in the same space uh camera Vector WS um so what we can do is just do a simple dot product between the two this will compare the two direction vectors and when they're facing the same
• 09:00 - 09:30 direction so when the surface we're looking at is is facing the camera we get white and when that surface is 90° we get black now again this has got um a little bit of detail you can see but we're going to just manipulate it a little to give it a bit more contrast so let's power it by let's say four and see what happens there we go oh let actually connect that that make a bit more sense there we go and so
• 09:30 - 10:00 you've got full control over how much um of that kind of edge darkening wrapping around you want to see um four gives us a decent result I think two four something like that again we can always make a parameter for this and scale it up later if I multiply these two together now what we should see is our scene looking pretty stylized maybe four is a bit High let's go back to two let's go back to two
• 10:00 - 10:30 there we go so what we're doing is taking our base color and our world normals converting our world world normals to a frenel using the dot product and multiply them together to get a kind of stylized Fel unlit which I think is a much easier or workflow or much easier View mode to be working in when we're doing set dressing compared to um a fully enlit
• 10:30 - 11:00 cool uh if we want the original views had uh particles in them you can see little things happening over here it's not really showing us the full particles um so this is because we're applying it as a oh no as a post process it should be there but yeah okay
• 11:00 - 11:30 so they are there in our post process but when we apply this as a um buffer visualization they won't be there so if we want to add them back in we can do a post uh what's it called see another scene texture it's going be postprocess input one so postprocess input zero that is the scene without any edits to it at all so that's what we're seeing um here if I change this to post process input one it will just give me the the
• 11:30 - 12:00 translucency elements and so we can get our waterfalls and snow and everything so I'm going to take that and I'm add it on top not necessary really we're looking at the environment but that will give us a little bit more of that kind of like particle elements to it so a bit similar to more the uh what the unlit Shader look like they are obviously quite different and
• 12:00 - 12:30 here you're getting um the water surfaces and all that kind of stuff but um but it's only supposed to be a working View mode so it doesn't need to be um exactly the same okay so whever you want to do your uh your sort of custom View mode set it up in here um something like this would work quite nicely I think yeah for environments where you want to see the environment without any lighting but you still want to see some detail of the model um this bit obviously being sort
• 12:30 - 13:00 of optional if you want to see the translucency in there as well uh and once we've saved that we want to add it into our View mode so we can edit this list uh and we're doing that in the base engine in file so I've got it open here uh if I remember correctly it is [Music] in uh the engine installation so epic
• 13:00 - 13:30 games u54 engine config base engine any file so uh where your engine is installed then the installation number so epic games 54 engine config base engine um and if we open this up there is a section well we Ed defined buffer visualization should find it for you I think it's just these ones let's see if buffer is anywhere else as well okay lots of places so let's do
• 13:30 - 14:00 buffer F there we are so engine. buffer visualization materials now you can see the list here hopefully that's not too small on the screen but um you can see the list I can zoom in slightly probably not um you can see in the list all the ones that exist already and all I've done is just taken one of the entries copy and pasted it um and filled in the details so let's do that again so uh let's use one of the original ones not one I've added so paste in there put a space
• 14:00 - 14:30 in and I'm going to call this test post process material is and then we want the um the path to where the engine material is saved so in this case it's in the buffer visualization um folder which makes this easier it's called test VP and if we go back to look at our any file so engine buffal buffer
• 14:30 - 15:00 visualization and it's test pp. test PP so you need the name of the file dot the name of the file and you can see that in here so post process tone color copy toass um localization text well we'll give it a thing and then this second last bit here is the um the actual visual name that we see so if you want to have spaces and things like that in it then you do that um we can also add this command here um so
• 15:00 - 15:30 after the closed brackets there apply Auto explosure to or false uh and so when we had it as our test material we had the auto exposure being applied it's making things quite bright if we turn that off we can do it here and that won't be applied um and that will help keep things a little bit more balanced color wise so again try that true or false up to you okay so if I save my any file I will
• 15:30 - 16:00 need to restart my engine save and check out all of this um and then let's restart my project okay 54 and if we've done this correctly what we should see is the buffer visualization mode now at the bottom we'll have that additional um view that we've added test PP um first thing I'm going to do is go into the postprocess and remove the test
• 16:00 - 16:30 material cuz we don't want to see it applied there so so this is our scene we're working in it it's a bit dark I want to see the world without the lighting in it I don't just want to use unlit it's too flat but if I go down to buffer visualization I can do my test post process and there it is so that's the one we've just made it's a bit dark well we can go in and once it's set up as a uh thing in the engine we can just go in and edit this live so if I were to take
• 16:30 - 17:00 my base color lighting and let's say multiply it by two so before we were seeing it too bright now we're seeing it too dark we can quickly go in and adjust that and make it there you are much brighter depending on what we what we want to do so this the water and things the translucency actually comes in quite faint so I'm going to take that and multiply that up
• 17:00 - 17:30 and now we should see our waterm stuff happening here and our snow Etc let's bring that down something like that again you can mess with it as much as you like cool so if you're happy with that and if you'd like to use that as your unlit default rather than your uh than the unlit in here now I'm not an environment artist I'm sure having unlit is really useful um but as a F artist I only only ever use unlit as a placement
• 17:30 - 18:00 tool or a debugging tool and so I'm not going to get rid of it obviously I can't stays there it's hardcoded but I am going to replace the default shortcut for it to be this one down here so uh alt 2 3 4 5 and six is the different views wireframe unlit uh lit detail lighting lighting um but I don't want to use the default unlit I going to use my own unlit and so to do that we can just go up to um editor
• 18:00 - 18:30 preferences keyboard shortcuts oh let me just adjust the engine size uh and now in here yeah so edit editor preferences keyboard shortcuts uh and if I type in test PP I'll be able to set a new binding for that um and so alt three I say is the uh is the default I'm not going to do that because I'm not going to use the test I'm going to use the one that I already had but but if we type in unlit you'll see you've got the
• 18:30 - 19:00 default unlit view mode binding and then the um the one that I've made here so I do maybe something like that alt shift three that's not a bad idea cool so now alt three will be uh my unlet ALT shift 3 will be the default unlet and we can quickly jump between them as well all right quite a nice little
• 19:00 - 19:30 useful tip I think um something I've been using a lot this week to finish up this scene with set dressing placing objects around um doing it in the default unlet I found I was struggling so I thought how can I do this better let's make my own unlet View mode um definitely going to going to help all right um as always if you've got any questions or comments about any of the things you've seen in this video or you want to learn more about postprocess materials uh do check out my courses on gumroad and udem me and
• 19:30 - 20:00 there's a whole course on postprocess materials um we do lots of different things about stylization and and kind of go into a lot more detail into the things we've used today um yeah big thanks to all the patreons for supporting the channel uh and I'll see you all next time with some more updates uh of this cave scene