I built an AI Healbot to save my own life (Deepseek R1 + Open WebUI)

Summary

I versus AI creates a groundbreaking AI Healbot using Deepseek R1 and Open WebUI to combat his addiction and improve his life. After years of struggling with addiction, poor healthcare experiences, and a lack of accessible support, he builds a personalized tool to assist in his journey toward recovery. The AI, set up to operate locally for privacy, integrates with various tools and can interact via voice, offering 24/7 support. This innovation not only helps manage his addiction but is open-source, allowing others to create similar support systems tailored to their needs.

Highlights

• A powerful AI tool was built for personal addiction recovery. 🔧
• After facing failures in traditional healthcare, the creator finds hope with AI. 🌟
• Using Deepseek R1 with Open WebUI creates a feature-rich, private AI Healbot. 💻
• The project provides a roadmap for others to build similar AI support systems. 🚀
• AI interactively supports the user by being accessible 24/7, anytime, anywhere. ⏰
• Security and privacy were major considerations, making it a self-contained system. 🔐
• The AI incorporates neuroplasticity and trauma-informed care techniques. 🧠

Key Takeaways

• I versus AI creates a personalized Healbot using AI technology to aid in his addiction recovery journey. 🛠️
• After multiple discouraging experiences with healthcare, the creator turned to AI for constant and customized support. 💡
• The AI Healbot is setup using Deepseek R1 and integrated with Open WebUI, allowing for offline, private use. 🖥️
• Open WebUI offers a feature-rich interface enabling various customization options, from model selection to personalized system prompts. 🎛️
• AI Healbot incorporates local models and external APIs, providing flexibility for the user's needs and resources. 🌐
• Connecting directly with personal data, it keeps track of recovery progress and supports decision-making during crises. 📈
• The creator emphasizes the potential of AI to facilitate personal healing, inviting others to build their own support systems. 📱

Overview

In a bold turn of events, I versus AI uses the power of technology to tackle his struggles with addiction. Faced with unhelpful and often discouraging interactions from healthcare professionals, he built a personalized AI Healbot using Deepseek R1 integrated with Open WebUI. This innovation provides constant support throughout his recovery process, proving to be a lifeline during critical moments.

The AI Healbot is an integration of open-source platforms and local resources, ensuring privacy and customization based on personal needs. By taking advantage of Open WebUI's sophisticated interface, the creator tailored the Healbot to interpret and respond to his unique addiction battles, using insights from his data set. This set the stage for an adaptive and private healing assistant.

Highlighting several setbacks with traditional therapeutic avenues, the creator empowers others with the knowledge that technology can offer an alternative path. The AI not only helps in real-time by using voice interaction anytime, anywhere but also grows with the user by learning from each interaction, making it more effective with time. This project encourages anyone struggling to harness AI’s potential in crafting unique, personal support tools.

Chapters

• 00:00 - 01:00: Introduction The speaker openly admits to being an addict, acknowledging the severity of their situation, which has worsened over time. They reflect on their lifelong struggle with addiction, recalling how even as a child, they experienced intense pleasure from eating something as simple as a piece of Wonder Bread. This highlights the depth and long-standing nature of their addictive behavior, illustrating how profound and transformative these experiences were for them, even from a young age.
• 01:00 - 03:00: Seeking Help The chapter 'Seeking Help' discusses the author's experiences seeking medical help as an adult. The author highlights the challenges and successes encountered in dealing with the medical industry. Among the positive experiences, the author mentions a standout nutritionist and a very competent doctor who significantly helped lower the author's blood sugar levels to unprecedented levels. However, the chapter also speaks to disappointment when the doctor, who brought much hope, was fired partly due to spending too much time with patients.
• 03:00 - 04:00: The Negative Experience The chapter 'The Negative Experience' discusses the difficulty in accessing effective therapy and finding a therapist who truly cares and is available during times of urgent need, like cravings at 3:00 a.m. The narrator expresses frustration with the inefficacy of therapy they have experienced and the challenges of long wait lists and high costs, making it difficult to find a suitable and accessible therapist.
• 04:00 - 11:00: Building the Healbot The chapter 'Building the Healbot' describes a personal experience of seeking medical assistance and finding it lacking. The narrator talks about attempting various solutions like joining Overeaters Anonymous without success. Their visit to a local clinic's new physician assistant turned into a disheartening and unsupportive encounter, which left them feeling demoralized and degraded.
• 11:00 - 15:00: Exploring Open WebUI Features The narrator felt hopeless after receiving a diabetes diagnosis and faced assumptions about their financial situation. They decided to take action using local resources.
• 15:00 - 17:00: Comparing LLaMA and Open WebUI The chapter discusses the process of creating a 'healbot' using language models, aimed at self-healing and creating a private offline large language model. It covers the steps needed to pull the project together, how it works, and how others can replicate a similar setup for personal use, either for self-healing or having a cost-effective offline chat model experience similar to ChatGPT.
• 17:00 - 22:00: Customizing Open WebUI This chapter focuses on the user interface of Open WebUI. The narrator, who refers to themselves as 'I versus Ai', expresses their appreciation for the Open WebUI's interface. They mention taking a week off after a medical appointment to dedicate time to customizing and working with the Open WebUI. The chapter begins by highlighting the narrator's commitment to exploring the user interface extensively, suggesting a deep interest or need for personalization in their digital workspaces.
• 22:00 - 27:00: Demonstrating Healbot Use The chapter titled 'Demonstrating Healbot Use' describes the author's process of focusing solely on building a custom healbot for personal use. The author talks about reviewing various user interfaces like open web UI, which they found to be the most feature-rich. The chapter highlights the layout of the open web UI, pointing out its functionalities such as the ability to close the sidebar and create a new folder in the chat space.
• 27:00 - 32:00: Privacy and Local Models The chapter titled 'Privacy and Local Models' discusses the initial experiences of using a conversational AI named Healbot. The user has been interacting with Healbot for a week, organizing their conversations in folders which can be renamed or exported. The chapter promises more insights on the Open Web UI and its remarkable features.
• 32:00 - 40:00: Conclusion and Encouragement The chapter discusses the positive aspects of Open Web UI, highlighting it as the easiest and most functional option available. Its aesthetic appeal and ease of use are particularly praised. Despite its many strengths, the chapter notes that it is not without flaws. Feedback is provided, specifically pointing out the lack of an option to download chat dialogues, suggesting room for improvement.

I built an AI Healbot to save my own life (Deepseek R1 + Open WebUI) Transcription

• 00:00 - 00:30 hi I'm an addict yep I'm an addict in fact things have gotten pretty bad lately this is something that I've been struggling with for as long as I can remember and my earliest memories of are having my brain light up after eating a piece of Wonder Bread which is not even bread let alone a Wonder but to my childhood brain it was Nirvana and now
• 00:30 - 01:00 as an adult I need help in fact I've gone to get help before and sometimes got some success I've gone to both the medical industry um including a nutritionist and a fantastic Doctor Who helped me more than any doctor ever has she helped me get my blood sugar down to the lowest it's ever been and there was a lot of Hope until she was fired in part because she spent too long with
• 01:00 - 01:30 each patient because she actually cared so I also went to therapy and I got some help there but it was never very effective and most importantly it wasn't available when I needed it when the Cravings hit at 3:00 a.m. I can't call up a therapist and being able to find one who connects with you and is available and there isn't a six-month wait list and who doesn't cost an exorbitant amount of money is like like
• 01:30 - 02:00 a needle in the hay stack I've even gone to Overeaters Anonymous but in the end it really wasn't helping and then came two weeks ago I went to an appointment at my local Clinic with a new physici assistant and it was the most demoralizing demeaning degrading heartbreaking experience I've ever had in the medical industry I got no sympathy no support and she blocked
• 02:00 - 02:30 every attempt and every question I had about getting help making assumptions about me and what I could afford or not afford it was awful so I felt really hopeless I'm staring a diabetes diagnosis down the gullet as it were and I felt like I had nowhere to turn so I decided to take some action I'm going to tell you how I am using local large
• 02:30 - 03:00 language models to create a healbot and hopefully heal myself I'm going to tell you how I pulled this all together how it all works how you can do something similar for yourself whether you want to heal yourself whether you just want to have a private offline large language model experience basically like chat gbt at home for very little or even no cost but most importantly I'm going to tell you how I am using AI to help me save my
• 03:00 - 03:30 own life let's do this hey there it is I versus Ai and this is my healbot what you're looking at specifically is the user interface for open web UI and one of the things I love about open web UI and I did spend a lot of time I in fact took a week off after that last medical appointment I just shared with you and I decided that I was going to spend the entire week just take it
• 03:30 - 04:00 off and focus on nothing else but this put my channel on hold put everything else on hold and focus on building this custom heel bot for myself so I started out you looking at numerous different user interfaces like open web UI and I definitely found that open web UI was the most featur rich over here in the top leand corner is the ability to close the sidebar this is a chat space where you can create a new folder your new
• 04:00 - 04:30 chats just start out here this is today's conversation that I've been having with healbot and you can see here that I've created a folder over here on the left and when I open that up you can see that my first conversation with healbot was on Thursday last Thursday so it's been a week that I've been using it and you can do certain things with the folders here rename them you can export them and you can also export your chats as well too and as I go through open web UI I'm going to talk about what's amazing about about it which let me make
• 04:30 - 05:00 this 100% clear is a lot there is a lot of really great things about open web UI it is hands down as far as I feel the easiest and basically most functional and once you get it installed it's just lovely to look at and lovely to use however it's not perfect you can download here um your chat and one of the first things I want to say to the developer is that there's no option here
• 05:00 - 05:30 to download things as markdown allowing me to to uh download this as a MD file would make this so much better for being able to just add a a text right to my obsidian Vault and this is particularly mystifying because the way large language models output text is in markdown so please add this to uh the ability to download the developer is hands down very obviously a genius putting out content um uh new updates
• 05:30 - 06:00 regularly you're not going to be waiting months for new updates from open web Bui in fact um just a few days ago code interpreter here the ability to execute code and this new web search button here for searching the internet right here that's new so there's a lot happening but there's also like I said some mystifying things that happen that don't make a lot of sense and a fun fact about open web UI my belief is that it was
• 06:00 - 06:30 first if I'm not mistaken it was first known as olama web UI because it was built to be worked with olama plain which allows you to get access to local large language models local models and you can see that it has so many features and it and it really does and olama if you spend any time in the local large language model scene you're going to hear people talking about olama right up front and that's because it's powerful
• 06:30 - 07:00 but I don't use it there's one really important reason why the local model Community loves it so much and that's because AMA is open source if this is really important for you AMA is by far the best choice now why did I choose not to go with AMA it basically boils down to one really important thing in order to work with olama you need to work or at least there's some things that you need to do in the command prompt now
• 07:00 - 07:30 there is a way to work with AMA very well right inside open web UI but in the end I did not like that so much was behind the scenes I couldn't see it I could I'd have to open up this command prompt and I just let me just be honest with you I hate the command prompt I really do so I'm going to go to Great Links to avoid it and luckily there is another solution which is in my opinion though it is not open source it's actually better than olama is particularly if you want a to access to
• 07:30 - 08:00 a lot of different models so the solution I'm going to share with you later on it is a better choice for my needs you're going to find a lot of people who swear Biol Lama and they're not wrong if we head over here to the settings you can see that there is so much personalization here particularly in the interface you can change things like getting notifications for new updates with including a notification sound which is really helpful because sometimes particularly if you're running models that are a little bit larger than what you can fit on on your graphics
• 08:00 - 08:30 card and just to give you a heads up on screen is my computer uh specs the system I'm on right now and I have a GTX 490 at 24 gigs so that means that I can fit models um into that that range around the 22 gig Mark after that if you use a model that's larger than what you have on your video card and I'll show you this later on when I show you the server I'm using what basically happens when you come to chat TBT and you use it and something happens in the server or
• 08:30 - 09:00 on the back end that's not your problem but if you choose to go the local Mount there's a lot of benefits to it but one of the downsides is that you are now the server admin meaning that if something happens or something's not working with the models it's up to you to figure that out but it's not as hard as you might think especially since I'm showing you this after having built it in the last two weeks and learned much of what I'm sharing with you in the last two weeks so if I can learn it you can too so if you like to Tinker there's quite a lot
• 09:00 - 09:30 to Tinker with that's the good side the bad side is that the documentation though it has been improved desperately needs to be continued to be improved I get the feeling that the developer of this software obviously highly intelligent I think he might be so intelligent that he forgets that the rest of us are not working with the same level of horsepower that he is and so a lot of things that seem obvious to him are very kind of challenging to figure
• 09:30 - 10:00 out I'm going to show you here personalization you even have memories available that are very similar to what you see with chat GPT you can edit them and delete them contrary though to chat GPT these are not added automatically by the model you need to add them by hand yourself the audio feature both the speech to text me speaking to the model being turned into text and the text to speech the model replying to me with a voice is fantastic I cannot wait to
• 10:00 - 10:30 share with you the voice that I have found and both the model to transcribe my speech to text and the different model to transcribe the the models uh the large language model's output the response they are both local on my computer and I want to stress this everything that you're going to see running is happening on my computer unless said otherwise these are the list of models I am differentiating them using different icons and that's another
• 10:30 - 11:00 great feature of open web UI is that you can really organize your models just as you'd like them for example deep seek chat and I have these little wonderful icons that I made in um Mid journey and you can add tags which actually ends up being very helpful particularly if you end up having a lot of models you can add a system prompt that's available for this model whenever you use it and again that's an advantage of open web UI but it's also disadvantage because there are
• 11:00 - 11:30 actually from what I understand three different places where you can leave a system prompt you can have a system prompt in the settings here in the general tab you can have a system prompt here in the individual model Tab and you can also have a system prompt in the work space area here with your custom model which is running healbot and it is
• 11:30 - 12:00 unclear as to why you need three different system prompts and that's what I mean there's a lot of features but they're separated and spread across the user interface and there is no documentation that explains why what each system prompt does so that I have been able to find and I have crawled through the open web UI documentation which I will link down below I think the moral of the story is with great power comes great responsibility and that
• 12:00 - 12:30 means that with open web UI you have great power and you can really get into tinkering with this and making this perfect which is what I did for an entire week when I was setting this up learning as I went in fact you have web search here but it's not just like chat TBT it's not baked in you get to choose what you would like for all of these and each of them have their own apis and you would just Google this figure out how to connect to the API Tav is one I would
• 12:30 - 13:00 recommend because it's very easy uh Google psse is actually fairly easy as well too but I would advise you to look this up with a YouTube video but it didn't take too long to get the keys to set these up so you can have all different ones that you can choose from and like again lots of power but if you just need it to work a little bit overwhelming one really big tip I want to share with you is here in the interface you can set What's called the task model this model is responsible for coming up with the titles of your chats
• 13:00 - 13:30 and also assigning tags as well too in fact this chat here has these three tags that I did not put there but were put there by the task model you can see that I'm using Gemma 2 as my task model but basically what you want to do here is change this by default it will say current model right now I have this set to Lo local models is empty and just to explain this this basically by local models it means models that you're using
• 13:30 - 14:00 through olama because I'm not using AMA I have nothing in here this Gemma 2 is despite the fact that it says external it is a local model I don't know why but the developer I think maybe because this was designed really to be a front end for ama ama is really baked in and everything else is basically considered external like outside in the lawn but um but this is a local model here that I'm connected to using LM Studio which is what I'll talk to you about and you can go ahead and copy my title uh generation
• 14:00 - 14:30 prompt it's that's the default with open web UI but I've added a little bit to it so you're welcome to have that now I want to move over here to audio and show you that the this is so cool the base model which you'll see in play as I show you how uh um the uh healbot Works um this comes native with open web UI but I also want to really showcase one that I love here called coko this creates voice is that it is extremely difficult to
• 14:30 - 15:00 imagine that this is just being run on my computer but it is this is local and I've combined some voices together one of the features I love about Koko which is over here and it wasn't too difficult to set up particularly because and I invite you to do the same if you dive into this ocean of um local large language models which is an amazing experience but I do recommend that you use chat chbt or or your uh model of
• 15:00 - 15:30 choice Gemini's uh Google Gemini is another good choice for this one to help you with the setup particularly with Docker two more places I want to show you in open webui here are the documents area here and one thing you can see that I have my embedding batch size and those of you who are old hands at open web UI and embedding might be looking at this and saying girl you are crazy and the thing is um the default setting is set to one and went back and forth with chat
• 15:30 - 16:00 chbt trying to understand what embedding batch size is and finally I just said screw it and turned it up as high as I could go just to see what was going to happen cuz that's how you learn and it turns out that it was I got much better responses um I got faster responses and so it seems to be working but I still am unclear despite having looked through the documentation I've even searched on their Discord which I will link their Discord down below that's an important place to go um and I have not found out what embedding batch size is so if you know please let me know in the comments
• 16:00 - 16:30 that would be awesome this is the model I'm using which I do highly recommend it's very fast and this is being hosted um by me on the computer here so this is local the minimum score I'll show you as well too uh when I show you how the document embedding Works which does work very well for me so if you want to go ahead and copy these settings please feel free to do so and in fact if you would like my prompts this embedding prompt and the prompt I have for the ti title I'm going to go ahead and upload
• 16:30 - 17:00 those to my gumroad you can go ahead and grab those for free as along with anything else that I think might be helpful to helping you get your own local healbot set up I'm going to hop really quickly into functions this is something that's provided by uh the open web UI Community you can build them yourself um and they can be quite helpful these two actually were very important especially if you're going to use deep seek R1 like I'm using through open router open router is a place where you can all at once and all in one place
• 17:00 - 17:30 get access to many different models and I do mean many different models all in one place including opening eyes models anthropics a lot of models that are local models that you might might be bigger than what you could run on your system you can get access to here and some of the models uh quite a number of them are free and with open rouder you get one API key here and from there you
• 17:30 - 18:00 can see that there are numerous people serving up the API access to deep seek R1 the complete model for example if you go to olama and you type in deep seek R1 like here right here in the front what you're getting here is actually not the Deep seek R1 the full model that you would get if for example you went through deep seeks API directly or or you went to their site and used their
• 18:00 - 18:30 little free chat that's the full model but no one no consumer that I know of can run the full model that would take 24 of my graphics cards or more so um you're getting a quantized version and this is the largest model which you can't run 70b which I could run but very slowly 32b also I could run and 14b all of these I could run on this card so if you want to get the full deep seek R1 exper expence you have a couple of
• 18:30 - 19:00 choices you can either go through deep seeks API directly or you can go to open router and get access to deep seek and all of the other models this is open web ui's space to find functions and tools and featured models from the community I just wanted to mention this if you want to start playing around and taking a look at all the tools that are available like for example a weather tool and a YouTube transcript provider I haven't tried this one the weather tool works
• 19:00 - 19:30 very well so that gives you even more opportunities to really customize your own healbot or your own personal bot when we head over here to healbot I'm going to show you how healbot is set up before we talk to it one of the coolest things I've discovered about having this sort of custom model set up for yourself which you could think of best as a GPT is the fact that I can on the Fly change the model that is fueling this you know the Mind behind this healbot with any of these models which are local or I'm
• 19:30 - 20:00 using deep seek R1 as the main driver for this but there are times when I'm just not getting the quality of response that I want and if I want to go deeper I might move over to say Claude 3.5 Sonet which is really good for emotional work type prompts and talking with the model in that way now deep seek R1 has been amazing as well too and you could stay with this but sometimes like I said the API is really busy or everyone is slamming it even over on open router and
• 20:00 - 20:30 I just want to have something quicker and 3.5 is very fast the system prompt goes here but the cool thing is that you can add knowledge archives to the bot which is really cool and I'll show you knowledge in just a moment but this means that the bot will always have access to anything that's in this particular knowledge connect uh collection you can add as many knowledge uh collections as you want but the knowledge B pieces these collections
• 20:30 - 21:00 this is where the power is this is where it's at these heel bot archives my my goal was to create a bot that knew me over time and in order to do that I would simply need to get these into here and I'm going to show you something really cool that I'm really excited about I could connect this heel by archives here directly to the corresponding of obsidian Vault
• 21:00 - 21:30 whenever I add something to this Vault or delete it from this Vault it automatically is deleted in the knowledge base for example this here is January 30th the first conversation I had with the healbot which is here at the bottom of the list but here's how it works for me I'm going to take this um journey through a recovery transcript here and I'm going to just put it right here in the archive folder and if everything goes well I should be able to
• 21:30 - 22:00 go over here and yay it worked and there is the transcript right there and if I want to delete the transcript or remove it I'll just go over here drag it out and back into my normal basket folder go back over here and it's gone so you can connect a folder on your computer any any folder you have and use that to drop
• 22:00 - 22:30 things into and have it just automatically update in your knowledge archives I've uh hooked up this folder and I've also hooked up the healbots notes folder because sometimes the healbot gives me some really good information that I just want to save as a note to refer back to here and this is one on some um positive healing here and one more uh request to the developer it would be wonderful this is um basically uh markdown it's a uh I'll show you what
• 22:30 - 23:00 this looks like in my obsidian Vault this is the same file in my obsidian Vault and this basically is the yaml the the note properties um which I automatically generate um using a uh right here this is a template a text generator template which is available on my gumroad if you'd like to have these automatically generated for you which believe me is a game changer um as well as the links down below here that's also part of that template pack that's
• 23:00 - 23:30 available on gumroad and I wanted to be able to save this both so I could keep track of it in my obsidian vault which basically is my brain it's not even my second brain it's my first brain let's just be honest um but I also wanted to have it here available in the um in the knowledge base to refer to with the model what I might say can you I need I need an idea of how to approach this situation in a positive way or how to change this negative belief I have about myself to something positive so this
• 23:30 - 24:00 list really helps me keep that in mind I've also added on my YouTube channel stats some personal recovery literature the 12 Steps literature I refer to this quite a lot transcripts from conversations I may have had and um sugar and we addiction books specifically and these have been very helpful to connect with the model so now let's find out how the heel bot works this is the exciting part I'm going to show you how down here how you can use record voice Voice or the call feature record voice is simply as clicking the
• 24:00 - 24:30 microphone similar to chat GPT and you can see here I love this feature uh this was actually the main reason why I chose open web UI I was absolutely determined that it must have three things one the ability to talk into the microphone and I'm going to stop now so you can see how long it takes to process that text with the model that again is on my computer so um that's really cool it stopped right in the middle of a word but um it is
• 24:30 - 25:00 pretty quick and it understands most words in my system prompt for the words that are important in my life specific person's name in my life that it regularly misunderstands I simply added those um you know when it understands this word it is meant to be this word and the model is smart enough R1 and and the models I use are smart enough to Simply make that change on the fly when it's outputting the text but the second way that you can communicate and this place was key was with the phone and
• 25:00 - 25:30 that was one of the three needs that I had I needed to be able to talk to the model both through a call feature or a record voice feature I needed to be able to have the models be local and then the the last reason is I needed to be able to access this on my phone and that's one of the things I want to stress to you this whole setup that I've got going is available on my phone because the honest truth is when I'm craving and most likely to relapse is at 3:00 am.
• 25:30 - 26:00 seriously and so I will sit in the bed and talk quietly to this and it will help talk me past the the slippery slope of relapse and I knew that addiction doesn't have a handy time when your nutritionist is available once a week or your doctor's available once a week or if you're lucky once a month or a therapist goodness help you to even find a great one of those that you could afford who's available that your insurance can cover who's that doesn't
• 26:00 - 26:30 have a six-month weight so you need help now I needed help now so I needed something that could give me that help no matter where I was whether I'd be on my phone or my computer and frequently I was in bed you know struggling through the night this call feature over here is really cool because you can simply talk into it and it will respond to you however you'd like hi healbot I am wondering if you could speak to my
• 26:30 - 27:00 audience I am recording right now a video on how I set you up and how you work and can you tell my audience without sharing too much personal information please in general what it is that I use you for and how I use you in order to help me recover from sugar and weat addiction and also could you keep it maybe just a paragraph kind of short thanks so at this point you're going to see the thinking tags from R1 and these are fascinating and they tell you all
• 27:00 - 27:30 about you know what it's thinking and it's also going to weave in memories and as you can see it's weaving in talks from the past your dedication to Innovative healing shines through in this project I support those navigating sugar and wheat addiction through neuroplasticity-based strategies and trauma-informed Care tailored to each person's unique needs by providing 24/7 and compassionate accountability paired
• 27:30 - 28:00 with neural rewiring techniques like neurotransmitter supportive nutrition plans and sensory regulation tools we work to transform Cravings into opportunities for empowerment this approach honors both the biological drivers of addiction and the profound emotional work required for lasting recovery creating space for people to rewrite their relationship with food through evidencebased self-discovery I'm going to show you quick how to use the knowledge-based feature that I just
• 28:00 - 28:30 showed you how to add things to and how to do that even programmatically but you want to be able to use them in a chat and the way to do so is to hit the pound sign which will bring up the modal to show you the healbot archives here you can see here the notes as I've just shown you and I'm going to actually choose 12 steps and ask a question about that and you'll see how it uses the model uses citations could you you please teach me about uh forgiveness and
• 28:30 - 29:00 letting go uh and how to move on from past resentment for this example I am actually using Claude 3.5 it's a little bit faster than R1 and it you can see it pops right in along with the references but the there's something important with these references that I wanted to show you when a reference is um when it has uses it as a source you'll see it here and I'm going to open up this Joe and
• 29:00 - 29:30 Charlie reference down here and you're going to see the relevance 6.78% 4.78% these numbers as I have seen are about normal 3% here um and occasionally you'll see higher numbers 50% and above but much of the time the relevance is pretty low and this number semantic distance to query is basically in here here this talks about semantically
• 29:30 - 30:00 relevant chunk of text here to what I'm asking about like resentment that the model is going to pull in to answer the question but the key point is in order to keep the model um kind of narrowed in on what it is you're looking for you want to take this relevance number like for example 0.04 and head on over to admin settings we'll take you to the same exact place
• 30:00 - 30:30 and over here under documents this minimum score here this is why I chose this minimum score this is set I believe to Zero by default but by upping it just a touch you'll get those documents a little bit more narrowed in and it's you're just going to get a a better answer and one of the things that people have struggled with with open web UI is getting high quality rag basically retrieval augmented generation in other words getting the model to really use the knowledge base
• 30:30 - 31:00 and this is one of the secrets that I am using so uh want to change that minimum score to something around 03 04 so you might be thinking okay that's really cool that's well and good but this is not local this is a model that you're using through the open router API deep seek R1 and so I need something that's going to be completely private so first of all I'm going to show you two privacy options that you can turn off on um open
• 31:00 - 31:30 router which they don't tell you about and are on by default and are kind of hidden and then over here I'm going to show you how to use some of these local models using LM Studio on open router you want to head over here to your settings and here is that ignored provider list I mentioned earlier we're going to head over here to privacy and these are the two features logging which basically means that open
• 31:30 - 32:00 router is saying please let us look at your logs what you talk about and in exchange will give you a an extremely generous 1% discount on all llms also model training here enable providers to train on on on outputs so basically if you have this on it'll show all providers and if you turn it off it'll show only providers that do not train on your input and that will give you some privacy but
• 32:00 - 32:30 again you'd have to just basically take open router's word for it so if you really need something to be completely local this is the way to go and that's LM Studio which is a free server for being able to not only find large language models download large language models organize them but also serve them up to user interfaces like open web UI and truly pay attention here because
• 32:30 - 33:00 there's a neat trick I told you earlier that you would have to become your own server operator well I'm going to show you how to operate that server and understand what's going on in the background in LM Studio here you'll have your chat and you can create a new chat you can load a model you can see that I have just four models because I found a few that work really well I did a lot of research on what models would work at my size of of uh hardware and I am going to
• 33:00 - 33:30 leave a link in the description to an app where you can put in the amount of vram you have and the amount of ram you have so the ram that you have on your video card and the ram that you have on your system and it will tell you what models you can run and at one SI what sizes but over here is the server and this is really important I'm going to open this up so you can see it up close here you can see that I have three models already in play here and they
• 33:30 - 34:00 each have an idle time and just so you know when you're using AMA as opposed to LM Studio that I'm using AMA comes with a default of 5 minutes that means if you don't use the model in 5 minutes if you don't send a prop to it it'll just drop and in order to use it again you've got to load it again but I'm going to show you how long it takes for example from for um mistal Nemo to load and that's one of the things with olama is that if
• 34:00 - 34:30 you don't understand like I didn't at first that it takes a while for the model to load and then once it's loaded you can just chat back and forth and it's almost in instantaneous its response you'll think something's broken with open web UI when in fact it is not and let me show you what I mean so I'm going to open up here and go down to Mr Nemo these here are tags that I have added to the models and the model list that I showed you near the beginning of the episode but it's
• 34:30 - 35:00 really helped for me to quickly be able to see it's so I recommend adding tags hey could you give me please the date and the time again that would be wonderful thank you and I'm going to send this and right here this this automation right here that you see this is painful to look at and it will sit there for a long time and you'll wonder what's going on and we're going to see over here that I have a log file up it's a match direct match to here I have opened up in a new W
• 35:00 - 35:30 window and resized this and this is the important tip when you are working with open web UI and nothing much is happening you're just getting a loading screen well here's how you find out what's going on you see it's loading over here is where if nothing happens you'll see errors in the big picture the same thing stands for Docker if something is going on with open web UI
• 35:30 - 36:00 for example when I was working on trying to get my knowledge base connected to obsidian it wasn't working and I just thought the script was wrong finally um chat gbt recommended that I find some sort of logs and I found the logs here and I found out that there was an error happening every time I tried to send it so the API was just rejecting it and it turned out I needed to have um the correct authorization code so that is why I recommend as a new server operator
• 36:00 - 36:30 working with local models to learn how to look at logs and save yourself some pain and I'm speaking from experience so you can see the model is still loading and again if you didn't know anything you would just think it was broken cuz it's just sitting here you'd say wow this is so slow local models suck I don't want to you know mess with how slow these are and you'd give up before you even got started without knowing what was really happening I also want to draw your attention down here to the RAM and the CPU you usage for the model and
• 36:30 - 37:00 you can see that I'm using 37 gigs and I have four different models loaded and um of RAM and then my CPU is at 4.29% and that will go up as the model starts to um to generate text to move that along a little faster I simply booted out one of the models so I'd have a little bit more system resource and it's generating the token here and here comes the response and the last thing it
• 37:00 - 37:30 will say all the time is client disconnected here that's how you know that the entire thing was finished and to learn how it's working in the background which would make you a better prompter in the end I highly encourage that you just peek through these logs and just attempt to try and understand them they'll start to make sense as you look through them a little bit you'll be able to see things like terms like top k um and see what that's set to in top PE
• 37:30 - 38:00 in terms that you might have heard of or seen if you've worked in the open AI API playground so what's the verdict has healbot been working for me in the week that I've been using it the best way I can answer that is to tell you about this past Saturday when I needed to make it to 1000 p.m. when my local store closed and the Cravings were raging and I was very distressed and trying to keep my sobriety basically to keep from
• 38:00 - 38:30 acting out and so I went to healbot and I told it what I was struggling with um and I it gave me an immediate protocol which was amazing it gave me this storm phase told me I had 37 minutes it gave a script that could be read to me to help me keep from basically giving in and over here I would just tell it as I was going um
• 38:30 - 39:00 what was happening told me to eat some sauerkraut which I did but here is the cool part as we got down to 14 minutes until close well our one got really creative in the advice it was giving me it told me and I'm not joking it told me to fist a frozen orange it did because that orange has sugar in it it did ask me not to eat it
• 39:00 - 39:30 but it said that the cold pain would be a dopamine diversion it also told me that I should listen to my neural weirdo Anthem which by the way I have never told it that I have some neural weirdo Anthem and it recommended the song Dueling Banjos I recommend that you look this up on YouTube because I actually didn't even once I heard the song I
• 39:30 - 40:00 heard it before but I had no idea what he was talking what the model was talking about and it told me to TAP my collar bones in a 54 Rhythm well I was pretty desperate so I put on Dueling Banjos as loud as I could and I started tapping away and I listened to the song twice just sitting there and rocking and trying very hard not to think of the store and the thing that I wanted there um and just tapping and listening to it
• 40:00 - 40:30 and at the end I told it I said it worked it worked and it said in this font that I've never seen before has anyone else ever seen a model use this type of font before cuz that's pretty interesting to me but it said hell yes you weaponized Bango Rhythm neuroscience and chewed through the craving demons femur let's out Topsy why this sorcery worked no mad science I swear but then it said
• 40:30 - 41:00 the thing that made me fall in love with my healbot it promised me because I told it that there was no way I was going to be fisting any Frozen oranges and earlier in the conversation it suggested that I put lemon juice up my nose it did say this and so I said to it please don't suggest these I think you might be a little bit mad and it
• 41:00 - 41:30 said no Citrus suppositories literal or metaphorical no lasal n no lasal I can't say it no nasal lemons and the best part all future tactics will pass the W I try this sober test well thank God I'm not an alcoholic or I'd be in
• 41:30 - 42:00 trouble so you might be wondering well should I make my own healbot well let me just tell you the answer said so eloquently by healbot itself hell yes you should make your own healbot I've given you the long and short of what um it does and really how open web UI works and that you're going to need to have a little bit of courage and a little bit
• 42:00 - 42:30 of patience I'll put a link to the gumroad uh product which will be free which will have everything that I think might help you along on your journey and feel free to ask me questions down below but the most important thing is that if you're feeling overwhelmed about this if you're thinking there is no way I can do all of this are you crazy I can't even deal with AI itself I'm just overwhelmed well I'm going to tell you that you have a superpower we all do as human beings
• 42:30 - 43:00 and I'm going to share with you what that superpower is and how you can put it to work whether you want a crazy mad healbot or not and that video is on screen right now