Kubernetes Monitoring with OpenTelemetry (Setup & Demo)

Summary

In this video, Bhoopesh Sharma walks us through the setup and demonstration of Kubernetes monitoring using OpenTelemetry. OpenTelemetry acts as a collector for application metrics, enabling seamless monitoring across various tools. The session covers the installation of a microservices application, an OpenTelemetry collector, Jaeger for tracing, and Prometheus for capturing metrics. Finally, the video concludes with the setup of Grafana dashboards to visualize the collected data, all illustrated with a practical demo.

Highlights

• Introduction to OpenTelemetry and its role in monitoring applications. 📡
• Explanation of OpenTelemetry's popularity due to its vendor-agnostic features. 🏆
• Step-by-step demo of setting up a microservices app and OpenTelemetry collector. 🚀
• Installing Jaeger for tracing application transactions. 🕵️‍♂️
• Deploying Prometheus to capture application metrics. 📈
• Setting up Grafana dashboards for data visualization. 🎨
• Discussion on how OpenTelemetry captures and sends data for monitoring. 🔄

Key Takeaways

• OpenTelemetry facilitates vendor-agnostic monitoring, allowing easy migration between different monitoring tools. 🎛️
• By acting as a collector, OpenTelemetry gathers data from various application endpoints before sending it to backends for analysis. 🗄️
• The video covers a practical, step-by-step installation of necessary components including microservices application, collector, Jaeger, Prometheus, and Grafana. ⚙️
• Grafana dashboards help visualize monitoring data, enhancing the observability of the system. 📊
• The setup enables capturing of both application and infrastructure-level metrics, offering a comprehensive monitoring solution. 🌐

Overview

Bhoopesh Sharma kicks off the session by introducing OpenTelemetry, a robust solution for monitoring applications, especially in Kubernetes environments. As a vendor-agnostic tool, it provides flexibility in choosing and switching between backend monitoring tools, enhancing its appeal in the tech community. The video emphasizes the efficiency of OpenTelemetry as a collector of application metrics and traces.

The tutorial then transitions into a comprehensive walkthrough of deploying a microservices application and integrating OpenTelemetry. Bhoopesh guides viewers through the step-by-step process of setting up an OpenTelemetry collector, detailing its role in capturing data from application endpoints. The setup also includes the installation of Jaeger for tracing transactions across the application.

To complete the monitoring setup, Bhoopesh demonstrates the deployment of Prometheus and Grafana. Prometheus captures the metrics, while Grafana offers a user-friendly interface for data visualization. Through this setup, viewers learn to create a holistic monitoring framework that can observe both application-level and infrastructure-level operations in a Kubernetes cluster.

Chapters

• 00:00 - 00:30: Introduction to Telemetry Concept The chapter introduces the concept of telemetry, focusing on its relevance for monitoring applications. The speaker encourages interacting with the channel (like and subscribe) to stay updated on topics such as Kubernetes, DevOps, CI/CD, monitoring, and OpenTelemetry. The introduction sets the stage for a deeper discussion on the concept of telemetry.
• 00:30 - 01:00: Understanding OpenTelemetry The chapter titled "Understanding OpenTelemetry" provides insights into what OpenTelemetry is and why it is gaining popularity in today's world. It explains that OpenTelemetry acts as a collector for various types of applications, such as Java, .NET-based, or Node.js applications, which require proper monitoring. Instead of sending data directly to monitoring agents, OpenTelemetry first collects metrics before sending them to the relevant agents.
• 01:00 - 01:30: OpenTelemetry's Vendor Agnostic Nature OpenTelemetry's main advantage is its vendor agnostic nature, allowing easy migration between different monitoring systems such as Grafana, App Dynamics, or Dynatrace. This is possible because the OpenTelemetry collector remains the same, enabling continuous transmission of metrics and traces without disruption, merely by changing the link to the new vendor's system.
• 01:30 - 02:00: Additional Features of OpenTelemetry The chapter 'Additional Features of OpenTelemetry' explores various extended capabilities of OpenTelemetry. It highlights that users can augment data collection by integrating infrastructure-level details, Kubernetes API information, and any cloud-related metrics into the OpenTelemetry collector. This process aims to provide a complete and holistic view of system performance and operational insights. The chapter includes a practical demonstration through the installation of a microservices application onto a Kubernetes cluster, followed by the deployment of the OpenTelemetry collector for comprehensive data collection and analysis.
• 02:00 - 02:30: Overview of Demo Setup The chapter begins with an explanation of the components involved in setting up a demo for tracing and data collection. The setup involves three main parts: a collector that gathers matrices and traces, employing Jaeger for tracing transactions, and using media to obtain matrices. After covering the theoretical aspects, the focus shifts to practical implementation, with a mention of a GitHub page created to follow along with the practical setup.
• 02:30 - 03:00: Preparing for OpenTelemetry Setup The chapter titled 'Preparing for OpenTelemetry Setup' appears to focus on the preliminary steps needed for installing and configuring the OpenTelemetry framework. The transcript from the chapter points out the readiness to dive into a demo or presentation related to this process, highlighting the use of README documentation as a guideline. Each folder in the setup directory contains development manifests required for the installation process. The first step involves installing and configuring the necessary components for the OpenTelemetry framework.
• 03:00 - 03:30: Setting Up OpenTelemetry Microservice In this chapter titled 'Setting Up OpenTelemetry Microservice', the discussion revolves around a Microsoft application configured as an OpenTelemetry app. The folder contains both deployment configurations and services, which incorporate numerous endpoints. These endpoints are structured to be deployed and exposed as services. A critical point discussed is that metrics are transmitted to an 'otel' exporter endpoint, also referred to as the 'hotel endpoint'. Primarily, the function of the application is to send data to The Collector, acting as the centerpiece of the setup.
• 03:30 - 04:00: Installing OpenTelemetry Collector This chapter covers the installation of the OpenTelemetry Collector. The process begins by setting up a route for the collector to send data to Jaeger. Following the route setup, the narrator installs the OpenTelemetry microprocess application. This involves first cloning the necessary Git repository and moving into the corresponding directory. Next, a Kubernetes namespace specifically for OpenTelemetry is created to ensure that the application is isolated within the cluster. The narrator confirms that this is a new cluster and proceeds to create the namespace before running the required commands to complete the installation.
• 04:00 - 05:00: Configuring Jaeger for Tracing This chapter covers the process of configuring Jaeger for tracing. The speaker discusses installing various manifests present in the current folder and checking the DNS. They mention having installed everything into a default namespace and demonstrate switching to a different namespace for the Hotel demo.
• 05:00 - 06:00: Configuring Prometheus for Metrics In this chapter titled 'Configuring Prometheus for Metrics', the focus is on installing and configuring Prometheus within a Kubernetes environment. The narrator encounters a common issue of mistakenly attempting to install components into the default namespace. To resolve this, they decide to delete existing configurations using a command ('kubectl delete -f') and choose to install all components into a specified foreign namespace instead. This approach helps in effectively managing and terminating deployments that are incorrectly installed in the default namespace. The importance of namespace management in Kubernetes is highlighted as a crucial step in the successful configuration of monitoring solutions like Prometheus.
• 06:00 - 09:00: Integrating Grafana for Dashboards The chapter 'Integrating Grafana for Dashboards' begins with a demonstration where the speaker navigates into a 'hotel' namespace within a system. At this point, the namespace appears empty. The speaker then executes a command to install necessary components into this namespace. As the installation process continues, the microservices application intended for this namespace starts running. The installation is confirmed as the application begins to come up successfully.
• 09:00 - 10:00: Conclusion and Summary The chapter provides a quick walkthrough of a microservices-based application, focusing on the setup and initial access. It involves logging into the application and observing how a PostgreSQL database and a queue are used for data storage. The chapter concludes with the application being initialized, particularly highlighting the raw detection service as a notable component of the system.

Kubernetes Monitoring with OpenTelemetry (Setup & Demo) Transcription

• 00:00 - 00:30 foreign Telemetry concept which is a very popular concept for monitoring your applications so yeah before we start uh onto this topic please like And subscribe this channel if you have not done it uh this will help us we help you to understand or get the latest videos related to kubernetes devops cicd monitoring open Telemetry and so on so yeah so let's discuss about this concept what
• 00:30 - 01:00 is this and why it is so becoming so popular in today's world so you might have on the left hand side can have any kind of an application it can be a java.net base or Java node based node.js based application that needs to be monitored properly so what this open Telemetry it will act as a collector so rather than sending those entry directly to your monitoring agent so it will first of all collect the matrices and then we'll send to the relevant uh the
• 01:00 - 01:30 back end it can be time series database it can be traced database it can be anything so why it is so popular because it is you know vendor agnostic when I when I spent agnostic it is I mean you can easily migrate from today's suppose your doing monitoring using grappana or app Dynamics or dynatory so tomorrow you can simply move from one vendor to another Vendor by simply changing the you know link so Hotel collector will remain same so application will continue sending your matrices and traces to the original
• 01:30 - 02:00 plan to the relevant backend now what additional things you can do it you can get the infra level uh details also the kubernetes API level details or any Cloud related methods can also be added to what we collector and finally it can send it to the uh to the end result so just to see a complete holistic picture we'll be doing a quick you know installation of first of all a micro Services application onto the cluster and then we'll install a
• 02:00 - 02:30 collector which will receive the matrices and traces from this app to this second section and then we'll use Jaeger for tracing or tracing the entire transactions which will take the data from collector basically this is the third section and then we'll have a media for getting the matrices so yeah this was all about Theory let's quickly jump on to the Practical implementation of it so I've created a GitHub page for this I will follow that page for our
• 02:30 - 03:00 today's demo and then you know let's let's quickly get started into the the installation of this entire piece so this is the readme I'll be following the readme documentation so specifically updated all these folders and in in each and every folder we have development manifests that are needed for installation of this entire open Telemetry framework okay so first of all you need to install and configure
• 03:00 - 03:30 Microsoft application so it is an open Telemetry app so basically you will find in this folder uh deployment and a service so basically there are a lot of endpoints that are there uh those endpoints are deployed as a deployment and the same thing is being exposed as a service now you will see in any any deployment file you'll see the matrices are being sent to this hotel exporter Hotel endpoint which is the one I'm talking about the centerpiece okay so first of all the met the application send data to The Collector
• 03:30 - 04:00 and then collector will send to Jaeger we'll follow the route and then first couple we will install the open Telemetry micro process application so I'll go to this folder after doing a you know a git clone of that repository okay I'm into that folder now I'll create a namespace if it is not there this is this is a brand new cluster so hopefully yeah the name Hotel membership has been created now I'll run uh this command
• 04:00 - 04:30 which will install all the manifests which are there in the current folder okay so let me see DNS oh I have installed everything into the let me see just again let me explain the Hotel demo kgp Oh by default I have installed everything into this world no problem let me just quickly change it to kns
• 04:30 - 05:00 default my bad just give me a moment let me just quickly Cube CTL delete minus f I don't know how to install anything into the default namespace so we will install uh everything into the um foreign is getting terminated
• 05:00 - 05:30 I'll pause the video for a second okay so I'm into the hotel name space the you know name space and now nothing is there so I'll run this command again and this will install everything to this namespace perfect so now my microsources application should be running here also it must be coming up yeah it is coming up and I'm into this namespace a what this will do it will install this
• 05:30 - 06:00 micro sources based application uh we will first of all quickly see that application how we can you know do a quick login it is installing a postgres database and a queue for storing the data so let me just quickly see uh the app is slowly coming up yeah it's almost up raw detection service is
• 06:00 - 06:30 let me just quickly do a hit service key put forward I'll do a front-end proxy 8080 oh this is the URL of your application basically yeah so this is my application which is you know already up so there's a wrapper of documentation which is already already there you can just follow this documentation for any kind of other understanding so my application is up
• 06:30 - 07:00 let me just quickly do some uh Plug and Play I'm able to do some dummy orders okay so my application is up and running so now for sending this application data to collector I need to install the open Telemetry collector so I will come back to here okay and then I'll open Telemetry collector I'll go to this folder I'm inside this folder now what this is doing is simply installing a doing a deployment pulling an image
• 07:00 - 07:30 and then it will just capture all the data which is being sent by the Microsoft service application and ultimately to assigned to the Jaeger port and all those things that will come in the third step first of all it will capture the data so qptl apply f so now this will install your collector I'll just go to the checking tab yeah you can see the open telemetric collector container is getting created
• 07:30 - 08:00 I can see the events yeah it is coming up okay my collector is up and running now and we see the logs also for the collector I'm just taking little bit time to explain each and everything so that it becomes clear for understanding so see now there are no error in The Collector logs it is able to capture the data from the tracing okay punch for your next episode
• 08:00 - 08:30 come on to the second step third step is relying to install Jaeger now what this Jager will do it will just see the deployment file okay it will capture the data and then it will push it to the Prometheus so first of all let me install Jaeger and then we'll install you know Prometheus Next Step
• 08:30 - 09:00 okay folder Cube CTL apply f the Jaeger has been installed let me hit the cube it will get power that is getting created see so first of all we install application then we install collector which contain the details from application to collector and then while you're installing in a Jager perfect so Jaeger is already also installed and let us see uh
• 09:00 - 09:30 then let's install the course over the primitius that will actually capture the matrices related to all that application so we'll go to Prometheus folder and then we'll do a cubital player now this will install Prometheus server only the same namespace and then we can see the kgp
• 09:30 - 10:00 so let us DD open television documentation in parallel so we have already installed the application let me do a port forwarding again here ah this is the front end clock C port forwarding so
• 10:00 - 10:30 you can see the application is up and running let me do some more hits on this application again to cart place order continue shopping this is this now we will just open the prepana is not installed here so I'll just open the Jager home page let me see if there's any kind of data which is coming here
• 10:30 - 11:00 is also installed and the traces are coming up let me see whether we have Prometheus installed okay Prometric is not there but I can do a port forwarding from here so just try to understand the concept first of all we have installed application and then collector and then Geiger and then Prometheus let me see since there are not visual server is
• 11:00 - 11:30 there foreign foreign service port forwarding I'll come here okay I can do up let's see whether things are coming or not okay data is coming
• 11:30 - 12:00 and the data is coming properly I can see the discovery also I have to targets Target point so yeah so permit it is also everything is up and running right now finally we need to install good corner to see the dashboard because that will also only make you know some significant so if you come onto the grafana folder we have a deployment file that will install you know grafana and it will have the
• 12:00 - 12:30 admin user ID and password and then we are having certain dashboards the predefined dashboards related to open Telemetry collector that we are installing you can just simply do copy waste on that we have a secret that is nothing but admin user ID admin password so this secret is used for logging to grappana and the config map that is used to test the graph that is up
• 12:30 - 13:00 yeah so that is that is finally the installing grafana so there is the final piece yeah so I'll come back here CD dot dot grafana and I'll do a cube CTL apply as okay you can ignore this because the test scrapana is trying to run onto the port 8080 which is already being used
• 13:00 - 13:30 here let me just quickly cancel this and then I love it and run this yeah so that is fine I'll do a kgp okay starting and grafana test is just you know creating the container Okay so I'll just Hep
• 13:30 - 14:00 clear GP yeah forget about this death dot board and it's just you know doing a hit on this uh you can simply delete spot that is not needed what does it is a testing code fine so my grafana is already so up let me see the grafana service as we see um service so I'll do a key port forward
• 14:00 - 14:30 foreign as I said the username is admin and the password is also admined okay I don't change the password now
• 14:30 - 15:00 just you can ignore this piece you can simply come here you can see explore the option and then you see Diego there are certain so traces are coming properly you can see now let's see something else the let's see the Matrix is also from Prometheus you can see a lot of matrices are coming let me just to open
• 15:00 - 15:30 okay one more uh two run one query to see how many matrices are coming yeah the query that I'll run to see what kind of matrices are coming basically how many matrices per job is coming yeah so I can see yeah so all these are endpoints that have been uh been monitored using open Telemetry
• 15:30 - 16:00 only and yeah so all these endpoints are nothing but your uh application endpoint you can see card service checkout service all those things that's part of the reason I want to run this piece I want to show it to you now let Theory finally the dashboards I'll click on the dashboards there are certain dashboard that we executed when we you know installed grafana so those dashboards are nothing but it is in this folder uh foreign
• 16:00 - 16:30 counts okay navigating some data to the back pan processor Also let's see this mother dashboard open television collector yeah yes we have started looking at some data onto the open Telemetry collector dashboard also okay now but ultimately collector data flow see so you have only one instance because it
• 16:30 - 17:00 is a single node you know k3d cluster so CPU memory you can see the uh the node graph also right and you can see the other related details like it's the pure open Telemetry collector dashboard and then finally see this pan metric stash code also you have so nutshell you're getting data everywhere the
• 17:00 - 17:30 proper linking has done so so the reason behind this the complete demos to we do uh understanding how it works for example we have installed uh application so you can see the application is sending details to this open Telemetry collector which is nothing but your this hotel call endpoint this and then it is sending it to four three one seven if you see any deployment sending data to this uh four three one seven now if I come here collector this is the collector that I'm talking
• 17:30 - 18:00 about this is the four three one seven is the one grp OTL otlp Port okay so here the entire data is coming back and then this collector is sending data in a Jager at this board one four two five zero if expand is eager deployment file you will find this one core uh the recipient host code and
• 18:00 - 18:30 yeah it should be there yeah so all those ports are there and ultimately it is eager is also sending data to over Media Services this is the one and then Prometheus server is getting the data for all these endpoints and it has a config map config map is nothing but the scraping thing yeah it is scrape interval not scrap interviews it is scraping that is happening
• 18:30 - 19:00 it is Handy having the name space and the relabel configs and finally SD grafana where it is having all these final details about the dashboard and the uh so basically what we say in a nutshell we have installed how many tools the factor for the application and then the collector second third post is simply five uh you can see five different charts of file five kind of bundle many class we
• 19:00 - 19:30 have installed so yeah I think we should wrap up this video we should not stretch it further so as a summary what we have done we have understood with a working example what is open Telemetry how we actually collect the traces from a working microsystem with application and how collector is sending data to a current parameters at the same time and now we can see the data back to the katana in terms of passwords so I think that is for now
• 19:30 - 20:00 maybe we'll have some more open Telemetry use cases in future so if you have not likened the families do that from here so that you can get similar kind of Channel sessions in future okay I would wrap up the session now thank you for now thank you okay bye