Lecture 48 "Docker Container – Overview Docker – Components Docker – Architecture"

Summary

This lecture explores the concept of container-based virtualization with a focus on Docker, an open-source and widely-used platform. The talk delves into Docker's architecture, advantages, key components, and how it facilitates software development and deployment. The use of Docker in streamlining traditional software engineering processes is highlighted, demonstrating how it helps overcome challenges associated with software deployment, management, and execution across various environments. Further, the session discusses Docker's key features, its replacement of virtual machines, and touches on the Docker architecture, encouraging the audience to engage with a recommended book on Docker to enhance understanding.

Highlights

• Docker is revolutionizing container-based virtualization, allowing applications to run smoothly across various platforms 🌟.
• It creates self-contained applications that can be shipped from your laptop to any server 📦.
• Docker aids in reducing software development complexities, handling dependencies efficiently 🔧.
• The platform is designed to be lightweight and easily replicable, which speeds up deployment processes 🏃.
• Docker architecture enables seamless software updates through layered file systems 🗂.

Key Takeaways

• Docker simplifies container-based virtualization, making software deployment seamless 🚀.
• It is an open-source platform that builds, ships, and runs applications anywhere 🌍.
• Docker eliminates many traditional software deployment challenges, boosting efficiency ⚡.
• Offers lightweight and faster alternatives to virtual machines ⏩.
• Docker supports continuous delivery, facilitating more agile software development 💡.

Overview

Docker represents a shift in container-based virtualization by providing a platform to build, ship, and run any application regardless of the environment. This open-source tool has become popular due to its ability to encapsulate applications along with their dependencies into containers, thus facilitating easier and more reliable software deployment.

One of the standout features of Docker is its capability to provide a lightweight alternative to traditional virtual machines. This means quicker startups and less overhead, which significantly aids developers and IT operations in rolling out applications swiftly across different systems. Docker's container ecosystem ensures that software behavior remains consistent regardless of the host environment.

The lecture further elaborates on Docker's efficient handling of complex configurations, making it easier for developers to manage and deploy applications. By employing a layered file system, Docker enables quick updates and efficient resource management, aligning with the needs of modern software development practices that prioritize agility and continuous delivery.

Chapters

• 00:30 - 06:00: Introduction to Docker This chapter introduces the concept of Docker as a popular service in the realm of container-based virtualization, which is a part of cloud computing.
• 06:00 - 12:00: Docker and Virtualization The chapter focuses on Docker, an open-source platform that plays a crucial role in container-based services. It begins with an exploration of Docker's fundamental philosophy, including its primary benefits and the architecture that makes it popular in the tech industry. The chapter emphasizes Docker's wide applicability and sets the stage for upcoming lessons and demos that will showcase practical applications of Docker. These demonstrations aim to reinforce the theoretical knowledge about Docker and show its versatility and effectiveness in real-world scenarios.
• 12:00 - 29:00: Advantages and Features of Docker This chapter introduces Docker containers, emphasizing their general advantages and features. It briefly mentions the basic architecture of Docker and highlights the key components and benefits. The focus is on understanding what makes Docker containers advantageous for various applications.
• 29:00 - 45:00: Docker Architecture and Commands The chapter explores Docker architecture and commands. It illustrates the containerization concept through a popular Docker image, explaining how containers are self-contained entities with all dependencies packaged within. These containers can be easily transferred and run across different environments, such as from a personal laptop to a server.
• 45:00 - 48:00: Conclusion and References The chapter discusses the concept of self-contained applications, which can run in the cloud or anywhere without relying on external dependencies. It likens shipping a self-contained application to transporting material in a container, suggesting that everything necessary for the application to run is included inside the container, ensuring seamless deployment from source to destination. This encapsulated format ensures that the application functions properly regardless of the environment in which it is deployed.

Lecture 48 "Docker Container – Overview Docker – Components Docker – Architecture" Transcription

• 00:00 - 00:30 [Music] [Music] hello ah so let us continue our ah discussion on count computing more specifically on ah container based virtualization rather we will be looking into one aspect ah one very popular service that is the docker
• 00:30 - 01:00 right which is open source and widely used and it will help us in understanding the basic say philosophy basic advantages and basic architecture of a typical container based services not only that in our subsequent lectures we will also have some demo using this docker to show it ah how applicable it is though we are looking with the docker but it is
• 01:00 - 01:30 generally true for any any type of containers right so ah we have ah this ah docker container which will be discussing and ah keywords this uh as we have seen that as we looking at the docker container it will be con docker container and architecture basic architecture advantages and so called benefits and key components of docker component
• 01:30 - 02:00 container what we will be covering so this is a very popular picture if you look for docker which also implicates right like yesterday last class we were discussing regarding [Music] this containerization service where we see that it is quantized all things are which are self content or ah with all it dependency a container which can be shipped from one to another like it can run to your laptop to server to
• 02:00 - 02:30 cloud anywhere so it is a self content application so which has the that application running and we along with the dependencies and other things right so it is more like a as we discussed that is shipping through a some material in a container so the whatever you put in the container should be a self contained so it is from the source to the destination it will go and there it will act in between whatever the way it is shipping or how
• 02:30 - 03:00 it is managed etcetera it is based on the how how the overall container thing so it ah it not only created a revolution in our overall transportation systems of goods and things ah that ah making a standardized approach it also what we see is making a revolution in the software world so today will be more ah concentrating with docker a open source platform so it is a it is a platform that allows to build ship run any app
• 03:00 - 03:30 anywhere very ah straight forward and direct thing so it is a it is a [Music] its a platform which will allow you to beat build ship and run any app any application anywhere so it irrespective dependency of the waste etcetera so you have this type of docker container so it is considered to be a standard way to solving one of the challenging aspects of the software not only
• 03:30 - 04:00 development development is sorry deployment development deployment and also we see that it helps in your testing and things like that right so it is this is ah containers attempts to address this problem ah additionally traditionally what we see that any software development process is a pipeline typically involved ah combination of various technologies for managing the movement of the software such as virtual machines or vms we have seen configure
• 04:00 - 04:30 configuring this management tool right ah package management complex wave of libraries dependencies etcetera right so this is those who are in the software industry or those who are working with the software and seen or even looked into our traditional software engineering approaches and documents you see that these are the things which come into play when we go for the development so all this tool needs to be managed and maintained by
• 04:30 - 05:00 specialist right specialist or trained personal or what we say trained engineers many of them or most of them having their own way of approaching the problem right so what it makes more difficult to interoperate right so when you ship one to another suppose you develop something in a variant of linux say ubundu and put it on red hat or sent ways or something and then it creates a problem you need to re-engineer the whole thing not not the whole thing you need to put your hand on the things to
• 05:00 - 05:30 see that how it need to be customized on that environment right this is a big hindrance in deployment right in especially large case deployment where you expect that the other end you cannot expect the other end to be ah versatile enough to handle all those things right and especially in today's world when ah this i t service purification is in a big way right starting from our banking to everything what we talk about in our life is now in this ah
• 05:30 - 06:00 is going in that way and that is a is serious challenge right if you if you need something to be configured at your end ah for getting that service it is a its a challenge right so this docker ah try to address this things rather container based services tries services try to address this [Music] this challenges right
• 06:00 - 06:30 so docker or containers has changed the traditional approach so everything goes through a common pipeline ah to a single output that can be used on any target there is no need to continue maintaining something that array of configuration mechanisms right so it is a thing so in other sense if i have a application self content say i have a database of say ah customize things say mysql and so like that and i develop out
• 06:30 - 07:00 here and give it to you to run on on based on your other application like it talks to your web server also so it should work right i do not have to go and look at the things that how my skill to be configured etcetera so my sql may be a more ah what we say more versatile or more widely used so it may be configuration may be easy but there are if there are customized things so it may be a challenge to do that right
• 07:00 - 07:30 so at the same time there is no need to throw away the existing substrate stack if it works for you right that is that is another important thing something working for you for a substrate stack for a good amount of time and you do not want to throw that right you can package it up in a docker container as is for others to consume right so this this is another way of looking at it when we make a containerized services right so so this is another ah important thing so something a software stack which is
• 07:30 - 08:00 working fine for you and not only that in some of the cases it may gone through some testing and some certification or some sort of a standardization process and then you do not want to throw it out and you want to make it as a continent so that you can be delivered to other to use right so additionally you can see how this containers are built ah so if you need to dig into the details you can right so in other sense that additionally if you want to see that
• 08:00 - 08:30 how this overall container is built you can look into the things right ok so this is the big picture so before going further let me ah refer to this ah book this is a very nice book it is good to read that docker in practice by ian mill and aidan ah hobson ah so this is a nice book so rather most of the things what we are discussing are referring to this
• 08:30 - 09:00 book so i kept that reference here but i encourage you to have a look in this book so it is a good to have a overall feel of the things very nicely written so what we see that before docker ah so we have different tools right so like if you see that the top one what this is the inputs this green is the input to the system requiring manual maintenance fewer inputs are here mean less the maintenance button right so these are the inputs and
• 09:00 - 09:30 that these are the tools this blue color that use the input to create environments for the software development right so these are the tools and stages of software development requiring the environment to rani right so this is the development taste life type of things and here also testing release to life but what we see that ah overall this with the docker we can reduce this middle term big way rather
• 09:30 - 10:00 what we can see that some of the configuration can be standardized which may not be so much thing right so it becomes quote unquote life little easier for software ah specifically for software deployment where for using docker or other any containerized services so one ah analogy ah or which it comes from like if you look at the docker is
• 10:00 - 10:30 typically a person or a so called quote unquote laborer who move the commercial routes in and out of the ship when they are docked at the ports right so it is doctor the port and docker so these boxes are items of different size shapes and experian dockers actually make that fit the goods in a appropriate way right cost effective way in the sense packaging is done right so at it it is first of all costly ah in getting those people and
• 10:30 - 11:00 making these things are ah challenging right now though we do not have that pretty early traditional things but in something shipping what we are doing we are talking more using cranes and etcetera nevertheless if there are variable number of sizes of box making then appropriate things forget about ship if i if you have a variable number of say ah gift boxes even say ah proper
• 11:00 - 11:30 means regular shape rectangular type of things then also if i if you want to pop put it on a big box its always a challenging because you need to optimize that thing in a big way so this is something what similar what we do in our software development so much time and intellectual energy is spent getting the metamorphological this odd safe software so ah into different size ah ships or carriers
• 11:30 - 12:00 full of other or safe software etcetera etcetera right so how to meta morph this type of things into this odds separate size software into the things so it is somewhat similar so that may be a reason where the name came from so if we look at the ah the benefits or why why docker so to say so before docker deploying software or to different environments required
• 12:00 - 12:30 significant efforts right even if you are not handling scripts of provisioning software to different machine you would have to handle the configuration management like even if you are not doing the software development but at least the configuration management is also a challenging you might have seen that if you have a working in organization or like even your institute so once you put one to put a software by a software and put on the things and then they are expert engineers comes and configured based on your available
• 12:30 - 13:00 resources right so and it ah gets a ah configured on the things it is not may not be very ah true for your ah normal software when you download about a laptop and two so i can convert still if you want to do a customized configuration right other than your ah default configuration then you have to need to think right that what is your
• 13:00 - 13:30 requirement fitting to your requirement and the software things is a again a challenge so those challenges are they are true so even this these things is little eased out when we went for ah vm type of solution like virtual machines and things like that so we ah try to ease out this ah type of ah challenges but even with this ah encapsulation in vms a lot of time was spent managing the deployment these ah
• 13:30 - 14:00 vms waste waiting for to boot managing the overhead resources they use and things like that right so though the vms tries to ease out this make some sort of a standardized way of approaching the things but still there are challenges in handling this stuffs so ah again referring to this particular book if we see so there are if we look at the life or
• 14:00 - 14:30 not ah before it is without a traditional way of looking at it this is very much existing out here now so there are typically three step to manage the deployment right one is ah that install configure and maintain complex application right and then install configure maintenance complex application in test server and install configure maintain application is live server right so if you want to manage it is like
• 14:30 - 15:00 this these are the things which you want to do whereas ah in docker ideally there may be a single effort right install manage ah maintain complex application in docker image so if you can make this docker image ready or things what you find in a lot of ah standardized image ah docker images in the docker hub those who are using or you can ah look at the docker hub
• 15:00 - 15:30 so you can docker run on development laptop test server and live server or what you have the production server type of things right so it makes ah definitely life easier for the specially for the deployment engineers who break their hand on lot of time on this type of things docker benefits with docker the configuration effort is separated from the resource management right and the
• 15:30 - 16:00 deployment effort is trivial right so docker is separate from the resource management so it is something like run your docker the environment image is pulled down ah ready to run right consuming fewer resources contain and content so that it does not interfere with the other environments right like i was telling that if i having docked into a mysql into a docker and put it in the things with a properly
• 16:00 - 16:30 configured then there it goes on running on the things so what we require is that the docker engine out here or the where the platform any other containerized platform which can house this docker right so rest of the things are ah need not to be looked into other dependencies and like this right so there is a big big advantage you do not need to worry about whether your container is going to be shipped into say a red hat machine or ubuntu or sent ways as long as the docker on it right
• 16:30 - 17:00 right so i do not much worry about the where it is going to house or the what is what he what we want to say that it is whether it is ah any flavor of this waste we if that docker as long as the docker is running on it it will run on the things right now we like to have some of the key advantages or of the things or key reasons one is replacing
• 17:00 - 17:30 ah i should say ah not only advantages key features may be the appropriate word so replacing virtual machine docker can be used to replace vms in many situation right if you do not if you only care about the application not the operating system docker can replace the vm rights where where because in these days this ah with wide scale deployment of the of this i t services this applications
• 17:30 - 18:00 are becoming major things so where docker may help not only in docker quicker than vm to startup it is more lightweight that we have discussed in the last [Music] class also the lightweight to move around right so it is lightweight so shipping to one to other is much easier and due to its ah layered file system right so it is approach a layer approach you can more easily and quickly say changes with others it is also rooted in the command line in and scriptable all
• 18:00 - 18:30 it is possible to script it so it is scriptable so prototype typing software if you want a quickly experiment with software without either disrupting your existing setup or going through the hassle of provisioning a vm etcetera so it is ah docker can give you a sandbox environment right to do a almost almost instantly right so it gives you a sandbox in environment almost instantly where you can prototype
• 18:30 - 19:00 your software for ah other than now otherwise the before prototyping the setting up of the whole environment takes a lot of time which can be effectively saved when new use the docker platform other features are is that packaging software because docker image has efficiently effectively no dependency right so a docker image do not have any dependency it is great way to package software you
• 19:00 - 19:30 can build your image and be sure it can run on any modern linux things right so ah anything ah it can run on the ah so packaging is much much easier enabling micro service architecture docker phase facilitates the decomposition of complex system to a series of composible parts right so that is a thing right so where it ah you can decompose your complex system in a in a set of ah which can be composed to ah
• 19:30 - 20:00 into this ah making this ah this complex system enabled right which allow you to reason about your services in more discrete way that is this allow you to restructure your software ah to make the parts manageable and pluggable and type of things right so in other sense if you have a complex system doc dockers do facilitate make them into ah manageable parts which can be separately managed but once it packaged it goes
• 20:00 - 20:30 other things right so it [Music] means ideally it enables a micro service architecture modeling network it is ah so once you have say several hundreds or more ah of isolated container can be initiated in one mission modeling ah network can be done efficiently so that also it is helpful so modeling is network enabling full stack productivity when offline right all parts of the system can be bundled into docker container you can
• 20:30 - 21:00 orchestrate this to run in your laptop and work on move even when the thing so in other sense ah on offline mode also if all the things are there you can basically bundle it and run ah work on the work on move right or ah this is has different type of advantages one is that this offline ah full stack productive environment otherwise otherwise testing something and doing something
• 21:00 - 21:30 on a offline mode also helps in reducing debugging over it ah right complex negotiation between the different teams about software delivered in a common place within the is a common common scenario in any industry right so docker allows you to state clearly even in script form the steps for debugging a problem or on a system and known with known properties right so making bug and environmental products
• 21:30 - 22:00 and much simpler etcetera so it it facilitates ah this debugging things and thus ah reducing the overall overhead cost of debugging right it you can clearly specify the stuffs steps of ah for debugging a problem and things like that and also overall the overall ah thus overall this particular platform or the the approach is helps you in documenting software dependencies right by building
• 22:00 - 22:30 your images in a structured way ready to be moved at different environment docker quote unquote forces you to document your software dependencies explicitly from this from the base starting point right so as as it need to be ah can be portable to anywhere and everywhere as we start our discussion with that statement so in in a sense this docker allows you to
• 22:30 - 23:00 or not allow is somewhat ah force you ah to document your software dependency explicitly from the ah means starting point that is the step 0 type of things and also it enable continuous delivery that is one important feature for any software thing that you one we you may like to have a enable days of continuous delivery ah is a paradigm what we say cd sometimes referred to as cd is a
• 23:00 - 23:30 paradigm of software delivery based on a pipeline that builds the system on every change and then delivers to the production on life through an automated or partly automated process so that means it is a the cd is a paradigm where approach fire where ah based on a pipeline that rebuilds the system on every change right so i have a i some change
• 23:30 - 24:00 request change the thing rebuild the systems and then leave it delivers to the production for the live system right and it may be automated or partially automated so this is this is a challenge and this is a [Music] feature to be addressed in a software deployment which dockers efficiently ah supports or it enable this ah cd right so docker builds more
• 24:00 - 24:30 producible and replicable than traditional software building methods right so these are or more ah so dropper if we look at the docker builds or docker itself is more producible and replicable right that is the basic philosophy we started with right if you core philosophy is that ah build ship and deploy anywhere so it is it is something which is ah it can be reproducible ah replicable and then our traditional building method so this
• 24:30 - 25:00 makes implementation ah continuous delhi very much much easier right so this is the if we look at the key concepts of the docker ah so if we see there is a layered thing so this is the things which are on the host mating is is stored that ah say [Music] this particular the deviant layer or the line x variant layer then my application
• 25:00 - 25:30 code layer they may be that is one application say v one and there is the application v two layer right so these are the two layers what we are having so this images and image images and collection of file system layers ah and more and some metadata right taken together they can be spun into a docker container right and this layer is a collection of ah changes ah to files right collection of changes to
• 25:30 - 26:00 files the difference of between v one and v two of the my application are stored in this layer right so i can have different type of say like my application version one my application version two and type of things which are in that those layers now this is on the ah your ah stored on the disk right when running the process i basically instantiate this right so that the container is a running instance right of an image you can have multiple containers running
• 26:00 - 26:30 the same image right the same image can be done on the multiple container they may have my application container my application container version 1 to my application container v one three and these are the multiple applications and you have this my application and that there may be again one or more of v two right so the other version so what we see that you you basically create that image ah and then deploy it as as and when it is
• 26:30 - 27:00 required right so if you have a new version of the things ah with some more applicability and changes etcetera you create a v three and which can be deployed into the things right so these are the things as it says that it ah looks for a registry or a name space where things will be maintained ah that how do you know that where things are there those are the things are there and if we look at the key commands so one is docker build build a docker
• 27:00 - 27:30 image ah docker run run a docker image in a as a container docker commit commit a docker container as a image right so and docker tag tag a docker image so these are the basic commands those who are working on the things some of the things will be using showing you ah during the demo that how things works right so [Music] that that will be
• 27:30 - 28:00 that you can see in the in our subsequent lectures ah at least one or two ah we will see show you that finally ah having rather what we are talking about is basically somewhere enclosed into the architectural thing but more formally telling so if you have a docker on your ah host machine is fit split into ah two parts right so this is the traditionally right one is the demand with a restful api and a client that
• 28:00 - 28:30 talks to the demon right so we have a demon with a responsible api and a client which talks to the devil so a private docker registry is a service that stores the docker image right so ah the i was telling that how do you find like you need to be this need to be a catalog or registry this can be requested from any docker demon and has the relevant accesses ah definitely you have to have the the registry is on the internal network and is in publicly
• 28:30 - 29:00 accessible so it is considered to be private right so that what are the different variations and other dock docker images are there that is there in the registry so one we see one invokes the docker client to get the information from or give instruction to the demand right the demand is a server that receives request and returns the response to the client using the http protocol right the so or piggyback on the http protocol in turn ah it will
• 29:00 - 29:30 request to other services to send and service images also using the http protocol so our basic underlining ah what we say that ah transport protocol you should not confuse with the transport layer of the ah of the network stack so the protocol which is piggyback here is the http the the server will accept request from the command line client or anyone else
• 29:30 - 30:00 authorized to connect to the things right the demand is also responsible for taking care of your images and containers behind the scenes whereas the client acts as the intermediary between you and the restful api right or between the your application and the respond api right so this is the overall working how it works and if we look at the same thing in a picture which is there in that reference book so what we have that here your the
• 30:00 - 30:30 host machine on which the docker is installed ah this is the host machine the host machine will typically sit on the private network right so you invoke the docker client program to get the information from an instruction to the docker demand right so the docker demand receive request and returns the responses to the docker client using the thing so the rocker client is there http request to the docker demand and it ah it is a request response type of things
• 30:30 - 31:00 right over http the private docker registry stores the docker images so it consult the docker images and ah the docker hub is public registry done by the docker information as we are talking about docker hub you can on the internet these are the docker hubs which can also be interacted with the things right other public registry can also exist so there can be other type of public registry so what we what is dictating here that one is that how how your docker are built but where this dockers how the
• 31:00 - 31:30 metadata and other connectivity information these are in the registry one is your within your thing that whatever the docker you are managing other can be the by the which is maintained by the docker that is the docker hub or you can have other public docker registry from where you can pull the things right so this is the broad picture though a broad picture but it gives a overall idea that how things work on the in overall mechanisms right so so
• 31:30 - 32:00 today's class what we try to discuss that more little more on the docker so that ah and any any other containerized services are more or less in the same philosophy their way of working may be different but their philosophy is same so what we try to see that what are the key feature or advantages using docker and what is the broad way how things works right so in our ah subsequent lecture or maybe
• 32:00 - 32:30 lectures we will see a some demo very small customized demo but try to see that how docker what we what we have [Music] conceptualized how it works in how it can be made to work in a ah say very in a lab type of scenario though synthetic but in a real life scenario ok so this is the references so the one is the docker
• 32:30 - 33:00 main site other is very nice book i encourage all of you can go through this book docker in practice especially if you want to work on docker this is a good thing to start with right thank you