Lecture 44 Cloud Federation

Summary

In this lecture, the concept of cloud federation is explored, explaining how multiple cloud providers collaborate to enhance service offerings. This collaboration can improve capacity utilization, reduce latency, enable better fault tolerance, and provide more seamless service to users. Various federation architectures, such as hybrid, broker, aggregate, and multi-tier, are discussed, highlighting their different levels of interoperability and coupling. Each architecture offers unique benefits in terms of resource optimization, load balancing, and global service provision, enabling cloud service providers to meet diverse business needs effectively. This overview provides insights into how cloud federation can be a strategic tool for expanding service reach and efficiency in cloud computing.

Highlights

• Cloud federation is about cloud providers collaborating to enhance services. 🤝
• This can improve fault tolerance and service reliability. ⚙️
• Federated clouds can better utilize resources and reduce latency. ⏱️
• There are various architectures like hybrid, broker, and multi-tier for different scenarios. 🌍
• Interoperability and capacity optimization are key benefits of cloud federation. 🔑

Key Takeaways

• Cloud federation enables cloud providers to join forces and enhance service delivery capabilities. 🤝
• It helps in optimizing resource utilization and improving service availability. 🌐
• There are several architectures like hybrid, broker, aggregate, and multi-tier, each serving different needs. 🏗️
• Federation aids in overcoming existing cloud limitations like service interruptions and lack of interoperability. 💡
• Federation can be either loosely or tightly coupled, depending on business requirements. 🔗

Overview

Cloud federation is a game-changer in the realm of cloud computing, allowing cloud service providers to collectively enhance their service delivery. By collaborating, these providers can pool resources to offer improved services, minimized downtime, and greater fault tolerance, making it a highly efficient system.

The federation is characterized by various architectural models including hybrid, broker, aggregate, and multi-tier, each offering distinct advantages. Whether it's about seamless resource allocation or efficient load balancing, these models are designed to cater to diverse business requirements and optimize the overall cloud service landscape.

Interestingly, these federations can be configured as loosely or tightly coupled networks, providing flexibility in terms of interoperability and resource sharing. This ensures that businesses can choose federation setups that best align with their operational needs, tackling traditional cloud limitations by leveraging collective provider strengths to enhance computing capabilities.

Chapters

• 00:00 - 03:00: Introduction to Cloud Federation This chapter provides an introduction to cloud federation, highlighting its significance in the field of cloud computing. The discussion includes an overview of basic approaches and architectures associated with cloud federation, preparing the foundation for understanding its role in cloud computing environments.
• 03:00 - 08:00: What is Cloud Federation The chapter provides an overview of cloud federation, focusing on its characteristics and architecture. It begins by discussing the intuitive meaning of cloud federation as a concept involving multiple cloud services.
• 08:00 - 15:00: Importance and Benefits of Cloud Federation The chapter discusses the concept of cloud federation, where multiple cloud service providers work together to deliver enhanced services. This collaboration aims to improve user experience by reducing latency, improving fault tolerance, and better meeting diverse user requirements. The chapter also explores the reasons behind the need for such federated systems in the cloud computing landscape.
• 15:00 - 23:00: Characteristics of Cloud Federation The chapter discusses the concept of cloud federation, also known as federated cloud. It describes cloud federation as the deployment and management of multiple external and internal cloud services to meet specific business needs. This involves collaboration among various cloud service providers to collectively achieve certain objectives.
• 23:00 - 34:00: Federation Architectures: Cloud Bursting, Brokering, Aggregation, and Multi-tier This chapter discusses various federation architectures for cloud platforms, focusing on cloud bursting, brokering, aggregation, and multi-tier systems. An example from IIT Kharagpur illustrates these concepts through their in-house cloud platform called Mega Mala. This platform is an internal system not accessible from outside their network, showcasing a practical application of such architectures.
• 34:00 - 43:00: Loosely Coupled Federation Architecture The chapter discusses the concept of a loosely coupled federation architecture within cloud computing. It focuses on how external service providers can be utilized when internal capacity is insufficient to handle demands, such as when there are 2000 first-year students. Despite challenges like timing constraints, applications must run seamlessly even when resorting to external services. The chapter emphasizes the importance of adaptability and continuity in running applications efficiently under varying capacity constraints.
• 43:00 - 51:00: Partially Coupled Federation Architecture The chapter titled 'Partially Coupled Federation Architecture' discusses the dynamics of federating cloud services among external providers or other clouds to achieve efficient load balancing and service provisioning. The concept of federation here is defined as a union of smaller parts acting towards a common function. This approach allows for better management and distribution of resources, illustrating a collaborative method of optimizing cloud service delivery.
• 51:00 - 59:00: Tightly Coupled Federation Architecture In this chapter, the focus is on 'Tightly Coupled Federation Architecture.' It discusses how multiple cloud service providers or multiple clouds collaborate to provide services in an integrated and transparent manner to the external world. These clouds come together to offer a unified solution, emphasizing the cooperation and seamless interaction between different cloud platforms.
• 59:00 - 63:00: Comparative Summary of Federation Architectures The chapter titled 'Comparative Summary of Federation Architectures' focuses on the collaboration between cloud service providers. It discusses the benefits of achieving capacity utilization through the federation of services, where one provider with surplus capacity can assist another that is constrained. This collaborative approach helps execute business rules seamlessly on a common service platform.
• 63:00 - 65:00: Conclusion of Cloud Federation The conclusion highlights the readiness to address cloud federation challenges. It acknowledges the need for enhanced interoperability among Cloud Service Providers (CSPs). Each CSP has strengths in different areas, and collaboration is essential. The summary also touches on the importance of cataloging services for effective cloud federation.
• 65:00 - 70:00: Upcoming Topics This chapter discusses service provision in a federated system, focusing on how providers catalog their services. It emphasizes the importance of having a global catalog to showcase available services, and discusses the role of providers and Service Level Agreements (SLAs) as a critical requirement.

Lecture 44 Cloud Federation Transcription

• 00:00 - 00:30 [Music] hello ah so welcome to this ah course on cloud computing today ah we will be discussing on cloud federation right so we will try to see that why federation is important and a overview of what are the basic approaches or architectures of cloud
• 00:30 - 01:00 federation right so we will have a overview of cloud federation the characteristics and federation architectures these are the ah typical keywords so ah when we talk about cloud federation so what we are what does it mean if even ah by ah if we go by just ah intuitively it means that there are there there are one
• 01:00 - 01:30 or more ah cloud providers ah who are trying to cooperate among themselves to give a ah enhanced service right so it may be better [Music] this [Music] serving the better user requirements it may be the less delay or latency in giving the service fault tolerant and different will see that what are the why we require this type of federalist federation
• 01:30 - 02:00 architectures right so when we talk about federated cloud also referred as cloud federation is a deployment and management of multiple external and internal cloud services to match a business need right so so i i have a broad business need and based on that it can be a a means a set of cloud service provider or set of cloud can ah work together and do the things right
• 02:00 - 02:30 like say if we take a example that iit kharagpur as we as i mentioned earlier we have a in-house ah open source cloud platform ah which is which we refer as mega mala which is a internal cloud of ah iit kharagpur which is not accessible from the external world outside iit kharagpur network but say some some some job is running on that say we are conducting a ah exam of say
• 02:30 - 03:00 say all first year students which is around 2000 students right so if it is not sufficient to handle by the cloud we can basically take a external from external service provider and though there are it is not that straight forward so that the application which is running has to run seamlessly there are timing constraint etcetera nevertheless when my capacity is ah full or my capacity in a constraint then
• 03:00 - 03:30 we can say take external or other cloud providers to do that or in some cases some of the providers can federate among themselves so that they can have a overall load balancing of the things better service provisioning and so and so forth right so a federation by by definition and by meaning also is a union of several smaller parts that perform a common accent right so this is the way we look at a
• 03:30 - 04:00 means meaning wise right similarly here also we see one or more cloud service provider or one or more clouds ah try to provide a a a some some service which is ah as such in a transparent to that ah external world and its gives a integrated form of the things right so different cloud ah are come together to give a common ah
• 04:00 - 04:30 common service platform right or common to have a typical business rule to be executed in a seamless manner now ah so ah collaboration between cloud service providers why we do that to achieve ah first of all may be cl capacity utilization right like ah somebody is constrained on the capacity somebody is having surface capacity so that it is ah i may if i am having a surplus capacity
• 04:30 - 05:00 i may say that i am i am ready to lent it right here definitely at some price and also sometimes required to have enhanced interoperability sometimes the federation itself is a challenge sometimes i i know that this portion can be done by better by this cloud c s p one other one is the c s p two and then we can at the back and talk to each other and this can be we can interoperate so also cataloging of services so these are
• 05:00 - 05:30 the type of services provided one this this ah c s p one two three n this ah providers can say that their catalog and say other catalog can have a as a ah what we say a ah global catalog of this ah federation and it shows that these are the services we are providing and inside about providers and sls that is one another requirement when
• 05:30 - 06:00 that how much we allow to during the federation so so we can have some ah some sort of a sharing the whether it is loosely coupled or tightly coupled based on that how much individual cloud expose their means infrastructure or ah they are back back or back end configurations and the slas what they maintain with others
• 06:00 - 06:30 those things make a there may be ah based on the architectures we are following but nevertheless there is some sort of a sharing is required when we federate so motivation as i was mentioning so first of all ah in order to make this federation different csps ah join hand and so to maximize resource utilization that is one of the
• 06:30 - 07:00 primary goal so if somebody is having surplus resources and somebody is resource constrained so that can be utilized or ah we can overall we can have a large resource pool to cater rate the variety of customers minimize power consumption sometimes it may so happen that overall requirement is distributed and we can achieve some efficiency in terms
• 07:00 - 07:30 of power consumption load balancing definitely ah that if if something is more loaded than the other so it can save that loading and global utility so utility wise also it is enhanced and having different service by the service providers and have a more ah say encompassing or more services can be provided and of ah overall ah it increases the
• 07:30 - 08:00 csps ah footprints right so like ah if you go to the other previously i say what we see that c s p one c s p two c s p three is there now if they are federating the csp one may be ah now footprint instead of his own footprint now it is share the footprints of all the three so it is a there may be a chance to expand ah individual footprints right so
• 08:00 - 08:30 which makes sense in case of ah their marketability and profitability of individual csps so characteristics as we are looking at so one is to overcome the current limitation of the cloud um such as ah service interruption ah where it is loaded all there are faults etcetera lack of interoperability degradation of the services those things can be ah
• 08:30 - 09:00 that can be handled if there is a ah defined or ah well established federation between more service provider so in case of overloading or if there are ah some sort of a ah fault or interruptions so that the other members of the federation can take care of those situations many entire cloud organization ah have been proposed like how this inter that
• 09:00 - 09:30 organizing or how this working of the inter cloud will work cloud federation is one such ah inter cloud organizations right so ah so in other sense it is a inter cloud organization involuntary characteristics like who wants to ah join the things may be voluntary it is ah usually not regulated by a things that you need to ah federate with these two other cloud provider type of things it is more of a voluntary individual things it should be maximum geographical separation so it is
• 09:30 - 10:00 a large ah geographical span or catering to a larger region or the means of the service area well defined marketing system and regulated federated agreement right so there should be a well defined ah market system and once we collaborate that should be a
• 10:00 - 10:30 what we say some agreement or ah some basis for collaborations right what would you say some regulations of ah or agreement over this federation right ah another important means things what just means that maximum geographical separations some what why one one of the thing what we try to give in a cloud environment is what we say um is
• 10:30 - 11:00 high degree of reliability right so that it is ah even so ideally so ideally this federation partner should be in different ah maybe power zone different network zone different seismic zone and type of things right so spread such a way that it is not like that a some some say environmental or some disaster may
• 11:00 - 11:30 affect all the federating partners right like so if so others will be ah working as usual ah so that that federation can ah effectively can ah support the their client base right and ah it is an environment where multiple ah service providers come together and share their resources right um so its a what we say a ecosystem so
• 11:30 - 12:00 called ah or ah a environment where the multiple survive ah service providers can share their resources to maximize ah what we say utilization of the resources ah satisfaction of the ah user community and so on so forth so when we talk about ah architecture or federation architecture so associated with several portability and
• 12:00 - 12:30 interoperability issues right so it is not pretty straight forward once once say two csp three csp or in number csp talk to each other or ah agreed upon federating and giving a single ah interface or sorry single ah say what we say view of that
• 12:30 - 13:00 globe of their federated structure for some ah purpose definitely for some goal and type of things so what is important at the back end though the issues like portability between service provider interoperability issues with respect to services data and different different challenges on the interoperability issues will need to be addressed right so if we look at typical
• 13:00 - 13:30 federation architecture so they those are one is cloud busting brokering aggregation and multi tire right so this is these are the typical ah federating federation architecture um [Music] which are um being practiced or which are ah [Music] are prominent or important architectures so these architectures can be classified according to the level of coupling
• 13:30 - 14:00 or inter operation among the cloud instances involved right ah ranging from loosely coupled that means no or little inter interoperability among the cloud instances ah i should not say ah no interoperability there is there has to be interoperability otherwise you cannot ah make the federation ah working but what is there that what it requires ah what what this
• 14:00 - 14:30 loosely coupled what is basically do not have much information about the other providers say infrastructure and things like that right so it is less ah what we say less transparent to each other right so it is not all all is all these features sets etcetera are exposed to the other providers right so that is loosely
• 14:30 - 15:00 coupled or we can have a tightly coupled where where the interoperability is very ah tightly managed among the cloud instances even they say that the type of ah architecture and other systems things what they are maintaining at their end so it is very tightly coupled so there are both need and advantage disadvantage whether we go for loosely coupled tightly coupled things based on it is
• 15:00 - 15:30 more based on the applications what you are looking for right some of the things may be loosely coupled some may be tightly coupled and if it is my federation at different levels say at the as a ah higher level or sas level then something is there or if it is in the infrastructure level then something else is there so there are different challenges and issues which need to be addressed now ah to
• 15:30 - 16:00 look at if we have loosely coupled federation architecture so limited interoperation between csps and cloud instances so there is not much interoperation between the csps and other cloud instances right ah in case of a loosely coupled like for example a private cloud ah clumpy complementing its infrastructure with resources for an external commercial cloud for some requirement
• 16:00 - 16:30 like we are telling that we are running exam and we found that the basic cloud infrastructure what we are having it not able to serve ah in a in a faithful manner ah so what we try to do that we purchase more ah cloud resources or subscribe to cloud resources which can be other public cloud or other private cloud but the challenge here is ah how ah how we show a integrated
• 16:30 - 17:00 fashion so as we do this type of federation these are all loosely coupled right so they are not exposing everything to you and you are also not exposing your feature set to x the external world but on a on the other hand you need to run the what we say quote unquote business process ah in this case may be running that exam in a particular ah stipulated time ah duration and that that makes ah
• 17:00 - 17:30 things pretty challenging so a csp has little or no control over the remote resources right so if it is a loosely coupled say iit kharagpur megawalk cloud with some other cloud so csp do not have ah much control ah to the resources right remote resources like this and about the vm placements ah are not known monitoring information is
• 17:30 - 18:00 limited for example only cpu memory disk consumption in each vms so these are limited right ah and usually no support for advanced feature such as cross side networks or vm migration and these are the things which ah modern day cloud supports those things are if you have a federation things that is difficult to support all those things right ah first of all ah they may be having their underlining ah
• 18:00 - 18:30 different sort of configuration at their end and type of things right so this this vm migration what we will see in subsequent lectures will be maybe a challenge out here the other is the partially coupled federation ah different csps or the what we say partnering clouds ah establish a contract or framework
• 18:30 - 19:00 agreement stating the ah terms and condition under which one partner cloud can use the resource order so it is partially coupled that means not so loose as loosely coupled not so ah strong as a tightly coupled but initial in between them partially coupled coupled where ah they establish a contract or a framework agreement stating that the the terms and conditions and under which one partner will share resources with the other and so and so forth right so this is also
• 19:00 - 19:30 [Music] good and this is ah also helping achieving better efficiency then may be pure loosely coupled thing so this contract can enable a certain level of control over the remote resources right so this contract as we have a contract previously loosely coupled we do not have any ah so to say any ah formal contract actually there should be some agreement definitely otherwise how
• 19:30 - 20:00 you how you federate but that is not that strong but here it allows to ah some control over the remote systems like ah allowing definition definition or say affinity rules to force ah two or more remote vms to place in the same this etcetera like you want to have that this for this particular working this three bm should be on the same ah physical cluster so those type of things are ah
• 20:00 - 20:30 may be allowed right it depends on ah installation to installation but this is things like that so may agree to interchange ah means more details ah monitoring information for example providing information of the host where the vm located energy consumption etcetera right so it may help in ah exchanging or interchange or more monitoring information right and may enable some advanced
• 20:30 - 21:00 networking feature among the partner cloud also it may enable some advanced networking features ah partner like creation of virtual network across boundaries and type of things right so what we see here that ah more flexibilities or more ah hand holding between the two cloud providers are given with respect to our loosely coupled thing
• 21:00 - 21:30 now we other thing what we are having tightly coupled right ah other totally other side tightly coupled in this case the cloud are normally governed by the same cloud administrator right so once you have a toughly coupled in other sense you require more information about the other party in other if we see the other sense it is basically the type of ah stringent requirement if it is there for strong for strong coupling then we have some
• 21:30 - 22:00 sort of a some single administrative authority at times right or if it is also different administration it reports to a single authority so that anything you do here the other party or other parties know immediately or a priority so that they can be ready with their configurations and things like that right so a cloud instance can have advanced control over remote ah remote resources
• 22:00 - 22:30 like for example allowing decision about exact placement of area of remote vm and type of things right so that type of reason that which cluster where this vm will be placed and so and so forth will be more stronger and can access all monitoring information available about the remote so ah in loose decouple we do not have any practically anything in ah partially coupled we have some control but here ah availability of the
• 22:30 - 23:00 information about remote services is much high may allow other advanced feature including creation of short cross site network ah cross side migration of vms right implementation of high availability techniques among the remote cloud instances creation of virtual storage systems across ah side boundaries right so when we have a tightly coupled we have other advance
• 23:00 - 23:30 features of cloud can be looked into like creation of cross side network cross side migrations of vms if there is a loading auto that i can migrate to the vm without ah much negotiation right it is already ah agreed upon right implementation of high availability techniques when we require ah among ah remote cloud instances right and creation of virtual storage so
• 23:30 - 24:00 as storage is one of the important factor of or important consideration in any ah implementation so this virtual storage systems across side boundaries all those things are maybe they are in the tightly coupled federations now ah if we ah see the a broad picture ah of the thing so we see there are four ah category one is
• 24:00 - 24:30 hybrid or architecture another is broker architecture right ah another is aggregated and that is multi-tier architecture right there are some commonalities between this ah architecture ah or between any two such architecture but nevertheless their overall ah [Music] focus is ah different or ah based on different type of business need the what type of
• 24:30 - 25:00 architecture you will be looking at may be important so we will i will quickly try to see that what they try to mean like so busting or ah hybrid architecture ah cloud busting or hybrid architecture combines the ah existing on premise infrastructure usually a private cloud with the remote resources from one or more public cloud to provide ex extra
• 25:00 - 25:30 capacity to satisfy peak demand ah periods right so what what we see that ah especially when the requirement is pretty high or what we say when it is ah reaching to the peak sometimes what happened we can ah predict some of the peaks some of the peaks we do not ah predict but if we as we have seen in earlier lecture if we design our my cloud based on my peak requirement then many of the time it may be underutilized rules right in other sense
• 25:30 - 26:00 if i have a federation in place and if required i can have a hybrid or bust out to the other things and get my work done definitely there will be some requirement of sla quality of service and of course there may be some ah financial ah constraint also the charging when you use the other things but if it is done seamlessly if you you can see that from the customer point of view its a its a something running and going on run even if there is a constant in the provider
• 26:00 - 26:30 right so as the as the local cloud os has no advance control over the virtual resources deployed in the external cloud beyond ah the basic operation and providers along right so has no advance control over the virtual resources ah so of a local or one size service provider do not have much advance control over the virtual resources the architecture is typically loosely
• 26:30 - 27:00 coupled so more most existing cloud managers support this hybrid architecture right so it is loosely coupled and it is ah well ah taken by this different c space the other one is a brokering which is more sounds to be more intuitively sounds to be ok or it is more natural so a broker that serves
• 27:00 - 27:30 various users has access to several public cloud infrastructure a simple broker should be able to deploy ah virtual resources in the cloud as selected by the user so in this case we have a broker or which acts as a managing the resources of one or more cloud service provider so broker is the common most common federation scenario so that makes more ah
• 27:30 - 28:00 easier to implement easier to maintain and easier to keep the what we say quote unquote ah privacy of this individual ah cloud providers right ah so that is that is ah there and ah say as we are looking at that advanced broker offering service management capabilities could ah make scheduling decision based on the optimization
• 28:00 - 28:30 criteria like cost performance energy consumption to automatically to automatically deploy virtual user service in the most suitable cloud so what is telling that the cloud as the broker is there so request coming there it can take a more global decision based on the what type of resources are there what is the requirement and type of things like that right it may even distribute the service component across multiple clouds this architecture is also loosely coupled not
• 28:30 - 29:00 that [Music] strongly coupled at times though people refer it is something say means not tight or loose it is in made in between that is partially coupled type of things since the public cloud typically do not allow advance control over the deployed virtual resources right so the public cloud may not allow so it is it is it is difficult to go there but definitely broker acts as a
• 29:00 - 29:30 thing and if we have multiple cloud infrastructure the broker will help in some sort of a load balancing finding out appropriate service requirement and responses to those things right so this is typically the broker architecture so we have a federation broker there the these are the external user it can be human user or system it does not matter and there are different csps and this broker interact
• 29:30 - 30:00 with them to give the overall increase the efficiency the other one is aggregated architecture involves two or more partner clouds that interoperate to the ah interoperate ah to the aggregates or rather interpret to aggregate the resources and provide users with a large virtual thing so what we do as the name sizes suggest so it is it is aggregation of the resources right and gives a a
• 30:00 - 30:30 aggregated view to the external world right so this architecture is usually partially coupled since partners could be provide with some kind of advance ah over the remote ah resources right so it is partially coupled unless you have some information about the things a priori or ah even on the fly aggregation all those things are not ah feasible usually the aggregation ah decent etcetera based on the job type of job etcetera are somewhat predecided or
• 30:30 - 31:00 done not on the flight it is possible that to do on the fly but there there are different constraints the partner clouds usually have a higher coupling level when the when they belong to the same cooperation and when they owned by different companies agree to cooperate and aggregate their resources right so this if you see this partner cloud ah
• 31:00 - 31:30 to have a ah so there are constant like they want to collaborate or it may be owned by different say it can be different csp all together but agree to cooperate and aggregate their resources so that means they need to share some information about the resources ah which is permissible based
• 31:30 - 32:00 on their own business rule or corporate rule and then it can aggregate the resources the other thing is the multitasker finally we have that multitask involves two or more cloud sites each running their own ah cloud os and usually belonging to the same ah corporation ah that are managed by a third ah cloud os instances ah following a hierarchical arrangement so
• 32:00 - 32:30 in the in case of a multitasker instead of having all flat things we have a hierarchical structure right so some sort of a root or top cloud os instance has a full control over other resources in different cloud size so it is a tightly coupled scenario and it is it exposes the resources available of the different cloud sites or the cloud providers domain and if they are located in a as if they are located in a single cloud so
• 32:30 - 33:00 in in hierarchical it is the control is in a higher call fashion and but they are ah more tightly coupled unless that that is otherwise it is difficult to maintain those ah thing ah for a for for a for executing a common goal so this architecture is beneficial for corporation definitely with geographically different cloud infrastructure because it provides uniform accesses right like if i say that i have a organizations which have
• 33:00 - 33:30 say in all major cities of india and some of the ah some second level cities etcetera so making them hierarchical it make me sense that ah to manage the whole thing and there may be different requirement at different regions so it can be in in like a hierarchy it can be maintained so it may be useful for implementing advanced management features such as high availability load balancing fault
• 33:30 - 34:00 tolerance right so once you have a this type of high multi tire high coil architecture so i can have we can have different advanced features which can be worked on right so in this today's discussion what we try to look at is this cloud federation that when one or more cloud service providers ah come together and say courton could join hands
• 34:00 - 34:30 say enhance to give a to deliver a common business process then we require federation we have seen that there are different sort of federation architectures which are being practiced by the things like hybrid or busting broker aggregated and multi-tier architectures which helps us in working with these things right so with this ah let us
• 34:30 - 35:00 end our discussion today we will continue with ah will in the subsequent lecture ah lecture or lectures one or two lectures we will see ah we will see another ah important aspects of ah this cloud computing which is v m migration right so that type of things will see so there are few ah references and which you may go through so let us end our discussion today thank you