Why DBMS?/1

Summary

Highlights

• Welcome to module 2 of DBMS course from IIT Madras, exploring the evolution of databases. 🎓🎬
• Discover how databases are integral to modern life, even in basic cell phone operations. 📱✨
• Explore the historical need for DBMS, from 300 years ago to the present digital era. 📜➡️💻
• Learn how societal and technological changes drive database evolution, with examples like Arogya Setu. 🌐📲
• Uncover the essential components of data management: storage, retrieval, transactions, audit, and more. 🔍💾
• Understand the transition from physical book-keeping to electronic data management. 📚➡️💻
• Highlight the evolution of DBMS from punch cards to relational databases and today's complex systems. 🕰️➡️💽
• Learn about key historical figures and technologies that have shaped database management. 🏛️👨‍🔬
• See how modern challenges like unstructured data and data science rely on DBMS. 📊🧩

Key Takeaways

• DBMS has evolved from physical records to sophisticated electronic systems! 📜➡️💻
• Data management is crucial for modern applications like health tracking apps. 🏥📱
• Databases ensure secure and scalable data handling. 🔒📊
• Our civilization's history is closely tied to advancements in data storage and retrieval. 📚📈
• Understanding DBMS is vital for today's data scientists. 🧑‍🔬💡

Overview

Chapters

• 00:00 - 01:00: Introduction and Overview The chapter serves as an introduction and overview to the second module of the Database Management Systems course offered by IIT Madras as part of its online BSc program. The chapter opens with welcoming music, setting a positive tone for learners. There is a brief mention of the coverage from Module 1, establishing continuity and building upon previous knowledge. The focus is on preparing learners for the concepts and details that will be explored in the subsequent sections of Module 2.
• 01:00 - 03:30: The Role of Databases in Modern Life This chapter explores the extensive integration and significance of databases in contemporary life. It highlights how databases influence every application and facet of daily activities, even in situations where their presence might not be overtly recognized. The discussion underscores the pervasive nature of databases, extending to basic technologies such as cell phones, thereby illustrating their fundamental role in shaping modern existence.
• 03:30 - 07:30: Historical Perspective on Data Management The chapter explores the indispensable role of databases in modern communication. It emphasizes how large database applications are crucial for various functions such as tracking call numbers, locations, connections, and the overall quality of service in telecommunications. The narrative highlights how databases support the infrastructure of towers and base stations, underlining their vital presence in everyday operations and technology.
• 07:30 - 12:30: Key Requirements of Data Management The chapter titled 'Key Requirements of Data Management' delves into the foundational aspects of Database Management Systems (DBMS). It is divided into parts one and two, focusing on providing a deeper understanding of the perspectives relevant to DBMS. The primary aim of this chapter is to explore the necessity of a DBMS, approaching it from a historical standpoint and discussing how it has evolved over time to meet various data management needs.
• 12:30 - 21:00: Challenges with Physical Data Management The chapter explores the historical evolution and necessity of database management systems. It questions the methods used for managing data at various points in time, including three hundred years ago, a hundred years ago, thirty years ago, and as recently as ten years ago. It emphasizes the continuous evolution of database management systems, highlighting the balance needed between different aspects.
• 21:00 - 34:30: Advancements in Electronic Data Management The chapter titled 'Advancements in Electronic Data Management' explores the evolving landscape of technology and its role in transforming society. It highlights the pivotal role technology plays in making more tools and resources available. This is particularly emphasized in the context of societal changes prompted by recent global challenges such as pandemics. The chapter delves into how these changes influence national and societal progress, encapsulating technology's impact on data management in this era of rapid transformation.
• 34:30 - 37:00: Conclusion The chapter discusses the use of the application 'Aragya Setu,' which is designed to provide information about one's health in relation to coronavirus. It highlights how the app can help users understand their own health status and that of people nearby, even those they do not personally know.

Why DBMS?/1 Transcription

• 00:00 - 00:30 [Music] welcome back to module 2 of database management systems course in the iit madras online bsc program in module 1 we have
• 00:30 - 01:00 taken a very basic look at how databases have pervaded all aspects of our life touch any part touch any application or even when you are not aware i mean just to tell you the extreme in a way even for people who just use a basic cell phone
• 01:00 - 01:30 to make calls you are using an extensively large database application which not only keeps track of the numbers the location tracks the connection track and so on of the caller and the quality and the overall infrastructure of towers and base stations and all of that so we cannot live without databases today in the
• 01:30 - 02:00 current module we again in the current module and the next ah which we have titled as part one and two of y dbms we take a little deeper look into perspectives in this module what we want to do is to understand the need for a database management system for somewhat of a historical perspective
• 02:00 - 02:30 asking ourselves fundamentally what is that database management systems are needed for what did we do say three hundred years back four hundred years back what did we do hundred years back what did we do thirty years back what did we do ten years back and why and how the database management systems are evolving keeping two sides in balance one is
• 02:30 - 03:00 one side is the availability of technology we are talking about a time where technology is the key thing which keeps on moving makes more things available and hand in hand with that on the other side are the way the society is transforming we have been part of a big pandemic and you know we all know in one of the ways or in some of the ways as a nation we
• 03:00 - 03:30 have tried to respond to it was to use say an application called aragya setu arugosetu is an application to find out about your health condition related to coronavirus and the health conditions of people potentially near to you ok so you can think of that these are people you do not even know
• 03:30 - 04:00 but it is possible to track and correlate this information so what kind of huge database go behind that which is keeping track of the conditions of several people who use the argus app and their locations and their movements and all that the other that all of you are aware of in this context is the
• 04:00 - 04:30 kovin app which has been tracking the entire nation's vaccination program it is tracking who is getting vaccinated where is she or he getting vaccinated what vaccine has been administered what is the first dose or the second dose a date when is the next vaccine to be given the other card or passport of that person and finally
• 04:30 - 05:00 the certification right another huge life changing database application now even if you look back 10 years ago this were not possible we did not have other card we did not have this kind of network we did not have this kind of scalability of database systems so in this module we try to take a little historical look on how have we evolved as a civilization
• 05:00 - 05:30 in terms of data and records management and in the next we will talk about specifically about why you need separate kind of systems for solving data related problems why doesn't simple computer programs can solve this problem so we will talk about evolution of data management practices and little bit of history of the dbmss so first about the evolution of data
• 05:30 - 06:00 management so if we if we ask in principle then what does the management of data or which earlier was used to be called management of records what all are needed in the human society what is the basic need the first of course is storage you need to remember things how much money i have how many
• 06:00 - 06:30 inventory i control how much of how many people an organization employs and so on so storage is important along with the storage comes retrieval that i should be whatever data is stored i should be able to get it back so storage and retrieval are the key requirements of data and records management i am not specifically talking about database systems as you know today but i am saying the need of management
• 06:30 - 07:00 of data and management of records of different things then you have transactions transactions is where something is being exchanged something is being done on that data either certain part moving from one place to the other or one so called account to another like i go to a ah showroom and buy a two wheeler so a transaction has happened
• 07:00 - 07:30 a two wheeler whose ownership was with the dealer or was with the company through the dealer is now transferred to my name right so this is a transaction and accordingly corresponding to that there are financial transactions the money that i had to pay to the company through the leader right so transactions are a very important part the fourth is audit you always need to know what happened did things happen properly was there any
• 07:30 - 08:00 mistake are all the rules and regulations of that particular segment properly followed was tax paid on the sale of the two healer was why are all these financial scams we are looking at because it is possible to dig out from the stored data with appropriate retrieval and audit to
• 08:00 - 08:30 see what who did certain things may not be right you have archival purposes that you need to just keep that data for maybe some possible future use right there could be more but i just talked about some of the salient requirements so we will have to the reason i am putting i know you all of you know this all of you know this i am not saying anything new but i just want to highlight these points because you have to always keep this in perspective when you learn this course
• 08:30 - 09:00 this is your these are your challenge points you are trying to make a system you are trying to improve a system you are trying to make changes to a system keeping these kind of requirements in mind right now the obviously related question is whom do you do it for you could do it for an individual for me for you for your friend for your parents or it could be for a small enterprise small
• 09:00 - 09:30 organization or for a big enterprise like google like reliance industries like tata motors and so on or it could be for a global organization it could be for united nations it could be for any other countries government or in more global organizations like apple so what you have to do what we need and for whom we need are
• 09:30 - 10:00 very clear the question is what should be the approach and if you look at the human history there has been primarily two major approaches in this practice which are very natural one is physical so the world started physical so from the day people started realizing the need for maintaining records to satisfy storage retrieval transaction or did archival these kind of requirements
• 10:00 - 10:30 people started maintaining records physically and then came the electronic era so most of the time the human civilization has been around we have seen physical forms of data and records management which is ah in in in certain purlins very formally known as bookkeeping if you if any of you have familiarity with accounting courses and so on you will know that there are courses like
• 10:30 - 11:00 accountancy and bookkeeping which basically talks about how you keep the physical ledgers how you i mean what in what style you put the data in it what style you write journals that is notes and so on now if you go back couple of ah centuries you will find that ah well this practice is coming from time immemorial the practice of having leisure such journals but things started streamlining somewhat
• 11:00 - 11:30 towards the end of 19th century when an american inventor by the name of henry brown first invented ah something which is called respectable receptable for storing and preserving papers which in today's time we know as file cabinet right how to organize and you have had a severe evolution based on that there is a significant correspondence
• 11:30 - 12:00 between this physical organization of the data to the electronic mimicry of that in terms of the database system so you know that transfer knowledge is very important to realize in even in terms of memory in terms of ah you know writing down everything is laborious as well as error prone so hollerith hermann hollerath who kind of started using adapting punch
• 12:00 - 12:30 cards that were used in the jacquard loom ah for weaving as memory so that you could do very simple mechanical computation based on these lasers based on this laser data that also happened somewhere around towards the end of 19th century and then so on i mean before that several centuries people followed the same practice following the getting into the 20th century also the
• 12:30 - 13:00 same practices continued till you came to almost the middle of the 20th century when computer programming started computer started happening programming started and in 60s mostly you started seeing the database management system or electronic management of data where you need need storage retrieval transaction audit archival this kind of so for storage which is the primary thing because you have to keep the records
• 13:00 - 13:30 where punch cards were used long punch tapes i do not know if any one of you have seen are were used then it evolved to magnetic tapes and so on went on for about 10 15 years till in 70s certain significant improvements and development happens in terms of cobol language being introduced it is still used 50 years on it is still used
• 13:30 - 14:00 coda seal approach thats one approach of doing data management came in apple first introduced vc cal vc calc is is kind of the forefather or great grand forefather of our spreadsheets today excel or google sheet as we see magnetic disks became prevalent and so on then in 80s we had relational database systems which changed the face of data management in 90s
• 14:00 - 14:30 internet came in so that started making the whole data and records management global in 2000s ah e-commerce really started booming all those most of the significant e-commerce activities you see today they started around two thousands and this some died some flourished and as that kept on happening there was more and more need and more and more
• 14:30 - 15:00 requirement to manage unstructured data data which is not just textual data which is just not numbers data about images the facebook profile photos data about videos data about audio tracks and so on so forth data about just freely written natural language text and so on and no sql was born in the last decade data science started riding high
• 15:00 - 15:30 and today you are there you will write the next future of data management using electronic systems so this this is a very broad evolution if you if you look at it now in this whole process there are several parameters on which this data and records management system particularly in the electronic form has to take care
• 15:30 - 16:00 things like durability i mean i store a data it must be available for ages together that was a basic you know drawback of the physical system papers wear out the termite will eat it up or even the basic acid in the paper will cause a decay of the paper on which you kept the records but electronic records must be more durable are they the big question and that will have to be ensured how do
• 16:00 - 16:30 you ensure that they remain durable as well it must be scalable it should work for say 10 20 100 people as well as ten thousand hundred thousand millions billions of people or transactions and so on it has to be absolutely secured of course depending on the application all of these will have a certain operating range you know the security of say the different information systems
• 16:30 - 17:00 that are associated with our defense and the security of a digital library are not equivalent like a digital library needs some security so that copyrighted stuff are not stolen but that security will be far less compared to the security of where the ward management is done so though but those are those are aspects
• 17:00 - 17:30 these aspects will have to always keep in mind keep questioning about them what is the retrieval you know how quickly you need to retrieve is it okay to get your data within a day within a couple of seconds under millisecond under nanosecond and so on so forth it should be easy to use of course otherwise it has not served the basic purpose consistency we i we talked little bit about consistency in the last class that always if i do something on that
• 17:30 - 18:00 database if i do some transaction then before the transaction and after the transaction the database must be in a consistent state it should not be that if a a transaction has been done to make a payment of 100 rupees to my friend and in that process hundred rupees get disappears from the system that is not certainly acceptable efficiency has to be high that efficiency as a whole not only of the its not only the time efficiency it must
• 18:00 - 18:30 be that that is what the speed of retrieval is but efficiency in terms of you know amount of storage requirement maintenance requirement efficiency and productivity of the staff those are involved the programmers those are involved because all of these lead to the cost considerations right so these are some of the not this is not a complete list the some of the key parameters on which data and record managements need to be looked at
• 18:30 - 19:00 so if we just you know more a fun discussion if you just look back into bookkeeping you know physical records management widely used everywhere till today you go to a grocery shop many a grocery store have a have a small pc some have migrated i mean they are more savvy they are migrated to doing ah business on mobile phone but there are several who will just write it down this is this is sold you go to particularly you go to a medicine shop they give you the medicine and then they
• 19:00 - 19:30 see that this particular they are low on that stock they will take out a leisure write down on the journal this medicine you know code opinion is not there in stock and so on so if we if we recall on that then we we all know we have seen that what are the problems there are several problems one durability is a problem physical damage i mean it can you can very easily get damaged you can get as you say with rodents termite humidity
• 19:30 - 20:00 wear and tear fire there are lot of ways physical records might get destroyed scalability is very difficult to maintain right you can manage a a medicine store for maybe 100 customers 200 customers on leisure you cannot maintain a medicine store of a million customers on paper its not possible security always because you always have to have
• 20:00 - 20:30 trusted people you need physical lock and key and so on so forth you know big ah safes and all that for maintaining the physical records retrieval is obviously time consuming you'll have to look up take out that big stuff from the file cabinet look up and so on consistency is certainly you can it is quite possible that you have made a payment ah that has been received but has not been entered on your part of the leisure right so there are several problems of this
• 20:30 - 21:00 physical book keeping systems somewhat of a better solution which many of us is widely used and kind of it came in about 40 years back i just mentioned vc calc so that was about 40 years back so it is a spreadsheet software which allow you to you know keep records in a simple terms you have you know paid applications like
• 21:00 - 21:30 excel or you can use google sheet which is free and it these also have different issues these are but these are less prone to durability issues of damage because you can keep a google sheet in a good cloud which has lot of backend infrastructure to make sure that it does not get lost then there is a scalability is easier it is easier to
• 21:30 - 22:00 search ah you can password password protect it is not a very good protection there are several ah you know systems available by which it can be broken but well some sort of protection can be there it is easy to use ah consistency of course is not much guaranteed in spreadsheets you will have to either write lots of formula or kind of make sure that you are using it in the right way
• 22:00 - 22:30 but it is it is a better than a a physical laser and journal based system right which according to this criteria you can assess it based on more criteria you know you can do it yourself so ah this is still used i mean this is widely used not still this is widely used for particularly single users or for small enterprise applications a good example of how a moderate range
• 22:30 - 23:00 data management solution can provide but certainly these are all spreadsheets or cvs files csv files and so on are all basically file system based we say these are flat file systems flat file solutions so they have lot of limitations as well if you their scalability is
• 23:00 - 23:30 limited like spreadsheets have a maximum limit on the number of rows consistency management we said is relatively a big challenge if we have certain constraints on the data its not easy amman you can you can try in excel then you will see that excel does allow you to put some constraints but they are just data entry constraints that is you cannot enter a negative value in this cell but if you have constraints resulting
• 23:30 - 24:00 from transactions it is very difficult to manage that in a in a file based system because you will have to write separate code for doing each one of that it the permission levels are limited you can give a password but you cannot have multiple people use the same file in multiple levels of authority right so these are
• 24:00 - 24:30 these are some of the typical limitations in the evolution which finally makes ah sure that the only solution is the database management systems which we are going to study here talking little about the history i will be very brief here ah because it is a long and deep history as i said it started between 1950s and 60s tapes and punch cuts then in late 90s 60s and 70s hard disks magnetic disks
• 24:30 - 25:00 ah came in and you know these are some of the people whose work you are going to read about like ah ted codd who define the relational data model you will we will learn about voice code normal form and so on and slowly the ah you know evolution towards more and more sophisticated data management system started happening 80 eighties really saw a boom
• 25:00 - 25:30 through which ah sql which we will learn about ah start learning about very soon in the coming modules became an industry standard and it is also saw the parallelization and distribution of database systems that before that database systems were centralized that you have one machine one server or one pc on which you are using it but in the 80s slowly across the enterprise you started using it you need to use it across the enterprise
• 25:30 - 26:00 you need to use it across the a local area network and so on and in the 90s as internet came in this use or this distribution certainly went all across the web in early 2000s and so on the need for unstructured data management came into being so we had xml and x query standards coming in and we will talk little bit about xml and those
• 26:00 - 26:30 kind of stuff slowly and from later 2000s really giant data storage systems google big table yahoo peanuts amazon these kind of things have started coming converging into the data science possibilities on the you know the foundation of data science is not database it is the foundational part in this is database system which give you data warehousing data mining all of that
• 26:30 - 27:00 and on top of that you have data science of which you are trying to gain an expertise so think about being a data scientist among many things what you have to be really good at is database management systems so there are different evolution models that i have shown here this is not this is more for you know ah keeping a picture about how things have moved from 60s to the 2010 this this
• 27:00 - 27:30 you know horizontal axis show you what has primarily been happening during those decades then how the technology has moved from based on the evolution of the computer technology and i will talk about this third part more subsequently is how the database architecture has ah move on and if you if you look at that way then basically this is
• 27:30 - 28:00 this is primarily the area of architecture on which we will be focusing talking about oracle system sql servers mysql postgas sql which is not named here psi base and so on and these are what you all see around this is your past and this is kind of what has been happening beyond that i am sorry
• 28:00 - 28:30 ah let me let me let me let me choose this so this is what we primarily intend to do this is the past stuff and this is all different types of what you see here are all different types of systems that are taking things forward particularly dealing with the big data and so on ok
• 28:30 - 29:00 so thats about the basic overview of how databases been playing a big role in our life the evolution of the data and records management and the history of the dbms thank you very much for your attention and see you in the next module