Master Data Management (MDM) - Chalk and Talk

Summary

Dave Does Demos' video delves into Master Data Management (MDM), discussing both its operational and analytical aspects. Dave explains the importance of having a unified data system for organizations and the role of Enterprise Service Bus (ESB) in ensuring data integrity across platforms. He differentiates between the needs of operational systems, which require real-time data accuracy, and analytical systems, which focus on historical data analysis. This insightful talk also covers best practices and legal compliance in data management.

Highlights

• Dave emphasizes the importance of distinguishing between operational and analytical MDM 🔍.
• Illustrates how an Enterprise Service Bus (ESB) functions as a central hub for data updates, ensuring consistency across platforms 🌐.
• Discusses how operational systems rely on real-time data accuracy for effective business processes 📅.
• Explains the necessity of maintaining historical data integrity in analytical systems for better decision-making 🎯.
• Highlights how proper MDM practices also ensure compliance with legal standards like GDPR 🔏.

Key Takeaways

• Master Data Management (MDM) has two sides: operational and analytical, each with distinct roles in a business 🎭.
• Using an Enterprise Service Bus (ESB) helps maintain consistent data across different systems efficiently 🔄.
• Operational systems require real-time data updates for accurate transactions and compliance 📈.
• Analytical systems benefit from historical data records to provide insightful analyses over time 📊.
• Understanding these differences in MDM can enhance internal communication and simplify complex processes 🗣️.

Overview

In the video 'Master Data Management (MDM) - Chalk and Talk,' Dave Does Demos explores the nuanced world of MDM. He breaks down its two main components: operational and analytical, and how each serves a unique purpose in data management. The operational side deals with everyday business transactions needing precise data, while the analytical side is more concerned with storing historical data for insights.

Dave introduces the Enterprise Service Bus (ESB) as an essential tool for synchronizing data across different systems. By subscribing to updates, different platforms can share a unified view of customer and transaction data, which is paramount for seamless operations and compliance with laws like GDPR. This ensures that customer data is consistently updated across all systems.

The video further explains how MDM systems, though often using the same terminology, serve different strategic goals depending on their focus. Operational MDM ensures data integrity for live systems, whereas analytical MDM is pivotal for deriving insights from past data. Dave's clear explanation helps viewers understand how to effectively employ these systems to improve business processes.

Chapters

• 00:00 - 00:30: Introduction to MDM The chapter introduces the concept of Master Data Management (MDM), highlighting its importance in both operational and analytical contexts. The speaker, Dave, mentions that while people often use the same terminology, they may be referring to different aspects of MDM. The purpose of this introduction is to ensure everyone has a common understanding of MDM's scope and nuances as it pertains to various applications.
• 00:30 - 13:20: Operational MDM Explained The chapter "Operational MDM Explained" delves into the benefits and reasons behind implementing Master Data Management (MDM) within an organization. It begins with a scenario-based approach to understand the motivations and advantages for the organization. The narrator promises a visual representation through a whiteboard session to elucidate the processes occurring in different areas. The chapter promises a detailed exploration of the distinction between analytics and operational aspects of MDM.
• 50:30 - 75:30: Benefits and Challenges of ESB in MDM The chapter titled "Benefits and Challenges of ESB in MDM" discusses the two sides of the business world: operational systems and the need for current, accurate information. It emphasizes the role of Enterprise Service Bus (ESB) in Master Data Management (MDM) for supporting live transactional systems and line of business operations.
• 75:30 - 80:30: GDPR and Legal Requirements for MDM The chapter discusses the importance of having a single system own information, such as customer data, to prevent discrepancies across different systems. This alignment ensures a cohesive business front and simplifies reporting processes, thereby enhancing business operations and compliance with GDPR and legal requirements.
• 80:30 - 133:20: Analytics MDM Explained The chapter titled 'Analytics MDM Explained' discusses the integration of multiple systems like CRM and web sales systems to manage data. It highlights the process where a salesperson enters customer information into the CRM, providing an example of how data is captured across various platforms including mobile apps.
• 255:00 - 310:00: Tools and Techniques for Analytics MDM The chapter discusses a system where customer data is managed across multiple platforms, such as a CRM and a web system. It describes a scenario where a customer, identified as 'Dave' and assigned a customer ID of 001, would not be recognized by the web system due to lack of prior interaction. The solution proposed involves the web system querying the CRM as a data source to retrieve customer information, albeit a loosely coupled integration.
• 310:00 - 320:00: Conclusion and Channel Subscription The chapter discusses the importance of maintaining a linked and consolidated customer account system. It warns against creating new records that are not linked to existing systems, which can result in having multiple customer accounts that do not provide a unified view of a customer's purchases. This disjointed approach can lead to inefficiencies, such as the inability to have consolidated billing. The focus is on ensuring all purchases and customer information are connected across different platforms to provide a seamless and comprehensive view of customer activity.

Master Data Management (MDM) - Chalk and Talk Transcription

• 00:00 - 00:30 hi there and welcome back to dave dust demos today i'm going to be talking to you about uh master data management the reason we're going to be talking about this is that there are two aspects to master data management there's the operational side and there's the analytical side so a lot of people talk about the same same words but they actually mean different things so i thought i'd throw a quick uh chalk and talk video together to talk you through the subject so that everybody's on the same page about what we're doing uh in various
• 00:30 - 01:00 scenarios why we're doing it what benefit that has to the organization um and and that kind of a thing so with that we'll jump into the whiteboard and and sort of talk through what is actually happening uh in in which area why we're doing it uh and that kind of stuff so i hope you enjoyed the video um so i'm gonna start by drawing the uh line between analytics and operational
• 01:00 - 01:30 so um and this kind of shows that the two sides of um of the world so by operational we mean line of business systems stuff that's actually in use day to day to run the business uh live transactional systems oltp that kind of a thing and on on this side of the world we need a current correct list of information
• 01:30 - 02:00 and something needs to own that information um and when that one thing owns information if a different system has the same uh kind of data stored for instance customer data then you don't want that information to be different in different systems because it makes you look like you're not joined up as a business but uh and you know it it just makes it better if it's all aligned and it's easier to report and you know that you've got
• 02:00 - 02:30 the actual truth so here we might have a crm system and then we might also have a web sales system so like a website uh this could be a mobile app it could be multiple things um and what's gonna happen is if a customer comes to me as a salesperson i'm going to enter them into the crm system so um i've got my customer there we will add
• 02:30 - 03:00 them in here and we'll create a record in there customer is dave and they are the first customer so we're gonna give that a customer id of zero zero one uh then if this customer came to the web system it wouldn't know anything about me because i've never interacted with the web system um so what we can do is have the web system look up my information on the crm because we know that might be a source of data but that's a very loose kind of
• 03:00 - 03:30 uh [Music] an interaction so uh we ideally don't want to do that um and the other thing that we don't want to do is when i come here to just create a uh a new record and not link that to anything because then i have two customer accounts if i look on the web system i know that i've bought a tv and if i look on the crm system i know that i've bought a computer um but i'm not able to see a joined up view of everything i've bought i'm not able to have consolidated billing all of
• 03:30 - 04:00 that kind of stuff is going to break down so what we actually want to do is before this transaction happens when i create a new user on the crm we will have an enterprise service bus this is basically just a fifo queue or a pub sub solution the crm system is then going to go out to the enterprise service bus and say i've got a new user is anyone
• 04:00 - 04:30 interested and the web server will subscribe to that and it will create a new record based on that same information so dave and over here a101 is going to be our um our web solutions customer id just because it's a different format um quite often this is just how things work in the real world that the developers created that uh it's baked
• 04:30 - 05:00 into the system you've no way of changing it um so when the web system has done that it's going to come back to the esb and say i just created a the same customer dave a101 and we're then going to create somewhere a kind of lookup table to say uh zero zero one is the same person as a101 and then for any time i as a customer come to the web solution if i updated my email address that can then go to the enterprise service bus and say
• 05:00 - 05:30 the master data record for dave the email address is changing everyone needs to update that and anyone that subscribed is going to be able to update that information based on on the change but you have to choose a system of record um and generally that's going to be something like your crm because it's more of a core system it could be your account system but whatever it is that is the owner that's the golden source of the truth that everybody can go to to check that their records are correct um
• 05:30 - 06:00 and it will tell everybody if that updates but everybody else will just tell it that they've done the updates so any other system that upstate user information tells the crm via the enterprise service bus usually that they've updated then the crm tells everybody else that that has changed so anyone that's interested in an email address of a customer will then update their own records internally rather than looking it up every single time off of that master data and that's
• 06:00 - 06:30 sort of how operational systems work is we want everything to stay up to date with the current information at the time that things are happening so that when our operational transactions are happening they've always got the correct current data um and that allows us to do uh to interact with business processes reliably this could be down to something like an hr system interacting with a pay system to make sure that that employee gets paid for the hours that they did and tying that up maybe with a time
• 06:30 - 07:00 recording system so that each of those are talking about the same person we know that they're talking about the same person because we've made sure that that's happening um the reason why we go via the esb is because uh if you directly talk from system a to system b you've created an interaction that uses apis or it might use scripting specific to that product if you ever swap that product out you might then have to update 30 different systems with new um with new integration
• 07:00 - 07:30 with that new product if you're using an enterprise service bus you only have to update once for the enterprise service bus to interact with it and everything else is still integrated with the enterprise service bus in the same way they always have been so you only change one thing rather than lots of things so that can change can save an awful lot of time on projects it can reduce costs but obviously it needs this sort of slight complexity in the middle of of keeping things up to date and this can also help with things like
• 07:30 - 08:00 compliance so in the case of gdpr and data protection laws you're required to keep customer data up to date and accurate so if a customer goes on to one web system or onto a mobile app and updates their data you're required by law to update all of your other records with that new data because they've made you aware of that change and you're not allowed to keep out of date data or inaccurate data that you know is inaccurate so there's a certain element of of legal requirement here
• 08:00 - 08:30 on the operational side when we move over to the analytics side the requirements are slightly different so it's no longer the case that if i accidentally didn't update something i might send an order to the wrong address it's no longer the case that i'm going to pay the wrong person or not pay the right person the amount of money that they're owed so the stakes are slightly lower but in terms of analytics the analytics are more useful if you uh tie things up properly
• 08:30 - 09:00 so where we're ingesting data from the operational systems we're going to be getting the same data from multiple different systems and this is pretty common because the web server knows about a customer the crm knows about a customer similarly warehousing systems know about stock supply chain systems know about stock and store systems know about stock and therefore you're going to get those same numbers from different sources sometimes with different information in them and it's important that
• 09:00 - 09:30 the analytics master data management doesn't rely on the operational side having master data management um sometimes the operational side is just a complete mess and we have to rebuild that picture the best that we can to do the best analytics that we can and just come up with a set of business rules to decide what we think is the best version of that data that we're going to store as our golden record and that will be what we reference and the reason we need this on the analytics side is we want historically
• 09:30 - 10:00 accurate reference data not necessarily the current correct data um and what i mean by that is if i have a store that is in one location and then that store moves the operational systems are all going to update to the new address of that store because they need to know if i send something to the store is the correct address if i'm dealing with that storage the correct address from an analytics point of view i need to know when that store moved what the old address was what the new address was
• 10:00 - 10:30 all of that kind of stuff so that over time i can see with the sales for this then we moved and the sales jumped up to here and that was the move that was responsible for it um and you know you're not saying i made a sale in this new location you're saying i made a sale here then i made new sales here but it was the same store that made the sales we just moved the store so sometimes it's useful to have all of that rich data captured over time operational systems don't generally do
• 10:30 - 11:00 that so we do that on the analytics side and this also slightly explains why the analytics world is is a bit removed from operational if you're doing reporting against operational systems you're going to lose a bunch of really useful historical information because they just simply don't store it it would be too expensive and complicated so they concentrate on keeping today's data accurate um so as this data comes in we're going to do a bunch of things to it to make sure that we're talking about
• 11:00 - 11:30 the same record so over here i've got my dave 101 and down here i've got dave a101 oh sorry that was zero zero one um and if i don't have a lookup table if if none of this uh stuff had happened on the operational side i've just got two records for some guy called dave um and i can make a bunch of assumptions here i can set business rules up so it
• 11:30 - 12:00 could be as simple as does this person have the same name and i can go yep that's dave and that's dave so therefore my master data is going to say i've got one record dave and it's those two things and then when i'm linking that up in my in my data model i know to just reference that same person um and anything that happened to this dave applies to this other dave as well that's really uh an easy example but it's quite easy also to see how that would be wildly
• 12:00 - 12:30 inaccurate in most scenarios because there's more than one dave in the world [Music] so actually what we what we would strive to do is keep that lookup table where i can in this process that's going to join these records say if i've got dave whose customer id is 101 does that match with dave there's a101 and that's a much better scenario because then i'm confident that those are the same uh record and i might be building this as part of my analytics solution to
• 12:30 - 13:00 say chances are these are the same people so i'm going to link them now if i find out they are actually different people in the future i can unlink them and make two records for that so that i know that they're different people and i should treat them differently in the future but generally speaking in an analytics world nothing is going to have gone terribly wrong because i did that and so we can make some assumptions here and there as long as they're not gonna drastically impact uh reporting or things like that um
• 13:00 - 13:30 so what i can also do is instead of that i can i can look at this one and then i'll have an email here and i'll have an email here and what i can say is oh this person is called dave and this person is called dave do the emails also match and if the emails match and the names match probably the same person and i can then have more rules such as addresses and phone numbers and things like that i can have complex complex rule matching engines that can say
• 13:30 - 14:00 if the email addresses don't match but the phone numbers do match it's still the same person and they've just got two email addresses and they've set up two accounts um and even you know if i've got another dave on this second system um and it's a 102 and they've got the same phone number i could then actually roll those into the same account as well and just be aware that this person has accidentally set up two accounts or or something like that
• 14:00 - 14:30 um if i were doing this in an operational capacity i might then come back to the web server and say next time this person logs on ask them if they've got two accounts um or send them an email from that account and say do you have two accounts or something like that so that you can rectify it join up that data um and in the operational world that's kind of useful because then that person has one account all of their sales might be in the same place so that they can see their historical ordering but in the
• 14:30 - 15:00 analytics world we don't mind so much because we can still report based on two different people if we're then doing proactive marketing they might not get quite such accurate results but that's kind of fine um so you know this is sort of less important but it does give us this golden record here um where when i then have a report down the line i would use this uh consolidated view of um
• 15:00 - 15:30 of customer data of store data of all of those things because i know that this i've put effort into cleaning up this data of joining it together matching merging verifying enriching so that if i've got some information from that system and some information from this system i'm going to join those together and enrich that so that i've got all of the information in a single golden record about that person so i know everything that's happened to uh that record from all of my systems um and then the the downstream reporting
• 15:30 - 16:00 doesn't have to do those things it's already done for them uh and the consumers of my reporting or my analytics don't really need to worry about those things um and in terms of tooling that there's a lot of tooling about around about this stuff and part of the reason why people get confused is both of these solutions are called master data management they work in very different ways they are for very different purposes as we've just discussed uh but each of the sources where you're reading about master data management likely has a bias based on who wrote it
• 16:00 - 16:30 so if there's a vendor that has a analytics mdm solution then they're going to be biased towards these kinds of things and if you've got an operational one that's selling an enterprise service bus mdm solution then they're going to be biased towards the other side and yet when you're internally talking within your business with different teams you're both going to be using the same words but you're talking about very different things um and so no understanding the differences here can really shortcut those
• 16:30 - 17:00 conversations to to get to that value that you need to get at the end of it so hopefully that's been useful to you and interesting to to learn a bit about master data management um i nearly said mobile data management that's another mdm that you might hear so beware with acronyms in in it because they are all over the place and we've reused them for decades and it it's not very helpful um but hopefully that's explained this uh the tooling that we can use will be
• 17:00 - 17:30 very very different so you might use data factory to achieve uh the matching engine on this side you might use python you might buy a specific product you might then apply quality checks to it as well with something like the great expectations framework um you can use what you want and the um the commercial offerings in the mdm space all they're doing is making it easier to create the rules engines that are linking the business systems together uh so the idea of if i've got
• 17:30 - 18:00 two people called dave they're the same person if i'm doing that in python i have to write a a library that has a function that does that matching if i'm doing it in a commercial mdm solution there's probably just drop down boxes to say if this is this then this um and that can make your life easier uh but in the long run it it makes less difference than you would think because once you've written those business specific python libraries they're yours forever you can copy and paste the code you can reuse those rules
• 18:00 - 18:30 so long term it's not really much more effort to do it that way if you've got the skills in-house then go for it but the tooling that helps with this in some instances can really help and it's a much easier way of getting into it because it kind of guides you through how to make those rules so i'll stop there hopefully that's been interesting to you if you if you found this interesting and you want to watch more hit the subscribe button down below all that does is it just notifies you and it puts it into your personal list
• 18:30 - 19:00 of things to watch so it's it's kind of like building your own tv channel uh hit the like button as well that just helps the channel to float to the top of the youtube algorithm and hopefully i'll see you next time for another video thanks for watching you