Event Sourcing • Greg Young • GOTO 2014

Summary

In this GOTO 2014 presentation, Greg Young delves into the intricacies of event sourcing, a method of data capture and processing centered around storing all changes as a series of events and deriving system state from them. With roots extending back to practices in industries like accounting and finance, event sourcing allows for detailed audit logs, flexible state derivation, and the ability to explore historical data in new ways. Young uses examples from banking and production environments to highlight the practicality and robustness of event sourcing, despite its perceived complexity compared to traditional CRUD systems.

Highlights

• Event sourcing is akin to keeping an audit trail like accountants do, storing every event as a permanent fact 🧾.
• The banking system exemplifies event sourcing, where account balances derive from all previous transactions 💰.
• Systems can derive their current state from a series of events, similar to executing a series of functions in a foldable stream ♻️.
• Old and new use cases can be tested against historical data without losing previous information or structure 🔎.
• Event sourcing provides a robust mechanism to protect against data loss or corruption, even dealing with super user attacks 🛡️.

Key Takeaways

• Event sourcing isn't a new concept but an evolution of age-old practices used in industries like accounting and finance 📜.
• By storing every change as an event, you can audit and derive current state making your systems more reliable and traceable 🔍.
• The concept of event sourcing leads to a more functional data handling model, moving away from traditional CRUD operations 🧩.
• Event sourcing allows the system to be replayed and analyzed at any point in history, offering great insights and adaptability 🔄.
• While setting up event sourcing can be complex, its benefits in terms of auditing and data integrity are significant and worthwhile 🌟.

Overview

Greg Young began his exploration of event sourcing by drawing comparisons to longstanding practices in industries like banking and accounting. He explained that instead of a static state, event sourcing involves logging every event that transpires in a system, resembling how a ledger records every transaction. This not only ensures a provable audit trail but allows for dynamic state computation and reevaluation at any given time.

Young emphasized how mature businesses like those in finance and insurance naturally align with event sourcing concepts. By treating state as transient and derived from historical events, organizations can gain a flexible and comprehensive understanding of their operations. This strategy also sheds light on the overall value and exploration of data not previously considered relevant.

The presentation highlighted both the operational and analytical advantages of event sourcing, despite its complexity. Young contrasted it with CRUD systems, pointing out that while CRUD simply updates static records, event sourcing retains every change as historical data. This approach not only enhances data integrity and audit capabilities but also allows for a more adaptable system finely tuned to future analytical needs and regulatory requirements.

Chapters

• 00:00 - 05:00: Introduction to Event Sourcing The chapter 'Introduction to Event Sourcing' begins with a personal anecdote from the speaker about their experience presenting new concepts at a conference in Copenhagen. The speaker reflects on their nervousness presenting to an audience that included notable figures like Martin Eric Evans and Gregor, focusing on themes such as domain-driven design and messaging.
• 05:00 - 10:00: Event Sourcing in Action The chapter "Event Sourcing in Action" opens with a humorous remark about the presenter consuming an excessive amount of coffee before delivering a talk. The speaker describes the experience as having rushed through the talk in 17 minutes, which they characterize as a 'really awful talk'. Despite the negative feedback from peers like Martin, who metaphorically 'gave a red card', and Eric, known for his restraint in critiquing, who admitted it was not a good talk, the key takeaway is about the enduring nature of 'not new ideas'. These ideas trace back as early as 2006 or 2007, highlighting the longstanding history beyond even those dates. The narrative sets a stage about rediscovering and presenting established concepts in a refreshed manner.
• 10:00 - 15:00: Immutable Data and Projections The interest in event sourcing stemmed from the necessity of maintaining a comprehensive log for proper auditing, which varies in importance across systems. Since 2007, a particular illustrative slide has been used to effectively convey this concept.
• 15:00 - 20:00: Avoiding Data Loss in Event Sourcing The chapter highlights the natural use of event sourcing in industries like finance, banking, gambling, and insurance. It suggests that these domains inherently use event sourcing principles. It poses a reflective question about whether your bank account balance is merely a column in a database table or a result of an equation. The implicit answer is that your balance is a summation of all transactions, reflecting the event sourcing approach.
• 20:00 - 25:00: Understanding Snapshots This chapter delves into the concept of snapshots in banking, specifically how account balances are affected by past transactions. It illustrates a scenario where an individual believes they have a certain amount in their account, while the bank reports a different figure. The explanation highlights the importance of understanding that current balances are a summative result of all previous transactions and encourages an approach of reviewing transactions to verify balance discrepancies.
• 25:00 - 30:00: Event Sourcing and Accidental Complexity The chapter discusses the concept of event sourcing and its growing adoption in businesses. It emphasizes the importance of having a first-level derivative that is provable or disprovable in business operations. The chapter introduces the idea that many industries have naturally adopted event sourcing, suggesting there are several reasons beyond the verification aspect that make this approach appealing. The details of these reasons are promised to be discussed later in the text.
• 30:00 - 35:00: Event Sourcing as a Teaching Tool The chapter 'Event Sourcing as a Teaching Tool' discusses the concept of event sourcing in systems. Event sourcing implies storing only facts as events, and all system states are derived from these stored facts. This concept is compared to how a bank balance is derived from transactions, emphasizing that every system state can be determined from these events. The chapter illustrates that regardless of the type of state needed, whether it is a read model, a domain object, or any other state, it can be derived from the stored events in the system.
• 35:00 - 40:00: Enterprise Considerations and CQRS The chapter discusses the application of event sourcing within the context of enterprise considerations, focusing on the CQRS (Command Query Responsibility Segregation) pattern. The speaker uses a purchase order example to explain the concepts, suggesting that the ideas can be applied to various problems. Event sourcing is introduced, and the state is represented by a purchase order containing line items and shipping information. The goal is to move swiftly through the definitions to reach more complex, interesting topics.
• 40:00 - 45:00: Implementing Event Sourcing in Existing Systems The chapter titled 'Implementing Event Sourcing in Existing Systems' begins by challenging the conventional focus on data storage and schema design. It references the common experience of dealing with SQL migration scripts and the problems that arise when altering data structures in traditional systems.
• 45:00 - 50:00: Q&A Session The chapter "Q&A Session" discusses the issue of reliability and rollback in the context of implementing new systems. It mentions the concern of taking a system down, bringing it back up, and experiencing delayed failures. The speaker humorously refers to two personas, a 'fireman' who responds to emergencies, and a 'cowboy' who takes risks, illustrating different approaches to handling unpredictable system behavior. The topic of migration scripts is also touched upon, emphasizing the challenges of reverting to an old schema after a new one has been adopted.

Event Sourcing • Greg Young • GOTO 2014 Transcription

• 00:00 - 00:30 it really was weird for me when I went to Copenhagen and they had it in the bleeding edge track so the first time I actually talked about event sourcing was at cuon 2007 um Martin was actually at this talk and I was scared senseless so I'm coming and talking about new ideas around domain driven design um messaging and these kinds of ideas and my front row was Martin Eric Evans and Gregor h
• 00:30 - 01:00 I think I probably had about 30 glasses of coffee before I started and I went through my entire talk in like 17 minutes it was a really awful talk uh to be fair I think Martin gave me a red card Eric even told me it was a bad talk which if you've never met Eric he doesn't really say bad things but it's it's not new ideas and even going back as far as like 2006 2007 these ideas go way back way past that
• 01:00 - 01:30 what really brought us towards looking at event sourcing was that we needed to have a log and we need to have a proper audit now for some systems this is very very important for other systems it's not so important that's what led us towards the idea of event sourcing and I've actually used a slide now since 2007 because the slide is so perfect what's interesting for me when we talk about event sourcing is that if you look at mature businesses and when I
• 01:30 - 02:00 say mature I do not mean Groupon every single one of them event sources conceptually Finance banking gambling Insurance all of them are naturally event sourced how many of you have a bank account do you think that your balance is a column in a table or is your balance an equation your balance is a summation of all the
• 02:00 - 02:30 previous transactions that have happened on your account correct what if it were just a column so you think that you should have 10,000 Crown the bank says you have 1,200 crown and you call them up on the phone and say I think my balance is wrong and they say I'll hail the column it's not really that useful but if I say that your balance is a summation of all the previous transactions that have happened on your account before can we take out a pen and paper and figure out what your balance should be
• 02:30 - 03:00 you can argue about whether or not a fact exists but you cannot argue about the overall result that's derived from them this is one of the main reasons businesses are actually going towards event sourcing having this first level derivative that is provable or disprovable is actually a very very valuable thing we're going to talk about it more but there's a lot of other reasons why all of these industries have gone and they're naturally event sourced before we get into those though
• 03:00 - 03:30 let's go with a definition of event sourcing when you talk about an Event Source system you only store facts All State in your system is a first level derivative off of your facts just like we said bank balance is an equation that runs off of your transactions All State can be worked this way we can say that we will only ever store events inside of a system and any state that you want to have I don't care whether it's a read model whether it's a domain object anything that you
• 03:30 - 04:00 want to have is a first level derivative off of that and we can do this to any problem I've been using this example for a long time and how many of you already know what event sourcing is okay good number um I'll try to go through the definitions and and showing people it fairly quickly so we can get to the more interesting things now this is my canonical piece of state so we have a purchase order with nline items and some shipping information associated with it this is
• 04:00 - 04:30 not the only way of storing information although this is what we've been taught to care about how many of you spend large portions of your day talking about the shape of data and if you start storing this you run into all sorts of interesting problems how many of you written a SQL migration script before fun right um to be fair I will never ever do a big bang release ever again where I take down a piece of software and bring it back up the reason reason why is not because
• 04:30 - 05:00 I'm worried that I will take it down and when I bring it back up it won't work that's really easy just do a roll back right what happens when you take it down and you bring it back up and then it works fine for a week and then it breaks how many of you write migration scripts to take the new schema with all the new data and bring it back to the old schema so we've had a joke that um when you have this problem where it runs for a week and then it dies you have your choice you can either wear the fireman hat or the cowboy hat um to be fair I
• 05:00 - 05:30 actually recommend a lot of teams to do this um the hats are are basically a token and if someone walks into your office and you're wearing the fireman hat they just walk out oh come on how many of you guys have worked in production before and then someone came in to ask you about the Christmas party and you're like get out of my office but I can't say this now this is our canonical piece of state but this is not the only way of dealing with a set of information I could also deal with it like this where we have cart created three items added
• 05:30 - 06:00 and then shipping information added now that three items added is actually three distinct events um if I put three distinct events there the boxes just got really little and you couldn't read them anymore but at any given point in time I could take these five events and I could replay them and I could give you back this in other words I can give you back a piece of state and what we're going to say in an Event Source system is that all state is transient don't get me wrong it may be persistent but it's transient what matters is my events
• 06:00 - 06:30 because any given point in time I can always replay my events to give you back any piece of State in the entire system my book of record is my events if people hear no functional programming another way we could say this is that current state is a left fold of previous behaviors I can fold over my previous behaviors and give you back some interpretation of it and that may be your current interpretation tomorrow it may be a completely different interpretation and this is one of the main values of EV sourcing is that this
• 06:30 - 07:00 concept of state becomes transient which changes more in your system how you view your use cases or your actual use cases your events align directly back to use cases now it took me forever to find that original slide because there's an accountant erasing in the middle of their Journal accountants don't do this unless they work for andron in which case they probably do
• 07:00 - 07:30 in an Event Source system we have no concept of an update or a delete you can only ever pen things and this works the same way that accountants work how many have taken an accounting class before maybe in University um they probably told you that accountants don't work in pencil they work in pen once you've put something into your Ledger you never ever take it out and we do the same thing in event sourcing now I'm just curious how many of you have an update or delete statement in your system
• 07:30 - 08:00 today okay now keep your hands up how many of you had a c-l board meeting to decide that that that information was meaningless and it was perfectly fine to get rid of how did you determine that that information had no value how do you even do a cost benefit analysis over information so I've got some data today that's happening inside of my system in order for me to do a cost benefit anal is I need to know how my business wants
• 08:00 - 08:30 to look at this information in 3 years how many of you can predict where your business is going to be in 3 years and what questions they'll ask you so how do you make this decision to update or delete data do you take out your magic eightball shake it up and try to figure out what the business is going to ask you how much does it cost to store one gigabyte of data for a year today yeah to be fair it's actually zero you can just open up a Gmail account and start Gmail
• 08:30 - 09:00 account but it costs almost nothing how many facts can you fit inside of a gigabyte worth of data let's imagine they're all 1K each that's a lot of facts that you can store you don't need to get a lot of benefit out of your data down the road and I'm going to talk about this a bit more later but I get this question a lot of if I only ever just store everything then my data is going to get really really big to be fair most systems I can fit
• 09:00 - 09:30 your data on a micr SD you're not big data even if you're pending you're talking about oh so over the next five years we're going to end up with 100 gigabytes worth of data okay how much does it cost to store 100 gigabytes we don't have an update or delete operation when we talk about Event Source systems there's only a pend and just like what accountants do we always append a new event so if I
• 09:30 - 10:00 were to go back and I were to look at accounting let's say that I made a mistake and I transferred you 10,000 Crown as opposed to 1,000 Crown because I fat fingered it well I just going eras the zero in The Ledger right no so I'm going to do one of two things I'm either going to do what accounts called a partial reversal in which case I'm going to put a journal entry that says I'm going to take 9,000 back from you and put it into the other account accountants don't like doing this okay if it's nice round numbers 10,000 and 9,000 you can figure out what
• 10:00 - 10:30 I originally intended to do as 1,000 what if there's six accounts involved and they're not perfectly round numbers well then you have to take out a pen and paper and try to figure out what my original intention was so instead what accountants will do is they'll do what's known as a full reversal I fat finger I give you 10,000 I take 10,000 back from you I Mark that the first thing was in mistake and then I do what I actually intended why because an auditor can come back through here and he can figure out exactly what I intended to do it's the same thing
• 10:30 - 11:00 when we talk about Event Source systems we tend to prefer full reversals as opposed to partials but let's say I want to remove an item I could say that we now have cart created three items added one item removed and shipping information added is this the same as cart created two items added shipping information added I always love this question because about half the room is going yes and half the room is going no and you haven't become a good
• 11:00 - 11:30 consultant if you haven't learned the words it depends there's actually a great page up on the C4 Wiki about it depends now if I were to look at it from this perspective yes those two are the exact same correct I'm going to get out an order with two line items and shipping information Associated to it from both of those streams what if I had a different perspective though what if I were counting how many times items were removed with they be the
• 11:30 - 12:00 same no and this actually a fun trick if you can go into your system today and you can come up with any set of use cases that leaves you at the same ending State as another set of use cases guess what you're losing information you just showed that you do not have a perfect hash how did you value that information who did you talk to about it this is one of the main benefits of event sourcing and why businesses actually go towards the concept of event sourcing regardless whether we talk about it in code or just
• 12:00 - 12:30 conceptually event sourcing does not lose information it is the only model that you can possibly use that does not lose information and you can have lots of different implementations of it a lot of accounting systems for instance they are a table with a descriptor column that says um type of row and then they join off to another table to get the rest of the information for that row that is at its heart event sourced they are storing facts and deriving State off
• 12:30 - 13:00 of their facts an event sourced system is the only possible way you cannot lose information when we talk about storing data any other form loses something now I see the real value of this what we have to do is we have to go through and try an example so let's imagine that we're using this model for I don't know someone like Amazon and our business user comes to us and he says you know I think people that remove their items from their carts within 5 minutes before they check out are more
• 13:00 - 13:30 likely to buy that item in the future than the other things we randomly show them well why do you remove an item from your cart five minutes before you check out so you go and look at the cart and it's going to be like 800 crown and you're like well I could buy all that stuff but my wife will kill me or I can remove three items from the cart I can get some of the items from the cart and my wife will not kill me so I will take some of those it's not that I don't want the other things it's that I'm deprioritizing them compared to the
• 13:30 - 14:00 rest so with this model we're now going to take this card off the wall we're going to try to implement it so I guess our first thing that we'll do is We'll add a new thing off the top called removed line items and then what we're going to do is we're going to write a report that looks for removed line items and does a subquery back to see if the person has bought this in the future so we put it out to production our business user says I want to run this report and they run it what do they see that reports starts from here and
• 14:00 - 14:30 moves forward let's try in this model now when we talk about deriving State off of an event stream it's known as a projection what we do is we project a set of State off of an event stream if you're in the functional world again this is a left fold it's a whole lot easier in the in the functional World they don't have all these fancy pattern names so what I'm going to do is as I fold over the event stream I'm going to
• 14:30 - 15:00 look for an item removed and when I find an item removed I'm going to put it into my state as I'm folding over and I'm going to say I found this item removed and now I am looking for the checkout when I get the checkout I compare the time that the item was removed to the time of the checkout if it was within 5 minutes I mark that I am now looking for this item in the future found equals false and then as I go through all the rest of the events over time if I find the person actually bought this item I Mark found equals
• 15:00 - 15:30 true easy enough now what I haven't told you about projections and this is a key idea in Event Source systems is that they must start at event number zero the very first event that your system put out is where a projection starts and you run it all the way forward until it comes to right now and then it continues into the future and this may take a while may take a weekend you can imagine you've got 500 million they they they don't go
• 15:30 - 16:00 that fast really um we we can currently get you up to about 700,000 per second coming off of persistence but even at that time it's going to take you a while to replay so we can imagine we started off because it's going to go into SQL Server start off on Friday coming on Monday we now have our read model there our projection is completed so now we write a report against our read model or as uh Martin call an eager read derivation beautiful term by the way
• 16:00 - 16:30 what does my user see when he runs his report well he sees all the data we never knew that he would have this feature but we didn't lose anything so we were able to actually give it to him not only that I can also go to my user and I can say you know this new report that you came up with this is what that report would have told you on August 14th 2011 at 1 34 in the
• 16:30 - 17:00 afternoon if you had this report at that point in time you can take any report and you can show it at any point in the history of the system and you can do this you can do it deterministically um if anyone happens to work with machine learning there's actually some very well-known algorithms for dealing with machine learning over event streams in particular it's called Alpha Beta training so what you're going to have is you can imagine Alpha is at the very beginning of the list beta is 10 events ahead and Alpha's job is to predict where beta is
• 17:00 - 17:30 but this is one of the major benefits of event sourcing that you can go back and you can take new ideas and apply them to any point in the history of the system you cannot change what happened in the past but you can have a new perception of what happened in the past let's just bring that back to a real world example and I don't recommend anyone does this but imagine if you could take your current mind and bring it back to when you were 13 years old at your first dance
• 17:30 - 18:00 you can't change anything but you can perceive it as you currently are don't recommend it in general most people are looking at event sourcing due to the these kinds of reasons these business reasons however there's a whole lot of programmer pornography associated with this how many of you have bought a hard drive before so there's two speeds on a hard drive right there's one for writing sequentially there's another one for
• 18:00 - 18:30 writing random okay to be fair on ssds it doesn't really matter that much but if I have an event log if I'm actually storing this do I write random or sequentially everything's sequential and you can become very very very fast um with a fairly naive setup on a single node we can linearize you to probably 50 to 100,000 transactions per second how many of you do more than 10
• 18:30 - 19:00 so we're a couple orders of magnitude above where you really need to be and you'd be amazed how simple things get when you can actually linearize everything but there's some other cool things that you can do for me the biggest one is this ability to go back in the past and does anyone know what this is this remote they used to be really popular in America they allowed you to go back and watch television too exactly actually to be fair the biggest thing that they did is they let you skip commercials and then they took that
• 19:00 - 19:30 functionality away they had the one button it would actually find the end of the commercial and you could just hit one button it would bring you right back to the show but you can basically go forwards and backwards through time inside of an Event Source system and time becomes explicit which is another big thing but there's some other really cool stuff that you guys can do how many of you write smoke tests so what I like to do is I store every command that my system has ever processed you told me to do something I wrote an event saying that I did
• 19:30 - 20:00 something and I replay them every week and I compare what I did this week when I reran every command my system has ever done to What It produced last week are these things that I expected to have change or are they things I didn't expect to have changed as I Chang Behavior over time this is a really good smoke test now this will not save you from all production bugs um there there are still black swans sorry but you should be reasonably comfortable pushing something to production if you've already taken
• 20:00 - 20:30 that software and reran every single transaction your system has ever done before through the new software and not have errors or maybe your bad coders I don't know you hide your bugs very well but there's some other cool things that we can do how many have heard of a super user Tech so the definition of a super user attack is a rogue system administrator or developer attacking your system they have root access and they
• 20:30 - 21:00 want to hack you they probably have access to Source they can do anything they want right years ago I actually had to deal with one of these um and we were building gambling systems and we were actually event sourced and we we had to go one step further in our security so that we could prevent it an Event Source system can prevent a super user attack and if you want to go look this up uh the guy's name is Chris harn and there's a Wikipedia page and there actually a 1 hour hbf special about this
• 21:00 - 21:30 guy um it's criminal masterminds which I never quite understood cuz all their criminal masterminds got caught so we had a pool it was called a pick six and basically a pick six is you have to pick the winners of six races in a row obviously that's hard and you get paid a lot of money for doing it this was way back in the CSU DSU days and what we were doing instead of moving all the bets over track cuz when you bet this kind of pool you normally bet like
• 21:30 - 22:00 a 100 bets at a time what we were doing was we would actually store them at your local track until the end of the fourth race and then we'd only ship over the ones that could still win back to the host track well that gets rid of 99% of your network traffic which if you're going over serial ports is fairly reasonable so what he was doing he would go and put a bet one two 3 4 all all at a remote track he would watch the first four races and then he would go edit his
• 22:00 - 22:30 bet on disc to change 1 2 3 4 with the actual winners of the first four races good scam right and he got caught um not because he was stupid but because he was unlucky how many have heard of Breeders Cup it's the second or third largest race in America and he was doing this scam during Breeders Cup and he went through and everything went normal put on the BET over the ivr got on the machine changed the BET one 2 3 four to the first four winners and then 2 99 to1
• 22:30 - 23:00 horses came in the fifth and six legs he was the only winning ticket in the world it was like a$3 million ticket now normally there's going to be let's say 30 40 winners to this kind of pool and you know you're just going to get paid here's your $100,000 payout no big deal what do you think happens when there's one winning ticket in the world and the mutual manager goes and looks at it and it's a bet like 537 all all no punter would ever bet this cuz if
• 23:00 - 23:30 you if any of the first four lost your entire all your tickets lose so he sees this couple million dollar ticket and he says let's let's get on the phone with Catskills figure out what's going on over there anything interesting oh there's a developer on the maintenance line that's interesting oh the audit tape is ejected yeah that's interesting but this is a fairly typical super user attack and and to be fair most super user Texs they will only ever
• 23:30 - 24:00 get caught if they're unlucky but we can prevent this with an Event Source system and we did how many have heard something called a worm Drive write once read many once you've written to the disc think about an old cdrom you can only physically write to the Thing Once you can't write to it multiple times now if my current state is a first level derivative off of my audit log you need to put some something in my audit log to change my state now if my audit
• 24:00 - 24:30 log is on a worm Drive how how do you attack me now to be fair there is a way if I'm writing it very slowly you could make an entire copy of the Worm Drive and you could switch them we can reasonably control that um I could write a message to the Worm Drive let's say every 100 milliseconds how many you can change a worm drive for another Worm Drive within 100 milliseconds I at least detect that somebody probably change a drive at this
• 24:30 - 25:00 point it's useful for this kind of situation now I'm not going to say that you should only ever use event sourcing if you happen to deal with super user attex um it's just one of those things if keep it in your toolbox if it happens to become useful it'll be very useful now conceptually we always start at event one and we go to the end of the stream and this works really well for most things when we talk about Event Source systems most Event Source systems are not having one giant log they have
• 25:00 - 25:30 millions of tiny logs think about a mortgage application in a bank so it comes in there's some approval process that it goes through and basically you get events that are written how many events happen to a mortgage application in a bank for a single mortgage application Millions for a single one that'd be weird they're very busy normally we might be saying 50 50 things will happen to this throughout it life cycle um uh we can think about it
• 25:30 - 26:00 as being a document in a document database so we have all the events for a given document now normally this is not going to be a problem to go from one to the end of the stream replaying it'll be very very fast um how much more expensive is it to read 20 rows from a database as opposed to reading one in terms of time to actually load it however there's some things this doesn't work so well with um what if I were to take the order book of Google in
• 26:00 - 26:30 the stock market at about 2:00 in the afternoon how many things have happened to Google throughout the day okay now you're millions and if I wanted to do that playing from one all the way to the end of the stream even though I do that conceptually would be very very slow so there's a related pattern where we can start dropping snapshots at given points in time inside the stream so here for instance as opposed to reading from one to the end I'm actually going to read backwards I'm going to start with six and then I'm
• 26:30 - 27:00 going to say are you a snapshot no five are you a snapshot no snapshot are you a snapshot yes so now I take my state at that point in time and then I apply five and six afterwards in general never ever store your snapshots like this um what happens when I store them like this and we're currently at version four and I say okay I'm going to write a new snapshot down then he writes version five concurrency problem is my Snapshot any less valid at version four because
• 27:00 - 27:30 he put version five in the event stream so you take your snapshots at a version and point back to the event stream you basically store them off to the side along with this you can have many many different types of snapshots pointing back to the same event stream um I can have 25 different folds that all go over the same event stream with their own perceptions of it and I can snapshot all of them independently in general of avoid snapshots if you can and you can't
• 27:30 - 28:00 always avoid snapshots if you have something like uh 5 million events for Google's order book you need to have a snapshot at some point but when you introduce a snapshot you're going all the way back and you're introducing this again and you will have all the same versioning problems that you end up with a SQL database or any uh structure that you actually store when I store events I have a lot less versioning problems I can bring up two servers side by side that each have their own ception of the
• 28:00 - 28:30 data and I don't need to talk about database refactoring patterns although I really love that book in the series when we talk about state state is very hard to verion now what's really interesting for me about event sourcing is there are certain businesses that are out there that are already naturally event sourced and if you introduce event sourcing to these people it will make total sense to them how many have talked to a lawyer before
• 28:30 - 29:00 so when I have a contract that's been written and then you want to make a change to it we just go edit the contract right or do we put addendums to the contract and over let's say five years we have 37 addendums to our contract and the only way to figure out what our contract is is we take the original contract and we apply all the addendums to them one after another in order that they happened sound familiar how many you been to a doctor before hopefully at least
• 29:00 - 29:30 once now when you go to the doctor do he take a picture of you and put it in your file and throw away the other picture of you or does he constantly append papers into your file and we wonder why they have a hard time understanding crud systems when their conceptual model is I have a file and I pen things into the file all the time we run into this a lot there are a lot lot of domains that are naturally event sourc and event sourcing is not a
• 29:30 - 30:00 new thing we're going to talk about this more but I've managed to track it all the way back to Mesopotamia Sumeria so when people ask like well how do I how do I deal with this new thing it's not new how many use a SQL database today guess what you're already using event sourcing there's this thing called a transaction log in your SQL database but you're going to run into a problem if I only store my events how do I write a query um I'm looking
• 30:00 - 30:30 for all customers with the first name Greg so I know we'll replay every event my system has ever done before we'll fold over every single stream and we'll put a filter at the end of The Fold to say whether or not it's currently Greg this will be fast right Big O of n is awesome when we have uh 500 million events and most times when you're talking about Event Source systems you you don't query your events you end up with a concept known as read models and
• 30:30 - 31:00 and this leads you back to cqrs we're going to talk more about cqs cqs is mainly a teaching pattern but I write events and then I I have n read models that I can actually read from and this will actually save you quite a bit how many have heard of these before you know you're not cool if you're not using these except for one of them one of them's really not that cool I'll leave it up to you guys to figure out which one
• 31:00 - 31:30 but when I look at this I I am reminded of about a decade ago especially in Scandinavia how many you remember these object database is going to take over the world they were 10 times faster on oltp operations than SQL databases they had no impedance mismatch between your domain model and your data storage they were awwesome or as my wife sometimes likes to say aasum
• 31:30 - 32:00 how many of you use an object database today one I thought they were going to take over the world does that remind you of these at all currently we're saying these things are going to take over the world because they're they're the new old thing no one had ever come up with a key value store before document database no one ever heard of oh wait Lotus Notes but why aren't people using these today
• 32:00 - 32:30 how many have used one of these so I've worked on quite a few projects that used one of these and it always ended up with the exact same failure so we would build out our object database and everything was beautiful with our domain model and we could go back and forth no impedance mismatch we really really happy and then one of these incompetent business people would come over and they would ask us for a report and they would want us to do something like roll up sales based on
• 32:30 - 33:00 customer and town and we'd go well that means I need to load up all those objects in memory I can't do that and then we remembered we had these things called SQL databases that were actually really good at this and we decide that object databases totally suck because they don't do that very well and therefore we should always use SQL and we run into this a lot where we come through and we want one tool that fits all of our
• 33:00 - 33:30 problems to be fair you will never have one tool that fits all of your problems no matter how hard you try and every single database on the planet sucks when we talk about databases they're they writing isn't very interesting most of them they work the same way as what we were talking about with vent sourcing they append to a log years ago I was giving a talk actually about a vent store which is I work on and in my talk someone actually
• 33:30 - 34:00 said you know you guys work the same way internally as Cassandra you're the same as Cassandra it's like yes we use a log structured merge so does SQL light so obviously SQL light also is Cassandra when you talk about a log structured merge what I do is I append to a log and then I've got chasers on the log that are updating in memory structures and then I batch update them down to dis this is how many many datab work think about SQL Server your transaction
• 34:00 - 34:30 file same general idea when we talk about databases what's interesting with them is not about their writing it's about their reading and we run into all sorts of accidental complexity if we have the wrong models so you guys might have heard this small startup before what Twitter is and its core is a topic based Pub sub correct we have topics names hashtags and people subscribe to
• 34:30 - 35:00 topics so if you ask me to build Twitter the first thing I'm going to do is I'm going to ignore everything that's been done in the financial industry for the last three decades and I'm going to build it with Ruby on Rails and my SQL because that's obviously how you should build a topic based Pub sub correct how many you remember the fail whale luckily they had enough Capital that they were able to get out of this problem but they introduced massive massive accidental the point they were managing hundreds and hundreds of MySQL
• 35:00 - 35:30 instances to build a topic based Pub up really bad idea I would not recommend going that way however if you do happen to make the next Twitter let me know I'll invest early but there other examples is this how many of you have created something like this before so we have ID parent ID and some data and what we're building here is essentially a tree and you go and you release it to
• 35:30 - 36:00 production and they say you know it's taking 9 minutes to bring up that report and you go but it works on my machine so what's the problem well on your machine you've got about 100 rows in that table and production they've got 800,000 and you have built a recursive query so you were smart and I really want to know how many of you are willing to admit this so you come up and say I'm going to do this so now we we have parent ID zero parent ID 1 parent ID 2
• 36:00 - 36:30 parent ID 3 and now you can do an or query and get back all the rows that are underneath a given thing just by using your ores on the parent IDs come on be fair how many of you have actually done this I'll admit it I've done it and this is a piece of mass of accidental complexity because we're trying to deal with this particular kind of read model we're forcing ourselves into it an alternative to this might be this thing called a graph database graph database is are really really good at dealing with
• 36:30 - 37:00 this and I want to be clear I'm not saying that you guys should go have a database per problem keep that as an opportunity and then you can talk about the operational complexity that you introduce by doing it uh at the end of the day if I go put a graph database and production somebody needs to maintain it operations needs to know how to back it up they need to know how to deal with it but these are all examples of accidental complexity I actually watched this exact problem at a in America and what they had was they had people and
• 37:00 - 37:30 then they had a table called people relationships that linked to different people together based on a certain relationship type and what they wanted to be able to query was so from this person to this person using only these relationship types can I find a path they had a half million dooll SQL Server doing this we replaced it over the course of two days with Neo forj and the laptop was faster than their production server this is normal when you choose the right
• 37:30 - 38:00 model keep in mind that the wrong models will introduce massive accidental complexity to your system when we talk about Event Source systems we can have as many read models as we want inside of the system all they have to do is subscribe to the events they're just a projection what we'll normally end up with is we'll store all of our events and then we'll end up with n read models that are off on the side now I know we're in
• 38:00 - 38:30 Scandinavia so I have actually done you guys the favor of giving you your questions and these are things I've heard over time like for instance Event Source systems need a service bus no almost nothing needs a service bus if you bring a service bus into the problem you're probably making a bigger problem in most circumstances um and to be in in most cases like especially if you bring in something like bis talk you now
• 38:30 - 39:00 have two problems you have the problem that you have bis talk and you have the bis talk consultants and they're going to take forever to get rid of now I see people try to apply event sourced systems with service buses all the time and they almost always fail and the reason they fail is because there's two different kinds of subscription models there's what's known as a consumer-driven subscription um think a Blog and there's what's known as a a a server driven subscription which think
• 39:00 - 39:30 amqp and normally when we talk about a service bus they're going to be using something like amqp rabid mq underneath or msmq or some web sphere queuing system now the problem that we run into is now we're publishing our messages off over the bus but what happens when we have a new model and our new model now wants to get our historical events through the service bus so now we need a control Channel it
• 39:30 - 40:00 needs to be able to talk to the thing on the other side to say hey by the way I want all my history and we got another problem what do you do with the events that are currently happening while you're replaying that history through the service bus the next thing you know you're in this massive pile of accidental complexity there's another way of doing this and it's a consumer-driven subscription which is what most people are using when they talk about event sourcing how many you heard of something called
• 40:00 - 40:30 Kafka Kafka is a consumer-driven subscription system and it works just like my blog how many of you have a Blog today how many of you read blogs okay so let we're going to use rabid mq now to distribute my blog so what's going to happen because we're going to work just like this when you want to read my blog you send me an email saying hi I'd like to read your blog can you create me a queue so I go through and create your queue for you now now now your email CL or sorry your client for the blog will
• 40:30 - 41:00 actually get the post from your your rabbit mqq cool but then you decide you actually want to go read my older posts so then you send me another email telling me to put all of the older posts into your queue for you this is what I mean when we say control Channel you need to have this other mechanism of talking back and forth with me in order to make this work a consumer-driven subscription works exactly like blogs when you come to my blog I give you an atom feed the atom feed says hey this is
• 41:00 - 41:30 where you currently are do you want to go to the oldest event this is where the oldest event is and then you can come forward in the Stream all the way up till you get to current maybe you want to work like Twitter so when you first come you only want to grab the last 50 that's perfectly fine as well the consumer remembers where they are in the Stream not the producer now what happens when I've got 200 things subscribing over a consumer cons based subscription well it's no problem what
• 41:30 - 42:00 happens if you have a consumer based subscription and you want to replay you want to pull down all of my old blog posts do I as a server care do I have to do anything this is the difference that we look at and most people are using service buses they end up in this trap where they end up building control channels around everything event sourcing is more complex more complex than
• 42:00 - 42:30 what um is it more complex to build an Event Source system than to build a typical crud app yes but not because of event sourcing it's because you have to go through and you have to figure out these things they're called use cases and you have to talk to people about how they want to use the software not just what data they want to store and manage and it's a different way of doing analysis if you talk about systems where you're actually going with a use case driven approach and comparing them event sourced versus non-event sourced
• 42:30 - 43:00 they're roughly equivalent in terms of effort to be fair if you come into an Event Source system for the first time you will have a learning curve you guys have been working with crud for how long you already know a lot of those little edge cases that you run into with crud you know how to handle them you know what the risks are how to deal with issues in production and there's a learning curve Associated to get the same information out this is one of my favorite questions what big companies use event
• 43:00 - 43:30 sourcing well basically all of your database companies do you're probably already using it you just don't know it and this is a really really bad way of making decisions um I I I actually die a little bit inside every time I read this on an email list because obviously if this big company is doing this then that makes total sense for us to do it because we're a big company and they're a big company people talk about Event Source systems
• 43:30 - 44:00 being slow and you can imagine if I've Got 5 million events and I have to replay 5 million events that's going to be really really slow but what if I were to store them in an actor and I would distribute my actors across a cluster and I would keep them all in memory when would I have to replay at this point when I lose one what if I were to drop a snapshot at every 1 million events so now we're C the most will ever have to replay is 1
• 44:00 - 44:30 million and that's only in a failure circumstance otherwise all my data is currently in memory does this sound like it'll be slow you can make these kinds of systems very very fast in fact if you start going you start looking at systems that need to be fast and they really care about things like latency oddly they tend to use these ideas things like trading one of the biggest ones that kills me is that people have now Associated event sourcing with object
• 44:30 - 45:00 orientation and in particular domain driven design to be fair an event sourced model is not objectoriented it is functional um when does an event change so we have entire immutable data and in order to get current state we left fold over whatever things that we have in the past this is obviously objectoriented yes but people have Associated Event Source
• 45:00 - 45:30 systems back to object-oriented and domain J and design based systems and it's totally not it is it is inherently functional it's a functional system of dealing with data storage and I've been asked this question a lot and the best framework that you're going to have is probably none when you start going back and you start saying I want to be able to replay these events to come forward okay it's a left fold do you really need a framework
• 45:30 - 46:00 to implement a left fold for you although to be fair there there are some guys here in orus that are building a cqrs framework so when their's breaks they they they live locally and you can annoy them so that may actually be a good one to look at and over time I've had a lot of people talk to me about this this data requirement and we're going to store everything we're going to pend everything in our system how many have ever looked at what
• 46:00 - 46:30 your current state is in your SQL database versus the size of your transaction log transaction log is a lot bigger normally but what you really need to ask yourself is not about how much data we're going to have what you need to ask yourself is at what current rate and what future rate will be we be retaining data and you need to take this and you need to bring it back and compare it to Mor's law if I were to go five years ago and buy a one terabyte SSD how much would that have
• 46:30 - 47:00 cost today it'll cost you about $500 I'm willing to bet you within three years we'll have two terabyte ssds so you need to figure out not only what is my retention but how does that curve fit when I I plot it next to Moors law and if you are slower than Mor law then guess what your data is going to get cheaper and cheaper and cheaper to store over time and there are some systems uh I can tell you today you need
• 47:00 - 47:30 to be sitting at about 3 to 5,000 messages per second before you actually have to start worrying about things once you hit that level you do need to worry about things going into the future if you're underneath that cheaper and cheaper and cheaper and there are some systems but literally how many of you guys have more than 10 transactions per second again we're like two or three hints most systems are doing very small of information and we are getting into premature optimization when we start
• 47:30 - 48:00 talking about all these kinds of issues and to be fair if I can put all of your data on a micr SD it's not big data now I've been talking with people about cqrs for a long time and cqrs was never really intended to be much of a pattern um I I actually agree with the functional people when they say that we don't have these things called patterns we just write code and I don't need a pattern for every single thing that we're going to
• 48:00 - 48:30 do cqrs the whole thing that was behind cqrs was really to make people start looking at their domains in a different way to lead them to everything else that we're talking about it also leads them to things like building out actors for their domain model which doesn't work very well if you're supporting all of your reads off your domain model with a om behind it once you start saying that we're going to our read models over here and we query them separately than what we actually write to everything turns and
• 48:30 - 49:00 it works out really well but the main thing I had to do in in bringing people towards this these ideas was to give them something to separate the reading and the writing part of the process without that they would never make the jump from objectoriented I I I use air quotes because they're not really objectoriented uh they're like C++ object orientation domain model two going all the way to something which is event sourced with read models on the side and cqrs has been a big part
• 49:00 - 49:30 of that but it's not something that you really need to go off and get really deep into um to be fair it's probably the dumbest pattern ever written if if you have a void return type you go here if you have a non-void return type you go there it's a whole pattern which also makes for a lot of fun when you hear people talking about how cqrs is crack for Architects and you're like uh it's probably the dumbest pattern ever written and that was already
• 49:30 - 50:00 there and I've gotten this one a lot um to be fair the only thing I can really say is thanks it's not Enterprise um at at the same time it will not cost you you know a half million dollars just to get in something really simple um you don't need a lot of the big Frameworks people are using or and if if you really want Enterprise stuff well IBM has a lot of stuff to
• 50:00 - 50:30 sell you and with that I'd like to thank all of you guys for actually coming out um I I do appreciate that everyone actually came to the talk and with that if anyone has any questions don't you have a problem when you need to decide which details to store in the event uh because you don't know which details you're going to need 5 years from now when you're going to build a new model okay that's fair um so
• 50:30 - 51:00 when we talk about the fact oh sorry um it was uh how do we determine which details to actually put inside of an event and it is the exact same analysis that you do for use case analysis um when we're we're talking about what is the fact that actually occurred inside of our system and what do we care about and this is absolutely an analysis problem and it takes some time uh of working with it to get it right um a normal follow-up question to this is what happens when what's in the the fact changes over time and how do I version
• 51:00 - 51:30 it over time um for I I'm sure someone will actually ask that question I always get it there's a 45 minute answer to that question um sorry uh it's on dddc qrs.ly recommend watching the entire video but that's unbiased um when we talk about facts over time and how they change generally what we
• 51:30 - 52:00 start talking about is using um hybrid or weak serialization in order to get that working as well but you can do it with strong serialization and I literally it's a 45 minute answer I was wondering about when you introduce uh event sourcing for already existing application already established changing domain model uh what would be the best uh way to kind of avoiding the Big Bang release cool that's actually a
• 52:00 - 52:30 great question um and when we talk about migrating existing data over to an Event Source system there are two ways of actually dealing with it the first way that we can deal with it uh and that's actually the same way I don't know if anyone here has ever worked with an accounting system so it's the exact same things that you would do with an accounting system so if at the end of the year I want to switch from Great Planes to sat for my accounting um I've got two options options I can either migrate all of my data out of Great
• 52:30 - 53:00 Planes and bring over all the transactions on all the accounts into the new system or I can bring over just the initial balance of all of my original data and and there's times that you will use both and for any problem you're going to use both of these Solutions on it um I will go and let let's just use DDD terminology quickly aggregate by Aggregate and some Aggregates I may I may bring over a snapshot uh a customer initialized event or customer
• 53:00 - 53:30 initialized from old system other ones I may be able to reverse engineer the history and in those cases I may actually bring over the history uh of the events as well and what this can allow me to do if I if I if I do it reasonably well I can actually start running side by side so I make the old system at least raising the events along the way and it may still be storing off to a third normal form as well but now the new system is also raising events what we can do is we can project those events back into the let's say third normal form
• 53:30 - 54:00 model so you can basically run side by side with these kinds of systems as you're moving um in terms of migration there are the two strategies um and you you can uh and you will use both of them concurrently um it's really a a thing that you come into and it's aggregate by aggregate or document by document depending how you want to look at it and you you will choose both strategies we're a little over time oh sorry okay
• 54:00 - 54:30 well then I will thank you guys for having come out and uh if you have questions I I know particularly in Scandinavia people like to ask questions oneon-one um so I'll be around probably most of the rest of the day today and at least in the morning tomorrow [Applause] thanks