Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel...
Transcript of Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel...
![Page 1: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/1.jpg)
IDA Driving IT Århus 2020Thomas Bøgh Fangel @tbfangel
Event Sourcing asthe Foundation of Traceability
![Page 2: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/2.jpg)
@tbfangel
MeThomas Bøgh Fangel
@tbfangel
Architect at Lunar since 2016
Distributed systems since 2004
Java, Scala, Typescript, Go
Event Sourcing for about a year
![Page 3: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/3.jpg)
@tbfangel
Agenda● Building a bank from scratch○ Tech vision
● Event Sourcing○ Why & How○ PatternsAgenda
![Page 4: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/4.jpg)
@tbfangel
100+Employees
25%Engineers
1M+Tx pr month
~100μ services
150kUsers
3K8S clusters
LunarFounded
2015Live2016
Smartphone onlyChallenger Bank
![Page 5: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/5.jpg)
@tbfangel
Building a bank from scratch
![Page 6: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/6.jpg)
@tbfangel
CardProcessor
NationalClearing
What does it mean to be a bank?
Pay with card
Transfer money
Keep money safe
![Page 7: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/7.jpg)
@tbfangel
Trustworthy Secure
Correct
Tech Vision
![Page 8: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/8.jpg)
@tbfangel
Traceability at all levels
nothing should happen without us knowing and our system should never be in a state
we cannot explain
Tech Vision
![Page 9: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/9.jpg)
@tbfangel
≠Traceability
Distributed Tracing
![Page 10: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/10.jpg)
@tbfangel
func DoSomeBusiness(s *State) (Result, error) {
if s.InImpossibleState() {//this should never happen - what to do?panic(“current state is impossible”)
}//happy cases below
}
![Page 11: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/11.jpg)
@tbfangel
![Page 12: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/12.jpg)
@tbfangel
Production=
The place where the impossible happens
![Page 13: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/13.jpg)
@tbfangel
Traceability=
Explain the impossible
![Page 14: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/14.jpg)
@tbfangel
“Something really awful happened to your money - but
we know exactly where the money is and we will fix it”
![Page 15: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/15.jpg)
@tbfangel
Event Sourcing
![Page 16: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/16.jpg)
@tbfangel
Shift of Focus
From state…...to events
![Page 17: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/17.jpg)
@tbfangel
Event Sourcing Components
Storage
Event stream
Projection
Side effects
Handler
Event stream
Projection
+
+Business logic
(pure functions)
Aggregate root
![Page 18: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/18.jpg)
@tbfangel
Implementation● Apogee: ES & CQRS* library in Go
(Open Source in 2020)● Postgres as event storage● In-memory views for AR● SQL backed views
* CQRS: Command Query Responsibility Segregation
![Page 19: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/19.jpg)
@tbfangel
Patterns
![Page 20: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/20.jpg)
@tbfangel
Adapter NationalClearing
Public APIs
Bank Coreasync sync
![Page 21: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/21.jpg)
@tbfangel
External Event Streams● Public API● Same guarantees and properties
as internal event stream● Derived from internal event stream● Essentially a handler writing to an
event stream ● May keep track of source events
![Page 22: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/22.jpg)
@tbfangel
External Event Streams
Storage
InternalEvent stream
Projection
Handler External Event stream A
External Event stream B
![Page 23: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/23.jpg)
@tbfangel
Distributed flows● Distributed transactions● Represented as aggregate roots
(state machines)● State determines handler action● Side effects in handlers● Public event streams as API
![Page 24: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/24.jpg)
@tbfangel
TRANSFER
Bank Core
TRANSFER
DK Adapter
CLEARING
ACCOUNT
TRANSFER
Handler Aggregate
Request Account Bank Transfer Adapter Transfer
1. TransferInitiated FundsReserved
1.
![Page 25: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/25.jpg)
@tbfangel
TRANSFER
Bank Core
TRANSFER
DK Adapter
CLEARING
ACCOUNT
TRANSFER
Handler Aggregate
Request Account Bank Transfer Adapter Transfer
1. TransferInitiated FundsReserved
2. Initiated
2.
![Page 26: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/26.jpg)
@tbfangel
TRANSFER
Bank Core
TRANSFER
DK Adapter
CLEARING
ACCOUNT
TRANSFER
Handler Aggregate
Request Account Bank Transfer Adapter Transfer
1. TransferInitiated FundsReserved
2. Initiated
3.
![Page 27: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/27.jpg)
@tbfangel
TRANSFER
Bank Core
TRANSFER
DK Adapter
CLEARING
ACCOUNT
TRANSFER
Handler Aggregate
Request Account Bank Transfer Adapter Transfer
1. TransferInitiated FundsReserved
2. Initiated 4. Initiated
4.
![Page 28: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/28.jpg)
@tbfangel
TRANSFER
Bank Core
TRANSFER
DK Adapter
CLEARING
ACCOUNT
TRANSFER
Handler Aggregate
Request Account Bank Transfer Adapter Transfer
1. TransferInitiated FundsReserved
2. Initiated 4. Initiated
5. TimedOut
5.
![Page 29: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/29.jpg)
@tbfangel
TRANSFER
Bank Core
TRANSFER
DK Adapter
CLEARING
ACCOUNT
TRANSFER
Handler Aggregate
Request Account Bank Transfer Adapter Transfer
1. TransferInitiated FundsReserved
2. Initiated 4. Initiated
5. TimedOut
6. Succeeded
6.
![Page 30: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/30.jpg)
@tbfangel
TRANSFER
Bank Core
TRANSFER
DK Adapter
CLEARING
ACCOUNT
TRANSFER
Handler Aggregate
Request Account Bank Transfer Adapter Transfer
1. TransferInitiated FundsReserved
2. Initiated 4. Initiated
5. TimedOut
6. Succeeded
7.
![Page 31: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/31.jpg)
@tbfangel
TRANSFER
Bank Core
TRANSFER
DK Adapter
CLEARING
ACCOUNT
TRANSFER
Handler Aggregate
Request Account Bank Transfer Adapter Transfer
1. TransferInitiated FundsReserved
2. Initiated 4. Initiated
8. Succeeded 5. TimedOut
6. Succeeded
8.
![Page 32: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/32.jpg)
@tbfangel
TRANSFER
Bank Core
TRANSFER
DK Adapter
CLEARING
ACCOUNT
TRANSFER
Handler Aggregate
Request Account Bank Transfer Adapter Transfer
1. TransferInitiated FundsReserved
2. Initiated 4. Initiated
9. TransferCompleted ReservationRemoved TransactionPosted
8. Succeeded 5. TimedOut
6. Succeeded
9.
![Page 33: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/33.jpg)
@tbfangel
TRANSFER
Bank Core
TRANSFER
DK Adapter
CLEARING
ACCOUNT
TRANSFER
Handler Aggregate
Request
9.
1.
2.
3. 4.5.+ 6.
7.8.
Account Bank Transfer Adapter Transfer
1. TransferInitiated FundsReserved
2. Initiated 4. Initiated
9. TransferCompleted ReservationRemoved TransactionPosted
8. Succeeded 5. TimedOut
6. Succeeded
![Page 34: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/34.jpg)
@tbfangel
Error handling● Failures are 1st class domain citizens● Idempotency* crucial both internally and
externally ● Only measure against timeouts and
crashes
* Idempotency: system state remains the same after one or several calls
![Page 35: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/35.jpg)
@tbfangel
Tech VisionRevisited
![Page 36: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/36.jpg)
@tbfangel
What about Correctness...?● 100% correctness is impossible● Understand your errors, then fix● Traceability leads to correctness by
explaining errors● Error correction is just another event
![Page 37: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/37.jpg)
@tbfangel
… and Consistency?● Ordered event streams● Empowers consumers ● Eventually is better than maybe
![Page 38: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/38.jpg)
@tbfangel
Thank You!https://tech.lunarway.com/blog/
https://tech.lunarway.com/talks/
https://tech.lunarway.com/opensource/
![Page 39: Event Sourcing as the Foundation of Traceability · Event Sourcing for about a year. @tbfangel Agenda Building a bank from scratch Tech vision Event Sourcing Why & How Patterns Agenda.](https://reader036.fdocuments.in/reader036/viewer/2022071113/5fe9cbfe73f6cb0c99009d62/html5/thumbnails/39.jpg)