Local cache copy vs requesting data By David Boyne

Local cache copy vs requesting data By David Boyne

Local cache copy vs requesting data

When using messages/events to communicate between boundaries/services, downstream consumers may want to request more information, this is common when events do not contain all the information required for downstream consumers (e.g notification events).

There are a few patterns engineers follow to solve this problem, more commonly they may go back to the producing service to request more information, or they change the payloads of the events (event-carried state transfer) and let other boundaries keep a local cache copy of the information, removing the need to go back to the producing service to get information.

Storing a copy of the data

  • When consumers read events, they may require more information (depending what’s in the event). Rather than requesting this through an API, the contract of the event may change (larger events) and a local copy of this data is stored.

  • When the consuming service needs information outside it’s boundary it can look into it’s local cache copy and get what it needs without going outside (e.g back to the producer or API).

  • Let’s look at the visual example. The order domain has all the information it needs about the user (from the user domain). When changes occur about the user this information is broadcasted (e.g change data capture) as an event and the order domain consumes this and stores a copy of the information.

  • This means we are not coupled to the users domain which can improve our availability, but the trade-off here is the data within the orders domain can be inconsistent/stale and eventually consistent, something to consider if you using this pattern.

Requesting information

  • When consuming an event that does not have all the information required (e.g notification events), downstream consumers may request information outside its boundaries (e.g. API back to get user information).

  • Having consumers make requests outside it’s boundary can create a coupling between that service and the service providing the information.

  • In the visual we see the order domain requesting more information about the user, they do this using a GET request back to the user domain. This creates a coupling between the services/boundaries.

  • When orders domain requests this information, it can create a back pressure onto the user’s domain, if you have high thoughput this can impact your availability.

  • This pattern allows the order domain to get relevant and up to date information about the user, which improves the consistency of the data, but at a cost of availability.

Extra Resources

  • Local cache with Ian Cooper - Ian Cooper from Just Eat gives us an example of using event-carried state transfer to share information between boundaries and keep a local cache state of the data.

  • Shared data with Randy Shoup - Randy Shoup gives us some patterns about sharing data with messaging/events. Sync vs Async and much more, great talk to dive deeper.

  • Local cache with Derek Comartin - Great collection of resources here to dive deeper with Derek Comartin, if you want to learn more with code examples check this out.

  • What is CAP theorem - In this visual we talk about consistency and availability trade offs, this comes back to the CAP theorem, worth reading and understanding the trade-offs this theory represents us with.

  • Event first thinking - This is a talk I gave back in 2022, I speak about event design but also the trade-offs between event types which is relevant to this visual.

  • Understanding eventual consistency - When dealing with copies of data, you may want to be aware of eventual consistency, this visual helps you dive deeper.

Download EDA Visuals

Join over 8,000 others learning EDA and download all the EDA visuals directly to your computer. This document updates everytime a new visual is added so make sure you come back to get the latest.

Download now →
Diagrams and thoughts by @boyney123 to help you learn.