Change data capture allows us to track changes made to a database and react.

We can observe changes written to a database and trigger downstream systems. This pattern is great for event-driven architectures as we can listen to changes and trigger downstream consumers or even replicate the data across our bounded context / microservices.

Listening to changes

• Change data capture allows us to listen to changes in a database.

• You can react to these changes to trigger downstream consumers or copy the data into other databases/views.

• It’s quite a common pattern to listen to changes in a database, change data capture can help us write some resilient, and scalable event-driven architectures, raising events for internal use but also domain events for business consumption.

• Change data capture patterns can help us reduce the amount of custom code we need to write. E.g. Rather than raising events straight after a Database insert, we can insert and let services/databases give us the change data capture event.

• Great example of this is using DynamoDB streams. Data is streamed to consumers in near real-time.

Keeping data in sync

• Change data capture can be used to keep data in sync across many boundaries of your architecture.

• Many architectures have many data sources, change data capture allows us to listen to a change on the “leader” and push these changes to “followers”.

Couple with other design patterns

• When consuming change data capture events, you may want to add layers in between the system consuming it, to enrich or filter the data.

• Example: if your consumer of the event requires additional information you can use enricher pattern to do this.

Replicate data across boundaries

• Your architecture may be split into many boundaries. You can use change data capture pattern to replicate data across boundaries making systems more resilient favouring availability over consistency.

• Replicating data may introduce eventual consistency, given no changes are made your database all listener’s will be eventually consistent. This means data changes across your architecture is async and can take time, it’s important to understand this.

Extra Resources

• Designing data-intensive applications - Book has a great insights into streaming and has a chapter around change data capture.
• Change data capture with DynamoDB - DynamoDB is a great NoSQL database that supports change data capture pattern. Many people are using this pattern to help scale their distributed architectures.
• Understanding bounded context - Your data may be replicated across your architecture, have you got the correct bounded context defined? Are you going back to producers to get information? Understanding bounded context can help.
• Migration with Change data capture - Maybe you are looking to migrate to an EDA architecture? Can you use change data capture pattern to help? Visual here to help you understand how you can use events to migrate.
• Unlocking value from your events - There is a huge amount of value in your events in your architecture, these include events from change data capture pattern. Can your business utilise these?
• Understand eventual consistency - If you are replicating data you may be favouring availability over consistency. It’s important to understand eventual consistency and what it means for your architecture.