Reduce pressure off downstream consumers

• Many people use message brokers to reduce back pressure from downstream services. The sender puts messages on the queue, and consumers can pick up these messages and process them in the batch size they want and the time they like.

• You can use message brokers to make sure that downstream services do not get too overloaded, which can improve system reliability.

• Example of this could be to use an SQS queue, to handle many messages, and a consumer to process these messages and delivery information to a third-party API.

Prevent messages/data being lost

• Many brokers offer the ability to retry messages/events if they fail to get processed or delivered. This can help prevent any information being lost.

• If your broker does offer retry and replay, you want to consider idempotency as your consumer may be triggered more than once, and you don’t want any unwanted side effects.

• This could be a great pattern, as you broker may do the heavy lifting for you (retry/reprocess), many are configurable and if all fails you can decide what you want to do with the messages after the retry period (example drop them or store them for later processing)

Parallel processing

• Some brokers offer pub/sub patterns allowing you to send events and notify downstream consumers. This pattern allows you to notify downstream processes that something has happened. Many downstream systems can listen to these events.

• If you are using message queues, you can still have many consumers processing messages from the queue, this allows you to scale processing downstream.

Reducing knowledge of systems

• Using messages/events to communicate between systems/services can help us decouple our architecture.

• Producers may not need to know about downstream consumers, this gives the producers the ability to isolated and focus within its own domain/boundary.

Extra Resources

• Designing data-intensive applications - Book has some great insights into streaming, and messaging. Highly recommened if you want to dive deep.
• Fundamentals of Software Architecture - A great book to learn more about the fundamentals of architecture, goes into event architecture and many patterns.
• 65 messaging patterns - Great resource to dive deeper into many different patterns. Highly recommended.