Asynchronous Microservice Architectures Using Message Queues

In the past two chapters, you learned how to build REST-based microservices with the Go programming language. The REST architectural style is both simple and flexible at the same time, which makes it an excellent choice for many use cases. However, being built on top of HTTP, all communication in a REST architecture will follow the client/server model with request/reply transactions. In some use cases, this might be restrictive and other communication models might be better suited.

In this chapter, we will introduce the publish/subscribe communication model, along with the technologies that you need to implement it. Typically, publish/subscribe architectures require a central infrastructure component—the message broker. In the open source world, there are many different implementations of message brokers; so, in this chapter, we will introduce two different message brokers that we feel to be among the most important ones—RabbitMQ and Apache Kafka. Both are suited for specific use cases; you will learn how to set up each of these two message brokers, how to connect your Go application, and when you should use one or the other.

We will then show you how to use this knowledge in order to extend the event management microservice that you have worked in the previous chapters to publish an event whenever something important happens. This allows us to implement a second microservice that listens on those events. You will also learn about advanced architectural patterns that usually work well alongside asynchronous communication, such as event collaboration and event sourcing, and how (and when) to use them in your application.

In this chapter, we will cover the following topics:

• The publish/subscribe architectural pattern
• Event collaboration
• Event sourcing
• AMQP with RabbitMQ
• Apache Kafka