RabbitMQ Bus implementation

Introduction

As part of the Distributed Data annex, I shall integrate RabbitMQ into my Ramses project to enable scalability, minimize coordination, redundancy & readiness for evolution. Within Ramses a few integration patterns are used(there are more, I don’t know each one as there are over 40.):

  • Topics and subscription(relevant)

  • Point to point: Queues

  • Micro Services

  • Messaging

  • Transformation

Theory

Within Ramses users can place comments on games. Lets say a new game is released, and loads of comments are placed under that game–perhaps users are trying to figure out the solution to the game cooperatively.

https://i.imgur.com/faupq97.png

Using an implementation like RabbitMQ enables me to use benefits of the various integration patterns. Such as pub/sub.

  • It decouples subsystems that still need to communicate. Subsystems can be managed independently, and messages can be properly managed even if one or more receivers are offline.

  • It increases scalability and improves responsiveness of the sender. The sender can quickly send a single message to the input channel, then return to its core processing responsibilities. The messaging infrastructure is responsible for ensuring messages are delivered to interested subscribers.

  • It improves reliability. Asynchronous messaging helps applications continue to run smoothly under increased loads and handle intermittent failures more effectively.

  • It allows for deferred or scheduled processing. Subscribers can wait to pick up messages until off-peak hours, or messages can be routed or processed according to a specific schedule.

  • It enables simpler integration between systems using different platforms, programming languages, or communication protocols, as well as between on-premises systems and applications running in the cloud.

  • It facilitates asynchronous workflows across an enterprise.

  • It improves testability. Channels can be monitored and messages can be inspected or logged as part of an overall integration test strategy.

  • It provides separation of concerns for your applications. Each application can focus on its core capabilities, while the messaging infrastructure handles everything required to reliably route messages to multiple consumers.

Source: https://docs.microsoft.com/en-us/azure/architecture/patterns/publisher-subscriber

Execution

In the context of my application, I chose to implement RabbitMQ messaging between my two components GameService & CommentService. I deployed RabbitMQ docker-container through my Portainer. Through Portainer, this container can be vertically scaled. As opposed to having it configured in a horizontal scaling; why?

The requirements of Ramses reflects upon the requirements of the real-life Twitter application, which users are estimated to posting 6000 thousand tweets every second.

Source https://www.internetlivestats.com/twitter-statistics/

A single instance of RabbitMQ can easily handle around 40.000 msg/s per second.

Source https://blog.rabbitmq.com/posts/2012/04/rabbitmq-performance-measurements-part-2

With this in mind, I deployed the RabbitMQ docker container on my Portainer.

https://i.imgur.com/xvk2bwI.png

And set up the exposed ports.

https://i.imgur.com/1OEWJgn.png

I initially had two deployed containers for RabbitMQ, one for the management interface & one for the messaging bus, but found out that they weren’t linked together so I merged the two containers.

https://i.imgur.com/2tNeNAj.png

And here is RabbitMQ running in prod 🎊