About RabbitMQ

1. RabbitMQ is a message broker written in Erlang. 
2. It is open source but commercial support is offered. 
3. RabbitMQ uses standardized messaging protocols. 
4. A protocol is similar to like hdp is a protocol just optimized for messaging. 
5. The protocol supported out of the box is AMQP(Advanced messaging queuing protocol) version 0.9 .
6. The other protocol are supported using the built in plugins system. 
7. There are RabbitMQ client framework available but because RabbitMQ uses the standard protocols you can interact with it using any programming language even the javascript. 
8. RabbitMQ also supports clustering ensuring the high availaibility of the broker service even if the hardware is failing. 

Messaging Scenarios

Besides setting out the complete microservices architecture , messaging can help out in many scenarios.

1. If you want the legacy applications to talk to each other without the data loss , messaging with queues is ideal. There are libraries from many programming languages for your message broker that lets you do that.

2. When data is coming from many places and need to power a dashboard Apps than messaging queue can easily be accumulated to give you real time insight on your dashboard scheme.

3. If you have big monolithic app and want to simplify it without rewriting it all at once first isolated one feature or layer to be handled by a service. Handle the communication between the monolith and new service with the messaging. Once done successfully take the next layer till the app is maintainable again. 

Microservice Architecture Example

The customer submits a new order from the WEBUI. From WEBUI a message has to be sent to the registration service which stores the order once completed it sends the message to the notification service confirming the customer that the message has been registered for example like sending an email.  

When to use the Microservices Architecture

The Architecture must have the capacity to scale. The project would start small but expectation is that they will grow fast. The Architecture must be flexible enough to add and remove features with minimal impact. Microservices are ideal for that because new services and old services can easily be added or removed if unsuccessful. Also when the demand rises enabling more capacity for each service should be an easy job. Since the messages are in the queue and services are picking up and processing messages one by one scaling the capacity up can be done by scaling another instance of the same microservice. Also the changes are rapid based on the requirement so deployment should be easy and reliable without affecting the overall system since each service is autonomous its easy to deploy it.

Much easier than deploying one big monolithic application. Also since there are many developers involved in the building of the whole system making all of them work on the same monolithic code would turn out to be a mess. With Microservices teams can be formed to work on microservices completely separate from other teams. But the biggest challenge is that the whole process must be reliable. Like if the orders and packages goes on missing its not an option. Since the message sits in the queue until the service processes it the reliability increases. Obviously we have keep focusing that the broker service keep functioning using clustering for example or else it becomes the single point of failure. 

Understanding the Microservices Architecture

Micro-services is building style for building distributed systems. Each service is a small independent process. Using the messaging service the microservices can be decoupled from each other. This is so decoupled that it is not even necessary to use the same programming language for each service. That's why the API has to be language agnostic. A message broker facilitates this language agnostic requirement. The services are called microservices because they are typically small. They focus on doing one isolated task in a distributed system. Each service should be autonomous. Microservices should keep the code sharing to minimum and they should have there own databases which doesn't need to be of the same type. One can use the Relational database while for the other service a nosql database might be more suitable. And because the each service is autonomous the building of the distributed system is very modular. Each microservice can be build by a separate team and can be separately deployed.

There is an important question over here that if each microservice has its own database what about the data integrity. In microservice architecture  each microservice maintains its own version of the entity with only the relevant data which is received with the message. For eg the registration service gets a new order from the user interface because it must be capable of producing the list of received orders later it stores the order in the database but it only stores the data necessary to produce the list. It than sends the message to the dispatcher and the finance service using a message broker. In the message it only keeps the data which the receiving service needs that service than processes the order and keeps the data of order it needs later. In this way the definition of order is different for each service. There is no single database which keeps all data that is to know about the order. The order data is distributed across the different microservices.

Messaging Patterns

The Messaging Patterns:-

1. Point-to-Point:- It sends the message directly one service to another specific service. The purpose of the message is to tell the service to do some task thats why it is known as command. For example a message register order which gives the command to register the order to service. For the service sending the message this is fire and forget the action. When the message is sent the processing is done by one or more services asynchronously. The sending service is not waiting for the result.

Introduction to Messaging Queues

A message broker is a intermediately between the services. It is capable of sending messages to and receiving messages from the services. It is the application running on some server and sending and receiving of the message is done with some API.

When a services sends a message to some other service it doesn't directly call the recipient service. It sends the message to the message broker and the message broker has internal mechanism to route the message to the receiver service. It doesn't send the message directly to the receiver service. But it holds it until the receiver picks the message up .

For example, if you consider the people as the services and the messages as the letter you send to other people. While sending the letter you do it through the mail service which act as the message broker which in turn routes the message to the receiver but the mail service doesn't handover the message to  the person personally but it is being stored in the persons mailbox until he is ready to take the message out. Such a mailbox is call the queue. It operates using the first in first out principle. 

Creating multiple S3 Buckets using AWS Cli

If you want to create multiple buckets in AWS than it can be created using the AWS Cli specifying the bucket name and the region the bucket should be created.

Create a text file specifying the list of the buckets name which are required to be created. For e.g

 cd /tmp  

 vi bucketlist.txt

We are going to create the buckets in the Singapore region , you need to change the region name if you want to create it in some other region

 for i in `cat bucketlist.txt`;do aws s3api create-bucket --bucket $i --create-bucket-configuration LocationConstraint=ap-southeast-1 ; done