So, I am a junior Java developer for a small to medium company and during our development, my 1 up has said we should really have a JMS system.
Now during my degree and the little googling I could do, I have a rough idea of how they work and am aware of implementations and frameworks that implement them.
The main issue I am having is grasping realistic situations where they can be applied. To clarify what I mean, a part of me says, use them for everything that accesses the db so that you are always providing feedback and just have the front end keep checking.
In a realistic sense that doesn't sound practical. I've been having some difficulties sourcing good materials that cover Messaging queues, and wonder if anyone can provide a concise way of summing them up, or point the way to a decent source that gives a good thorough explanation.
Mario Hartman
Software and coffee lover!
I've killed JMS in all our projects, AMQP is much more reliable and also works across the board across a large number of languages, not just on the JVM like JMS does with some great other features you don't get in JMS.
Where to use it? Everywhere you can where it makes sense to do so.
Need to send an email? Throw it on the queue, now it's no longer your application's responsibility and another application that deals only with email can pick up the email and send it. Need to register a user, throw it on the queue, let the accounts application register that user for you. Need to update account details? Throw an UpdateAccountDetailsRequest on the queue. Need to get a list of all stock that has not been sold? Throw an UnsoldStockRequest on the queue and wait for an UnsoldStockResponse from the queue.
With AMQP you get the further benefits of exchanges ... need to notify all applications that they need to refresh their cache? Send a RefreshCache request to a fanout exchange and all applications will receive the RefreshCache request. Want to get notified every time an error occurs in any of your applications? Throw all the logs on the log queue and add a subscription that listens for error messages on the log exchange which will forward you all error messages to you application in realtime.
When building a Service Orientated Architecture, every action you do becomes a message you can throw on a queue. The nice thing about this is that you get to decouple your application pieces to a point where all you need to know is where the queue is sitting and what format the message needs to be that you want to send. I don't care how that email is sent, as long as it is done - my way of getting that email sent is simply by throwing an EmailRequest with the content on the queue and my code can continue without caring how it is done or who does it or when it's done.
Read the book Enterprise Integration Patterns, it's packed with predesigned solutions that all requires some sort of message queue whether it's AMQP, JMS, TIBCO, MSMQ, etc
Im no Java developer (I come from the NodeJS world) but I use a lot of messaging queuing. This pattern is used to delegate tasks to other services, while yours keeps responsive as much as possible.
Let's say we are building an API that allows signed in users to upload a picture and add some fancy filters to it (like black and white and other instagram like things), also see other users and their albums as well.
Well, image processing can take some heavy processing and if we leave it for the main thread (or main API) to handle this you will only be able to process one image at the time. Not only that, no users will be able to signup or see other's albums because the API is busy applying some fancy filter to a picture using its main thread.
How to solve this? a message queue. A user wants to apply a filter to a picture, the API issues a B&WFilterTask for that picture and places it on a queue.
In the mean time, the user can get some kind of loading message telling him that the image is being processed. At this point your API is not busy because it trusts the task to something or someone else (a worker) who will handle it so it can allow more users to signup, get pictures from albums etc, you get the point.
But... who will handle that task? That could be another service or a worker which tells the API when the task has been completed to notify the user.
How cool is this? it is very cool and convenient! Lets say your site becomes super famous and thousands of users now upload pictures and apply styles to them. The only thing you need to do now is to spawn more workers to grab more tasks from the queue, and kill them (sadly but true) when you dont need them, allowing you to scale and save money.
This same pattern can be applied to the Signup emails you are sending to your users when they create their accounts.
Delegating these tasks allow you to scale your system, keep it responsive and also cost effective as you grow when needed.
We can improve even more... Lets say... NodeJS or Java is not the best for heavy processing so the workers built with it are slow... we could totally rewrite the workers in Python or any other language that is a lot faster for image processing and just let it take tasks from that queue.
Now you have an amazing hybrid service that uses the best-fitted programming languages to do the heavy-lifting and take advantage of them.
Message queues are a big thing...
You should use a queue wherever you might otherwise experience a bottleneck in your application. A common example is handling image uploads... this can include resizing an image to various standard sizes. This is an example of a compute heavy task, there are similar tasks that may take a lot of memory, time or effort, and can be serialized away. Another example may be a persistent logging system, if you have spikes in activity, and MQ can act as an output buffer.
You have limited resources available for these types of tasks, and you don't want to slow your main service(s) for these types of work. This work can be offloaded efficiently.
Beyond these obvious examples it depends on load. You may want to inject an intermediary service that acts as a caching layer for certain types of requests, and an MQ can allow for distribution of the work (RabbitMQ supports an RPC request type that's VERY useful here).
In the end, it's best to avoid these types of systems as much as possible as they add complexity to the system, which comes at a cost of administrative overhead. Keep things as monolithic and simple as much as possible, and only worry about adding real complexity when it is really needed. Most systems are replaced long before they actually need some of these kinds of patterns in place. You can also scale vertically a LOT before you need such system distribution with modern computers (and even those of the past 5-6 years).
Hi, I have been working on systems in that the usage of messages queues it's almost mandatory. I will talk about messages bus or messages queues instead of JMS that it's an implementation of this.
I think that main usage of messages queues it's when you want to release the processing thread from the message handling, i.e let's say that you have a trading system that receives orders from customers, each time that you receive an order you need to parse the data, validate the fields and sending to the market if you do all this in the incoming thread you will block that thread for this customer so the user won't be able to post a lot of orders per second, so if you want to fix that, instead of do all that processing in the incoming thread you can just parse the order and put it on a queue so you release the thread and you are ready to receive a new request. Then another thread in your app (or in another app) will take the messages from the queue and process them.
You will find a lot of scenarios like this one where messages queues are the best approach.
And more these days where the idea it's to move from monolithic apps to micro services.
Hope this give you a little more details of real apps.
Regards
Mario