So, I have a distributed system, lets say 2 hosts. Both run RESTful web servers. I need to implement a subscribe/publish mode for HostA to HostB.
I am anticipating a low traffic on the communication channel. My first thought is to implement a REST endpoint for subscription on HostA and for HostB to expose a REST endpoint to receive updates.
How does it sound? What other options do I have, lets say if there high level of updates that need to get published.
Thanks in advance.
If you have low traffic, then using a ready-to-scale later solution could still be cheap, and won't force you to re-factor your solution when scaling issues will start knocking at the door.
Google PubSub, AWS SQS are there to provide easy solution for you. If you want to avoid vendor lock-in, RabbitMQ sounds like a very reasonable choice, and could be cheap for low volumes (look at CloudAMQP plans, for basic but fast and reliable pubsub, the free plan could maybe be enough for you (and Rabbit MQ libs are available in any languages) There's a small learning curve, but then you'll be set for a long while!
All the responses seem to lean toward some kind of socket based solution, if i am understanding the comments correctly. I guess I just have a problem with stateless servers communicating to each other using some persistent mechanism. Is this bias baseless?
Sounds like something socket.io would be good for. It uses websockets for async sub/pub schemes. It's free and awesome. If you want a hosted solution, pusher.com is another good option.
Philip Davis
Software developer
There are several good answers here already (Google PubSub, AWS SQS, etc).
I'll just add that you can do pub/sub through Redis as well. i.e. don't make your servers pub/sub to eachother (e.g. what if one crashes, needs to be updated, etc)... instead, make them pub/sub to a Redis instance (or AWS Elastic Cache if you don't want to be tied down to AWS SQS).
http://redis.io/topics/pubsub