“Guaranteed Delivery” Messaging - advise : MQTT or ZeroMQ?

Asked 1 year, 8 months ago Active 8 months ago Viewed 3k times

We need a lightweight client based messaging solution. We used AMQP, RabbitMQ before, but in C++ we
have problems.
1 We would like to choose ZeroMQ with malamuteserver or MQTT ? Our IoT will publish data ( 45 kb )
almost every 5 min.
We need to deliver this message 100% and do not want to loose any message.
1 We tried MQTT QoS level 2, but when server disconnected or main server client has a problem we are
loosing published messages.
We need exactly RabbitMQ task / worker model. Messages should queued in the server until consumers
connected if anything happens.
Any suggestion, direction and examples welcome.
P.S.: This will be production so we want to chose less problematic way :)
Thanks a lot.
c++ rabbitmq mqtt zeromq distributed-system

edited Jun 2 '18 at 12:57 asked Jun 2 '18 at 11:02

user3666197 2adnielsenx xx
23.4k 3 34 70 368 3 12

2 Answers

I think MQTT is overhyped , sure the result I think is due to available open source servers. Zeromq does
offer a lot of features to build something that will meet the requirements. The more I look at the options
3 available , the more i am leaning towards zeromq. In our case, we will be receiving data at random
intervals from a very large number of devices ( gateway's in a mesh network or End nodes themsevles ).
Our finalized message size is very simple , a delimited string , binary encoded , zipped ( <100 Bytes) and
sent over the wire. I am decided on zeromq on the server to receive messages. Reasons are not merely
based on zeromq as a broker but also how We can utilize its curveZmq features to make provisioning of
devices easy and allowing for a simple ZTP (Zero Touch Provisioning system) and key manageability . I am
at the preset time debating on using pub/sub vs push /pull patterns where in each end device is a
publisher or pusher with a proxy subscriber on the cloud server. while typically pub in a pub/sub there are
fewer publishers and more subscribers in a typical large scale IOT deployment there are more publishers
and fewer subscribers so it makes me wonder if i should go with pub/sub and there there is the problem
of loosing messages due to slow joining subscriber - what if we bring down the server for maintenance ,
devices in the field would keep publishing messages until HWM is reached. Guess there is always a risk of
loosing messages irrespective of anything ex - backhaul network is down and the device Hits the HWM -
this is out of control.
Malamute does not have much documentation else i would have explored it a bit more.
 So , have you decided on what to use ? If you want to persist messages until they are consumed , i
strongly suggest zeromq as a proxy with workers pushing messages into a persistent store .. you can get
creative here as well with including a sequence # etc..and allowing clients to request messages given a
sequence range etc if they are lost .
answered May 25 '19 at 20:28
Chansdad
60 4

thanks for your time and elagent informative reply. I will surely take a look and test zmq. what is worried me that
development looks like little bit old. –  2adnielsenx xx May 31 '19 at 9:42

A: We need to deliver this message 100% and do not want to loose any message.
Messages should queued in the server until consumers connected if anything happens.
1
B:
C: This will be production so we want to choose a less problematic way :)
A: is doable
A: + B: is doable, harder, yet still doable
A: + B: + C: is not, this composition of requirements does come at cost

D: Any suggestion, direction and examples welcome.

ZeroMQ comes at hand, as being lightweight, out of question, tunable / tweakable way beyond the cited
throughput of ~ 45 [kb / 5 min] , yet the Devil comes at proper understanding of the strenghts of the
Zen-of-Zero, the package, as-is, by design strives to provide ZERO WARRANTY and lets all kind users
design their own, use-case-specific ( read as a "just-enough"-warranty one needs, so not losing a single
bit of efficiency for the rest of the world use-cases, gee aha ).
So, the D: goes into spending a proper amount of design-efforts for covering the "costs"-of- C: and
you have met the design target.
So easy, Maria, this is a common task for any and all CTO-s to face this and decide next steps.

Bonus part :
If in a need to minimise the IoT-devices' requirements, may go and compare these costs -of- C: with a
similar custom-adaptation of even a more lightweight framework for Scalable Formal Communication
Pattern Archetypes, that was designed by Martin SUSTRIK et al, as a younger sister of the ZeroMQ one --
the nanomsg , there might be some saving on low-power / scarcer resources, as commonly present in
massive cohorts of IoT-devices.

Anyway, keep us posted, where have your Production implementation turned out to be, ok?
answered Jun 2 '18 at 12:56
user3666197
23.4k 3 34 70

Any comparison between MQTT and ZeroMQ regarding to this A,B,C concept ? – Rahibe Meryem Jun 2 '18 at 19:43
Highly active question. Earn 10 reputation in order to answer this question. The reputation requirement helps protect
this question from spam and non-answer activity.