MTM-64360 User guide for MQTT Service Pulsar client #3642
Conversation
|
Preview available here |
scmi-c8y
requested review from
bbyreddy,
rthrippleton,
rohitjoshi-c8y,
abor-c8y,
jato-c8y,
schmiel-cumulocity and
a team
content/device-integration/mqtt-service-bundle/pulsar-client.md
Outdated
Show resolved
Hide resolved
content/device-integration/mqtt-service-bundle/pulsar-client.md
Outdated
Show resolved
Hide resolved
| Microservices and external applications can connect to the {{< product-c8y-iot >}} Messaging Service to receive messages from devices connected to the MQTT Service, and send messages to those devices. | ||
| The Messaging Service is a modified deployment of Apache Pulsar, and your applications will use the Pulsar client protocol directly to connect to it. | ||
|
|
||
| ### Connecting to the Messaging Service |
There was a problem hiding this comment.
nitpick: A quick prerequisites section helps perhaps?
There was a problem hiding this comment.
Yes, good thought. What would you have as pre-requisites?
| weight: 30 | ||
| layout: redirect | ||
| title: Connecting microservices and applications | ||
| --- |
There was a problem hiding this comment.
thought: Could a simple architecture diagram showing how these components fit together help? Maybe too much to ask for now, and perhaps we can come back to it later.
There was a problem hiding this comment.
I was thinking about updating the diagrams we already have in the intro section of the doc. I guess I might be inspired to copy parts of those into the later sections and add more detail.
| <p> | ||
|
|
||
| * Code snippet showing how to publish | ||
|
|
There was a problem hiding this comment.
issue/suggestion: It might be useful to add a section on error handling so readers know how to deal with common error scenarios.
There was a problem hiding this comment.
Yes, that's another good idea. Not sure what I would put in it exactly... probably I should try to remember all the ways I mess up in writing some example code.
content/device-integration/mqtt-service-bundle/pulsar-client.md
Outdated
Show resolved
Hide resolved
| <p> | ||
|
|
||
| * Code snippet showing how to shut down cleanly | ||
|
|
There was a problem hiding this comment.
throught: Would it be helpful to include a short “common pitfalls” list (e.g., forgetting to acknowledge messages -> backlog quota filled)?
There was a problem hiding this comment.
That is really what this "best practices" section was supposed to be, and I think I already mentioned the consequences of not acknowledging a few lines up from here. It should be clearer when all these bullets are turned into actual prose.
Co-authored-by: Rohit Joshi <89976252+rohitjoshi-c8y@users.noreply.github.com>
content/device-integration/mqtt-service-bundle/pulsar-client.md
Outdated
Show resolved
Hide resolved
content/device-integration/mqtt-service-bundle/pulsar-client.md
Outdated
Show resolved
Hide resolved
content/device-integration/mqtt-service-bundle/pulsar-client.md
Outdated
Show resolved
Hide resolved
content/device-integration/mqtt-service-bundle/pulsar-client.md
Outdated
Show resolved
Hide resolved
content/device-integration/mqtt-service-bundle/pulsar-client.md
Outdated
Show resolved
Hide resolved
content/device-integration/mqtt-service-bundle/pulsar-client.md
Outdated
Show resolved
Hide resolved
content/device-integration/mqtt-service-bundle/pulsar-client.md
Outdated
Show resolved
Hide resolved
content/device-integration/mqtt-service-bundle/pulsar-client.md
Outdated
Show resolved
Hide resolved
Co-authored-by: Richard Thrippleton <richard.thrippleton@softwareag.com>
scmi-c8y
requested review from
a team,
rthrippleton,
rohitjoshi-c8y and
jato-c8y
|
@rohitjoshi-c8y it would be good to get your thoughts on the 4 comments of your that I left open. I might have covered these in the re-written text already, but if not I probably need some more details of what you were looking for. |
| To do this, your microservice or external application will connect to the {{< product-c8y-iot >}} Messaging Service, a modified deployment of [Apache Pulsar](https://pulsar.apache.org/), and use the Pulsar protocol to publish and consume MQTT messages. | ||
| The diagram below shows the important interfaces and data flows used when interacting with the MQTT Service through Pulsar. | ||
|
|
||
| <p align="center" width="100%"> |
There was a problem hiding this comment.
suggestion: I think this wonderful diagram should be moved/condensed/changed+reused for Overview / Architecture - the diagram there is very legacy-oriented. Maybe now, or maybe in a later ticket, not sure.
| The diagram below shows the important interfaces and data flows used when interacting with the MQTT Service through Pulsar. | ||
|
|
||
| <p align="center" width="100%"> | ||
| <img width="80%" src="/images/mqtt-service/mqtt-service-pulsar-connections.svg" alt="MQTT Service Pulsar connections"> |
There was a problem hiding this comment.
praise: Wonderful diagram, just the right amount of fiction to help a user model it in their mind.
There was a problem hiding this comment.
thought: Now that we're exposing a reliable messaging system more explicitly, used to communicate with another (potentially) reliable messaging service, I wonder if there's scope for confusion. As a user I might think that when I publish a message to Pulsar, when I get an acknowledgement from it it means that the MQTT device has acknowledged it. A similar delusion in the other direction perhaps.
Somewhere we might want to disillusion them of this, and be clear about it being decoupled. And another consequence of decoupling is a lack of errors - publishing to a non-existent MQTT topic or device via Pulsar will just complete and drop the message into a blackhole.
| The message will only be published to a client that is connected at the time the MQTT Service processes the published message. | ||
|
|
||
| In order to enforce device-level isolation, the message will be published **only** to the specific MQTT client identified by the `clientID` message property, provided that client has an active subscription to the relevant MQTT topic. | ||
| If the `clientID` property is empty, the message will be sent to **all** connected MQTT clients with active subscriptions to the MQTT topic. |
There was a problem hiding this comment.
Or at least that's what https://cumulocity.atlassian.net/browse/MTM-64283 says.
| If the `clientID` property is empty, the message will be sent to **all** connected MQTT clients with active subscriptions to the MQTT topic. | |
| If the `clientID` property is missing, the message will be sent to **all** connected MQTT clients with active subscriptions to the MQTT topic. |
Opening this up for the full review now that we have a complete first draft plus work-in-progress sample code in Cumulocity-IoT/cumulocity-examples#158.
The relevant part of the preview is: https://cumulocity-iot-c8y-docs-build-pr-3642.surge.sh/device-integration/mqtt-service/#pulsar-client
Some open questions that we should answer collectively, and make sure the implementation agrees with the doc:
clientproperty should be treated the same as empty, or as a bad message to be ignored?