Distributed Data Logger
The full article was originally published by Stefano Della Valle on Medium. Read the full article here.
Advanced MAM explorer for Industry 4.0 applications
Distributed Data Logger (DDL) was created with the purpose of extending and boosting the well-known Masked Authenticated Messaging (MAM) second IOTA layer, allowing a MAM channel reader to access additional features.
More specifically a MAM channel has these properties:
– it can be read only starting from a known root id (the unique message identification number) up to the last message in the channel;
– it can be read only sequentially, because the fetch function, given a root id, returns one message, a requested number of messages, or all messages in the channel starting from that root;
– there is no way to know in advance how many messages are stored in a MAM channel from a given root id, without reading all the channel up to the last messages.
These properties are not reducing MAM’s usefulness, but in some cases they are inconvenient.
For example both MAM writer and receiver, communicating on a channel for a long time, need to store the root id of the last message exchanged safely, in order to be able to start again from that point if the system needs to be restarted losing the state.
In doing this, they lose the opportunity to retrieve all the messages older than the last one they have managed.
To solve this issue, the first root id of the channel can be stored too, allowing the parties to start from the end or from the beginning to retrieve some data from the channel.
But what happens if the channel contains thousands of messages?
The transmitter is able to skip all the channel’s content and restart from the last root id, but the receiver, who needs to retrieve a specific message in a time frame, needs to process all the channel from the beginning with a lot of effort.
We found it really unpractical, and that’s why we started searching for a way to not only speed up the search, but also filter it: this is basically how our Distributed Data Logger came to life.
Distributed Data Logger looks like generic public MAM decoders and can actually decode a generic MAM channel like any other. If the messages in the channel (the MAM message payload) have a specific format however, DDL is able to provide some useful additional features.
The UI is pretty simple and intuitive. Distributed Data Logger lets you choose a provider (otherwise a default one will be used) and then asks you for the entity root (first root of your MAM).
Once you copy/paste the entity root id (the root id of a MAM channel produced by an entity) you only need to press the Tab and the tool will automatically check if your MAM has the right characteristics to unlock the advanced features and filters.
If not, you’ll still be able to use the standard fetch which is similar to the one of the MAM decoder
How to enable the advanced features?
The secret behind the boost is a simple JSON structure.
Every message published on the MAM will contain a field “count”, which basically represents its position in the MAM. Besides, starting from the first message, a field “prevRoot10” and “root10” will be added every 10 messages and will respectively contain the root of the 10th previous message and the root of the 10th following message.