Not possible to access nested JSON MQTT data

[acpenela]

Apache StreamPipes version

Other StreamPipes version (please specify below)

Affected StreamPipes components

Processing Elements

What happened?

When a MQTT adapter with a nested JSON data format is configured and started, the data isn't accessible in the pipeline nor possible to visualize in Dashboard.

Data schema configuration:

Live preview of the data:

Even using a pipeline element, such as javascript eval, we cannot access the properties of nested objects.

When creating dashboard visualizations, it´s not possible to access values from nested object properties.

How to reproduce?

Create a MQTT adapter in Streampipes with nested JSON data. In the pipeline try to view (using live preview) or access atribute values in the pipeline. After creating and starting both the adapter and the pipeline, try visualize object property values in dashboard visualizations.

Expected behavior

In the pipeline access values from nested object attributes.
In the Dashboard, access and visualize data using nested objects.

Additional technical information

Streampipes version - 0.95.0
On a dashboard, if we select the raw widget to display the message, the nested objects appear as java.util.map, which implies that they are passing through the pipeline , but we have no way of accessing the data.

Are you willing to submit a PR?

No, I don't want to submit a PR.

[tenthe]

Hi @acpenela,

thanks for opening this issue.

Nested structures are not completely supported in all StreamPipes components. However, you can flatten the events in the adapter, enabling you to move nested properties to the top level.

In the schema editor (as shown in your screenshot), you can drag and drop the entries to the top level, placing them next to the timestamp. As a result your event shoud look like this:

{
  "eixo_1": 4
  "eixol posição": 0.15,
  "eixol_velocidade": 17.1,
  "eixol aceleração": 1.6,
  "eixo1_ temperatura": 50.7
  "timestamp": 171992775938
}

If you encounter any issues during this process, please let us know. Additionally, we are planning to include a flatten feature in the adapter to assist you with this process in the future.

It is also possible to change the runtime names of your properties by clicking on Edit field.

Cheers,
Philipp

[KunJakob]

@tenthe Can you show exactly how you do that? I am not able to flatten a JSON structure in the adapter in any way. Dragging the elements to the root does not cause a transformation in the output.

[tenthe]

Hi @KunJakob,
Thanks for reaching out! I just tested it, and it looks like this feature is currently not working.

When I adjusted the structure using drag and drop, my event in the preview broke.

Is this the same issue you're experiencing?

[KunJakob]

@tenthe thats exactly what I've had, yes!