Skip to content

Commit 66a853d

Browse files
VivianBassVivianBass
committed
add guide to use jq and jsonpath
1 parent 776cdb9 commit 66a853d

File tree

6 files changed

+211
-1
lines changed

6 files changed

+211
-1
lines changed

Diff for: ‎SPA_howto.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
layout: default
33
title: Graphical UI
44
parent: For users
5-
nav_order: 3
5+
nav_order: 4
66
permalink: /usage/spa
77
---
88
# Graphical UI

Diff for: ‎figures/jp1.png

22.8 KB
Loading

Diff for: ‎figures/jq1.png

105 KB
Loading

Diff for: ‎figures/jq2.png

117 KB
Loading

Diff for: ‎figures/jq3.png

58.9 KB
Loading

Diff for: ‎jq_jsonpath.md

+210
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,210 @@
1+
---
2+
layout: default
3+
title: GraphQL filtering using jq and JsonPath
4+
parent: For users
5+
nav_order: 3
6+
permalink: /usage/graphql/filtering
7+
---
8+
# GraphQL & API
9+
{: .no_toc }
10+
11+
## Table of contents
12+
{: .no_toc .text-delta }
13+
14+
1. TOC
15+
{:toc}
16+
17+
---
18+
19+
20+
## What is jq?
21+
22+
[jq](https://stedolan.github.io/jq/) is a lightweight and flexible command-line JSON processor. It is used to extract and manipulate data from JSON documents with simple and powerful filters, and it allows you to transform JSON data in various ways.
23+
24+
Why use jq?
25+
- **Filtering**: You can quickly filter, slice, and extract specific parts of JSON.
26+
- **Transformation**: jq allows you to modify JSON structures, making it useful for cleaning, formatting, or transforming JSON data.
27+
- **Simplicity**: With a simple syntax, you can do complex manipulations easily.
28+
29+
### Why jq is used with GraphQL:
30+
31+
When working with GraphQL, the response data is returned in JSON format. Since GraphQL allows for flexible queries, the structure of the returned data can vary significantly, making it necessary to manipulate the response data for specific use cases. This is where jq comes in.
32+
33+
After querying a GraphQL endpoint, you might need to isolate or transform the returned JSON data for processing. `jq` makes this easy, enabling you to select only the necessary fields from the complex JSON response.
34+
35+
## What is jsonpath?
36+
37+
[jsonpath](https://goessner.net/articles/JsonPath/) is a query language for JSON, similar to XPath for XML. It allows you to extract specific parts of a JSON document based on a path expression. jsonpath is often used when interacting with JSON data in various programming environments.
38+
39+
Why use jsonpath?
40+
- **Simplifies JSON querying**: It enables easy navigation and extraction from nested JSON structures using path expressions.
41+
- **Integration with APIs**: Many tools and libraries support jsonpath for working with JSON-based APIs.
42+
- **Efficient Data Extraction**: With jsonpath, you can extract only the necessary data, saving time and resources.
43+
44+
### Why jsonpath is used with GraphQL:
45+
46+
GraphQL responses are JSON-based, and the data can often be deeply nested or include numerous fields. jsonpath provides a simple and readable way to extract specific data from this structure, making it an excellent tool for working with GraphQL results.
47+
48+
## Example Usage
49+
50+
In real-world scenarios, especially when dealing with GraphQL APIs, tools like `jq` and `jsonpath` can help you interact with the responses more efficiently.
51+
52+
Zendro GraphiQL integrate these filtering techniques directly into their user interfaces.
53+
54+
### Filtering in Zendro
55+
56+
Zendro's GraphiQL interface provides a flexible and interactive way to query the data. Within this interface, you can apply filters directly to your queries to refine the response data. The interface allows you to easily request only the fields you need and presents the data in a JSON format.
57+
58+
In the results, you can see the JSON output, which can be further processed with `jq` or `jsonpath` for specific tasks, such as:
59+
- **Filtering Specific Fields**: You can extract only the fields that matter to you from the GraphQL response.
60+
- **Nested Data Extraction**: The interface allows you to deal with nested data structures, and with additional tools like `jq` or `jsonpath`, you can query deeper levels of the data tree.
61+
62+
Once you have the query, at the top of the interface, there are several buttons. If you click on `Filter`, a section will open at the bottom of the screen where you can select the option you want to use: `jq` or `JsonPath`, the space to write your filter, and the window where you can see the results.
63+
64+
65+
![Filter.png](figures/jq1.png)
66+
![Filter_options.png](figures/jq2.png)
67+
68+
The filters you use will be applied to the query results.
69+
70+
You can reproduce the results by accessing to the [Zendro-BrAPI data warehouse](https://brapi-graphiql.zendro-dev.org/).
71+
72+
### Filtering with jq
73+
74+
For example, if we want to get the names of the studies in the trial example, since we are only calling 2 `(limit:2)`, we expect to have 2 names:
75+
76+
![Filter_example.png](figures/jq3.png)
77+
78+
#### Explanation:
79+
80+
- `.trials[]`: Here, `.trials` accesses the `trials` property of the root object. The `[]` at the end of `trials` indicates that we are selecting all elements of that property, which is an array. In `jq`, `[]` is used to iterate over an array and extract all the elements from it.
81+
82+
- `.studiesFilter[]`: Just like in the previous step, we select the `studiesFilter` property inside each element of `trials`, and again use `[]` to access each element within `studiesFilter`.
83+
84+
- `.studyName`: Finally, we access the `studyName` property inside each `studiesFilter` object.
85+
86+
87+
Or in case you want to get only the variable name and its value, ignoring those values that are null:
88+
89+
```
90+
[.trials[].studiesFilter[].observationsFilter[] | select(.value != null) | { name: .observationVariable.observationVariableName, value: .value }]
91+
```
92+
93+
#### Explanation:
94+
95+
- `.trials[]`: Accesses the `trials` property of the root object. The `[]` indicates that we are selecting all elements of the `trials` array.
96+
97+
- `.studiesFilter[]`: Within each `trials` element, we access the `studiesFilter` property and use `[]` to iterate over the elements of the `studiesFilter` array.
98+
99+
- `.observationsFilter[]`: Within each `studiesFilter` element, we access the `observationsFilter` property and again use `[]` to iterate over each element of the `observationsFilter` array.
100+
101+
- `select(.value != null)`: This filters the elements of `observationsFilter` by checking if the `value` property is not null. Only the elements where `value` is not null will be passed along to the next step.
102+
103+
- `{ name: .observationVariable.observationVariableName, value: .value }`: This creates a new object for each filtered element. It extracts the `observationVariableName` from the `observationVariable` object and the `value` from the current object, renaming them as `name` and `value` respectively.
104+
105+
To get the result:
106+
107+
```
108+
{
109+
"data": [
110+
{
111+
"name": "fresh root yield|CO_334:0000013",
112+
"value": 5
113+
},
114+
{
115+
"name": "germination count|CO_334:0000166",
116+
"value": 93.3
117+
},
118+
{
119+
"name": "harvest index variable|CO_334:0000015",
120+
"value": 0.2
121+
},
122+
{
123+
"name": "initial plant vigor assessment 1-5|CO_334:0000220",
124+
"value": 4
125+
},
126+
{
127+
"name": "plant height measurement in cm|CO_334:0000018",
128+
"value": 240
129+
},
130+
{
131+
"name": "plant stands harvested counting|CO_334:0000010",
132+
"value": 9
133+
},
134+
{
135+
"name": "rotten root percentage|CO_334:0000229",
136+
"value": 0
137+
},
138+
{
139+
"name": "selected variety boolean 0&1|CO_334:0000232",
140+
"value": 0
141+
},
142+
{
143+
"name": "fresh root yield|CO_334:0000013",
144+
"value": 9
145+
},
146+
{
147+
"name": "germination count|CO_334:0000166",
148+
"value": 60
149+
},
150+
{
151+
"name": "harvest index variable|CO_334:0000015",
152+
"value": 0.37
153+
},
154+
{
155+
"name": "initial plant vigor assessment 1-5|CO_334:0000220",
156+
"value": 3
157+
},
158+
{
159+
"name": "plant stands harvested counting|CO_334:0000010",
160+
"value": 9
161+
},
162+
{
163+
"name": "selected variety boolean 0&1|CO_334:0000232",
164+
"value": 0
165+
}
166+
]
167+
}
168+
```
169+
170+
### Filtering with JsonPath
171+
172+
Now if we want to get the IDs of the studies in the trial example, since we are only calling 2 `(limit:2)`, we expect to have 2 IDs:
173+
174+
![Filter_example.png](figures/jp1.png)
175+
176+
#### Explanation:
177+
178+
The syntax of JSONPath is very similar to file paths or regular expressions. It is used to navigate through a JSON object or array and extract specific elements from them.
179+
180+
- `$`: The dollar sign `$` represents the root object. It is the starting point of the query.
181+
182+
- `.trials[*]`: This means "access the `trials` property of the root object and select all elements of that property, which is an array." The asterisk `[*]` indicates that we are selecting all elements from the array.
183+
184+
- `.studiesFilter[*]`: Next, we select the `studiesFilter` property within each `trials` object. Again, the asterisk `[*]` selects all elements within the `studiesFilter` array.
185+
186+
- `.studyDbId`: Finally, after traversing all the elements of `studiesFilter`, we select the `studyDbId` property within each of those objects.
187+
188+
#### Key Differences from jq:
189+
190+
JSONPath does not support creating new objects like `jq` does with `{ name: .observationVariable.observationVariableName, value: .value }`. To get both `name` and `value` in a combined result, you would typically need to perform separate queries and combine the results programmatically.
191+
192+
193+
```jsonpath
194+
$.trials[*].studiesFilter[*].observationsFilter[?(@.value != null)].observationVariable.observationVariableName
195+
196+
$.trials[*].studiesFilter[*].observationsFilter[?(@.value != null)].value
197+
```
198+
199+
#### Explanation:
200+
201+
- `$.trials[*]`: Accesses the `trials` property of the root object and selects all elements in the `trials` array.
202+
- `$.studiesFilter[*]`: Within each `trials` element, selects the `studiesFilter` property and iterates over the `studiesFilter` array.
203+
- `$.observationsFilter[?(@.value != null)]`: Within each `studiesFilter` element, selects the `observationsFilter` property and filters the elements where `value` is not null (similar to `select(.value != null)` in `jq`).
204+
- `.observationVariable.observationVariableName`: Selects the `observationVariableName` property inside the `observationVariable` object.
205+
- `.value`: Selects the value property within the filtered `observationsFilter` elements.
206+
207+
208+
---
209+
210+
By combining GraphQL with tools like jq and jsonpath, you can efficiently retrieve and manipulate data, making your API interactions even more powerful.

0 commit comments

Comments
 (0)