Many changes. Go files moved around, interpolate variables function w…

…ith test, retrodaredevil is organization, docker compose uses oss image

.github/workflows/is-compatible.yml

@@ -1,5 +1,9 @@
 name: Latest Grafana API compatibility check
-on: [pull_request]
+on:
+  pull_request:
+  push:
+    branches:
+      - main
 
 jobs:
   compatibilitycheck:

DEVELOPMENT.md

@@ -103,3 +103,30 @@ This section contains notes about dependencies.
   * https://github.com/graphql/graphiql/issues/2405#issuecomment-1469851608 (yes as of writing it says it's closed, but it's not)
   * It's not a bad thing that we include this dependency because it gives us a couple of types that we end up using
 
+
+## To-Do
+
+* Queries that result in errors should display nice errors messages on the frontend
+* Add ability to have multiple parsing options
+  * Add "advanced options" section that has "Partition by" and "Alias by"
+    * Including these might be necessary as you may want to partition by and alias by different things for different parts of a GraphQL query
+    * Advances options can also include the ability to add custom labels to the response - this allows different parsing options to be distinguished by Grafana
+* Add support for secure variable data defined in the data source configuration
+  * The variables defined here cannot be overridden for any request - this is for security
+  * Also add support for secure HTTP headers
+* See what minimum Grafana version we can support
+* Add support for variables: https://grafana.com/developers/plugin-tools/create-a-plugin/extend-a-plugin/add-support-for-variables
+* Add metrics to backend component: https://grafana.com/developers/plugin-tools/create-a-plugin/extend-a-plugin/add-logs-metrics-traces-for-backend-plugins#implement-metrics-in-your-plugin
+* Support returning logs data: https://grafana.com/developers/plugin-tools/tutorials/build-a-logs-data-source-plugin
+  * We could just add `"logs": true` to `plugin.json`, however we need to support the renaming of fields because sometimes the `body` or `timestamp` fields will be nested
+* Publish as a plugin
+  * https://grafana.com/developers/plugin-tools/publish-a-plugin/publish-a-plugin
+  * https://grafana.com/developers/plugin-tools/publish-a-plugin/sign-a-plugin#generate-an-access-policy-token
+  * https://grafana.com/legal/plugins/
+  * https://grafana.com/developers/plugin-tools/publish-a-plugin/provide-test-environment
+* Publish on a private plugin repository
+  * https://volkovlabs.io/blog/installing-grafana-plugins-from-a-private-repository-805b54a1add3/
+* Create a GraphQL button panel (or a SolarThing app) that has a button panel that can be used to
+  * If we create an app, we can follow https://github.com/RedisGrafana/grafana-redis-app
+    * https://github.com/RedisGrafana/grafana-redis-app/blob/e093d18a021bb28ba7df3a54d7ad17c2d8e38f88/src/redis-gears-panel/components/RedisGearsPanel/RedisGearsPanel.tsx#L314
+* Auto-populate the data path field by using `documentAST` to recognize the first path to an array

README.md

@@ -8,18 +8,29 @@ This datasource tries to reimagine how GraphQL queries should be made from Grafa
 
 Requests are made in the backend. Results are consistent between queries and alerting.
 
+## Features
+
+* Complex GraphQL responses can be turned into timeseries data, or a simple table
+* Includes [GraphiQL](https://github.com/graphql/graphiql) query editor. Autocompletion and documentation for the GraphQL schema available inside Grafana!
+* This is a backend plugin, so alerting is supported
+* `from` and `to` variables are given to the query via [native GraphQL variables](https://graphql.org/learn/queries/#variables)
+* Variables section of the query editor supports interpolation of string values using [Grafana variables](https://grafana.com/docs/grafana/latest/dashboards/variables/add-template-variables/). (\*not supported in alerting or other backend-only queries)
+* [Annotation support](https://grafana.com/docs/grafana/latest/dashboards/build-dashboards/annotate-visualizations/)
+
 
 ## Variables
 
 ### Provided Variables
 
 Certain variables are provided to every query. These include:
 
-| Variable      | Type   | Description                                                | Grafana counterpart                                                                                                |
-|---------------|--------|------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------|
-| `from`        | Number | Epoch milliseconds of the "from" time. Passed as a number. | [$__from](https://grafana.com/docs/grafana/latest/dashboards/variables/add-template-variables/#__from-and-__to)    |
-| `to`          | Number | Epoch milliseconds of the "to" time. Passed as a number.   | [$__to](https://grafana.com/docs/grafana/latest/dashboards/variables/add-template-variables/#__from-and-__to)      |
-| `interval_ms` | Number | Milliseconds of the interval. Equivalent to `to - from`.   | [$__interval_ms](https://grafana.com/docs/grafana/latest/dashboards/variables/add-template-variables/#__interval)  |
+| Variable        | Type   | Description                                                                    | Grafana counterpart                                                                                               |
+|-----------------|--------|--------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------|
+| `from`          | Number | Epoch milliseconds of the "from" time                                          | [$__from](https://grafana.com/docs/grafana/latest/dashboards/variables/add-template-variables/#__from-and-__to)   |
+| `to`            | Number | Epoch milliseconds of the "to" time                                            | [$__to](https://grafana.com/docs/grafana/latest/dashboards/variables/add-template-variables/#__from-and-__to)     |
+| `interval_ms`   | Number | The suggested duration between time points in a time series query              | [$__interval_ms](https://grafana.com/docs/grafana/latest/dashboards/variables/add-template-variables/#__interval) |
+| `maxDataPoints` | Number | Maximum number of data points that should be returned from a time series query | N/A                                                                                                               |
+| `refId`         | String | Unique identifier of the query, set by the frontend call                       | N/A                                                                                                               |
 
 An example usage is shown in the most basic query:
 
@@ -106,22 +117,3 @@ References:
   * This error indicates that the query returns more fields than just the time and the datapoint.
   * For alerts, the response from the GraphQL query cannot contain more than the time and datapoint. At this time, you cannot use other attributes from the result to filter the data.
 
-## To-Do
-
-* Add ability to have multiple parsing options
-  * Add "advanced options" section that has "Partition by" and "Alias by"
-    * Including these might be necessary as you may want to partition by and alias by different things for different parts of a GraphQL query
-    * Advances options can also include the ability to add custom labels to the response - this allows different parsing options to be distinguished by Grafana
-* Add support for secure variable data defined in the data source configuration
-  * The variables defined here cannot be overridden for any request - this is for security
-  * Also add support for secure HTTP headers
-* See what minimum Grafana version we can support
-* Add support for variables: https://grafana.com/developers/plugin-tools/create-a-plugin/extend-a-plugin/add-support-for-variables
-* Add metrics to backend component: https://grafana.com/developers/plugin-tools/create-a-plugin/extend-a-plugin/add-logs-metrics-traces-for-backend-plugins#implement-metrics-in-your-plugin
-* Support returning logs data: https://grafana.com/developers/plugin-tools/tutorials/build-a-logs-data-source-plugin
-  * We could just add `"logs": true` to `plugin.json`, however we need to support the renaming of fields because sometimes the `body` or `timestamp` fields will be nested
-* Publish as a plugin
-  * https://grafana.com/developers/plugin-tools/publish-a-plugin/publish-a-plugin
-  * https://grafana.com/developers/plugin-tools/publish-a-plugin/sign-a-plugin#generate-an-access-policy-token
-  * https://grafana.com/legal/plugins/
-  * https://grafana.com/developers/plugin-tools/publish-a-plugin/provide-test-environment

docker-compose.yaml

@@ -2,15 +2,15 @@ version: '3.0'
 
 services:
   grafana:
-    container_name: 'wildmountainfarms-wildgraphql-datasource'
+    container_name: 'grafana-dev-wildgraphql'
     platform: 'linux/amd64'
     build:
       context: ./.config
       args:
-        grafana_image: ${GRAFANA_IMAGE:-grafana-enterprise}
-        grafana_version: ${GRAFANA_VERSION:-10.0.3}
+        grafana_image: ${GRAFANA_IMAGE:-grafana-oss}
+        grafana_version: ${GRAFANA_VERSION:-10.0.10}
     ports:
       - 3000:3000/tcp
     volumes:
-      - ./dist:/var/lib/grafana/plugins/wildmountainfarms-wildgraphql-datasource
+      - ./dist:/var/lib/grafana/plugins/retrodaredevil-wildgraphql-datasource
       - ./provisioning:/etc/grafana/provisioning

pkg/main.go

@@ -17,8 +17,8 @@ func main() {
 	// from Grafana to create different instances of SampleDatasource (per datasource
 	// ID). When datasource configuration changed Dispose method will be called and
 	// new datasource instance created using NewSampleDatasource factory.
-	if err := datasource.Manage("wildmountainfarms-wildgraphql-datasource", plugin.NewDatasource, datasource.ManageOpts{}); err != nil {
+	if err := datasource.Manage("retrodaredevil-wildgraphql-datasource", plugin.NewDatasource, datasource.ManageOpts{}); err != nil {
 		log.DefaultLogger.Error(err.Error())
 		os.Exit(1)
 	}
-}
+}

pkg/plugin/datasource.go

@@ -9,7 +9,10 @@ import (
 	"github.com/grafana/grafana-plugin-sdk-go/backend/httpclient"
 	"github.com/grafana/grafana-plugin-sdk-go/backend/instancemgmt"
 	"github.com/grafana/grafana-plugin-sdk-go/backend/log"
-	"github.com/wildmountainfarms/wild-graphql-datasource/pkg/plugin/util"
+	"github.com/wildmountainfarms/wild-graphql-datasource/pkg/plugin/parsing"
+	"github.com/wildmountainfarms/wild-graphql-datasource/pkg/plugin/querymodel"
+	"github.com/wildmountainfarms/wild-graphql-datasource/pkg/plugin/queryvariables"
+	"github.com/wildmountainfarms/wild-graphql-datasource/pkg/util/graphql"
 	"net/http"
 )
 
@@ -85,16 +88,6 @@ func (d *Datasource) QueryData(ctx context.Context, req *backend.QueryDataReques
 	return response, nil
 }
 
-// queryModel represents data sent from the frontend to perform a query
-type queryModel struct {
-	QueryText string `json:"queryText"`
-	// The name of the operation, or a blank string to let the GraphQL server infer the operation name
-	OperationName string `json:"operationName"`
-	// The variables for the operation. May either be a string or a map[string]interface{}
-	Variables      interface{}          `json:"variables"`
-	ParsingOptions []util.ParsingOption `json:"parsingOptions"`
-}
-
 func statusFromResponse(response http.Response) backend.Status {
 	for _, status := range []backend.Status{} {
 		if response.StatusCode == int(status) {
@@ -112,8 +105,8 @@ func (d *Datasource) query(ctx context.Context, pCtx backend.PluginContext, quer
 
 	log.DefaultLogger.Info(fmt.Sprintf("JSON is: %s", query.JSON))
 
-	// Unmarshal the JSON into our queryModel.
-	var qm queryModel
+	// Unmarshal the JSON into our QueryModel.
+	var qm querymodel.QueryModel
 
 	err := json.Unmarshal(query.JSON, &qm)
 	if err != nil {
@@ -128,40 +121,10 @@ func (d *Datasource) query(ctx context.Context, pCtx backend.PluginContext, quer
 	}
 	log.DefaultLogger.Info("Query text is: " + qm.QueryText)
 
-	// Although the frontend has access to global variable substitution (https://grafana.com/docs/grafana/latest/dashboards/variables/add-template-variables/#global-variables)
-	//   the backend does not.
-	//   Because of this, it's beneficial to encourage users to write queries that rely as little on the frontend as possible.
-	//   This allows us to support alerting later.
-	//   These variable names that we are "hard coding" should be as similar to possible as those global variables that are available
-	//   Forum post here: https://community.grafana.com/t/how-to-use-template-variables-in-your-data-source/63250#backend-data-sources-3
-	//   More information here: https://grafana.com/docs/grafana/latest/dashboards/variables/
-
-	var variables = map[string]interface{}{}
-	switch value := qm.Variables.(type) {
-	case string:
-		// This case happens when the frontend is not involved at all. This is likely an alert.
-		// Remember that these variables are not interpolated
-
-		err := json.Unmarshal([]byte(value), &variables)
-		if err != nil {
-			log.DefaultLogger.Error("Got error while parsing variables! Error", err)
-			log.DefaultLogger.Info(fmt.Sprintf("Value of variables from parsing error is: %s", value))
-
-			// continue executing query without interpolated variables
-			// TODO consider if we want a flag in the options to prevent the query from continuing further in the case of an error
-		}
-	case map[string]interface{}:
-		// This case happens when the frontend is able to interpolate the variables before passing them to us
-		//   or happens when someone has directly configured the variables option in the JSON itself
-		//   and storing it as an object rather than a string like the QueryEditor does.
-		//   If this is the ladder case, variables have not been interpolated
-		variables = value
-	default:
-		log.DefaultLogger.Error("Unable to parse variables for ref ID:" + query.RefID)
-	}
-	util.AutoPopulateVariables(query, &variables)
+	// use later: pCtx.AppInstanceSettings.DecryptedSecureJSONData
+	variables, _ := queryvariables.ParseVariables(query, qm.Variables)
 
-	graphQLRequest := util.GraphQLRequest{
+	graphQLRequest := graphql.GraphQLRequest{
 		Query:         qm.QueryText,
 		OperationName: qm.OperationName,
 		Variables:     variables,
@@ -183,7 +146,7 @@ func (d *Datasource) query(ctx context.Context, pCtx backend.PluginContext, quer
 	}
 	status := statusFromResponse(*resp)
 
-	graphQLResponse, responseParseError := util.ParseGraphQLResponse(resp.Body)
+	graphQLResponse, responseParseError := graphql.ParseGraphQLResponse(resp.Body)
 	if responseParseError != nil {
 		return &backend.DataResponse{
 			Error:  err,
@@ -222,7 +185,7 @@ func (d *Datasource) query(ctx context.Context, pCtx backend.PluginContext, quer
 
 	// add the frames to the response.
 	for _, parsingOption := range qm.ParsingOptions {
-		frame, err := util.ParseData(
+		frame, err := parsing.ParseData(
 			graphQLResponse.Data,
 			parsingOption,
 		)
@@ -242,7 +205,7 @@ func (d *Datasource) query(ctx context.Context, pCtx backend.PluginContext, quer
 func (d *Datasource) CheckHealth(ctx context.Context, req *backend.CheckHealthRequest) (*backend.CheckHealthResult, error) {
 	// test command to do the same thing:
 	//   curl -X POST -H "Content-Type: application/json" -d '{"query":"{\n\t\t  __schema{\n\t\t\tqueryType{name}\n\t\t  }\n\t\t}"}' https://swapi-graphql.netlify.app/.netlify/functions/index
-	graphQLRequest := util.GraphQLRequest{
+	graphQLRequest := graphql.GraphQLRequest{
 		Query: `{
 		  __schema{
 		    queryType{name}
@@ -260,7 +223,7 @@ func (d *Datasource) CheckHealth(ctx context.Context, req *backend.CheckHealthRe
 		return nil, err
 	}
 
-	graphQLResponse, responseParseError := util.ParseGraphQLResponse(resp.Body)
+	graphQLResponse, responseParseError := graphql.ParseGraphQLResponse(resp.Body)
 	if responseParseError != nil {
 		if resp.StatusCode == 200 {
 			return &backend.CheckHealthResult{

pkg/plugin/util/parsing.go → pkg/plugin/parsing/parsing.go

@@ -1,24 +1,18 @@
-package util
+package parsing
 
 import (
 	"errors"
 	"fmt"
 	"github.com/grafana/grafana-plugin-sdk-go/data"
+	"github.com/wildmountainfarms/wild-graphql-datasource/pkg/plugin/querymodel"
 	"reflect"
 	"strings"
 	"time"
 )
 
 // the purpose of this file is to parse JSON data with configuration from a ParsingOption
 
-type ParsingOption struct {
-	// The path from the root to the array. This is dot-delimited
-	DataPath string `json:"dataPath"`
-	// the time path relative to the data path.
-	TimePath string `json:"timePath"`
-}
-
-func ParseData(graphQlResponseData map[string]interface{}, parsingOption ParsingOption) (*data.Frame, error) {
+func ParseData(graphQlResponseData map[string]interface{}, parsingOption querymodel.ParsingOption) (*data.Frame, error) {
 	if len(parsingOption.DataPath) == 0 {
 		return nil, errors.New("data path cannot be empty")
 	}

pkg/plugin/util/parsing_test.go → pkg/plugin/parsing/parsing_test.go

@@ -1,4 +1,4 @@
-package util
+package parsing
 
 import (
 	"bytes"

pkg/plugin/querymodel/querymodel.go

@@ -0,0 +1,18 @@
+package querymodel
+
+// QueryModel represents data sent from the frontend to perform a query
+type QueryModel struct {
+	QueryText string `json:"queryText"`
+	// The name of the operation, or a blank string to let the GraphQL server infer the operation name
+	OperationName string `json:"operationName"`
+	// The variables for the operation. May either be a string or a map[string]interface{}
+	Variables      interface{}     `json:"variables"`
+	ParsingOptions []ParsingOption `json:"parsingOptions"`
+}
+
+type ParsingOption struct {
+	// The path from the root to the array. This is dot-delimited
+	DataPath string `json:"dataPath"`
+	// the time path relative to the data path.
+	TimePath string `json:"timePath"`
+}