Create Transformers for Analytics Manager queries#

Transformers intercept and change queries right before they are sent to the Analytics Manager.

In the default flow, Default Transformers are called by the API's query module (/forepaas/qb), in order to inject the values of the Dynamic Parameters to the filters and scales of the query.

#Typical use case

For a better understanding of how transformers work and can be created, let's use an example that is often found in data Projects.

Let's say that most of the displayed data is filtered on the same date attribute. However, let's imagine some tables do not have a date attribute, but instead a previous_date attribute.
With default behavior, queries made to this table would return a No table found for this query error.

To solve this, a custom transformer is needed.

• Create a custom transformer
• Configure the Front App
• Customize a transformer
• Check the transformation

#Create a custom transformer

Just like custom endpoints, creating a new transformer requires adding files in a specific directory: /src/qb/transformers.

There are 3 types of transformers available for each part of a query :

• filter: for which you can create a filter.js file
• data: for which you can create a data.js file
• scale: for which you can create a scale.js file

In the case described above, create ./src/qb/transformers/filter.js, copying the default one from ./forepaas/qb/transformers/filter.js

./src/qb/transformers/filter.js

module.exports = {
  default:(query,{reference,value})=>{
    if(value === null) return Promise.resolve(query)
    if(!Array.isArray(value)) value = (value||"").split(';')
    query.filter = query.filter || {}

    if (reference.search(/^range_/) !== -1) {
      let rangeref = reference.replace(/^range_/, '')
      query.filter[rangeref] = query.filter[rangeref] || {}
      if(Array.isArray(value)) value = value[0]
      if(value) query.filter[rangeref].between = value.split(',')
      return Promise.resolve(query)
    }

    query.filter[reference] = query.filter[reference] || {}
    query.filter[reference].in = query.filter[reference].in || []
    query.filter[reference].in = query.filter[reference].in.concat(value)
    return Promise.resolve(query)
  }
}

Transformers are objects with at least one default method.
This method takes the following parameters:

• query: the full query modified by the function
• reference: as defined in the dynamic parameter configuration in the Front App
• value: value(s) passed by the Front App

This function is called for each Dynamic Parameter passed with the Query.

At this point, the Front API keeps the same behavior, going through this custom transformer rather than the default one.

#Configure the Front App

With that in mind let's have a look at a regular request.
Here is the configuration of a chart:

{
  ...
  "chart": {
    "requestId": "xyz123-xyz123-xyz123-xyz123-xyz123",
    "dynamic-parameters": [
      "datepicker"
    ]
  },
  ...
}

The generated HTTPS request is a POST call on the endpoint /qb/query/xyz123-xyz123-xyz123-xyz123-xyz123 with the following body:

{
  "dynamic_parameters": [
    {
      "type": "filter",
      "transform": "default",
      "value": [
        "1582502400,1582588799"
      ],
      "params": [],
      "reference": "range_date"
    }
  ]
}

Here only 2 things need to change in order to get the needed feature:

• default: needs to be changed to a custom value, for instance changeRef
• params: needs to include previous_date so our custom transformer may use it.

The dynamic-parameters configuration allows you to do both those changes. Going back to the chart's configuration, the transformer type will be set using a | and the params using a :.

{
  ...
  "chart": {
    "requestId": "xyz123-xyz123-xyz123-xyz123-xyz123",
    "dynamic-parameters": [
      "datepicker|changeRef:range_previous_date"
    ]
  },
  ...
}

And the POST body becomes :

{
  "dynamic_parameters": [
    {
      "type": "filter",
      "transform": "changeRef",
      "value": [
        "1582502400,1582588799"
      ],
      "params": ["rnge_previous_date"],
      "reference": "range_date"
    }
  ]
}

It is now necessary to handle it in the filter transformer.

#Customize transformer

A new method changeRef needs to be implemented in the object taking the same input as default, plus the paramsarray described above.

module.exports = {
  ...
  changeRef: (query, { reference, value, params }) => {}
}

In that case, it is suited to factorize default method, so changeRef may be implemented by calling it with different rules for reference.

Therefore, the full /src/qb/transformers/filter.js ends up looking like this:

const defaultTransformer = (query, { reference, value }) => {
  if (value === null) return Promise.resolve(query)
  if (!Array.isArray(value)) value = (value || "").split(';')
  query.filter = query.filter || {}


  if (reference.search(/^range_/) !== -1) {
    let rangeref = reference.replace(/^range_/, '')
    query.filter[rangeref] = query.filter[rangeref] || {}
    if (Array.isArray(value)) value = value[0]
    if (value) query.filter[rangeref].between = value.split(',')
    return Promise.resolve(query)
  }

  query.filter[reference] = query.filter[reference] || {}
  query.filter[reference].in = query.filter[reference].in || []
  query.filter[reference].in = query.filter[reference].in.concat(value)
  return Promise.resolve(query)
}

module.exports = {
  default: defaultTransformer,
  changeRef: (query, { reference, value, params }) => {
    return defaultTransformer(query, {
      reference: params[0] || reference,
      value
    })
  }
}

The changeRef method calls the defaultTransformer function, changing reference in the process.

#Check the transformation

The last step is to make sure that query uses the changeRef option. In the end, the query sent by the chart will have the following body:

{
  "filter": {
    "previous_date": {
      "between": ["1582502400","1582588799"]
    }
  }
}

💡 Using these techniques on filter, scale or data coupled with Front App's Dynamic parameters allows you to thouroughly customize your apps!

#Go further

If you need training or technical assistance to implement our solutions, contact your sales representative or click on this link to get a quote and ask our Professional Services experts for a custom analysis of your project.

Ask questions, give your feedback and interact directly with the team building the Data Platform on the dedicated Discord channel.

If you need support with your OVHcloud services, create a request in our Help Centre.

Join our community of users.