Plugin Dev Tutorial

[intapiuser]

Intro To Plugins

PLUGINS - WHAT AND WHY?

While the native visualizations, or widgets, that are provided natively in Sisense cover a wide range of use cases and functionality, some may want to have a very specify use case, interaction or a totally new type of visualization, this is where Sisense’s plugins infrastructure comes into play.

The plugin infrastructure allows us to create, adjust and manipulate aspects of the system to fit a customized use case, in order to create these plugins we will need coding knowledge with basic javascript, html and css, there are more options to expand and create really complex and intricate plugins, in this course we will cover the basics on how to create a new plugin, work with the data flow and display our result on the dashboard.

This course will not cover:

• DB connections
• Data creation
• Data modeling
• API calls

OFFICIAL PLUGINS:

Sisense Marketplace offers officially supported and partner created plugins, free or paid options for different use cases and solutions, this should be the first place to look for a solution if you have a custom requirement that needs to be created\solved without ‘out of the box’ options.

COMMUNITY PLUGINS:

Sisense’s community forum is lively and active, this should be your second source when looking for a custom solution for a use case, many clients and partners post solutions they have created and allow you to download and use them in your own Sisense instance.

TOOLS IN THE SYSTEM:

Sisense is using visualization libraries such as Highcharts and D3 for some of the native visualizations, you can check what versions these are by simply opening the dev tools and typing highcharts.version or d3.version, current versions for L8.2.4.446 stand on:

Highcharts:6.0.4

D3:3.3.8

future Sisense versions may have library updates in them.

PLUGINS FOLDERS AND INSTALLATION:

Most plugins require simple installation, just drop the plugin folder into the plugins folder in your system and check that it is on in the Add-ons page under the admin panel.

You can simply use the ‘File Management’ (Linux only) under the system management in the admin panel or locally on the machine itself (linux/windows).

In the plugins folder you will find an ‘entry.json’ file, this file holds a setting called ‘isProd’ and is true by default, this setting tells the system to clean the plugins from and break points and debugger lines when it is in a production environment, we want these to show so we will change this setting to isProd:false this will allow us to debug our new widgets!

In this course we will use the Template widget i have created, you can find the source code here, download it and follow along as we explore the code! (you can also just use this in your environment if you were looking for Semi Circle Donut visualization!)

Plugin File Structure

STYLE.HTML:

The style file will populate the right sidebar under the design tab, this file is a basic html file and will require simple html components.

This is where we will populate all of our dynamic styling and design options for our widget, examples for that may include, line type, line thickness, legend location, point radius, etc.,

• Note: the style.html will sometimes be saved to the cache of the machine and will not display changes when saved, if that happens it will be an easy temp fix by renaming it to ‘style1.html’ for example, don’t forget to also update that name in the ‘widget.js’ file!

STYLERCONTROLLER.JS:

This file will be our bridge between the UI components of the design panel and the widget data. We will handle the changes and interactions from the user when a design has changed and will apply it accordingly to the widget data. This is a javascript file and will require javascript knowledge in order to create, manipulate and use.

WIDGET.CSS:

The basic widget.css will be applied to the widget when rendered. Make sure you test this out and if there are other css files affecting the dashboard with same names as what you are trying to use, you may see irregular behavior.

WIDGET.JS:

This is the file that we will use to register the widget with, hold the widget object and info, basic widget functions and handlers.

This is where we will hold most of the basic widget display information such as:

• Name
• Family
• Title
• Icon
• Styler template
• Styles object

In the data object of the widget we will have:

• Panels information
• buildQuery function
• processResults function

Also we will have the options object and functions of:

• beforequery
• render

PLUGIN.JSON:

The structure file of the plugin, this is where we will define the files the plugin is depending on and version number.

New Plugin

PLUGIN.JSON SETUP:

In this file we will specify the files the plugin is using. If you have more js files or css files that the plugin is using, this will be the place to specify them. The system will only load the files if they exist here and won’t be loaded if they are in the folder of the plugin but are not specified here.

• Name: name of the plugin
• Source: js files the plugin is using(libs, scripts, etc)
• Style: css files to load with the widget (these files will be loaded to the system and will affect the system as a whole, make sure you have unique style names, or if you want to use system styles find the class names)
• Folder name: as in the plugins folder
• Last update: a date generated by the system
• Is enabled: if the plugin is ticked as ‘on’ in the plugins page under the admin tab
• Version: the version on the plugin, what will be displayed in the plugins page, format should be x.x.x

{
    "name": "templatewidget",
    "source": [
        "<other js lib files>",
        "widget.js"
    ],
    "style": [
        "widget.css"
    ],
    "folderName": "templatewidget",
    "lastUpdate": "<generated by the system>",
    "isEnabled": true,
    "version": "1.0.0"
}

WIDGET.JS - BASE PLUGIN INFO SETUP:

First we have to create the basic information that the plugin uses

name: "templatewidget",
family: "Indicator",
title: "Template Widget", // name that will be displayed for user selection
iconSmall: "/plugins/templatewidget/WidgetName-icon-small.png",
styleEditorTemplate: "/plugins/templatewidget/styler.html",
hideNoResults: true,

• Name: the name of the plugin in the system, usually the same name as the folder name
• Family: used by some other plugins\scripts to detect the type of widget and apply other changes to it as required
• Title: the display name of the plugin
• iconSmall: the icon location for the plugin, usually in the same folder
• styleEditorTemplate: the editor for the design panel, this is the template that will be used on the right side of the screen
• hideNoResults: set to true if you wish to hide the default message of no results when a query returns no data

The style object is where we’ll store different settings for the plugin such as point radius, line type, ticks on axis and other options we would like to expose to the designer with our style.html template. These settings will be saved with the plugin on the dashboard

style: {
    /* object structure for styling use, available to be changed
       using the design panel*/
},

Under the data object we will find the panels settings, this is the location that will populate the panel items on the left side of the designer, populate with dimensions, values and break by for example, some settings like which panel allows for coloring and text formatting will be added here, for now we will keep it simple

panels: [{

name: 'Dimensions panel',

type: "visible",

metadata: {

types: ['dimensions'],

maxitems: 1

},

visibility: true

},

{

name: 'Values panel',

type: "visible",

metadata: {

types: ['measures'],

maxitems: -1

},

visibility: true

},

{

name: 'filters',

type: 'filters',

metadata: {

types: ['dimensions'],

maxitems: -1

}

}

],

• Name: represents the panel name and will also be the display name that is used
• Metadata: this is the type of data we handle with the panel and how many items can be added to the panel, -1 means unlimited panel items

The filters panel will show up on the right panel of the design window, this is by default.

Data Flow

STAGES OF DATA FLOW:

The path of data goes like this:

1. The build query is called - this is where the plugin handles the query creation using the panels specified before. We’ll have to collect the panels settings and put these jaql parameters into our query
2. Before query function is called - an optional function in our flow, this function is also popular amongst dash\widget scripts. Here we will have the completed query right before it’s sent to the server as this will include dashboard filters and other options that were added after our build query function. If you wish to change anything right before the query is sent to the server, this is where you’ll do it
3. Process result function is called last - an optional function in the data flow, this function is usually used when a specific structure of data is needed for a visualisation or if there is a need to format\modify any result. This is also accessible with a widget script.

BUILD QUERY:

In the build query function we will gather the jaql objects from our panel items and populate a new query jaql

1. Create a query jaql object with:
2. The widget’s datasource
3. Metadata as an empty array
4. Gather the panels data, check all items in the panel in case there is more than one and push the panel item into the metadata array
5. Add the filters to the metadata array
6. Return the query data

/**
         * build query will handle the jaql building for the widget request,
         * in this function we will gather the data from the panels and create a jaql query to be used
         * @param {*} widget 
         */
        buildQuery: function (widget) {
            // creating the jaql object
            var query = {
                datasource: widget.datasource,
                metadata: []
            };

            // populating the jaql object with the panels info
            if (widget.metadata.panel("Dimensions panel").items.length > 0) {
                widget.metadata.panel("Dimensions panel").items.forEach(curPanel => {
                    query.metadata.push(curPanel);
                });
            }

            if (widget.metadata.panel("Values panel").items.length > 0) {

                widget.metadata.panel("Values panel").items.forEach(curPanel => {
                    query.metadata.push(curPanel);
                });
            }

            if (defined(widget.metadata.panel("filters"), 'items.0')) {
                widget.metadata.panel('filters').items.forEach(function (item) {
                    item = $$.object.clone(item, true);
                    item.panel = "scope";
                    query.metadata.push(item);
                });
            }

            return query;
        },

BEFORE QUERY:

In the before query function we will get the widget and event as parameters. If there are last minute changes and modifications that are needed (effecting dashboard filters for example), this is where we’ll do it

/**
     * OPTIONAL
     * before query will be called right before the execution, this is the place to manipulate the jaql query further after
     * it was updated with dashboard filters and such
     * @param {*} widget 
     * @param {*} event 
     */
    beforequery(widget, event) {},

PROCESS RESULT:

In the process result, we will get the widget and the query result as parameters. If there is a manipulation of the data or restructure DO IT HERE.

This function will only be called when new data is arrived, sometimes this may be skipped for example when the “readjust” function is called, the path will skip fetching the data and move to render right away.

Do not process the data in the render function as it may be called without running through the process result path, using this instead of render will ensure that the render function always has the data and the process isn’t happening when it is not necessary.

/**
         * OPTIONAL
         * process result will handle the functionality needed for structuring the data correctly for a UI component to consume
         * @param {*} widget 
         * @param {*} queryResult 
         */
        processResult: function (widget, queryResult) {}

Render

Render function will be called after the data flow process has finished (widget added, filters changed and so on). It can also be called with a redraw but without running through the data process again, using the data that was already received (resizing of the widget for example)

The render function will receive the widget and event where the widget will hold the information including the data that was collected while the event will hold the cause of the render and the element to render in (event.element).

This function is synchronic and will not allow for async functions to be called, in order to create more complex plugins that require async functionality we would have to call an external function and use that to call async functions in it.

/**
     * render function will be called right after process result function, this is where the UI component will interact with the data
     * this function can also be called after a filter change, redraw event or readjust event
     * @param {*} widget 
     * @param {*} event 
     */
    render: function (widget, event) {
        // the widget dom element, this is the element the plugin will populate with the UI component
        var element = $(event.element);
    },

Semi Circle Donut - Plugin Example

The Semi Circle Donut plugin is using the highcharts configuration of pie with some customization, you can check out the original code here highcharts semi circle donut.

There are a few changes we had to do to this template code in order to make our widget work in the environment:

• Process result function
• Render function
• Styler page
• Styler controller page

STYLER PAGE:

Updating the styler page in order to give us a simple check box that will define if we will show the labels inside of the visualization parts or floating, here is a quick code for the change:

<div data-ng-controller="plugin-semicircledonut.controllers.stylerController">
    <div class="settings-pane-host">
        <div class="pane-row">
            <div class="type-title">Labels</div>
            <div class="cbox-pane-row selected"
                data-ng-click="ShowInnerLabelsTick()" data-ng-class="{selected: model.isInnerLabels}">
                <div class="custom-cbox-btn" data-ng-class="{selected: model.isInnerLabels}"></div>
                <div class="pane-separator"></div>
                <div class="pane-caption">Inner Labels</div>
            </div>
        </div>
    </div>
</div>

You can see we are using the widgets style object here as “model.isInnerLabels”, this will also check the box according to the setting that is saved so it will display that in the future.

STYLER CONTROLLER:

Updating the styler controller in order to handle the changes from our styler page, using a simple “ShowInnerLabelsTick” in order to switch the value between true and false.

$scope.ShowInnerLabelsTick = function () {
        $scope.model.isInnerLabels = !$scope.model.isInnerLabels;
        _.defer(function () {
            $scope.$root.widget.redraw();
        });
    };

After each selection change we will force a redraw of the widget and it will apply and display our new settings

PROCESS RESULT:

The pie data structure is different from the raw data that is returned from our query so we will have to build a new array with the right structure for the data, after the data is collected we will divide the count and make a quick percent out of the total and add that to the data.

processResult: function (widget, queryResult) {
            let postProcessResult = [];
            let sumValue = 0;

            // Restructure the data and collect sum for % calculation
            queryResult.$$rows.forEach(curVal => {
                let dimName = curVal[0].text;
                let value = curVal[1].text;

                sumValue += curVal[1].data;

                postProcessResult.push([dimName, 'percent', value]);
            });

            // Calc of percent using the collected sum
            postProcessResult.forEach(curResult => {
                curResult[1] = (curResult[2] / sumValue);
            });

            return postProcessResult;
        }

RENDER:

The render function is basically the configuration of the highchart element we want to create and a call to highchart to render it.

Using JQuery to get the relevant element and quickly empty it from previous content, we will usually do this when we want to render something completely new. If there is an option for you to not re-render the whole element it may be a better option in terms of performance and resource consumption. This option usually takes more code to define if a whole rerender is needed or not. So, for this example, we will just render everything again.

Creating a simple object as chartConfig and then populating it with the settings that we want will be an easy task after we handle all of the plugin settings and result processing, what do we need and how would we get it?

• Gather panels titles in order to display in our chart titles:
• Gather the dimensions panel title with this code

widget.metadata.panel("Dimensions panel").items[0].jaql.title

Gather the values panel title with this code

widget.metadata.panel("Values panel").items[0].jaql.title

Gather information for the tooltip according to the point and data attached, we will use the formatter in order to create the tooltip with this code

tooltip: {

  formatter: function (d) {

 return this.point.name + ' ' + this.series.name + ':<br><b>' + this.point.percentage.toFixed(1) + '% (' + d.chart.options.series[0].data[this.point.index][2] + ')</b>';

 }

}

The processed data we will gather with this code

widget.queryResult

And last we will apply the simple setting selection we creating with our design panel and get the value using this code

widget.style.isInnerLabels

Each chart may require different data structure and as of that may have a different path to reach each value, that means that not all points of data will be available at the same location in every chart type that you will create, make sure you check and handle these changes when creating new plugins!

For more settings and options i will leave you to explore the highcharts documentations and come up with new, intricate and fascinating designs! Check out the highcharts documentation here 

render: function (widget, event) {
        // the widget dom element, this is the element the plugin will populate with the UI component
        let element = $(event.element);
        element.empty();

        chartConfig = {
            chart: {
                plotBackgroundColor: null,
                plotBorderWidth: 0,
                plotShadow: false
            },
            title: {
                text: widget.metadata.panel("Values panel").items[0].jaql.title + "<br> By <br>" + widget.metadata.panel("Dimensions panel").items[0].jaql.title,
                align: 'center',
                verticalAlign: 'middle',
                y: 60
            },
            tooltip: {
                formatter: function (d) {
                    return this.point.name + ' ' + this.series.name + ':<br><b>' + this.point.percentage.toFixed(1) + '% (' + d.chart.options.series[0].data[this.point.index][2] + ')</b>';
                }
            },
            accessibility: {
                point: {
                    valueSuffix: '%'
                }
            },
            plotOptions: {
                pie: {
                    dataLabels: {
                        enabled: false
                    },
                    startAngle: -90,
                    endAngle: 90,
                    center: ['50%', '75%'],
                    size: '110%'
                }
            },
            series: [{
                type: 'pie',
                name: widget.metadata.panel("Values panel").items[0].jaql.title,
                innerSize: '50%',
                data: widget.queryResult
            }]
        };

        if (widget.style.isInnerLabels) {
            chartConfig.plotOptions.pie.dataLabels = {
                enabled: true,
                distance: -50,
                style: {
                    fontWeight: 'bold',
                    color: 'white'
                }
            }
        }

        Highcharts.chart(element[0], chartConfig);
    }