🐍 PySisense: Programmable Sisense Environment Management
⚠️ Experimental Project Notice PySisense is an experimental, community-contributed Python SDK from Sisense Field Engineering, shared “as-is” for learning and experimentation. It is not a GA Sisense feature and is not supported through standard Sisense Customer Support. PySisense runs locally in your environment. You control where it runs, which credentials it uses, and what permissions it has. Use a sandbox first and validate behavior before broader adoption. PySisense is a Python SDK designed for structured, repeatable interaction with the Sisense API. It turns common “environment work” (governance, migrations, lifecycle management, and audits) into clean Python methods so teams can manage Sisense like a platform, not a set of manual clicks and one-off scripts. GitHub Repository: PySisense PySisense's Purpose As Sisense environments scale, operational work becomes harder to manage in three common areas: Efficient Governance: Reduce repetitive audits across large numbers of assets and folders. Smooth and Reliable Migrations: Promote assets across dev, staging, and production with speed and consistency. Straightforward Workflow Development: Avoid stitching together multiple API calls and handling pagination, retries, and permission edge cases manually. PySisense was created to remove that friction by packaging real-world Sisense admin workflows into a consistent SDK that is easy to automate, debug, and extend. PySisense Capabilities 1) Environment and asset management at scale Use Python to search, inventory, and manage Sisense assets and metadata without writing raw REST calls each time. Examples of the type of work PySisense supports: Finding dashboards and folders quickly by name or metadata patterns Auditing ownership, access, and sharing posture Standardizing repeatable admin workflows across environments 2) Cross-environment migrations (dev to prod and beyond) PySisense supports repeatable migrations where you can move users, dashboards, and models across Sisense environments with predictable behavior and structured outputs. This is particularly useful when teams need: Controlled promotion from dev to prod Tenant-to-tenant migration and consolidation Repeatable rollout patterns across multiple environments 3) Governance and bulk operations with consistent logging Operational actions are only useful if they are traceable. PySisense is built for automation with: Structured logging (debug/info/error) Repeatable method interfaces and return shapes A design that is friendly for CI/CD and scripted execution 4) WellCheck: Visibility into complexity and risk PySisense includes a WellCheck module that helps you evaluate dashboard and model health, focusing on complexity and patterns that create operational risk. This can include checks such as: Dashboard structural complexity (widget density, pivot usage, etc.) Data model patterns (many-to-many relationships, island tables, RLS datatype checks, import query patterns) Unused columns and other common cleanup opportunities Intended Audience for PySisense Sisense platform owners who need repeatable governance and operational control Analytics engineering teams managing multiple environments or tenants Developers automating Sisense workflows through scripts and pipelines Advanced Sisense users who prefer “as-code” control over manual UI-heavy operations Organization of the SDK PySisense follows a class-based structure to keep workflows clear and modular: SisenseClient: Base API wrapper for consistent HTTP operations AccessManagement: Users, groups, roles, and permissions workflows Dashboard: Dashboard lifecycle tasks including shares and ownership patterns DataModel: Dataset/schema/security operations and model-level tasks Migration: Cross-environment workflows (users, dashboards, models) WellCheck: Health checks for dashboards and data models Utils: Helpers for export, formatting, and data operations Contributing and Support This is an experimental, community-contributed project maintained by Sisense Field Engineering and provided “as-is.” Do not open a GSS ticket (this is not a GA Sisense feature). For usage questions or help getting started, contact your Customer Success Manager (CSM), who will route feedback to the Field Engineering team. For bugs and improvements, use GitHub Issues or submit a Pull Request. For feature requests, comment below or open a GitHub Issue with details.Exploring RAG: A Sisense-Based Approach with BigQuery and Gemini
Exploring RAG: A Sisense-Based Approach with BigQuery and Gemini Continuing from our previous article, Automated Machine Learning with Sisense Fusion: A Practical Guide, where we discussed how Sisense and AutoML simplify complex machine learning workflows, I am now excited to introduce an advanced feature: a chatbot interface powered by Retrieval-Augmented Generation (RAG) and a large language model (LLM) like Gemini. This enhancement allows users to interact with their data in natural language through Sisense's Native Blox widget. Imagine asking questions like "How many products did I sell per category?" and receiving insightful answers instantly. This not only improves data accessibility but also bridges the gap between technical data repositories and business decision-making. In this article, I’ll cover the first step of enabling this functionality: setting up a seamless integration pipeline between Sisense and Google BigQuery, embedding table and column metadata, and preparing the data for efficient querying. Important Note: Experimental Project: This implementation is an experimental project and not an official Sisense feature. It demonstrates how Sisense’s functionalities can be utilized to build a custom Retrieval-Augmented Generation (RAG) pipeline or how you can use this example by importing the provided notebooks. Google Ecosystem: The example provided is specific to the Google ecosystem, utilizing services like BigQuery (BQ) and Gemini for query generation and processing. Associated Costs: Please note that each prompt sent to the chatbot incurs a cost, which will be billed to your Google Cloud Platform (GCP) account. This includes charges for LLM processing and database queries. Building the Foundation: Data Preparation and Embeddings To enable natural language interactions with your data, we first need to establish a robust data infrastructure. This includes creating a vector store for embeddings in Google BigQuery (BQ) and preparing metadata about tables and columns. The process is fully automated using a pre-built Sisense custom code notebook, which simplifies embedding generation and management. Dataset in BigQuery The journey begins with your dataset in Google BigQuery. The notebook retrieves metadata such as table and column names, descriptions, and schema information from BigQuery’s Information_Schema. If any descriptions are missing, the system automatically generates them, ensuring comprehensive metadata coverage. Setting Up the Vector Store in BigQuery The vector store acts as the backbone for similarity searches within the RAG system. Using the custom notebook, the system: Creates the vector store: A structured repository to store embeddings for tables and columns. Organizes metadata: Ensures all table and column descriptions are structured and accessible. Generating Table and Column Descriptions Missing descriptions can hinder data understanding. The notebook includes a Description Agent that automatically generates meaningful text for: Tables: Contextual descriptions based on schema and usage. Columns: Descriptions highlighting their role within the dataset. These enhancements ensure the metadata is both informative and ready for embedding generation. Creating Embeddings with Vertex AI To enable semantic search, metadata descriptions are converted into numerical embeddings using Vertex AI’s TextEmbeddingModel. This is facilitated by the EmbedderAgent, which: Accepts strings or lists of strings (e.g., column descriptions). Generates embeddings through Vertex AI. Handles both single and batch processing for efficiency. Efficient Embedding with Chunked Processing For large datasets, embeddings are generated in chunks using the get_embedding_chunked function. This ensures: Scalability: Handles datasets of all sizes without performance issues. Parallel Processing: Processes text chunks simultaneously to speed up the workflow. Structured Outputs: Embeddings are returned in a structured DataFrame for storage or analysis. Storing Embeddings in the Vector Store The final step is storing these embeddings in the BigQuery vector store. This ensures that: Similarity searches are fast and efficient. Metadata is always accessible for chatbot interactions. ALT text: A screenshot of a data table displaying various columns such as "table_schema," "column_name," "data_type," "source_type," and several other attributes. The table shows sample values and metadata related to a database structure, organized in rows and columns. How the Chatbot Interface Works Now that the foundation for embeddings and metadata storage is set, let’s explore the chatbot interface in action. Imagine opening the chatbot in the Blox widget and asking a question about your dataset. Within moments, the chatbot responds in natural language, providing actionable insights. But what exactly happens under the hood to generate this seamless interaction? RAG Notebook and the Chatbot Workflow The chatbot operates using a pre-built RAG custom code transformation notebook, which orchestrates the end-to-end process. With this notebook, the entire pipeline—from understanding the query to generating the response—is automated. The notebook uses multiple specialized agents, each responsible for a specific task, ensuring precision and efficiency at every step. SQL Query Builder Agent BuildSQLAgent This agent specializes in constructing SQL queries for BigQuery. It uses the LLM to analyze the user’s natural language query and matches it with table schemas and column details from the vector store. It outputs a fully formed SQL query tailored to the user’s dataset and question. SQL Validation Agent ValidateSQLAgent The ValidateSQLAgent validates the SQL query before execution using a Large Language Model (LLM). Validation ensures the query adheres to essential rules, including: The presence of all referenced columns and tables. Proper table relationships and join conditions based on the schema. Formatting and compliance with BigQuery-specific SQL standards. Validation occurs during the debugging process, specifically within the DebugSQLAgent, to identify potential errors before attempting a dry run or execution. It provides a detailed JSON response: If valid, the process moves to the next step (dry run or execution). If invalid, the DebugSQLAgent uses the error details to refine the query iteratively. SQL Debugging Loop Agent DebugSQLAgent This agent runs the debugging loop to refine queries that fail validation or execution. The process includes: Validation: The query is passed to ValidateSQLAgent to check syntax, schema compliance, and structure. If valid, the query is ready for execution. Dry Run: If validation passes, the query is tested using a dry run via the test_sql_plan_execution function to confirm execution readiness. Execution: Once validation and dry runs succeed, the final query is executed using the retrieve_df function, which returns results as a DataFrame. Iterative Refinement: If the query fails either validation or the dry run, the DebugSQLAgent uses the LLM to troubleshoot and generate an alternative query. The loop repeats until a valid query is generated or the maximum debugging rounds are reached. This agent ensures the final query is: Correctly structured and semantically valid. Optimized for performance and aligns with the original user intent. Response Agent ResponseAgent This agent translates the SQL query results into natural language. It bridges the gap between technical SQL outputs and user-friendly communication. By combining the query results with the user’s original question, it crafts a clear and relevant response. How the Workflow Executes Here’s the step-by-step process for generating a response: User Query Embedding The EmbedderAgent converts the user’s natural language question into a numerical embedding. Using BigQuery native vector search, the system retrieves similar embeddings from the vector store created in the first phase. Schema and Content Retrieval Based on the retrieved embeddings, the system fetches relevant table and column schema details from the vector store. SQL Query Generation The BuildSQLAgent uses the retrieved schema details to construct an SQL query that aligns with the user’s question. SQL Validation and Execution The ValidateSQLAgent checks the generated SQL for accuracy and potential errors. If the SQL passes validation, it is executed against the BigQuery database. Debugging (if needed) If the query fails or generates an error, the DebugSQLAgent refines it iteratively until a valid query is produced. Response Generation The ResponseAgent uses the query results and the user’s original prompt to generate a natural language response. If the system fails to generate a valid response, it communicates the issue to the user. Conclusion By combining the foundational embedding process with this RAG-powered workflow, the chatbot transforms how users interact with their data. From seamless SQL query generation to delivering natural language responses, the system exemplifies the power of Sisense Fusion and advanced AI tools to simplify data-driven decision-making. As always, please reach out to your Customer Success Manager (CSM) if you would like to implement this in your own environment.Automated Machine Learning with Sisense Fusion: A Practical Guide
Automated Machine Learning with Sisense Fusion: A Practical Guide In this article, we’ll explore how using Sisense and AutoML (Automated Machine Learning) can simplify the process of applying machine learning to real-world business problems. AutoML takes care of tasks such as data preprocessing, model selection, and hyperparameter optimization without requiring deep expertise in machine learning. Let’s dive into some practical business challenges where machine learning can make a significant impact. Understanding the Business Use Cases To illustrate how machine learning (ML) solves business challenges, we’ll look at two real-world use cases: Optimizing Inventory for a Popular Retail Product (Regression Problem): Imagine a popular clothing store trying to manage stock for a trendy item that frequently sells out. By applying machine learning, the store could predict the future demand for this product. This is an example of a regression problem, where the model forecasts continuous values—such as the number of items to stock—based on historical sales patterns, seasonal trends, and customer behaviors. This allows the store to optimize its inventory, avoid shortages, and maximize sales, demonstrating the power of machine learning to enhance operational efficiencyto stock) based on historical data and other influencing factors. Improving Customer Retention in Subscription Services (Classification Problem): For subscription-based businesses, predicting customer churn is essential. By analyzing data such as usage patterns, customer engagement, and support history, machine learning can predict whether a customer is likely to cancel their subscription. This classification model enables businesses to proactively target at-risk customers with personalized offers or support, helping to improve retention and customer satisfaction. The predictive power of machine learning transforms how businesses engage with their users, reducing churn and increasing long-term loyalty. Difference Between Regression and Classification Regression models are used to predict continuous values, such as quantities (e.g., how many products to stock) or prices. Classification models are used to predict categorical outcomes, such as Yes/No (e.g., whether a customer will churn) or other distinct categories (e.g., fraud/no fraud). Hands-On: Using Sisense Fusion and AutoML Everything demonstrated in this video, including the integration of machine learning models, is powered by Sisense Fusion’s native APIs and features. The web app is simply a wrapper that adds a shiny interface, but all actions performed in the demo—whether selecting data models, training machine learning models, or making predictions—can be done entirely within Sisense Fusion’s native platform without the need for external code or API calls. The power of Sisense Fusion lies in its seamless ability to manage and integrate these machines' learning tasks natively, making it easy for users to build, deploy, and interact with models without needing deep technical expertise or external integrations. The web app just provides a visually engaging way to demonstrate the capabilities of the Sisense Fusion platform. Step 1: Selecting the Data Model and Dataset In the first part of the video, we use the web app that leverages Sisense Fusion’s native API features to select the data model we want to work with. Here’s a breakdown of what happens: Selecting a Data Model: After starting the app, we selected the data model available in Sisense that contains our data. Choosing the Dataset: Once the data model is selected, the app displays all the tables or datasets contained in that model. We then select the dataset we want to train the machine learning model on. Target Variable Selection: After selecting the dataset, the app presents all the columns within the dataset. We select the target variable (the column we want to predict). For example, in customer churn prediction, this could be the Exited column, which indicates whether a customer has churned (1) or not (0). Selecting the Prediction Type: Next, we select whether the task is a regression or classification problem, based on the target variable. Since we are predicting customer churn, we select a classification. Storing Information: Once all selections are made, the app stores this information. This data will later be used when we select the machine learning model for training. Step 2: Exploratory Data Analysis (EDA) After submitting behind the scenes, a Flask application generates an Exploratory Data Analysis (EDA) report based on the dataset. This report provides important insights, such as: Number of customer records. Missing values in the dataset. Relationships between variables. These insights help us select relevant columns to ensure the machine learning model performs optimally. Step 3: Model Training Options We can have multiple options for training our model within Sisense Fusion for example: Auto-Sklearn: This is an open-source AutoML library that automates the model training process. Since it runs locally within Sisense, data never leaves the platform, ensuring data security. However, model training can be computationally expensive, meaning your Sisense cluster should have adequate resources to handle it. AWS Autopilot: This option leverages Amazon Web Services (AWS) infrastructure to train the model, offering more reliable performance. However, it incurs additional costs and requires your data to be sent to AWS for processing. After selecting the model training method, the process begins automatically, and you’ll see the status on the screen as it progresses. Part 4: Integration with AWS SageMaker and Dashboard Creation After selecting the AWS option for model training in the web app a new Custom Code Table is added to the Sisense data model. This Custom Code Table automates the training and deployment of the machine learning model using AWS SageMaker Autopilot. Here’s how it works: Input Parameters for the Notebook The custom code notebook contains a set of input parameters that are passed based on the selections made earlier in Part 1 (dataset and target column). Other parameters include: Dataset: The table you selected for model training. Target Column: The data column you want to predict (churn, in this case). Drop Feature: Columns you wish to exclude from model training (optional). AWS Credentials: Paths to AWS access keys and secret keys to authenticate with AWS. S3 Bucket Name: A unique S3 bucket where the dataset is stored for training. AWS Role ARN: The role with the necessary permissions to access S3 and SageMaker. The notebook code reads these parameters and uses them to call the AWS SageMaker Autopilot API, which automates the model training and deployment process. The trained model is deployed as an endpoint on SageMaker, allowing for online predictions. Creating a Blox widget with dynamic input fields The custom code notebook also contains code that dynamically creates a Sisense dashboard and a Blox widget based on the dataset selected for model training. Here’s what happens: Dynamic Input Fields: Based on the feature columns in the dataset, the Blox widget dynamically generates input fields (boxes) for each feature. This is crucial for online predictions, as it allows users to input new data for the model to predict outcomes in real time. Predict Button: A predict button is added to the widget. When a user inputs new data into the input boxes and clicks the predict button, the system requests the SageMaker endpoint, passing the input data. The model processes this data and returns the prediction, which is displayed in the widget. This setup enables real-time, online predictions directly from the Sisense dashboard, with predictions being powered by the AWS SageMaker endpoint. The dynamic nature of the widget allows the interface to adjust based on the dataset used for training, making the system flexible and user-friendly. The custom code table outputs key information about the trained model and its deployment status, which includes the following columns: Model Name: The name assigned to the trained machine learning model. Metric Name: The evaluation metric used to assess the model's performance, such as accuracy, precision, or recall. Score: The metric score that indicates how well the model performed during evaluation. Local Path: The path within the Sisense environment where the model is stored. Model S3 Location: The S3 location where the trained model is saved after deployment. AWS Model Name: The name of the model is registered in AWS SageMaker. Endpoint Name: The name of the deployed SageMaker endpoint used for making real-time predictions. This output allows users to track key details about the model, including where it's stored, its performance, and the endpoint used for predictions. Step 5: Saving Model Versions The notebook not only trains the model but also saves important metadata within Sisense’s file management storage. This allows you to maintain version control over your models. For each model training session, I store details such as the model metrics (accuracy, precision, etc.) and save each model in a folder based on the timestamp. This ensures easy traceability and allows for multiple versions of models to be stored and retrieved as needed. Step 6: Making Predictions Once the model is trained, we move on to predictions. There are two ways to handle predictions: Batch Predictions (Offline): This method allows you to process thousands of records at once. It's suitable for scenarios where real-time predictions are not required, and predictions can be generated in bulk. Online Predictions (Real-Time): For real-time applications, you can provide individual customer records and receive immediate predictions. This is ideal for real-time decision-making, such as predicting whether a new customer will churn based on their current attributes. Online Predictions (Real-Time): When a custom code table was built it automatically generated a Sisense dashboard and a Blox widget based on the input features used during model training. This integration allows predictions to be embedded directly into Sisense dashboards, enabling users to interact with the model seamlessly. Here’s how it works: The Blox widget takes the input data from the user and sends an API request to Sisense’s custom code transformation. In the case of Auto-Sklearn, the pre-trained model is loaded locally within Sisense, as the model was trained and stored in the local environment. For AWS SageMaker, instead of loading a local model, the system sends a request to the SageMaker endpoint (where the model is deployed) for predictions. The prediction results, whether generated locally with Auto-Sklearn or through the SageMaker API, are returned to the dashboard and displayed within the Blox widget in real-time. This process ensures that predictions are fully integrated into the Sisense environment, providing an interactive and real-time experience for users, with the flexibility to use either local or cloud-based models depending on their needs. Conclusion Sisense Fusion, combined with AutoML, offers an efficient and powerful way to integrate machine learning into real-world business applications. Whether using Auto-Sklearn for local, cost-efficient model training or AWS Autopilot for cloud-based scalability, Sisense provides seamless version control and easy integration into dashboards, making it a comprehensive platform for automating machine learning at scale. If you’re interested in integrating this solution into your Sisense deployment, please reach out to your dedicated Customer Success Manager (CSM) for further assistance. Related Content: https://docs.sisense.com/main/SisenseLinux/ai-overview.htm https://academy.sisense.com/gen-ai Related Content:Using the InternalHttp Function Within Scripts and Plugins
The InternalHttp function is a Sisense function within the Sisense internal Prism object. The prism object and the InternalHttp function is present on all Sisense pages, and can be used in scripts and plugins, including when embedded with various forms of Sisense embedding. It facilitates custom additional API requests to the Sisense server by applying the same request headers used for internal Sisense requests to handle details of API requests such as authentication, CORS, and CSRF.
Introduction To Hyperparameter Optimization - Machine Learning
There are lots of knobs (a.k.a hyperparameters) we can turn when coming up with a Machine Learning model. in the script below, we take the well-known iris dataset, and play around with different hyperparameters. First, a few notes: In machine learning, we generally split our data into 3 sections A training dataset A dev dataset A test dataset We train the model on the training dataset, tune hyperparameters based on the dev dataset, and only run the test dataset when we're evaluating our model. Note that we don't want to tune hyperparameters based on the output of our test dataset in order to avoid overfitting both the test and training dataset if you want to quickly iterate through many different hyperparameters, I recommend using a smaller subset of your data to allow for quick processing. Once some of the hyperparameters have been narrowed down, you can dedicate more time and computational resources to running the full training dataset and creating the model in your production workflow. Without further ado, here is a hyperparameter optimization on K Nearest Neighbors using the corresponding classifier from the Python sklearn library. The below example uses Python 3.6 code. First, we import our libraries: import pandas as pd import numpy as np from sklearn.model_selection import train_test_split from sklearn.datasets import load_iris from sklearn.neighbors import KNeighborsClassifier from sklearn.utils import shuffle from sklearn.metrics import f1_score import matplotlib.pyplot as plt import random I set a random.seed() here to make results reproducible random.seed(123) In this example, we are building a dataframe which contains the iris dataset. However, for your own purposes, you can very well use your SQL output, which will get passed into the Sisense for Cloud Data Teams Python/R editor as a dataframe named df. Your final dataframe must have a list of features (these are the predictor components, you can think of these as your "X," and the corresponding label, you can think of this as your "y"). This dataframe below has 3 features: the sepal length, sepal width, and petal length. We also have a column, 'target,' which contains the name of the iris type. iris = load_iris() df = pd.DataFrame(data= np.c_[iris['data'], iris['target']], columns= iris['feature_names'] + ['target']) df = df.drop('petal width (cm)', axis = 1) Next, we shuffle the dataframe to ensure we are getting a representative sample of our data in the training, dev, and test dataset. df = shuffle(df, random_state = 300) Now, we want to split our target column (y) from all of our features (our x values) df_features = df.drop('target', axis = 1) features = df_features.values target = df["target"].values Now we split our data into training, dev, and test datasets num_rows = df.shape[0] train_cutoff = int(num_rows * 0.6) dev_cutoff = int(num_rows * 0.8) features_train = features[:train_cutoff,:] features_dev = features[train_cutoff:dev_cutoff,:] features_test = features[dev_cutoff:,:] target_train = target[:train_cutoff] target_dev = target[train_cutoff:dev_cutoff] target_test = target[dev_cutoff:] Now, we loop through all our hyperparameters. In this example, we are looping through all values of n_neighbors from 1 to 10. This determines how many of our "neighbors" we are using to classify a given point outside our training dataset. Additionally, we will compare the effectiveness of using "uniform" versus "distance" weights for our model. Note that: A "uniform" weight takes a vote between the N closest neighbors of a point to classify it. A "distance" weight gives more importance to those neighbors that are closest to the point. For example, let's say our KNN is looking at n_neighbors of 5. if of the 5 closest neighbors, the closest of them all is Category A, that will be given more weight than the furthest of the 5 neighbors. To evaluate the dataset, we will run a .predict() function on model using the dev dataset, and score the model using the F1 score (F1 is better at capturing false positives and false negatives, more info on this here). Note that you can achieve this logic with GridSearchCV as well all_k = range(1,11) uniform = [] distance = [] # Looping through all values of k for nbrs in all_k: knn_uni = KNeighborsClassifier(n_neighbors = nbrs, weights = 'uniform') knn_dist = KNeighborsClassifier(n_neighbors = nbrs, weights = 'distance') pred_uni = knn_uni.fit(features_train, target_train).predict(features_dev) pred_dist = knn_dist.fit(features_train, target_train).predict(features_dev) f1_uni = f1_score(target_dev, pred_uni, average = 'macro') f1_dist = f1_score(target_dev, pred_dist, average = 'macro') uniform.append(f1_uni) distance.append(f1_dist) Finally, we plot our results fig, (ax1, ax2) = plt.subplots(1, 2, sharey=True) ax1.plot(all_k, uniform) ax1.set_title('Uniform Weights') ax1.set_ylabel('F1 Score') ax2.plot(all_k, distance) ax2.set_title('Distance Weights') # Use Sisense for Cloud Data Teams to visualize a dataframe, text, or an image by passing data to periscope.table(), periscope.text(), or periscope.image() respectively. periscope.image(fig) Now we analyze the output. The number of neighbors used to generate the model is on the x axis, with the F1 score on the y axis. An F1 score closer to 1 is more desirable here. We see that there are more fluctuations in the uniform weights scoring compared to the distance weights. This is expected as we would anticipate the closest neighbors to be more informative when classifying an iris. Therefore, distance weights looks like a better option. Secondly, it looks like distance weights with 6-8 neighbors yield the highest F1 score. We would go on the lower end of our range here as n_neighbors of 6 is less computationally intensive than n_neighbors of 8 (we have fewer neighbors to account for when classifying each point). Of course, we used a very very small dataset here, so we can expect the lines above to be smoother for a larger dataset. Any other parameters you like to play around with for KNN? Now that we have found our desired hyperparameters, let's run this on our test dataset! Let's put this in a Sisense for Cloud Data Teams view so it's easy to leverage this logic multiple times without rewriting code. See post here for further details!Prepping A Dataframe To Hold Your Training And Testing Data - Machine Learning
Once you have optimized your hyperparameters, it may be nice to create a dataframe that has some of your testing and training data to train your model. Here, we are going to use a KNN model with n_neighbors = 6 and "distance" weights on the iris dataset. We will be creating a dataframe with the following columns: Columns for each of the features (a.k.a. the inputs of your model) Columns for the actual iris classification (the target) Columns for the predicted iris classification (note, this would be just be the actual iris classification for the data that was in the training dataset) A column that labels whether the row is in the "test" or "train" dataset With this data, we can create charts like: Plotting the training data against the testing data Displaying the final accuracy / F1 score as a number overlay on a dashboard With that, below is the Python 3.6 code used. We are building this in a Sisense for Cloud Data Teams view so we can materialize our result on the cache and be able to refer to it in other future analyses. Note that if you have a very large training dataset, you may need to truncate the number of records in this dataframe. Alternatively, you can create a dataframe for just your test dataset. First, we import our desired libraries import pandas as pd import numpy as np from sklearn.datasets import load_iris from sklearn.neighbors import KNeighborsClassifier from sklearn.utils import shuffle import random Next, we set a random seed so we can use the same split from tuning our hyperparameters (see post here). random.seed(123) Then, we create our dataset. Note that you can very easily skip this step and use your SQL output. Be sure that your SQL output has columns for your features (these are the elements used to predict the classification of your data), and the target (this is the classification). iris = load_iris() df = pd.DataFrame(data= np.c_[iris['data'], iris['target']], columns= iris['feature_names'] + ['target']) df = df.drop('petal width (cm)', axis = 1) We then shuffle the dataframe, also with a random_state to ensure we are consistent with the split from hyperparameter tuning df = shuffle(df, random_state = 300) We then remove the target column to get just our feature names df_features = df.drop('target', axis = 1) features = df_features.values target = df["target"].values Next, we split the data into testing and training data. We are using the split of 60% training data - 20% dev data, and 20% test data. We used dev data for tuning hyperparameters, so we won't be putting this in our final dataframe. num_rows = df.shape[0] train_cutoff = int(num_rows * 0.6) dev_cutoff = int(num_rows * 0.8) features_train = features[:train_cutoff,:] features_dev = features[train_cutoff:dev_cutoff,:] features_test = features[dev_cutoff:,:] target_train = target[:train_cutoff] target_dev = target[train_cutoff:dev_cutoff] target_test = target[dev_cutoff:] Now, it's time to generate the model knn_dist = KNeighborsClassifier(n_neighbors = 6, weights = 'distance') pred_dist = knn_dist.fit(features_train, target_train).predict(features_test) Then, we build our dataframe. Note that to do this, we are creating 2 dataframes (one for the test and one for the training data), and concatenating them together train_data = np.concatenate((features_train,np.array([target_train]).T, np.array([target_train]).T), axis = 1) test_data = np.concatenate((features_test,np.array([pred_dist]).T, np.array([target_test]).T), axis = 1) cols = list(df.columns.values) cols.remove('target') cols.extend(['estimated_target','actual_target']) d_train = pd.DataFrame(train_data, columns = cols) d_train['dataset'] = 'train' d_test = pd.DataFrame(test_data, columns = cols) d_test['dataset'] = 'test' # Combine Dataframes df_final = pd.concat([d_train, d_test]) Now, we finally materialize our output periscope.materialize(df_final)Predictive analytics with AutoML for time-series forecasting using Custom Code
Sisense forecasting relies on Sisense’s Cloud Service and machine-learning algorithms. With Sisense, users have the ability to forecast based on historical data in the Sisense widget. Using Custom Code Notebooks, a user can define their input parameters based on their data and train their machine learning model for use as predictive analytics locally meaning without sending the data to Sisense’s Cloud Services.
