Uploading Files Locally Using Rest (Linux)
In Sisense Linux distribution you are able to upload files to the local environment using a 3rd party tool, which is embedded in the Sisense platform.
Power-Shell 1 file upload Example - Version <=L20121.3
The script below will allow the user to upload files programmatically, using REST command. The script itself is written in Powershell.
Change the first 4 parameters
dns - The URL you use for your Sisense site
RemoteLocation - The location of the target file within File Manager
FilePath - The local location of the file from the Local Server
<span>$dns= 'https://myhost.sisense.com'
</span><span>$RemoteLocation= 'data/Test'
</span><span>$FilePath = 'C:\myfolder\myfile.csv';
</span><span>$AUTH_TOKEN = 'Bearer <Token>'
</span>
<span>###############################################################
</span><span>$pos = $FilePath.LastIndexOf("\")
</span><span>$URL = $dns+'/app/explore/!/upload?vId=0&rename=0&to=/'+$RemoteLocation;
</span><span>$filename = $FilePath.Substring($pos+1)
</span><span>$fileBytes = [System.IO.File]::ReadAllBytes($FilePath);
</span><span>$fileEnc = [System.Text.Encoding]::GetEncoding('UTF-8').GetString($fileBytes);
</span><span>$boundary = [System.Guid]::NewGuid().ToString();
</span><span>$LF = "`r`n";
</span><span>$Headers = @{'Authorization'=$AUTH_TOKEN};
</span>
<span>$bodyLines = (
</span><span> "--$boundary",
</span><span> "Content-Disposition: form-data; name=`"filename`"; filename=`"$filename`"",
</span><span> "Content-Type: application/octet-stream$LF",
</span><span> $fileEnc,
</span><span> "--$boundary--$LF"
</span><span>) -join $LF
</span>
<span>Invoke-RestMethod -Uri $URL -Method Post -ContentType "multipart/form-data; boundary=`"$boundary`"" -headers $Headers -Body $bodyLines</span>Python Example For Folder Syncing (Upload Only)
In this example you can set a local folder with sub-folders to sync with a File Management virtual library.
First run will upload and update all files and create a lastRunTime.json file for reference of last run.
On the next run only files that the update date is larger than the last run time of the application will be uploaded/updated
Config.ini file sample:
<span>[DEFAULT]
</span>
<span># Sisense URL
</span><span>host = https://test.sisense.com
</span>
<span># Path to store files on target machine
</span><span>remoteLocation= data/Test
</span>
<span># Path to folder or file which should be transferred
</span><span># If folder is specifeed, all subfolders will be also proccessed
</span><span>path = C:\Users\Documents\Test\
</span>
<span># Sisense API token
</span><span>token = Bearer TOKEN
</span>
<span># Script will store last modified time for each processed file. This will allow it to upload only modified files on next executions
</span><span>lastRunFilename = lastRunTime.json
</span>Droppy.py file sample:
Droppy.py example Version <=L2021.3
Version >= L2021.5
<span>import uuid
</span><span>import requests
</span><span>import io
</span><span>import json
</span><span>import os
</span><span>import time
</span><span>from datetime import datetime
</span><span>import configparser
</span><span>
</span><span>
</span><span>config = configparser.ConfigParser()
</span><span>config.read('config.ini')
</span><span>
</span><span>lastRunFilename = config.get('DEFAULT','lastRunFilename')
</span><span>host = config.get('DEFAULT','host')
</span><span>remoteLocation = config.get('DEFAULT','remoteLocation')
</span><span>path = config.get('DEFAULT','path')
</span><span>token = config.get('DEFAULT','token')
</span><span>
</span><span># Cut slashes at the end of paths if they are exist
</span><span>if path[-1] == '\\':
</span><span> path = path[:-1]
</span><span>if remoteLocation[-1] == '/':
</span><span> remoteLocation = remoteLocation[:-1]
</span><span>
</span><span># Function to compare stored and file last modified time
</span><span>def compareModificationTime(filePath):
</span><span>
</span><span> if (filePath in lastRunTime and os.path.getmtime(filePath)>int(lastRunTime.get(filePath))) or (filePath not in lastRunTime):
</span><span> return True
</span><span> else:
</span><span> return False
</span><span>
</span><span># Function to upload file to Droppy via Sisense API
</span><span>def uploadFile (filesNames, folderPath):
</span><span> def get_auth(token):
</span><span> headers = {
</span><span> 'Authorization': token,
</span><span> }
</span><span>
</span><span>
</span><span> x_auth = requests.post(f'{host}/app/explore/api/login', headers=headers)
</span><span> if x_auth.status_code==200:
</span><span> return x_auth.text
</span><span> else:
</span><span> print ('Cannot get x-auth token. Check API token validity')
</span><span> return False
</span>
<span> # # SERVICE FUNCTION
</span><span> # # Do NOT modify
</span><span> x_auth = get_auth(token)
</span><span> if os.path.isfile(path):
</span><span> globalFolderPath = os.path.dirname(path)
</span><span> else:
</span><span> globalFolderPath = path
</span><span>
</span><span> if folderPath != globalFolderPath:
</span><span> remotePath = remoteLocation + folderPath.replace(globalFolderPath,'')
</span><span> else:
</span><span> remotePath = remoteLocation
</span><span>
</span><span> remotePath = remotePath.replace('\\','/')
</span><span>
</span><span> files = []
</span><span> header = {'Authorization': token, 'x-auth': x_auth,}
</span><span> notSentFiles = []
</span><span> for filename in filesNames:
</span><span> filePath = '%s\\%s'%(folderPath, filename)
</span><span> if compareModificationTime(filePath):
</span><span> files.append(filePath)
</span><span> lastRunTime [filePath] = int(time.time())
</span><span> else:
</span><span> notSentFiles.append(filePath)
</span><span> if files:
</span><span> for file in files:
</span><span> with open(file,'rb') as ff:
</span><span> file=os.path.basename(file)
</span><span> r = requests.post(f'{host}/app/explore/api/resources/{remotePath}/{file}?override=true', headers=header,data=ff)
</span><span> print(remotePath+'/'+file, r.status_code, r.reason)
</span><span>
</span><span> return notSentFiles
</span><span>
</span><span>
</span><span>try:
</span><span> f = open(lastRunFilename, "r")
</span><span> timestamp = f.read()
</span><span> lastRunTime = json.loads(timestamp)
</span><span>except:
</span><span> lastRunTime = {}
</span><span>
</span><span>
</span><span>
</span><span>notSentFiles=[]
</span><span>if ( os.path.isdir(path)):
</span><span>
</span><span> for root, subdirs, files in os.walk(path):
</span><span> if files:
</span><span>
</span><span> notSentFiles += uploadFile (files, root)
</span><span>
</span><span> time.sleep(0.1)
</span><span>elif ( os.path.isfile(path)):
</span><span>
</span><span> notSentFiles += uploadFile ([os.path.basename(path)], path.rsplit('\\', 1)[0])
</span><span>
</span>
<span>elif not os.path.exists(path):
</span><span> print ("Specified path doesn't exist")
</span><span>
</span><span>if notSentFiles:
</span><span> print ('These files were not uploaded because they were not modified:')
</span><span> for file in notSentFiles:
</span><span> print (file)
</span><span>
</span><span>
</span><span>with open(lastRunFilename, 'w') as f:
</span><span> json.dump(lastRunTime, f)
</span>
0 comments
