Builds fail because libraries used in Custom Code tables are not installed
Use case You have specific libraries in Notebooks that have to be installed to run Custom code tables. However, these libraries can be uninstalled automatically (during upgrade or other system changes). Solution Add the following Python code into the first cell of each Custom Table Notebook to check if necessary libraries exist and install them if not, before executing the code: Adjust 'libraries' values to the libraries you need to check. import importlib import pandas as pd # List the libraries you want to check libraries = ['numpy', 'pandas', 'matplotlib', 'openpyxl', 'scipy', 'pyarrow'] # Initialize an empty list to store results results = [] for library in libraries: lib_found = importlib.util.find_spec(library) if lib_found is not None: results.append({"library": library, "status": "Previously Installed"}) else: results.append({"library": library, "status": "Newly Installed"}) print(f"{library} NOT FOUND. Installing now...") !pip install {library} Related Content: Sisense Docs: https://docs.sisense.com/main/SisenseLinux/transforming-data-with-custom-code.htm Sisense Academy: https://academy.sisense.com/notebooks-courseResolving SystemError and Memory Allocation Issues in Notebooks
Resolving SystemError and Memory Allocation Issues in Notebooks Summary This article addresses common issues encountered when running notebooks, specifically SystemError and memory allocation errors. It provides step-by-step instructions to resolve these issues and ensure smooth notebook functionality. Main Content Step-by-Step Instructions to Resolve SystemError Identify the Failing Line: Review the notebook to locate the line in the configuration cell that is causing the SystemError. If the line has been commented out, uncomment it to identify the exact issue. Check for Syntax or Logic Errors: Ensure that there are no syntax errors in the notebook code. Verify the logic of the code to ensure it aligns with the intended operations. Run the Notebook Again: Execute the notebook to see if the SystemError persists. If a different error appears, proceed to the next section. Troubleshooting Memory Allocation Issues Identify the Memory Allocation Error: If the error message indicates that the output exceeds the resources allocated to the Notebooks Compute Instance, note the memory allocation details. Increase Memory Allocation: Access the Configuration Manager: Navigate to Admin > System Configuration / System Management > Configuration. Select the 5-click config menu > Warehouse-Service > notebooks.compute.defaultMemoryInMibs. Increase the memory allocation from the default value (e.g., 256 MiB) to a higher value (e.g., 1024 MiB). Verify the Changes: After increasing the memory allocation, run the notebook again to ensure the error is resolved. Additional Tips Testing with Real Data: When testing notebooks with real data, ensure that the sample size is manageable within the allocated resources. For larger datasets, consider increasing the memory allocation proportionally. Feedback and Support: If the issue persists, contact support with detailed information about the error and the context of its occurrence. Providing feedback on your experience can help improve the support process. Check out this related content: Academy Documentation
