Usage analytics model build failing – troubleshooting and resolution [Linux]

Step-by-Step Guide: 

Common Symptoms

• The Usage Analytics Model build fails each time you run it
• Reports are outdated (typically by several weeks or more)
• Error messages reference issues such as:
• "Failed to read row: [number], Error from CSV. Unexpected character."
• "Headers do not match, corrupted file."
• "Malformed content" or "connector cannot process CSV"

Probable Causes

1. Old or Incompatible CSV Files
1. The build may attempt to process outdated files beyond your configured retention policy (commonly 30 days).
2. These files may use a different schema, missing columns, or malformed content.
2. Schema Mismatch After System Upgrade
1. Upgrades (e.g., to Sisense L2025.2+) may introduce new columns to files, resulting in inconsistent headers between older and newer CSVs in your /usage/query/FinishQuery/ directory.
3. Malformed Content in CSV Columns
1. CSV rows may contain improperly escaped JSON or illegal characters, which can disrupt the parsing process.

Step-by-Step Resolution

Step 1: Check Usage Analytics Retention Settings

• Navigate to your Sisense Usage Analytics settings.
• Verify that "days to retain" is set appropriately (default: 30 days).
  

Step 2: Remove Outdated or Incompatible CSV Files

• Access the File Management..
• Go to /opt/sisense/storage/usage/query/FinishQuery/.
• Delete any CSV files older than the retention period, especially those predating recent upgrades (e.g., files before April 2025 if your upgrade was in May 2025).

Step 3: Validate File Consistency

• Ensure all remaining CSV files have identical headers (column names).
• If you identify files with extra or missing columns:
• Remove those files from the directory.
• Optionally, compare headers of a few recent files to confirm schema alignment.

Step 4: Address Malformed CSV Content

• For files noted in error messages (with line number references), open and inspect the problematic row.
• Correct formatting where possible:
• Nested JSON should be properly escaped and quoted.
• Columns should be separated only by commas, not other characters.
• If corrections aren’t feasible, remove the affected file.

Step 5: Verify Disk Space

• Usage Analytics build may also fail if disk usage exceeds capacity.
• Ensure sufficient free disk space for the build to complete.

Step 6: Run Usage Analytics Build

• After cleanup, re-run the Usage Analytics Model build.
• The build should now succeed with the latest, schema-consistent files only.

Step 7: Confirm Data Range

• Check your dashboard to verify the updated date range in reports (should show recent usage, typically up to the present).: 

Supplementary Information and FAQs

• Why were files outside my retention policy being read?
  • In rare cases, leftover or misplaced files from previous periods may remain. Cleaning up ensures that only relevant, up-to-date files are processed.

• What caused the schema mismatch?
  • System upgrades sometimes add new columns to logs/exports. If both old and new format files exist together, connectors can’t parse the mixed schemas.

• Should I automate this cleanup?
  • Sisense normally rotates and deletes old files per retention settings. If this fails (e.g., due to permissions, disk issues, or interrupted upgrades), manual cleanup may be required.

• What if errors persist after cleanup?
  • If you continue to see malformed row errors, further check the reported line number in the CSV referenced by the error and address any formatting problems.
• Can I request help from Sisense support?
  • Yes. If needed, compile error messages and affected files, and contact Sisense support with detailed information.

Summary of Resolution

By cleaning up old or malformed CSV files, ensuring header consistency post-upgrades, and confirming disk space, Usage Analytics Model builds will resume normal operation, and reporting data will be current. If issues persist, reach out to Sisense support with error details and relevant files.

Add a disclaimer for custom solutions. DO NOT CHANGE IT THIS DISCLAIMER!!! ⤵️
Disclaimer: This post outlines a potential custom workaround for a specific use case or provides instructions regarding a specific task. The solution may not work in all scenarios or Sisense versions, so we strongly recommend testing it in your environment before deployment. If you need further assistance with this, please let us know.