Logging best practices

Recommendation

Collibra maintains a 30-day archive of log files by default, but in special circumstances CollibraSupport can temporarily extend this period.

Impact

  • Comply with industry-specific log retention requirements.
  • Keep your environment performant, to avoid potential risk such as:
    • Issues with restores.
    • Issues with Import jobs.

Recommendations

  • When setting up JDBC logs, set parameters to follow the same folder structure as Jobserver logs to ensure they are written to easy to find locations with the appropriate permissions.
  • In general operation, the default logging level is usually sufficient. You should only set logging levels to a higher level of detail when you are troubleshooting issues with Collibra Support or Engineering teams. You should only maintain this logging change until the issue has been duplicated and captured in the logs. You should then return the logging levels to their default, because higher levels of logging can exhaust the available disk space faster than monitoring tools can detect.
  • By default, the system retains 30 days of logs in the archive. If there is a business or regulatory requirement to retain log files for longer than 30 days, we recommend external scripting to call the Collibra Support REST API (<base console url>/docs/rest/index.html#/support) to move archived files to a non-Collibra storage space, such as an S3 bucket, every 2 weeks.

Validation criteria

Use Collibra Console → Settings → Logs to monitor these recommendations.

Additional information

For more information, go the following resources: