Importing logs in an observability tool

You can open log files of all Collibra services in Collibra Console. Instead of analyzing log files in Collibra Console, you can also collect them in the observability tool of your choice. To know where to find the log files and what the content is, go to the Platform configuration section.

The path to the log files is typically /path/to/collibra_data/<service name>/logs.

The following is an example configuration for OpenTelemetry collector to collect the logging of the Collibra Platform service. This may be different for the observability tool that you are using.

receivers:
  # See more details: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/filelogreceiver
  filelog/collibra_dgc:
    include:
      - ${env:COLLIBRA_DATA_PATH}/dgc/logs/dgc.log
    resource:
      service.name: collibra-dgc
    multiline:
      line_start_pattern: \d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}.\d{3}
  
  # Add more filelog receivers for each log that you want to collect

processors:
  # See more details: https://github.com/open-telemetry/opentelemetry-collector/tree/main/processor/batchprocessor
  batch:

  # See more details: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/resourcedetectionprocessor
  resourcedetection:
    detectors:
      - env
      - system

exporters:
  # See more details: https://github.com/open-telemetry/opentelemetry-collector/tree/main/exporter/debugexporter
  debug:

  # Add appropriate exporter(s) here
  # See all available ones: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter

service:
  pipelines:
    logs:
      receivers:
        - filelog/collibra_dgc
        # Add more filelog receivers for each log that you want to collect
      processors:
        - resourcedetection
        - batch
      exporters:
        - logging
        # Add appropriate exporter(s) here