Schedule Looker ingestion jobs

You can use Task Scheduler on Windows or Crontab on Mac and Linux to make the lineage harvester run scheduled jobs. In a scheduled job, the lineage harvester uploads the Looker data source information to Collibra.

Collibra automatically creates new assets and relations of the type "Data Element targets / sources Data Element" at specific times, dates or intervals, using the information in your configuration file.

Warning When you run the lineage harvester, Collibra Data Lineage creates all Looker assets in the specified domain (or domains) in Collibra. We highly recommend that you do not move these assets to other domains. If you move assets to other domains, they will be deleted and recreated in the initial Data Catalog BI domain (or domains) when you synchronize Looker. As a consequence, all manually added data of those assets is lost.

Warning Relations that were manually created between Looker assets and other assets via a relation type in the Looker operating model, are deleted after a refresh of the Looker metadata.

Example You created a configuration file with connection information to your Looker environment. You schedule the lineage harvester job to run each Sunday at 23:00. As a result, your Looker metadata is automatically refreshed on a weekly basis.