Technical lineage password manager integration design

When you run the lineage harvester, you can either:

  • Enter the passwords in the console. The passwords are then encrypted and stored in /config/pwd.conf.
    Note Lineage harvester 2022.05 includes an internal format change to the password manager pwd.conf file. This means that if you use Lineage harvester 2022.05, you can no longer use the pwd.conf file with an older lineage harvester version.
  • Provide the passwords via command line, in a prescribed JSON structure via stdin. This allows you to store the passwords locally in your password manager, instead of in your lineage harvester folder.

This topic provides guidance on how to structure the JSON file and which commands to use, to store the passwords locally in your password manager.

Structure of the JSON file

If you prepare a JSON file with your passwords, you have to name the file passwords.json.

The JSON file must have two sections:

  • The catalogs section defines the connection information and credentials to your Collibra Data Intelligence Platform instance.
  • The sources section defines the connection information and credentials to your data sources. You use the same "id" as the id property in the lineage harvester configuration file.

The JSON file must have the following structure:

{
 "catalogs": [
  {
   "url" : "<url-to-collibra-cloud>", 
   "username":"<username-to-sign-in-to-collibra>",
   "password": "<password-to-sign-in-to-collibra>"
  }
 ],
 "sources": [
  {
   "id": "<id-of-your-database>",
   "username": "<database-username>",
   "password": "<database-password>"
  }
 ]
}

Examples of commands

When you run the lineage harvester, you can use one of the following commands to provide the passwords:

Passwords location

Command
a locally stored JSON file cat passwords.json | ./bin/lineage-harvester full-sync --passwords-stdin
a custom script, for example from a password manager <prepare-passwords-command> | ./bin/lineage-harvester full-sync --passwords-stdin

Note Depending on your password manager, you may need different parameters. For example, see the LastPass documentation for the parameters needed by LastPass.