Lineage harvester system requirements
To install and run the lineage harvester, you have to meet the following requirements.
Software requirements
Java Runtime Environment version (JRE) 17 or newer, or OpenJDK 17 or newer.
Hardware requirements
You need to meet the hardware requirements to install and run the lineage harvester.
Minimum hardware requirements
You need the following minimum hardware requirements:
- 2 GB RAM
- 1 GB free disk space
Recommended hardware requirements
The minimum requirements are most likely insufficient for production environments. We recommend the following hardware requirements:
- 4 GB RAMTip 4 GB RAM is sufficient in most cases, but more memory could be needed for larger harvesting tasks. For instructions on how to increase the maximum heap size, see the advice on how to resolve Java heap space errors, in the Collibra Support Portal.
- 20 GB free disk space
Network requirements
The lineage harvester uses the HTTPS protocol by default and uses port 443.
You need the following minimum network requirements:
- Firewall rules so that the lineage harvester can connect to:
- The host names of all data sources in your lineage harvester configuration file.
- All Collibra Data Lineage service instances in your geographic location:
Region Current service instances New service instances
IP address DNS name DNS name aws-ca 15.222.200.199 techlin-aws-ca.collibra.com techlin-ca-central-1.collibra.com aws-eu 18.198.89.106 techlin-aws-eu.collibra.com techlin-eu-central-1.collibra.com aws-sg 13.228.38.245 techlin-aws-sg.collibra.com techlin-ap-southeast-1.collibra.com aws-us 54.242.194.190 techlin-aws-us.collibra.com techlin-us-east-1.collibra.com gcp-au 35.197.182.41 techlin-gcp-au.collibra.com techlin-australia-southeast1.collibra.com gcp-ca 34.152.20.240 techlin-gcp-ca.collibra.com techlin-northamerica-northeast1.collibra.com gcp-eu 35.205.146.124 techlin-gcp-eu.collibra.com techlin-europe-west1.collibra.com gcp-sg 34.87.122.60 techlin-gcp-sg.collibra.com techlin-asia-southeast1.collibra.com gcp-uk 35.234.130.150 techlin-gcp-uk.collibra.com techlin-europe-west2.collibra.com gcp-us 34.73.33.120 techlin-gcp-us.collibra.com techlin-us-east1.collibra.com Important We are migrating the Collibra Data Lineage service instances to new DNS names. The migration will be completed by October 31, 2024. You can already start referring to the new DNS names in addition to the existing ones. If you have network infrastructure that requires traffic to be explicitly configured, we recommend that you adjust your configurations to also accommodate the new DNS names by October 31, 2024, to mitigate any interruptions of service. Removing any existing DNS names is not required yet during this migration, and should not be done before the end of October 2024.The IP addresses for the new DNS names are different from the existing ones. We recommend that you only use the new DNS names in your network configurations, as the IP addresses for the new instances are subject to change periodically. If you need to use the IP addressed for the new instances in your network configuration, we recommend using a command line utility like nslookup to query the DNS and obtain the mapping between domain name and IP address. For more information, see the Technical Lineage DNS Change article in the Support Portal.
NoteThe lineage harvester connects to different Collibra Data Lineage service instances based on your geographic location and cloud provider. If your location or cloud provider changes, the lineage harvester rescans all your data sources.
- You have to allow all Collibra Data Lineage service instances in your geographic location.
-
Regardless of your geographic location, you have to allow one of the following. This is required to retrieve the API Key. No other data is transferred.
Region Current service instances New service instances
IP address DNS name DNS name aws-ca 15.222.200.199 techlin-aws-ca.collibra.com techlin-ca-central-1.collibra.com aws-eu 18.198.89.106 techlin-aws-eu.collibra.com techlin-eu-central-1.collibra.com aws-us 54.242.194.190 techlin-aws-us.collibra.com techlin-us-east-1.collibra.com gcp-us 34.73.33.120 techlin-gcp-us.collibra.com techlin-us-east1.collibra.com In addition, we highly recommend that you always allow the techlin-aws-us instance as a backup, in case the lineage harvester cannot connect to other Collibra Data Lineage service instances.
Region Current service instances New service instances
IP address DNS name DNS name aws-us 54.242.194.190 techlin-aws-us.collibra.com techlin-us-east-1.collibra.com