This article is specific for an on-premises Jobserver, and the corresponding Jobserver, that connect with Collibra Platform.
Installation
- Ensure that you use the latest available installer version that matches your Collibra environment. Collibra is deployed on a monthly basis while the on-premises installer versions are only available on a quarterly basis.
Example Collibra Platform is released on a monthly basis while the on-premises environments can only be upgraded on a quarterly basis. For example, Collibra 2020.11 has a corresponding on-premises version 5.7.7. This on-premises version will remain the latest available one for the next two monthly Collibra Platform releases.
- A user with the correct permissions to execute the installation:
- Linux: Ideally, a root or sudo user should be used. The user must be able to execute the installation script on the mounted file system.
- Windows: An administrator user must be used. This user must have full rights on the intended installation drive and directories.
-
Ensure that all required ports are open/listening and not in use by other programs or processes on the new Jobserver.
For the list of default TCP ports, see Overview default ports in Collibra. - Ensure that you can connect to Collibra Console that is used to manage the on-premises Jobserver.
- Ensure that the server that will run the Jobserver meets the minimum system requirements.
- Ensure that the server that will run the Jobserver can communicate with Collibra Platform. For more information, see how you can check the communication.
Configuration
- Ensure that all JVM properties are entered correctly into Collibra Console.
- Ensure that there are no spaces at the beginning or end of each JVM property added. These empty spaces will cause the Jobserver to enter a Failed state and prevent the creation of the spark-jobserver.log file.
- Ensure the name specified in the proxy.url JVM parameter matches the name of the Jobserver defined in Collibra Console under section 18.a and 18.c.
Note This value is case-sensitive, so the name should match between both Collibra Console for Collibra Platform entries and Collibra Console for on-premises Jobserver entries.
- On Linux, the Jobserver may fail to start if the keystore.p12 file is not owned by the "Collibra" user and group. You can verify this by looking at the last entry in the spark-jobserver.log once the Jobserver is in error state.
If this is the issue, you can fix it as follows:- On the server, go to the directory where the p12 file is stored.
- Run the following command:
chown collibra:collibra keystore.p12
- Restart the Jobserver.
Jobserver usage
- Ensure that all drivers are configured properly in accordance to the related documentation. This includes applying all of the required properties, based on the Data Source and Driver type:
- Ensure that your on-premises Jobserver can communicate with the servers that host your data sources. Please work with your Network/Server teams to resolve any network blockages or restrictions that would prevent the Jobserver from successfully connecting to the intended data sources.
Use the Jobserver behind a reverse proxy
See Connection from an on-premises Jobserver to a Collibra Platform.
If you have configured everything correctly, you will see a notice on the Jobserver:
INFO [I/O dispatcher 1] reversehttp.gateway.PollingController - proxy -> no requests polled (204)
In the logs of the Collibra Platform service, you will see the following message stream:
[http-nio-0.0.0.0-4400-exec-12] INFO c.c.r.proxy.ReverseHttpProxyServlet - client -> eea3e5fc-3ed4-4095-beb8-f365cd984973: GET /processes [ajp-nio-0.0.0.0-8080-exec-7] INFO c.c.r.proxy.ReverseHttpProxyServlet - gw <- gw polled eea3e5fc-3ed4-4095-beb8-f365cd984973 [ajp-nio-0.0.0.0-8080-exec-2] INFO c.c.r.proxy.ReverseHttpProxyServlet - gw -> respond eea3e5fc-3ed4-4095-beb8-f365cd984973 [http-nio-0.0.0.0-4400-exec-12] INFO c.c.r.proxy.ReverseHttpProxyServlet - client <- eea3e5fc-3ed4-4095-beb8-f365cd984973: 200