Configure AWS passwordless authentication using Amazon RDS PostgreSQL as the Metastore
This page shows you how to configure passwordless authentication for the Metastore in Amazon RDS.
AWS passwordless authentication is an optional feature that allows Collibra Data Quality & Observability to connect to a database without specifying a password.
Typically, database passwords are set in the application configuration, but by leveraging the AWS passwordless option, Collibra Data Quality & Observability retrieves a temporary password from AWS instead. This temporary password is most commonly referred to as a token, which is a lengthy credential containing a password and non-password information, such as how long the token is valid. When configured for the purpose of passwordless authentication, the AWS RDS PostgreSQL database accepts this token instead of a traditional password.
Collibra Data Quality & Observability notes how long the token is valid and stores it in its memory. Whenever a database request is made, Collibra Data Quality & Observability verifies whether the token is still valid, and if it isn't, a new token is requested via an HTTP call from Collibra Data Quality & Observability to AWS.
Note AWS passwordless authentication is currently only supported for EC2 Instance Profile-based authentication with an Amazon RDS Metastore for Collibra Data Quality & Observability standalone and cluster-based deployments. IAM pod role-based authentication support will be available in a future release.
Prerequisites
- Standalone
- Cloud Native
- The Amazon RDS instance must have the IAM authentication method enabled.
- A user must be created in the Amazon RDS database and given the role rds_iam.
- A matching user must be created in AWS with permission to connect to Amazon RDS.
- The ec-2 instance to which Collibra Data Quality & Observability is deployed must have permission to connect to Amazon RDS.
Tip For more information, see the official AWS documentation on Using IAM Authentication with PostgreSQL and Creating and using an IAM policy for IAM database access .
- The Amazon RDS instance must have the IAM authentication method enabled.
- A user must be created in the Amazon RDS database and given the role rds_iam.
- A matching user must be created in AWS with permission to connect to Amazon RDS.
- The EKS instance to which Collibra Data Quality & Observability is deployed must have permission to connect to Amazon RDS.
Tip For more information, see the official AWS documentation on Using IAM Authentication with PostgreSQL and Creating and using an IAM policy for IAM database access .
Setting up AWS passwordless authentication
- Standalone
- Cloud Native
- Open the owl-env.sh file.
- Export the Spring profile "AWS_ENV" environment variable with the following command:
export SPRING_PROFILES_ACTIVE=AWS_ENV
- Export the Spring data source URL environment variables with the following commands, replacing the parameterized
${RDS-url}
sections with your Amazon RDS JDBC URL: export SPRING_DATASOURCE_URL=${RDS-url}
export SPRING_AGENT_DATASOURCE_URL=${RDS-url}
- Export the Spring data source username environment variables with the following commands, replacing the parameterized
${RDS-username}
sections with the Amazon RDS username configured with the profile rds_iam: export SPRING_DATASOURCE_USERNAME=${RDS-username}
export SPRING_AGENT_DATASOURCE_USERNAME=${RDS-username}
- Export the Spring data source password environment variables with the following commands, replacing the parameterized
${value}
sections with a non-empty value: export SPRING_DATASOURCE_USERNAME=${value}
export SPRING_AGENT_DATASOURCE_USERNAME=${value}
- Save and close the owl-env.sh file.
- Open the owl.properties file.
- Append the
authenticationPluginClassName=com.owl.common.datasource.aws.AwsPostgresAuthenticationPlugin
argument to the spring.datasource.url property in the configuration file, similar to the valid Spring data source URL shown in the following example: - Add the following properties to the configuration file:
- Save and close the owl.properties file.
- Restart the Collibra Data Quality & Observability Web App.
Note The password properties are not used for the connection, but Collibra Data Quality & Observability requires them to be present.
Example
spring.datasource.url=jdbc:postgresql://my-rds-instance.xyz.us-east-1.rds.amazonaws.com:5432/mydatabasename?authenticationPluginClassName=com.owl.common.datasource.aws.AwsPostgresAuthenticationPlugin
# Replace {RDS_username} with the Amazon RDS username configured with the profile rds_iam
spring.datasource.username={RDS_username}
# Replace {value} with a non-empty value
spring.datasource.password={value}
- Deploy Collibra Data Quality & Observability with the following Helm Chart variables, replacing the parameterized
${value}
sections with your AWS RDS Metastore details: - Validate that the dq-agent and dq-web pods restarted with the appropriate configuration values after the deployment completes.
--set global.configMap.data.active_profile=AWS_ENV
--set global.configMap.data.metastore_url="${value}"
--set global.configMap.data.metastore_user="${value}"
--set global.configMap.data.metastore_pass="${value}"
Troubleshooting
Problem | Possible solution |
---|---|
I cannot connect to the database. I see a PAM Authentication error in my stacktrace. |
Generally, this indicates an issue with the AWS configuration. Confirm that you can connect to the database using AWS authentication via the AWS cli or a different method outside of Collibra Data Quality & Observability. You can use AWS tools to generate a token and connect. Follow the instructions in the official AWS documentation to verify database connectivity . Note Depending on your database configuration you may need to perform these steps from within an EC2 instance that has appropriate access to your RDS IAM-enabled database. |
I have verified that I can connect to my database via IAM authentication, but I still see authentication errors in my stack trace. |
Closely analyze your IAM policy from the resource running Collibra Data Quality & Observability to the RDS resource. See the official AWS documentation to using an IAM policy for IAM database access . |
After one hour, I notice that the Connections section of Collibra Data Quality & Observability is missing connections. When I have developer tools open in my web browser, I see the failed connections in the network tab. | This is a known issue that affects certain configurations. It will be fixed in a future release. As of today, the only workaround is to restart your application. |