Configure AWS passwordless authentication using Amazon RDS PostgreSQL as the Metastore

This page shows you how to configure passwordless authentication for the Metastore in Amazon RDS.

AWS passwordless authentication is an optional feature that allows Data Quality & Observability Classic to connect to a database without specifying a password.

Typically, database passwords are set in the application configuration, but by leveraging the AWS passwordless option, Data Quality & Observability Classic retrieves a temporary password from AWS instead. This temporary password is most commonly referred to as a token, which is a lengthy credential containing a password and non-password information, such as how long the token is valid. When configured for the purpose of passwordless authentication, the AWS RDS PostgreSQL database accepts this token instead of a traditional password.

Data Quality & Observability Classic notes how long the token is valid and stores it in its memory. Whenever a database request is made, Data Quality & Observability Classic verifies whether the token is still valid, and if it isn't, a new token is requested via an HTTP call from Data Quality & Observability Classic to AWS.

Note AWS passwordless authentication is currently only supported for EC2 Instance Profile-based authentication with an Amazon RDS Metastore for Data Quality & Observability Classic standalone and cluster-based deployments. IAM pod role-based authentication support will be available in a future release.

Prerequisites

  • The Amazon RDS instance must have the IAM authentication method enabled.
  • A user must be created in the Amazon RDS database and given the role rds_iam.
  • A matching user must be created in AWS with permission to connect to Amazon RDS.
  • The ec-2 instance to which Data Quality & Observability Classic is deployed must have permission to connect to Amazon RDS.

Tip For more information, see the official AWS documentation on Using IAM Authentication with PostgreSQL and Creating and using an IAM policy for IAM database access .

  • The Amazon RDS instance must have the IAM authentication method enabled.
  • A user must be created in the Amazon RDS database and given the role rds_iam.
  • A matching user must be created in AWS with permission to connect to Amazon RDS.
  • The EKS instance to which Data Quality & Observability Classic is deployed must have permission to connect to Amazon RDS.

Tip For more information, see the official AWS documentation on Using IAM Authentication with PostgreSQL and Creating and using an IAM policy for IAM database access .

Setting up AWS passwordless authentication

  1. Open the owl-env.sh file.
  2. Export the Spring profile "AWS_ENV" environment variable with the following command:
    • export SPRING_PROFILES_ACTIVE=AWS_ENV
  3. Export the Spring data source URL environment variables with the following commands, replacing the parameterized ${RDS-url} sections with your Amazon RDS JDBC URL:
    • export SPRING_DATASOURCE_URL=${RDS-url}?authenticationPluginClassName=com.owl.common.datasource.aws.AwsPostgresAuthenticationPlugin
    • Tip If your URL already has a parameter, use & instead of ?.

    • export SPRING_AGENT_DATASOURCE_URL=${RDS-url}
  4. Export the Spring data source username environment variables with the following commands, replacing the parameterized ${RDS-username} sections with the Amazon RDS username configured with the profile rds_iam:
    • export SPRING_DATASOURCE_USERNAME=${RDS-username}
    • export SPRING_AGENT_DATASOURCE_USERNAME=${RDS-username}
  5. Export the Spring data source password environment variables with the following commands, replacing the parameterized ${value} sections with a non-empty value:
    • export SPRING_DATASOURCE_USERNAME=${value}
    • export SPRING_AGENT_DATASOURCE_USERNAME=${value}
    • Note The password properties are not used for the connection, but Data Quality & Observability Classic requires them to be present.

  6. Save and close the owl-env.sh file.
  7. Open the owl.properties file.
  8. Append the authenticationPluginClassName=com.owl.common.datasource.aws.AwsPostgresAuthenticationPlugin argument to the spring.datasource.url property in the configuration file, similar to the valid Spring data source URL shown in the following example:
  9. Example 
    spring.datasource.url=jdbc:postgresql://my-rds-instance.xyz.us-east-1.rds.amazonaws.com:5432/mydatabasename?authenticationPluginClassName=com.owl.common.datasource.aws.AwsPostgresAuthenticationPlugin

  10. Add the following properties to the configuration file:
  11. Copy
    # Replace {RDS_username} with the Amazon RDS username configured with the profile rds_iam
    spring.datasource.username={RDS_username}

    # Replace {value} with a non-empty value
    spring.datasource.password={value}
  12. Save and close the owl.properties file.
  13. Restart the Data Quality & Observability Classic Web App.
  1. Deploy Data Quality & Observability Classic with the following Helm Chart variables, replacing the parameterized ${value} sections with your AWS RDS Metastore details:
  2. Copy
    --set global.configMap.data.active_profile=AWS_ENV
    --set global.configMap.data.metastore_url="${value}"
    --set global.configMap.data.metastore_user="${value}"
    --set global.configMap.data.metastore_pass="${value}"
  3. Validate that the dq-agent and dq-web pods restarted with the appropriate configuration values after the deployment completes.

Troubleshooting

Problem Possible solution
I cannot connect to the database. I see a PAM Authentication error in my stacktrace.

Generally, this indicates an issue with the AWS configuration. Confirm that you can connect to the database using AWS authentication via the AWS cli or a different method outside of Data Quality & Observability Classic. You can use AWS tools to generate a token and connect.

Follow the instructions in the official AWS documentation to verify database connectivity .

Note Depending on your database configuration you may need to perform these steps from within an EC2 instance that has appropriate access to your RDS IAM-enabled database.

I have verified that I can connect to my database via IAM authentication, but I still see authentication errors in my stack trace.

Closely analyze your IAM policy from the resource running Data Quality & Observability Classic to the RDS resource.

See the official AWS documentation to using an IAM policy for IAM database access .

After one hour, I notice that the Connections section of Data Quality & Observability Classic is missing connections. When I have developer tools open in my web browser, I see the failed connections in the network tab. This is a known issue that affects certain configurations. It will be fixed in a future release. As of today, the only workaround is to restart your application.