Report this

What is the reason for this report?

Is the connection to hosted postgres TLS 1.2 on public net and TLS 1.3 on private net?

Posted on June 17, 2020

I have a docker image built with my batch code (running deno and pgc4d) and the DO CA cert for a hosted postgres instance.

If I run the docker image (Alpine) locally on my machine (MacOS) using the public connection string, everything works as expected. It can connect to the database, read and write some values and exit normally.

If I deploy the image into a DO hosted Kubernetes cluster using the private connection string, I get an error when it tries to establish a connection to the database.

WARN RS - rustls::session:718 - Sending fatal alert DecodeError
error: Uncaught InvalidData: invalid certificate: BadDER
    at unwrapResponse ($deno$/ops/dispatch_json.ts:43:11)
    at Object.sendAsync ($deno$/ops/dispatch_json.ts:98:10)
    at async Object.startTls ($deno$/tls.ts:70:15)
    at async startTlsPostgres (file:///bin/project/bundle.js:28100:24)
    at async Object.connectPg (file:///bin/project/bundle.js:28073:28)
    at async Object.qipMetricsForDay (file:///bin/project/bundle.js:28428:20)
    at async execute (file:///bin/project/bundle.js:28573:17)
    at async gExpA (file:///bin/project/bundle.js:79:14)
    at async file:///bin/project/bundle.js:28601:1

Now this CA cert is bundled in the docker image, so it’s not changing between my running it locally or in the kube cluster. I have code written in node and other languages that connects to DO postgres just fine. It’s just this code using deno.

The section in rustls/session.rs dealing with DecodeError mentions that warnings are nonfatal for TLS1.2, but are outlawed in TLS1.3. Which leads me to believe that my db client code is connecting via TLS1.2 when I run over the public net but connecting via TLS1.3 when using DO’s private net.

Something is not validating correctly with the CA cert. But I don’t know what.

I have other code that does this sort of thing fine, running in docker locally as well as in the kube cluster and talking to DO’s postgres. But the difference here is that this is the first code to use deno, while forcing validation of the pg host certificate using a CA.

Any ideas for tracking this down?



This textbox defaults to using Markdown to format your answer.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

Heya,

I know it’s been quite a while since the question was originally posted, but I would like to post an answer in case someone runs into it now in 2023.

It’s likely a TLS-related issue, but there are a few things you can check to diagnose and potentially fix the problem.

1. Check the CA certificate: Make sure that the CA certificate is correctly bundled within your Docker image and loaded by your Deno application. You mentioned that the CA cert is the same between your local machine and the Kubernetes cluster, which is good. However, it’s important to confirm this.

2. Check the TLS version: It’s possible that the issue is related to TLS version differences. TLS 1.3 introduced stricter validation rules compared to TLS 1.2. To address this, you can try explicitly specifying the TLS version your Deno application should use.

In your Deno code, before making the connection, try setting the minimum TLS version to 1.2 like this:

JavaScript

  1. import { connect, Tls } from "https://deno.land/std/http/server.ts";
  2. const tls: Tls = {
  3. // Other TLS settings
  4. minVersion: "TLSv1.2", // Specify the minimum TLS version
  5. };
  6. // Establish the connection with the above TLS configuration
  7. const conn = await connect({
  8. hostname: "your-postgres-host",
  9. port: 5432,
  10. tls,
  11. });

This may help ensure that the connection uses TLS 1.2, which might resolve the issue if there’s a TLS version mismatch.

3. Add more logging: To diagnose the issue more specifically, add more detailed logging to your Deno application. You can log the certificate validation process to see where it fails. For example, log the certificates being used, their expiration dates, etc.

4. Check DNS resolution: Make sure that the DNS resolution is working correctly in your Kubernetes cluster. Sometimes, connectivity issues can arise from DNS misconfigurations.

6. Check Kubernetes service configuration: Double-check the configuration of your PostgreSQL service in Kubernetes. Ensure that it’s correctly configured to listen on the private network.

7. Check for proxy: If there’s a proxy or reverse proxy involved, it might interfere with the TLS handshake. Make sure your Docker image and Kubernetes cluster are set up to handle proxies correctly.

8. Check CA certificate format: Ensure that the CA certificate is in the correct format and readable by Deno. It should be in PEM format.

9. Check Kubernetes Pod Security Policies: If you’re using Pod Security Policies in your Kubernetes cluster, they might be preventing certain network operations. Verify that the policies allow your application to establish outgoing network connections.

10. Check Kubernetes service names: Make sure that the PostgreSQL service name or IP used within your Kubernetes cluster is correct.

Hope that this helps!

The developer cloud

Scale up as you grow — whether you're running one virtual machine or ten thousand.

Get started for free

Sign up and get $200 in credit for your first 60 days with DigitalOcean.*

*This promotional offer applies to new accounts only.