Question

Connecting to managed PostgreSQL from functions using psycopg2

Hi,

I just can’t wrap my head around this issue. Any time I try to connect to managed postgres from a function using psycopg2 the function times out.

2022-11-19T18:45:33.700648Z    stderr: The action did not initialize or run as expected.

The action exceeded its time limits of 15000 milliseconds.

Some facts:

  • Functions and databases are part of the same App
  • Database Trusted sources are off, it can be reached from anywhere
  • Function memory limit has been raised to 1024MB
  • Function timeout has been raised to 15000ms
  • It builds and deploys fine. I can import psycopg2 in code without errors.

Is there something missing? This should be the most basic usecase there is. Could you provide a good example/tutorial on this like you have with JS and MongoDB?

My project.yml:

packages:
  - name: sample
    environment:
      DATABASE_URL: <removed>
    functions:
      - name: hello
        runtime: python:default
        limits:
          timeout: 15000
          memory: 1024

My build.sh:

#!/bin/bash

set -e

virtualenv virtualenv
source virtualenv/bin/activate
pip install -r requirements.txt
deactivate

My requirements.txt:

psycopg2-binary==2.9.5

My __main__.py:

from urllib.parse import urlparse
import os
import psycopg2


def main(args):
    print("hello from hello")
    db_url = os.environ.get("DATABASE_URL")
    assert db_url is not None, "DATABASE_URL is missing!"
    print("db_url found")
    p = urlparse(db_url)
    print("parsed")
    conn = psycopg2.connect(
        dbname="name",
        user=p.username,
        password=p.password,
        port=p.port,
        host=p.hostname,
        sslmode="require",
    )
    print("connection created")
    conn.close()
    print("connection closed")
    return {"body": str(44)}

It runs fine if I remove database connection command from it. All the print statements work as expected. But if I have the connection statement there it does not even print anything. Running it locally works just fine.

from urllib.parse import urlparse
import os
import psycopg2


def main(args):
    print("hello from hello")
    db_url = os.environ.get("DATABASE_URL")
    assert db_url is not None, "DATABASE_URL is missing!"
    print("db_url found")
    p = urlparse(db_url)
    print("parsed")
    return {"body": str(44)}

Submit an answer
Answer a question...

This textbox defaults to using Markdown to format your answer.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

Sign In or Sign Up to Answer

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

Accepted Answer

Getting back to this. I wrote a simple test function in go and it had the same issue.

package main

import (
	"context"
	"fmt"
	"os"

	"github.com/jackc/pgx/v4"
)

func Main(args map[string]interface{}) map[string]interface{} {
	fmt.Println("starting")
	conn, err := pgx.Connect(context.Background(), os.Getenv("DATABASE_URL"))
	if err != nil {
		fmt.Fprintf(os.Stderr, "Unable to connect to database: %v\n", err)
		os.Exit(1)
	}
	defer conn.Close(context.Background())

	var greeting string
	err = conn.QueryRow(context.Background(), "select 'Hello, world!'").Scan(&greeting)
	if err != nil {
		fmt.Fprintf(os.Stderr, "QueryRow failed: %v\n", err)
		os.Exit(1)
	}
	res := make(map[string]interface{})
	res["body"] = greeting
	fmt.Println(greeting)
	return res
}

But it seems that it is really a issue with Digital Ocean. After creating a database on another cloud provider and suplying it as a connection string it started to work flawlessly. So it seems that DO functions can’t interact with DO PostgresQL…

Bobby Iliev
Site Moderator
Site Moderator badge
November 21, 2022

Hi there,

Seems like you are reaching the timeout time set in your profile settings. Have you tried increasing this a little bit?

Also if you print the p variable, do you see the correct details being parsed after the urlparse(db_url)?

Best,

Bobby