Question

Connecting to managed PostgreSQL from functions using psycopg2

Hi,

I just can’t wrap my head around this issue. Any time I try to connect to managed postgres from a function using psycopg2 the function times out.

2022-11-19T18:45:33.700648Z    stderr: The action did not initialize or run as expected.

The action exceeded its time limits of 15000 milliseconds.

Some facts:

  • Functions and databases are part of the same App
  • Database Trusted sources are off, it can be reached from anywhere
  • Function memory limit has been raised to 1024MB
  • Function timeout has been raised to 15000ms
  • It builds and deploys fine. I can import psycopg2 in code without errors.

Is there something missing? This should be the most basic usecase there is. Could you provide a good example/tutorial on this like you have with JS and MongoDB?

My project.yml:

packages:
  - name: sample
    environment:
      DATABASE_URL: <removed>
    functions:
      - name: hello
        runtime: python:default
        limits:
          timeout: 15000
          memory: 1024

My build.sh:

#!/bin/bash

set -e

virtualenv virtualenv
source virtualenv/bin/activate
pip install -r requirements.txt
deactivate

My requirements.txt:

psycopg2-binary==2.9.5

My __main__.py:

from urllib.parse import urlparse
import os
import psycopg2


def main(args):
    print("hello from hello")
    db_url = os.environ.get("DATABASE_URL")
    assert db_url is not None, "DATABASE_URL is missing!"
    print("db_url found")
    p = urlparse(db_url)
    print("parsed")
    conn = psycopg2.connect(
        dbname="name",
        user=p.username,
        password=p.password,
        port=p.port,
        host=p.hostname,
        sslmode="require",
    )
    print("connection created")
    conn.close()
    print("connection closed")
    return {"body": str(44)}

It runs fine if I remove database connection command from it. All the print statements work as expected. But if I have the connection statement there it does not even print anything. Running it locally works just fine.

from urllib.parse import urlparse
import os
import psycopg2


def main(args):
    print("hello from hello")
    db_url = os.environ.get("DATABASE_URL")
    assert db_url is not None, "DATABASE_URL is missing!"
    print("db_url found")
    p = urlparse(db_url)
    print("parsed")
    return {"body": str(44)}

Submit an answer


This textbox defaults to using Markdown to format your answer.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

Sign In or Sign Up to Answer

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

Accepted Answer

Getting back to this. I wrote a simple test function in go and it had the same issue.

package main

import (
	"context"
	"fmt"
	"os"

	"github.com/jackc/pgx/v4"
)

func Main(args map[string]interface{}) map[string]interface{} {
	fmt.Println("starting")
	conn, err := pgx.Connect(context.Background(), os.Getenv("DATABASE_URL"))
	if err != nil {
		fmt.Fprintf(os.Stderr, "Unable to connect to database: %v\n", err)
		os.Exit(1)
	}
	defer conn.Close(context.Background())

	var greeting string
	err = conn.QueryRow(context.Background(), "select 'Hello, world!'").Scan(&greeting)
	if err != nil {
		fmt.Fprintf(os.Stderr, "QueryRow failed: %v\n", err)
		os.Exit(1)
	}
	res := make(map[string]interface{})
	res["body"] = greeting
	fmt.Println(greeting)
	return res
}

But it seems that it is really a issue with Digital Ocean. After creating a database on another cloud provider and suplying it as a connection string it started to work flawlessly. So it seems that DO functions can’t interact with DO PostgresQL…

Bobby Iliev
Site Moderator
Site Moderator badge
November 21, 2022

Hi there,

Seems like you are reaching the timeout time set in your profile settings. Have you tried increasing this a little bit?

Also if you print the p variable, do you see the correct details being parsed after the urlparse(db_url)?

Best,

Bobby

Try DigitalOcean for free

Click below to sign up and get $200 of credit to try our products over 60 days!

Sign up

Get our biweekly newsletter

Sign up for Infrastructure as a Newsletter.

Hollie's Hub for Good

Working on improving health and education, reducing inequality, and spurring economic growth? We'd like to help.

Become a contributor

Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.

Welcome to the developer cloud

DigitalOcean makes it simple to launch in the cloud and scale up as you grow — whether you're running one virtual machine or ten thousand.

Learn more
DigitalOcean Cloud Control Panel