Question
Functions: Cannot import a module inside a virtual environment
- Posted on October 18, 2023
- DigitalOcean FunctionsPythonPython Modules
Asked by Ivan Montilla Miralles
Hello,
When using DO Functions, I require to import a Python module and once I do, the Function stops working with an The function exhausted its memory and was aborted. error.
When using doctl, the following message appears: Request accepted, but processing not completed yet. All functions invocation >= 30s will get demoted to an asynchronous invocation.
My example function:
from tronpy import Tron
def main(args):
client = Tron()
wallet = client.generate_address()
return {
"body": {
"wallet": {
"base58check_address": wallet["base58check_address"],
"hex_address": wallet["hex_address"],
"private_key": wallet["private_key"],
"public_key": wallet["public_key"],
},
},
}
This code is valid when run separately in my IDE but doesn’t work when using DO Functions.
I confirm the execution stops at from tronpy import Tron because I also tested this by removing further library function calls.
I also tried this with just the mysql-connector-python library, and it also doesn’t work, so that rules out tronpy failing.
My build.sh file:
#!/bin/bash
# Stop on the first error found
set -e
# Clean build and virtual environment
rm -rf venv
rm -f __deployer__.zip
# Create virtual environment and install dependencies
python3.11 -m venv --without-pip venv
source venv/bin/activate
# --compile --no-cache-dir
pip install "tronpy==0.4.0" --target venv/lib/python3.11/site-packages
deactivate
Notes about my build.sh file:
- if the directory was
virtualenvas docs say, the function doesn’t even run with standard library code. - I’m not using
-r requirements.txt, but the result is equivalent if I did that.
My .include file just in case:
venv
__main__.py
Thanks in advance,
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.
Featured on Community
Get our biweekly newsletter
Sign up for Infrastructure as a Newsletter.
Hollie's Hub for Good
Working on improving health and education, reducing inequality, and spurring economic growth? We'd like to help.
Become a contributor
Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.
I found out a workaround (but it’s not the proper, real solution), I proceed to explain:
In your Function source directory, you may have the
venvdirectory, and it’s not included in Python’ssys.path; therefore, Python cannot see it to import (and running it, will fail).Failure version
Workaround that works for me
Real fix: Additional scenario with
virtualenvbuild.shIn this scenario you don’t need to append anything to
sys.pathas DigitalOcean does it for you:However, be careful with local builds as
Ubuntu 22.04.3 LTSships withPython 3.10.12and thereforevirtualenvwill result in the wrong Python version and your Python Function will not run, at all. Using--remote-buildfixes this, until you can use the appropriate Python version locally.Lesson learned: DigitalOcean will append the directory
/tmpintosys.path, but that’s also the home of/tmp/virtualenv...which if the directory is not called exactlyvirtualenv, it will not work.The docs mention it should be called
virtualenv, but the remote builder logs will not tell you it failed to appendvirtualenvbecause it doesn’t exist. 8 hours hunting the root cause, in case you were wondering.It also doesn’t help that the failure to import a library, appears as memory exhaustion error.
This comment has been deleted