Here is an example that is structured much the same way as my project, this follows all the standards as outlined in the documentation.
.
├── lib
│ └── shared
│ └── common.py
├── packages
│ └── my-namespace
│ ├── my-function
│ │ ├── .include
│ │ ├── __main__.py
│ │ ├── build.sh
│ │ └── requirements.txt
packages/my-namespace/my-function/.include
../../../lib/shared
__main__.py
packages/my-namespace/my-function/build.sh
#!/bin/bash
set -e
virtualenv --without-pip virtualenv
pip install -r requirements.txt --target virtualenv/lib/python3.11/site-packages --compile
packages/my-namespace/my-function/__main__.py
from shared.common import Common
def main(event, context):
return Common.foo()
packages/lib/shared/common.py
import os
class Common:
def foo(self):
return {
'statusCode': 200,
'headers': {
'Content-Type': 'application/json'
},
'body': {
'cwd': os.getcwd()
}
}
However, when calling the function it simply fails with the generic error message:
{
"code": "69c77f0d4dca6d6f2a9a79ad065f2212",
"error": "There was an error processing your request."
}
I have identified that the problem occurs when trying to import the python script that is included from lib. I have ensured: that ./shared/common.py
exists within the functions directory on deployment and, that the code in ./shared/common.py
is free from errors and that the scripts are executable.
If I forgoe the use of lib
and instead copy shared
directly into the my-function
folder (effectively manually performing the include step, resulting in an identical folder structure at runtime), then the import works without issue and the code runs correctly, without any other changes required.
This defies logic and I can only conclude that there is some issue with the build process on DigitalOcean’s end that is resulting in scripts copied by the build process being inaccessible to python at runtime.
Speculatively, perhaps by way of some sort of pre-compilation/caching process rendering them as invisible/invalid to the runtime.
Whatever the cause, it renders it impossible to use lib
functionality in any meaningful way.
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
Upon further testing it appears the issue is that when using .include
the build.sh
script doesn’t appear to run, meaning no virtualenv
is created and import fails for any not built-in packages meaning my example is incorrect.
Instead modifying shared/common.py
to the following:
import os
import numpy
class Common:
def foo(self):
return {
'statusCode': 200,
'headers': {
'Content-Type': 'application/json'
},
'body': {
'cwd': os.getcwd()
}
}
And adding numpy
to the requirement.txt
file would demonstrate the issue. Now from shared.common import Common
would fail with the error No module named 'numpy'
when using .include
- but it would work correctly if no .include
file is used and the shared
folder is manually copied.
Hey Jordan! 👋
Is your function by any chance open-source? If so can you share a link to your GitHub repository so I could look into this further?
I think that it is possible that you might need a build.sh file within the lib
directory in order for this to work as expected. Can you confirm if you have that in place already?
As far as I can see in the docs here, to include the lib/shared/common.py
in your function’s deployment, you need to explicitly reference it in the .include
file. Update the .include
file in packages/my-namespace/my-function/
to:
../../../lib/shared
This tells the build process to include the shared
directory in the deployment package.
Also, your build.sh
looks fine, but make sure it has execute permissions which I’ve seen cause issues in some cases. You can set the execute permission using:
git add --chmod=+x build.sh
If the problem persists, try using the --remote-build
flag to force a fresh build:
doctl serverless deploy --remote-build
Let me know how it goes!
- Bobby
Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.
Full documentation for every DigitalOcean product.
The Wave has everything you need to know about building a business, from raising funding to marketing your product.
Stay up to date by signing up for DigitalOcean’s Infrastructure as a Newsletter.
New accounts only. By submitting your email you agree to our Privacy Policy
Scale up as you grow — whether you're running one virtual machine or ten thousand.
Sign up and get $200 in credit for your first 60 days with DigitalOcean.*
*This promotional offer applies to new accounts only.