Question

Problem running python/Jinja as a build step for a static site.

Posted March 25, 2021 618 views
PythonDigitalOcean App Platform

I have a static site on App Platform that I cannot get to build correctly.

I am using Jinja to generate a static site from some templates. I have a script called build.py in the root directory of the app that does this. It works locally with python 3.8.5, so I know it’s not an issue with the script itself, but maybe the environment. I’ve also gotten the App to run just fine with just the pre-built html made by running the build script locally instead of as a part of the deployment process.

The initial section of the deployment always runs as follows:

=> Initializing build
=> Retrieving source code to /workspace
=> Selecting branch "main"
=> Checking out commit "[a commit hash]"
=> Validating environment

=> Building app using buildpacks
=> Configuring custom build command to be run at the end of the build:
     [see below after this section]
=> Running buildpack detection

digitalocean/python-appdetect 0.0.2
heroku/python                 0.188.4
digitalocean/procfile         0.0.3
digitalocean/custom           0.1.0

=> Building app

-----> Installing python-3.8.7
-----> Installing pip 20.1.1, setuptools 47.1.1 and wheel 0.34.2
-----> Installing SQLite3
rm: cannot remove '/workspace/.heroku/python/apt/state/lists': Directory not empty
Sqlite3 failed to install.
-----> Installing requirements with pip
       Collecting Jinja2==2.11.3
         Downloading Jinja2-2.11.3-py2.py3-none-any.whl (125 kB)
       Collecting MarkupSafe==1.1.1
         Downloading MarkupSafe-1.1.1-cp38-cp38-manylinux2010_x86_64.whl (32 kB)
       Installing collected packages: MarkupSafe, Jinja2
       Successfully installed Jinja2-2.11.3 MarkupSafe-1.1.1

What comes next fails a few different ways:

When I use python build.py as the build command, it fails with ImportError: No module named site. Running python -V as the build command shows that the command invokes version 2.6.

Switching to python3 build.py as the build command instead fails with

Fatal Python error: Py_Initialize: Unable to get the locale encoding
ModuleNotFoundError: No module named 'encodings'

Running python3 -V as the build command shows it invoking version 3.6.

Switching to python3.8 shows bash: python3.8: command not found

From the logs it looks like python3.8 is installed as a part of building the app, but I can’t seem to invoke it.

Is it possible to run a python script with python3.8 as a part of the build step for a static site on App platform, and if so, how can I invoke it correctly here? Not even sure this would fix the issue, but it certainly isn’t working with python 2.6

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

×
Submit an Answer
1 answer

👋🏼 @Treelimb

Thanks for reporting this. I am able to reproduce and it seems to be bug with the build environment. I’ve logged an internal bug report so we can fix it. As a workaround until then, you should be able to use .heroku/python/bin/python instead of python in the build command.

UPDATE: This has been fixed.

  • Hey thanks for the suggestion! Glad it might be something that just gets fixed internally.

    After trying .heroku/python/bin/python build.py I’ve now got the following failure:

    ...
    => Configuring custom build command to be run at the end of the build:
         .heroku/python/bin/python build.py
    => Running buildpack detection
    
    digitalocean/python-appdetect 0.0.2
    heroku/python                 0.188.4
    digitalocean/procfile         0.0.3
    digitalocean/custom           0.1.0
    
    => Checking for cache from a previous build
    Restoring metadata for "heroku/python:shim" from cache
    Restoring data for "heroku/python:shim" from cache
    
    => Building app
    
    -----> No change in requirements detected, installing from cache
    -----> Installing pip 20.1.1, setuptools 47.1.1 and wheel 0.34.2
    -----> Installing SQLite3
    -----> Installing requirements with pip
    Running custom build command: .heroku/python/bin/python build.py
    Python path configuration:
      PYTHONHOME = '/home/apps/.heroku/python'
      PYTHONPATH = '/home/apps'
      program name = '.heroku/python/bin/python'
      isolated = 0
      environment = 1
      user site = 1
      import site = 1
      sys._base_executable = '/workspace/.heroku/python/bin/python'
      sys.base_prefix = '/home/apps/.heroku/python'
      sys.base_exec_prefix = '/home/apps/.heroku/python'
      sys.executable = '/workspace/.heroku/python/bin/python'
      sys.prefix = '/home/apps/.heroku/python'
      sys.exec_prefix = '/home/apps/.heroku/python'
      sys.path = [
        '/home/apps',
        '/home/apps/.heroku/python/lib/python38.zip',
        '/home/apps/.heroku/python/lib/python3.8',
        '/home/apps/.heroku/python/lib/python3.8/lib-dynload',
      ]
    Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
    Python runtime state: core initialized
    ModuleNotFoundError: No module named 'encodings'
    
    Current thread 0x00007f08ecb47200 (most recent call first):
    <no Python frame>
    building: exit status 1
    ERROR: failed to build: exit status 1
     ! Build failed (145)
    

    Some searching shows this might be something do with the PYTHONHOME var, but maybe that doesn’t apply here.

    https://discuss.python.org/t/fatal-python-error-init-fs-encoding-failed-to-get-the-python-codec-of-the-filesystem-encoding/3173

    Maybe this indicates that I am missing a package/dependancy somewhere, but when I run this locally I do so in a python virtual environment so I don’t think it’s the case that I just happen to already have a necessary package installed.

    Also, unrelated I assume but I just noticed that SQLite3 didn’t fail to install this time. I don’t use it for this app anyway, but I thought it was interesting that it failed to install every time previously but worked this time.