How to make presigned urls in python without AWS libs?

Dear support/colleagues.

I am trying to generate my own presigned url for DO spaces and has started from Amazons example here:

https://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html

but without luck and I am now asking ou for help.

This presigned url

https://fra1.digitaloceanspaces.com/cpi-germany/cpi-green-transparent-700x700.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=IB6Q77Q7CAUEX3PJ4DE6%2F20190906%2Ffra1%2Fs3%2Faws4_request&X-Amz-Date=20190906T072212Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=2a1a17581e0c0e59d3b188972c3848dc3867f7c5d8a44fa429c7d300acbcbea1

is generated by the web interface and works for me. I can download the png with curl.

I am then trying this:

import sys, os, base64, datetime, hashlib, hmac, urllib
import requests # pip install requests

# ************* REQUEST VALUES *************
method = 'GET'
service = 's3'
host = 'fra1.digitaloceanspaces.com'
region = 'fra1'
endpoint = 'https://fra1.digitaloceanspaces.com'

spacename = "cpi-germany"   #Or bucket
objectname = "cpi-green-transparent-700x700.png"

# Key derivation functions. See:
# http://docs.aws.amazon.com/general/latest/gr/signature-v4-examples.html#signature-v4-examples-python
def sign(key, msg):
    return hmac.new(key, msg.encode('utf-8'), hashlib.sha256).digest()

def getSignatureKey(key, dateStamp, regionName, serviceName):
    kDate = sign(('AWS4' + key).encode('utf-8'), dateStamp)
    kRegion = sign(kDate, regionName)
    kService = sign(kRegion, serviceName)
    kSigning = sign(kService, 'aws4_request')
    return kSigning

# Read AWS access key from env. variables or configuration file. Best practice is NOT
# to embed credentials in code.
access_key = os.environ.get('AWS_ACCESS_KEY_ID')
secret_key = os.environ.get('AWS_SECRET_ACCESS_KEY')
if access_key is None or secret_key is None:
    print('No access key is available.')
    sys.exit()

# Create a date for headers and the credential string
t = datetime.datetime.utcnow()
amz_date = t.strftime('%Y%m%dT%H%M%SZ') # Format date as YYYYMMDD'T'HHMMSS'Z'
datestamp = t.strftime('%Y%m%d') # Date w/o time, used in credential scope


# ************* TASK 1: CREATE A CANONICAL REQUEST *************
# http://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html

# Because almost all information is being passed in the query string,
# the order of these steps is slightly different than examples that
# use an authorization header.

# Step 1: Define the verb (GET, POST, etc.)--already done.

# Step 2: Create canonical URI--the part of the URI from domain to query 
# string (use '/' if no path)
canonical_uri = '/'+spacename+'/'+objectname

# Step 3: Create the canonical headers and signed headers. Header names
# must be trimmed and lowercase, and sorted in code point order from
# low to high. Note trailing \n in canonical_headers.
# signed_headers is the list of headers that are being included
# as part of the signing process. For requests that use query strings,
# only "host" is included in the signed headers.
canonical_headers = 'host:' + host + '\n'
signed_headers = 'host'

# Match the algorithm to the hashing algorithm you use, either SHA-1 or
# SHA-256 (recommended)
algorithm = 'AWS4-HMAC-SHA256'
credential_scope = datestamp + '/' + region + '/' + service + '/' + 'aws4_request'

# Step 4: Create the canonical query string. In this example, request
# parameters are in the query string. Query string values must
# be URL-encoded (space=%20). The parameters must be sorted by name.
# use urllib.parse.quote_plus() if using Python 3
canonical_querystring = ''
canonical_querystring += 'X-Amz-Algorithm=AWS4-HMAC-SHA256'
canonical_querystring += '&X-Amz-Credential=' + urllib.quote_plus(access_key + '/' + credential_scope)
canonical_querystring += '&X-Amz-Date=' + amz_date
canonical_querystring += '&X-Amz-Expires=30'
canonical_querystring += '&X-Amz-SignedHeaders=' + signed_headers

# Step 5: Create payload hash. For GET requests, the payload is an
# empty string ("").
payload_hash = hashlib.sha256(('').encode('utf-8')).hexdigest()

# Step 6: Combine elements to create canonical request
canonical_request = method + '\n' + canonical_uri + '\n' + canonical_querystring + '\n' + canonical_headers + '\n' + signed_headers + '\n' + payload_hash


# ************* TASK 2: CREATE THE STRING TO SIGN*************
string_to_sign = algorithm + '\n' +  amz_date + '\n' +  credential_scope + '\n' +  hashlib.sha256(canonical_request.encode('utf-8')).hexdigest()

# ************* TASK 3: CALCULATE THE SIGNATURE *************
# Create the signing key
signing_key = getSignatureKey(secret_key, datestamp, region, service)

# Sign the string_to_sign using the signing_key
signature = hmac.new(signing_key, (string_to_sign).encode("utf-8"), hashlib.sha256).hexdigest()


# ************* TASK 4: ADD SIGNING INFORMATION TO THE REQUEST *************
# The auth information can be either in a query string
# value or in a header named Authorization. This code shows how to put
# everything into a query string.
canonical_querystring += '&X-Amz-Signature=' + signature


# ************* SEND THE REQUEST *************
# The 'host' header is added automatically by the Python 'request' lib. But it
# must exist as a header in the request.
request_url = endpoint + canonical_uri + "?" + canonical_querystring

print request_url;

print('\nBEGIN REQUEST++++++++++++++++++++++++++++++++++++')
print('Request URL = ' + request_url)
r = requests.get(request_url)

print('\nRESPONSE++++++++++++++++++++++++++++++++++++')
print('Response code: %d\n' % r.status_code)
print(r.text)
 

but get the following output

python main.py
https://fra1.digitaloceanspaces.com/cpi-germany/cpi-green-transparent-700x700.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=VPXIVAYZ5QOPAITB2VA4%2F20190906%2Ffra1%2Fs3%2Faws4_request&X-Amz-Date=20190906T072814Z&X-Amz-Expires=30&X-Amz-SignedHeaders=host&X-Amz-Signature=1e5c9c73150eb3095e694a2bbab6220402aa596d9ea62b72e3a89550cd20e686

BEGIN REQUEST++++++++++++++++++++++++++++++++++++
Request URL = https://fra1.digitaloceanspaces.com/cpi-germany/cpi-green-transparent-700x700.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=VPXIVAYZ5QOPAITB2VA4%2F20190906%2Ffra1%2Fs3%2Faws4_request&X-Amz-Date=20190906T072814Z&X-Amz-Expires=30&X-Amz-SignedHeaders=host&X-Amz-Signature=1e5c9c73150eb3095e694a2bbab6220402aa596d9ea62b72e3a89550cd20e686

RESPONSE++++++++++++++++++++++++++++++++++++
Response code: 403

<?xml version="1.0" encoding="UTF-8"?><Error><Code>SignatureDoesNotMatch</Code><RequestId>tx000000000000009440137-005d720a8e-c8079e-fra1a</RequestId><HostId>c8079e-fra1a-fra1</HostId></Error>

When I compare the url I generate with this python script with the one I got from the DO webinterface, they are almost identical. The access key is different, obviously I assume, and the signature is different.

Now, I am basically doing whatever the Amazon example does, so I am a bit out of ideas.

Can anyone spot my obvious error or does anyone have a snippet that works?

Best regards, Morten

P.S.: The python script is only used to get to understand the process, as I need to use Dart for production. As far as I can tell, there are no snippets on making presigned urls for dart, so I feel a bit on my own here. If anybody has a snippet for dart that I would be happy to see it.