I’m trying to upload an image using a pre-signed POST URL but it fails somewhat randomly.
Sometimes it returns the correct 204 code, but in roughly 50% of cases it returns 400 with a message:
<?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidArgument</Code><Message>Malformed JSON</Message><BucketName>XXXXXX</BucketName><RequestId>tx00000f65f55d800802749-00658c2492-d1de9a73-fra1b</RequestId><HostId>d1de9a73-fra1b-fra1-zg02</HostId></Error>
I’ve tried different boto3 and requests package versions but to no avail.
The issue started to appear on 24th December and is still occurring.
Here is the test code:
import requests
import requests
from botocore.exceptions import ClientError
import boto3
AWS_S3_ENDPOINT_URL = f"https://fra1.digitaloceanspaces.com"
do_spaces_client = boto3.client(
"s3",
aws_access_key_id="XXXXXXXXXXX",
aws_secret_access_key="XXXXXXX",
region_name="fra1",
endpoint_url=AWS_S3_ENDPOINT_URL,
)
def create_presigned_post(
bucket_name, key, fields=None, conditions=None, expiration=10 * 60
):
try:
response = do_spaces_client.generate_presigned_post(
bucket_name,
key,
Fields=fields,
Conditions=conditions,
ExpiresIn=expiration,
)
except ClientError as e:
print(e)
return None
return response
def upload_image_to_s3(
s3_obj_name: str,
img_bytes: bytes,
presigned_url: str,
presigned_fields,
timeout: int = 180,
) -> int:
files = {"file": (s3_obj_name, img_bytes)}
resp = requests.post(
presigned_url,
data=presigned_fields,
files=files,
timeout=timeout,
)
print(resp.text)
resp.raise_for_status()
return resp.status_code
resp = create_presigned_post(bucket_name="XXXXX",
key="test_image.jpg",
fields={"Content-Type": f"image/jpeg"},
conditions=[{"Content-Type": f"image/jpeg"}],
)
upload_image_to_s3(s3_obj_name="test_image_8.jpg", img_bytes=b"fejifejfejofe",
presigned_url=resp['url'], presigned_fields=resp['fields'])
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
I’m experiencing the same issue, also starting roughly in the time range you mentioned. Pre-signed POST requests to Spaces from browsers fail sporadically, and I don’t see any malformed request data when trying to reproduce this issue.
I just submitted a support ticket for this issue (#08501990).
have the same issue, any prgoress with the support ticket?
I seem to be hitting this as well. Anxious to understand what’s going on!
Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.
Full documentation for every DigitalOcean product.
The Wave has everything you need to know about building a business, from raising funding to marketing your product.
Stay up to date by signing up for DigitalOcean’s Infrastructure as a Newsletter.
New accounts only. By submitting your email you agree to our Privacy Policy
Scale up as you grow — whether you're running one virtual machine or ten thousand.
Sign up and get $200 in credit for your first 60 days with DigitalOcean.*
*This promotional offer applies to new accounts only.