I plan to purchase Space for storage purpose, and Droplet to run app code.
App code will perform authentication, uploading and downloading operations.
However, I have some doubt on the correct way to write App code. I’m using Flask + Boto3
I was wondering, should I create a new instance of boto3 client for each file upload request, or use a shared instance? Which is the correct way to do so?
Create a new instance each upload request
@app.route('/', methods=['POST'])
def upload_file():
file = request.files['file']
file.save('/tmp/file.ext')
client = boto3.client('s3')
client.upload_file('/tmp/file.ext', # Path to local file
'my-space', # Name of Space
'file.ext') # Name for remote file
Shared instance for each upload request
client = boto3.client('s3')
@app.route('/', methods=['POST'])
def upload_file():
file = request.files['file']
file.save('/tmp/file.ext')
client.upload_file('/tmp/file.ext', # Path to local file
'my-space', # Name of Space
'file.ext') # Name for remote file
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
Hi there,
In most cases, reusing the Boto3 client instance (i.e., the second approach) is the preferred method. The Boto3 library is designed to be thread-safe and to manage its own internal connection pooling, so you don’t need to worry about creating a new instance for each request. From the docs:
Unlike Resources and Sessions, clients are generally thread-safe. There are some caveats, defined below, to be aware of though.
This practice has several benefits:
-
Performance: Creating a new client for every request can add significant overhead, especially in a highly concurrent environment. Reusing the same client instance allows the underlying HTTP connection to be reused, which can result in substantial performance improvements.
-
Resource utilization: Each new client instance consumes resources (e.g., memory, file descriptors for connections). In a high-throughput environment, creating a new client for each request can quickly exhaust available resources.
However, if you are processing requests with differing configurations (different access keys, regions, etc.), you might have to instantiate a new client for each of those configurations.
In conclusion, unless there’s a specific need for different configurations, it’s generally better and more efficient to reuse the Boto3 client.
Best,
Bobby
Become a contributor for community
Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.
DigitalOcean Documentation
Full documentation for every DigitalOcean product.
Resources for startups and AI-native businesses
The Wave has everything you need to know about building a business, from raising funding to marketing your product.
The developer cloud
Scale up as you grow — whether you're running one virtual machine or ten thousand.
Get started for free
Sign up and get $200 in credit for your first 60 days with DigitalOcean.*
*This promotional offer applies to new accounts only.