Managing large video / audio file uploads and streaming (Laravel project)

Question

Hi all,

maybe someone can point me in the right direction. I have developed a Laravel PHP project which allows people to upload and watch large video files (as well as audio files). Those files are confident and specific to a group of users, i.e. they shall not be publicly accessible. For now, the PHP code takes care of the permissions as to who can upload a video for which group and who can watch a video. It’s also important that those videos can be streamed in a speedy fashion for the users who are authorized to watch them.

If I think about the traditional way, managing such files requires me to e.g. offer a “resume upload” function, ideally converting the file to the best format (MP4 or such) to conserve storage space, then storing the file in a file directory under a unique name, and finally offering it for download or viewing in the authorized user’s browser. But I’m not sure if this will scale and it seems very old school to handle it all in PHP.

I heard about Azure Media Services which might help with handling such large video files but I would like to try something like this on DO first (sensing better documentation and community support since I’m new to all this).

Here are my specific questions:

Which DO file storage can be recommended for storing and accessing large video files (several GB per file and easily quite a few TB overall)?
How can end users upload new videos to that storage? I would assume that they log in to my PHP app, and then start uploading from there (do I need to build in upload resume functions in case of network interruptions?). In the background, the file gets a unique name and is being sent to storage or (ideally) gets processed first into a uniform video file format (such as MP4 or something)…?
Is the server-side handling of uploads a potential issue due to network bottlenecks or other limitations? How can I make sure that several users can upload large videos at the same time?
Accordingly, how do I deliver files for streaming through my app?

Ideally, I would like to use an endpoint which takes care of uploads (including resuming of uploads), compilation of the video files, naming them in a unique fashion, storing them, and streaming them. The only info I would require is the name of the file and a URL that can be given to the authorized user to watch the video.

Many thanks in advance!

Cheers, Ingo

Rexford Nkansah · Answer

Which DO file storage can be recommended for storing and accessing large video files (several GB per file and easily quite a few TB overall)? DigitalOcean Spaceshttps://www.digitalocean.com/products/spaces/ I would assume that they log in to my PHP app, and then start uploading from there (do I need to build in upload resume functions in case of network interruptions?). In the background, the file gets a unique name and is being sent to storage or (ideally) gets processed first into a uniform video file format (such as MP4 or something)…? Yes. You’ll need to build resume functionality. No idea the complexity of implementing such a feature, but I believe it’s something you’ll have to handle in your Laravel code. The logic of how that will happen, no idea. Is the server-side handling of uploads a potential issue due to network bottlenecks or other limitations? How can I make sure that several users can upload large videos at the same time? If you expect the number of uploads to be as big as YouTube today, then you’re gonna need YouTube’s kinda infrastructure and scale. Depending on your user base, you can always scale your backend server and performance, and increase your bandwidth demands accordingly on DigitalOcean. Accordingly, how do I deliver files for streaming through my app? DigitalOcean CDNhttps://www.digitalocean.com/community/tutorials/using-a-cdn-to-speed-up-static-content-delivery

KFSys · Answer

Here’s how you can approach managing large video/audio uploads and streaming in your Laravel project using DigitalOcean:

1. Storage Recommendation

DigitalOcean Spaces is a great option for storing large video files. Spaces is a scalable object storage solution that integrates with Laravel via the Flysystem adapter. Here’s why it’s suitable:

Scalability: Handles several GB per file and scales to TBs effortlessly.
Integration: Compatible with Laravel for seamless upload and retrieval.
Access Control: Supports fine-grained permissions, ensuring files are accessible only to authorized users.

To integrate Spaces:

Use the Laravel Filesystem package.
Add the DigitalOcean Spaces configuration to your Laravel project (config/filesystems.php).

2. Upload Handling

To allow users to upload large files with resume capabilities:

Use Tus protocol or FineUploader. These libraries enable resumable uploads with network interruption recovery.
Implement chunked uploads in the frontend to break large files into smaller parts before uploading.
On the backend, Laravel can handle merging these chunks into a single file.

Here’s a possible workflow:

User initiates the upload via the Laravel app.
Chunked/resumable uploads are sent to the server using a library like Tus PHP.
Once the upload completes, process the file (e.g., convert to MP4 using FFMPEG).
Store the file in DigitalOcean Spaces with a unique identifier.

3. Processing and Conversion

Convert uploaded videos into a uniform format (e.g., MP4 for compatibility):

Use a queue system (e.g., Laravel Horizon with Redis) to offload processing tasks.
Integrate FFMPEG for server-side video processing. This allows you to compress files and optimize them for streaming.
Process the file after the upload is complete to avoid blocking the user interface.

4. Handling Network Bottlenecks

For simultaneous uploads:

Use a load balancer to distribute traffic across servers.
Optimize server configuration (e.g., Nginx/Apache) for handling large file uploads.
Consider DigitalOcean’s managed Kubernetes for autoscaling your upload and processing services.

5. Secure Streaming

To deliver secure streaming:

Use signed URLs from DigitalOcean Spaces to allow temporary access to files.
For on-the-fly streaming, integrate a video streaming server like NGINX RTMP or Wowza Streaming Engine.
Serve video streams via HLS (HTTP Live Streaming), which is efficient for adaptive bitrate streaming.

6. Full Workflow Example

Upload:
- User logs in to your Laravel app and starts an upload.
- The frontend uses a resumable upload library to send the file in chunks.
- The backend processes the chunks and stores the final file in DigitalOcean Spaces.
Processing:
- A queue processes the file into MP4 using FFMPEG.
- The processed file is stored back in Spaces.
Streaming:
- Generate a signed URL for the requested video.
- Serve the video using HLS or a similar protocol for adaptive streaming.