Question

How do you download multiple files/videos, simultaneously, on a basic digitalocean droplet/VPS?

How do you download multiple files/videos, simultaneously, on a basic digitalocean droplet?

Let’s say I create a basic digitalocean droplet/VPS, that runs Ubuntu

I start the console

I do

apt install python (to install a package that is needed by a command i’m going to use)

and

wget https://youtube-dl.org/downloads/latest/youtube-dl and chmod 777 youtube-dl

(to install a command called youtube-dl that lets me download youtube videos)

There are some files that I want to download e.g. these two video files

https://www.youtube.com/watch?v=iXFvkJLqPqo (the johnny depp one, about 11min long)

and

https://www.youtube.com/watch?v=-QwtKBC6iQY (a ricky gervais one)

I’ve got a file with links to the videos I want to download

user@basicdroplettest1:~# cat vidfilelist.txt https://www.youtube.com/watch?v=iXFvkJLqPqo https://www.youtube.com/watch?v=-QwtKBC6iQY

This command

while read in; do ./youtube-dl “$in”; done < vidfilelist

will execute youtube-dl on each line of that file, so it’ll do

youtube-dl https://www.youtube.com/watch?v=iXFvkJLqPqo

and

youtube-dl https://www.youtube.com/watch?v=-QwtKBC6iQY

The problem is

it downloads the first one, then when that one has completed, it starts downloading the second one.

So it’s not simultaneous

If I put an ampersand at the end of each line, we see it still doesn’t work.

user@basicdroplettest1:~# cat ./vidfilelist.txt https://www.youtube.com/watch?v=iXFvkJLqPqo & https://www.youtube.com/watch?v=-QwtKBC6iQY &

user@basicdroplettest1:~# while read in; do ./youtube-dl “$in”; done < vidfilelist.txt [youtube] iXFvkJLqPqo: Downloading webpage [download] Destination: Top Johnny Depp Comebacks & Reactions to Questioning While Testifying-iXFvkJLqPqo.mp4 [download] 2.5% of 20.67MiB at 50.65KiB/s ETA 06:47

[download] 3.0% of 20.67MiB at 50.90KiB/s ETA 06:43

[download] 3.2% of 20.67MiB at 50.95KiB/s ETA 06:41

[download] 5.5% of 20.67MiB at 51.04KiB/s ETA 06:31

See above, i’m hitting ENTER, but it’s not going into the background and onto the next one…

So that hasn’t worked.

Any ideas how I can get it to download each of the video files in the file, in the background?

(as opposed to downloading one, followed by the other).

Subscribe
Share

Submit an answer
You can type!ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

Hello,

In addition to what has already been mentioned you can add the & symbol right after the download command, that way it will send the process in the background and without waiting for it to complete, it will start a new download and so on:

while read in; do ./youtube-dl "$in" &; done < vidfilelist

Hope that this helps!

Best,

Bobby

Hi @userdo,

Well, to download them simultaneously, you’ll need to run the command twice. That can be achieved by either creating a second SSH session or using SCREEN.

Check the following article to see what exactly is screen and how it can be used:

https://www.digitalocean.com/community/tutorials/how-to-install-and-use-screen-on-an-ubuntu-cloud-server