How do you download multiple files/videos, simultaneously, on a basic digitalocean droplet?
Let’s say I create a basic digitalocean droplet/VPS, that runs Ubuntu
I start the console
I do
apt install python (to install a package that is needed by a command i’m going to use)
and
wget https://youtube-dl.org/downloads/latest/youtube-dl and chmod 777 youtube-dl
(to install a command called youtube-dl that lets me download youtube videos)
There are some files that I want to download e.g. these two video files
https://www.youtube.com/watch?v=iXFvkJLqPqo (the johnny depp one, about 11min long)
and
https://www.youtube.com/watch?v=-QwtKBC6iQY (a ricky gervais one)
I’ve got a file with links to the videos I want to download
user@basicdroplettest1:~# cat vidfilelist.txt https://www.youtube.com/watch?v=iXFvkJLqPqo https://www.youtube.com/watch?v=-QwtKBC6iQY
This command
while read in; do ./youtube-dl “$in”; done < vidfilelist
will execute youtube-dl on each line of that file, so it’ll do
youtube-dl https://www.youtube.com/watch?v=iXFvkJLqPqo
and
youtube-dl https://www.youtube.com/watch?v=-QwtKBC6iQY
The problem is
it downloads the first one, then when that one has completed, it starts downloading the second one.
So it’s not simultaneous
If I put an ampersand at the end of each line, we see it still doesn’t work.
user@basicdroplettest1:~# cat ./vidfilelist.txt https://www.youtube.com/watch?v=iXFvkJLqPqo & https://www.youtube.com/watch?v=-QwtKBC6iQY &
user@basicdroplettest1:~# while read in; do ./youtube-dl “$in”; done < vidfilelist.txt [youtube] iXFvkJLqPqo: Downloading webpage [download] Destination: Top Johnny Depp Comebacks & Reactions to Questioning While Testifying-iXFvkJLqPqo.mp4 [download] 2.5% of 20.67MiB at 50.65KiB/s ETA 06:47
[download] 3.0% of 20.67MiB at 50.90KiB/s ETA 06:43
[download] 3.2% of 20.67MiB at 50.95KiB/s ETA 06:41
[download] 5.5% of 20.67MiB at 51.04KiB/s ETA 06:31
See above, i’m hitting ENTER, but it’s not going into the background and onto the next one…
So that hasn’t worked.
Any ideas how I can get it to download each of the video files in the file, in the background?
(as opposed to downloading one, followed by the other).
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.
Hello,
In addition to what has already been mentioned you can add the
&
symbol right after the download command, that way it will send the process in the background and without waiting for it to complete, it will start a new download and so on:Hope that this helps!
Best,
Bobby
Hi @userdo,
Well, to download them simultaneously, you’ll need to run the command twice. That can be achieved by either creating a second SSH session or using SCREEN.
Check the following article to see what exactly is screen and how it can be used:
https://www.digitalocean.com/community/tutorials/how-to-install-and-use-screen-on-an-ubuntu-cloud-server
Want to learn more? Join the DigitalOcean Community!
Join our DigitalOcean community of over a million developers for free! Get help and share knowledge in Q&A, subscribe to topics of interest, and get courses and tools that will help you grow as a developer and scale your project or business.
Sign up now