How to download a file concurrently/simultaneously from different sources in Ubuntu

In order to achieve this, you need to download an install Aria2, a lightweight, multi-source download utility. It can be installed from it’s package ‘aria2′ from the universe repositories.

sudo apt-get install aria2

Now we need to create a file containing the source URLs for the file(s) we need to download. The source/mirror URLs should be separated by a tab or newline. The mirrors can also be used directly as a command line argument, but that would just make things clumsy.

I’m using Lucid Netbook Edition as an example here. My sources.txt contains the following :

http://mirror.uoregon.edu/ubuntu-releases/lucid/ubuntu-10.04-netbook-i386.iso     http://mirrors.us.kernel.org/ubuntu-releases/lucid/ubuntu-10.04-netbook-i386.iso     http://ubuntu.mirrors.hoobly.com/lucid/ubuntu-10.04-netbook-i386.iso     http://mirrors.pavlovmedia.net/ubuntu-releases/lucid/ubuntu-10.04-netbook-i386.iso     ftp://mirror.vcu.edu//pub/gnu+linux/ubuntu-releases/lucid/ubuntu-10.04-netbook-i386.iso     ftp://ftp.gtlib.gatech.edu/pub/ubuntu-releases/lucid/ubuntu-10.04-netbook-i386.iso     http://ftp.chg.ru/pub/Linux/ubuntu/releases/lucid/ubuntu-10.04-netbook-i386.iso     http://mirror.yandex.ru/ubuntu-releases/lucid/ubuntu-10.04-netbook-i386.iso     http://ubuntu.mirror.ac.za/ubuntu-release/lucid/ubuntu-10.04-netbook-i386.iso     http://ubuntu.hugi.is/releases/lucid/ubuntu-10.04-netbook-i386.iso     http://mz.releases.ubuntu.com/lucid/ubuntu-10.04-netbook-i386.iso     http://releases.ubuntu.nautile.nc/lucid/ubuntu-10.04-netbook-i386.iso


Now we need to start the download using Aria2. Type this in the terminal.

aria2c -i /path/to/sources.txt -s 12

The -i argument specifies the input file and -s argument specifies how many connections will be used to simultaneously download the file. Here, we have 12 mirrors for the .iso file, so I used 12 in the argument. According to the aria2c man page, if the value specified in the -s argument (say, N) is less than the number of mirror URLs, the first N URLs will be used and the rest will be used as backup. Otherwise, if the number of mirrors is less, a single source is used more then once, so that N connections can be made.

Aria2 is a wonderful download utility that support Metalinks and BitTorrent other than the usual http, ftp. It can be used to do a whole range of things, but we will keep ourselves limited to the topic at hand. Maybe we will try out those possibilities in an other post.

One Comment

  1. Bro_isthill says:

    Great one!