Tech Titbits » Internet

How to download a file concurrently/simultaneously from different sources in Ubuntu

Ellimist — Thu, 13 May 2010 03:06:19 +0000

In order to achieve this, you need to download an install Aria2, a lightweight, multi-source download utility. It can be installed from it’s package ‘aria2′ from the universe repositories.

sudo apt-get install aria2

Now we need to create a file containing the source URLs for the file(s) we need to download. The source/mirror URLs should be separated by a tab or newline. The mirrors can also be used directly as a command line argument, but that would just make things clumsy.

I’m using Lucid Netbook Edition as an example here. My sources.txt contains the following :

http://mirror.uoregon.edu/ubuntu-releases/lucid/ubuntu-10.04-netbook-i386.iso     http://mirrors.us.kernel.org/ubuntu-releases/lucid/ubuntu-10.04-netbook-i386.iso     http://ubuntu.mirrors.hoobly.com/lucid/ubuntu-10.04-netbook-i386.iso     http://mirrors.pavlovmedia.net/ubuntu-releases/lucid/ubuntu-10.04-netbook-i386.iso     ftp://mirror.vcu.edu//pub/gnu+linux/ubuntu-releases/lucid/ubuntu-10.04-netbook-i386.iso     ftp://ftp.gtlib.gatech.edu/pub/ubuntu-releases/lucid/ubuntu-10.04-netbook-i386.iso     http://ftp.chg.ru/pub/Linux/ubuntu/releases/lucid/ubuntu-10.04-netbook-i386.iso     http://mirror.yandex.ru/ubuntu-releases/lucid/ubuntu-10.04-netbook-i386.iso     http://ubuntu.mirror.ac.za/ubuntu-release/lucid/ubuntu-10.04-netbook-i386.iso     http://ubuntu.hugi.is/releases/lucid/ubuntu-10.04-netbook-i386.iso     http://mz.releases.ubuntu.com/lucid/ubuntu-10.04-netbook-i386.iso     http://releases.ubuntu.nautile.nc/lucid/ubuntu-10.04-netbook-i386.iso

Now we need to start the download using Aria2. Type this in the terminal.

aria2c -i /path/to/sources.txt -s 12

The -i argument specifies the input file and -s argument specifies how many connections will be used to simultaneously download the file. Here, we have 12 mirrors for the .iso file, so I used 12 in the argument. According to the aria2c man page, if the value specified in the -s argument (say, N) is less than the number of mirror URLs, the first N URLs will be used and the rest will be used as backup. Otherwise, if the number of mirrors is less, a single source is used more then once, so that N connections can be made.

Aria2 is a wonderful download utility that support Metalinks and BitTorrent other than the usual http, ftp. It can be used to do a whole range of things, but we will keep ourselves limited to the topic at hand. Maybe we will try out those possibilities in an other post.

.htaccess archived on AskApache.com

Ellimist — Sat, 20 Mar 2010 02:47:31 +0000

An old copy of my .htaccess for AnimorphsfanForum.com seems to have been archived on AskApache.com.

http://static.askapache.com/htaccess-files/htaccess-sample.txt

I hope that it is helping some people.

Update : Following this post, the .htaccess was removed from the site. You can still search for it using Google, though.

The AskApache.com admin has kindly informed us that the file was mistakenly not moved over during a host switch. It has been restored now.

phpBB: Redirect broken links after uninstalling an SEO mod

Google Public DNS – An alternative to OpenDNS?

Ellimist — Sat, 05 Dec 2009 07:01:34 +0000

Google has recently launched Google Public DNS, a public DNS resolver service with an aim to make web browsing faster and secure. OpenDNS, providing the same services and more, has been in the market since 2006. So, which one is better, and is it beneficial to switch from using OpenDNS to Google Public DNS? We will find out.

Features :

Performance – Both GPDNS and OpenDNS employ quite a number of tweaks to increase performance of the end users. Both load balance user traffic and implement “smart” caching techniques to speed up domain resolution.
While this does not directly affect performance, OpenDNS has a nifty tool called CacheCheck, which can be used to check the status of a domain in OpenDNS’s cache and optionally update the domain’s DNS records in the cache. This does score some brownie points for OpenDNS.
Security – Even though Google Public DNS has protection against cache-poisoning, DOS and amplification attacks, it blandly admits that it doesn’t provide any content-filtering/anti-phishing/anti-malware protection. Google says :

Google Public DNS is not a malware-blocking service. Google Public DNS does not perform blocking or filtering of any kind

However, OpenDNS delivers a lot in this sector. In addition to protection against cache-poisoning and other vulnerabilities, it offers web-content filtering, “PhishTank” anti-phishing, malware-protection and whitelisting/blacklisting of domains, all though a web-based dashboard. In addition to this, OpenDNS also features a community based domain tagging system, where users can tag domains into appropriate categories and check the status of an already-tagged domain.

Real-world Performance

I used Namebench to perform DNS benchmarks. The results are as follows:

Mean Response Duration

Fastest Individual Response Duration

The tests proved that Google Public DNS resolver is 5% faster than the OpenDNS resolver.

Conclusion

Even thoughthe real world test found Google Public DNS to be 5% faster than OpenDNS, it actually makes a very little difference in speed. Considering the advanced features like anti-phishing, anti-malware, content-filtering and custom-error pages, OpenDNS is recommended for both network administrators and end users.

Special Note :

Google says that it shows correct results for NXDOMAIN results, ie, due to mistyping or otherwise, if a non-existent domain is tried to be resolved, it will return an error message. OpenDNS, however, redirects such requests to it’s search page where it shows similar domains along with some sponsored links. I couldn’t decide whether this is a good or a bad move, so I didn’t include this in the evaluation process.

No related posts.