Group test: download managers
Download managers exist for two reasons. Firstly, they help organise your downloads, moving them to a single, central location on your desktop. Secondly, they help to improve download performance. But with so many around, which to choose? Let us help you...
Organisation is very important if you need to download many files from different locations. The kind of downloads we're talking about are those you normally come across using a web browser. BitTorrent and other peer-to-peer transfer protocols normally include their own download manager clients anyway.
Both Gnome and KDE will open a separate window for each file you copy from a remote system, for instance. Firefox features its own download window, but you only have control over cancelling and clearing the download queue. The download managers we'll look at will pool downloads into a single window, enable you to pause, resume and reorder the download queue, as well as enable certain advanced features such as parallel and segmented downloads.
Parallel and segmented downloads are used to speed up the download process by tackling two bottlenecks in the client-server relationship. Many servers will limit the bandwidth on a single connection so that a single connection doesn't swamp other connections to the same server. By splitting the file into segments and using a different connection for each one, this limitation is sidestepped.
This might not always be ethical, as your multiple connections may be blocking other users from downloading the file, but it will improve your download speed if the server bandwidth isn't inundated with demand. If it is, parallel downloads will help, especially with a new distro release. A parallel download will grab sections of the same file from different remote locations, spreading the download burden across several servers. This is ideal with Linux distributions, because most will use a mirror system to offer the download on several machines at once.
How we tested
The ideal download manager will be easy to use and increase download efficiency. If it adds too much complexity, we're far more likely to stick with the default download option rather than jump through a couple of extra hoops just to add a file to a download queue. And that's how we approached this group test.
Some of the more advanced managers with the most features were also the hardest to use, which would stop most people using them. As a result, our verdict is based on that delicate balance between features and function. We wanted faster downloads, but we didn't want to have to think too much about it. So we judged our seven download managers on the number of features they provide, ease of use, download efficiency and user-interface design, even on those run from the command line.
KGet let's you download exactly what you need, which makes it a great choice if you're a KDE user.
KGet is only a practical solution for KDE users. Its core functionality is tightly woven into the KDE desktop, and specifically, into Konqueror. This means that if you use neither, there's very little in KGet to warrant its use. If, on the other hand, you are a KDE user, this level of integration is KGet's strongest feature, and offers a significant advantage over most of the other download managers we're looking at.
For this reason, it's surprising that KGet is seldom installed by default. You nearly always have to install it after the event through your distro's package manager. When you launch it for the first time, you're asked whether you want to make KGet the download manager for Konqueror. The sensible answer is yes, as this will give KGet permission to transparently take over download duties for Konqueror.
Clicking on a link from within Konqueror will send the download request to KGet, which will automatically add the file to the queue and download the file. This really helps declutter your desktop, because the download status for each file is now listed in a single window, rather than one for each download.
And it's from this window that you can pause and resume downloads, either globally or one by one. Downloads will also automatically continue if the connection happens to drop, or if you log out of your desktop and log in again. This simple overview is a clear and concise way of seeing exactly what's being downloaded, and how much bandwidth each download is consuming.
The other change you'll find after installation is a new item added to Konqueror's Tools menu. From here you can choose to 'Show Drop Target' or 'List All Links'. The first option really shouldn't be in the menu; it's a general function that can also be selected from the KGet applet sitting in your desktop panel. When enabled, it draws a large blue arrow icon on to your desktop background.
You can now drag HTTP and FTP URLs directly from your web and file browser onto this dock (Firefox included), and KGet will obligingly download the files in the background. By default, KGet will ask where you want them stored, but you can easily pre-configure this option in the KGet configuration window. A clever addition is the ability to save to a location that's dependent on file type. You could save images to one directory and music files to another, for example.
But we've saved the best feature till last. The other option in the additional Konqueror menu is a lifesaver. Choose the List All Links option and the window that opens will show every file linked to from the current web page. This is invaluable if you often need to download a glut of files from a single page. It could be a project's main website, for instance, littered with additional files and dependencies.
The List All Links window helps you download exactly what you need without resorting to a single click on every vague blue link in a web page. Click on the Audio button, and you'll see every music file linked to from the current web page. Type '*.mp3' into the search box, and you'll see every MP3 file. This is the single best feature in KGet, making it an essential install if you use the KDE desktop and download plenty of files.
Verdict: A good choice for KDE users, but sadly, it's missing any advanced download acceleration. 8/10
Wget: the trouble with even the most useful command line utilities is that they nearly always look boring in a screen grab.
Yes, we do mean the command line tool that often needs to be installed manually and dates back to a time when the world wide web was but a budding begonia. But Wget is as essential today as it must have been to our great-grandparents, when they were hand-winding the internet way back in 1996. On more than one occasion, Wget was the only tool in this group test that could save a broken X configuration and download the Nvidia drivers to fix our display.
It's fantastically simple to use too. Type 'wget' followed by the URL of the file, for instance, and Wget simply grabs the files and saves it to the current location. For a command line tool, even the user-interface isn't so bad. A progress bar stretches across the terminal, and this drops to a new line when certain milestones in the downloaded amount have been reached.
Wget is also very easy to understand and gives a clear indication of exactly how well your download is progressing, including the download speed, as well as an ETA for the completed file. It will even resume and restart interrupted downloads, retry broken links and run in the background if you specify the --background argument at runtime.
And that's not the limit to its download capabilities. Wget can also be used to mirror and download a website. The -r -l 0 argument will recursively traverse the entire website, parsing the HTML and downloading everything you need to recreate those pages locally. This makes Wget perfect for automated backups as well as simple scripting. The -N argument, for instance, will only download copies of files if the online version is newer than one stored locally.
If you connect to a server using SSH, Wget is much easier to use than FTP, and is eminently more flexible in the type of files that it's able to download.
Verdict: There's still a place in our hearts for one of the first download managers ever to include HTTP. 7/10
One thing GWget inherits directly from Wget is the Mark Rothko inspired design aesthetic.
As the name suggests, this a version of Wget wrapped around a Gnome GUI. It's designed to help normal people take advantage of Wget's excellent feature set and stability, without resorting to terminal emulation or 70s-style beards.
And it manages this fairly well. Click on the New button, type in the URL of the file you want to download and GWget will go and get it. Alternatively, drag a link from Firefox on to the main window and the file will be added to the queue. There was once even an additional Firefox extension that helped you send files to the GWget queue, but this has fallen behind recent browser releases and won't install on either version 2 or version 3 of Firefox.
But if you're looking for browser integration, Epiphany might be worth considering. This is Gnome's built-in web browser, and uses the same rendering engine as Firefox (for now). An Epiphany extension can be installed that adds integral GWget downloading from the browser view, like the combination of Konqueror and KGet. As GWget inherits functionality from Wget, only HTTP and FTP connections are supported, but simple configuration pages let you enable recursive downloads, set up proxy servers and limit the speed of downloads.
You can also view the graphical queue manager, which is another significant improvement over the original. You can easily see the speed, progress and ETA of your downloads, as well as cancel any you no longer need. MD5 checksum generation is also easy to use, which is a feature we found on only one other utility.
If you want the straightforward simplicity of wget, but don't want the command-driven interface, GWget is an improvement. It's just a pity it doesn't take the idea any further by implementing some kind of acceleration technique.
Verdict: It might be Wget wrapped up in Gnome eye candy, but there are better desktop alternatives available. 6/10.
Downloader for X
If you've used a Windows download manager, you should feel right at home with Downloader for X.
When you first launch Downloader, you'll notice that it seems to be modelled after popular download managers for Windows. Most of the window area is taken up by the download status display. As with KGet, each file you add to the download queue is listed here, along with essential statistics such as the speed, completion and an ETA. Also like KGet, Downloader supports both FTP and HTTP connections, and these can either be added manually from the New Download button, or automatically by copying and pasting a URL from the clipboard.
A desktop icon called a 'drop basket' can also be enabled, and this accepts drag-and-drop requests from browsers and file managers if you want to add files to the Downloader queue. One nice addition is that you can use wildcards to download files from an FTP server, making it easy to grab a load of images, for instance.
Downloader can parse HTML files and download HTML files linked from a web page in a similar way, as well as traverse directories and subdirectories. Downloads like these are configured from the Edit Download window, which enables you to control everything from the password you may need to use for the connection, to the number of parallel threads you want to hammer the download server with.
There's an interesting automation panel too. From here you can schedule downloads for a certain time and date. You can also perform a variety of other options, including speed limiting, executing a command and cleaning the download queue. If you run into problems, the error log features multi-coloured error messages that can genuinely help if you get into problems constructing wildcards and filters, as well as tracking down broken links. A comprehensive application, then, but Downloader is still missing download performance improvements.
Verdict: Easy to use if you're looking for a general manager with scheduling, but not all that original either. 7/10.
There's not much to see, but here's Aria2 downloading an Ubuntu ISO in two segments at twice the normal speed.
Linux is a brilliant platform for small, powerful, high performance utilities. And that's exactly what Aria2 is. At first glance, this command-line tool looks little different from the venerable Wget. You add the URL to the end of the command, the file is downloaded automatically. But Aria2 is a generation apart from Wget, offering more advanced features.
For example, add the -s2 argument to the initial command, and the download will be split into two connections. If the server you're connecting to limits the speed of each connection, this will double your download speed. There's a smarter option still: use two separate URLs pointing to the same file and Aria2 will amalgamate the data from both into a single download. You can even share connections across the FTP and HTTP protocols for maximum flexibility. In the background, Aria2c assembles chunks of data from each download into a single file.
Over the last few years, the protocol of choice for downloads has switched from HTTP and FTP to the much-maligned BitTorrent, something that hasn't been missed by the Aria2 developer, Tatsuhiro Tsujikawa. Type 'aria2c file.torrent', and the files pointed to by the torrent will be downloaded automatically. You can limit the upload and download bandwidth, as well as specify a torrent file on a remote server without downloading it first.
It's even possible to limit both the seeding ratio and the time spent seeding a download. This helps to ensure you're sharing as much of a torrent as you've downloaded, without crippling your connection for a long time. Command line tools aren't for everyone, and there are a couple of under-developed GUIs for Aria2, but it's easy enough to use anyway, and it's often quicker to use than the graphical equivalent.
Verdict: Even from the command line, Aria2 is a straightforward to use and ultra fast download manager. 9/10
For speed and usability, WXDownload Fast can't be beaten. It's just a shame that it's lacking in stability.
Any download manager brave enough to sport a performance graph has got to be taken seriously. And that's exactly what WXDownload Fast has, straight across the middle of the main window. While downloads are in progress, the graph draws the cumulative speed of all downloads over time. It's a little gimmicky, but it can help you see exactly how fast your connection is.
And improving speed is obviously what this application is designed for. Downloads are added by selecting the URL in your browser or email client and clicking on the New button in 'WXDFast', as this app is also known. You can add multiple URLs for mirrors of the same file, as well as splitting the file into downloadable segments, and WXDFast will parallelise the download so that various sections can be downloaded simultaneously - vastly increasing the potential download speed.
The graph does make it easier to see if there's any difference in performance between a segmented download and downloading as a single chunk, but you can also do this with other managers by simply making a note of the download speed. If you need space to be able to watch YouTube videos at the same time, there's an option to limit the download bandwidth or even schedule the download to start at a specific data and time.
All of these functions are tied to a clear and easy-to-understand user interface, and this is the only tool we found that offered advanced segmented download options from a GUI. But it was also the only tool in our group test where we encountered stability problems. Adding certain URLs would crash the application, and because the URL had already been added to the configuration file, you couldn't even start the application again as it would attempt to resume from the same broken URL. The only solution is to delete the configuration file and start again.
Verdict: Accelerated downloads and queue management through a GUI. If only it would run long enough for an ISO. 8/10
If you start seeing a mess of digits on your screen while using Curl, you're doing something wrong.
If you think Aria2 and Wget sound geeky, you've not tried Curl. Sure, it's got the command line interface we're beginning to get used to - type curl followed by the URL of the file, and the download is handled automatically. But unlike both Wget and Aria2, Curl will pipe the output from the download directly to the console, spraying your screen with corrupt characters and binary artefacts if you're not prepared for it. The answer is to pipe the output from the command to a separate file, as you do with the cat command, for example. You can also use the -o argument to specify an output file -you'll even see a progress meter as a bonus.
This unfriendly approach to interface design belies Curl's real forte - as a programmer's tool. It's a more Unix-like utility than all of the other managers we've looked at. The accompanying library is used by many programmers to embed download functionality into their applications, and you can find Curl modules for PHP, FTP filesystems, Python and dozens of other utilities.
But it's the scale of the options and protocols that scares us. HTTP, HTTPS, FTP, FTPS, SCP, SFTP, TFTP, DICT, Telnet and LDAP are all supported. Our collective mind boggles at why you'd need to use the likes of DICT, LDAP and Telnet to transfer files, but if you're shaking your head in disagreement, presumably these additions apply to you. There's no segment or parallel download support, no recursive traversal of servers, and no HTML parsing.
A script called mcurl gets around the segmented download problem by implementing multi-part simultaneous threaded downloads using Curl, and you can even use Curl to upload files. But the bottom line is that this download manager is best suited to hand-coded scripts and automation.
Verdict: Comprehensive and complicated. The typical Curl user will already know what it's about. 7/10.
Our choice: Aria2
This has been a tough group test from which to reach a conclusion. Our winner is a command line utility with only a couple of rudimentary GUIs - none of which we can honestly recommend. These aren't the normal characteristics we'd want in a winner. But the bottom line is that if you're looking at a download manager in the first place, you're going to want a tool that boosts performance and is easy to use.
Despite being run from the command line, Aria2 satisfies both of these criteria. But there's something else that's also helped Aria2 come out on top. Nearly all the download managers we've featured in the group test can be used with a fantastic Firefox extension called FlashGot.
FlashGot is a kind of meta-download manager. It manages download managers, and you can tell it to use almost any of the managers you might have installed - including our winner, Aria2. This means that if you use Firefox, you won't even have to wrestle with the command line. Just choose Aria2 as your download manager within FlashGot and you're good to go.
Best of the rest
Our runner-up is also worth a mention, because KGet was very close to winning. Its desktop integration and filtering tools are hard to beat, and it has to be the easiest download manager to use.
The problem is that KDE users are the only ones who can realistically take advantage of it, and even they can't use KGet to accelerate downloads, because it's missing a few important features. If KGet developers are reading this and they're still adding features for the imminent 4.1 update of the desktop, please take note: some form of parallel/threaded download would really help!
Of the other tools we looked at, Curl is obviously an important application because it's used by so many other utilities. If you want to script your own solution, or include uploading as well as downloading, that's the tool to use. Just don't expect to be able to make any sense out of it without spending at least half an hour with the manual first.
Finally, if the WXDownload Fast team can update their app to be a little more stable, it would be the de facto choice for desktop-friendly, super-fast download management. Which is why we're left with Aria2. It's fast and it works.
FlashGot puts your favourite download manager right into Firefox.
Most of us find files we want to download while we're browsing the web. This is why several download managers integrate with one particular web browser - KGet into Konqueror and GWget into Epiphany, for example. Both these managers enable you to add a downloadable file from the browser without switching to a different application. But the situation is slightly different with Firefox, thanks to its support of extensions.
Extensions use the reverse approach, adding functionality directly to the web browser rather than through a third-party utility. If you search for download manager extensions on the Mozilla website, there are dozens of possibilities. But few offer any form of acceleration or the flexibility found in most of the standalone managers we've looked at. Which is why FlashGot is so popular.
With over 250,000 weekly downloads, it's the perfect choice for Linux users who regularly download files. Rather than re-invent the wheel, FlashGot can be configured to use almost any download manager, including many Windows versions if you're happy to use Wine. Which means that if you use Firefox, Aria2 and FlashGot are the perfect combination.
First published in Linux Format magazine