Review

GNU Wget

Overview

Synopsis

GNU Wget is a free software package for retrieving files using HTTP, HTTPS and FTP, the most widely-used Internet protocols. It is a non-interactive commandline tool, so it may easily be called from scripts, cron jobs, terminals without X-Windows support.

Category

Web Scraping Tools

Features

Can resume aborted downloads, using REST and RANGE
Can use filename wild cards and recursively mirror directories
NLS-based message files for many different languages
Optionally converts absolute links in downloaded documents to relative, so that downloaded documents may link to each other locally
Runs on most UNIX-like operating systems as well as Microsoft Windows
Supports HTTP proxies
Supports HTTP cookies
Supports persistent HTTP connections
Unattended / background operation
Uses local file timestamps to determine whether documents need to be re-downloaded when mirroring

License

Proprietary

Price

Free

Pricing

Subscription

Free Trial

Available

Users Size

Small (<50 employees), Medium (50 to 1000 Enterprise (>1001 employees)

Company

GNU Wget

PAT Rating™

Editor Rating

Aggregated User Rating

Rate Here

Ease of use

7.6

8.5

Features & Functionality

7.5

8.2

Advanced Features

7.5

8.2

Integration

7.6

8.4

Performance

7.4

8.7

Customer Support

7.4

—

Implementation

—

Renew & Recommend

—

Bottom Line

GNU Wget has many features to make retrieving large files or mirroring entire web or FTP sites easy, including: resume aborted downloads, using REST and RANGE and use filename wild cards and recursively mirror directories.

7.5

Editor Rating

8.4

Aggregated User Rating

1 rating

You have rated this

The recursive retrieval of HTML pages, as well as FTP sites is supported -- the user can use Wget to make mirrors of archives and home pages, or traverse the web like a WWW robot (Wget understands /robots.txt). Wget works exceedingly well on slow or unstable connections, keeping getting the document until it is fully retrieved.

This allows freedom of movement as the user does not always need to be in a region where wifi has great signal. Re-getting files from where it left off works on servers (both HTTP and FTP) that support it. Matching of wildcards and recursive mirroring of directories are available when retrieving via FTP. Both HTTP and FTP retrievals can be time-stamped, thus Wget can see if the remote file has changed since last retrieval and automatically retrieve the new version if it has.

Wget supports proxy servers, which can lighten the network load, speed up retrieval and provide access behind firewalls. If the user is behind a firewall that requires the use of a socks style gateway, you can get the socks library and compile wget with support for socks. Most of the features are configurable, either through command-line options, or via initialization file .wgetrc. Wget allows you to install a global startup file (etc/wgetrc by default) for site settings.

Filter reviews