Linux wget Command to Download and Mirror a Website in Static Local Copy

• Tweet

• Tweet

wget is such a wonderful command in Linux you can ever get. Consider blessed to have it in your SSH arsenal. Now, not only does it allow you to download something neatly from the command line to the current working directory — that’s why WordPress always puts the latest version of the blog script at http://www.wordpress.org/latest.zip, that way you don’t even have to visit the site before using wget to get it in no hassle:

wget http://www.wordpress.org/latest.zip

And the latest version of WordPress will be in your local working directory in no time — but it can also help you to create a local mirrored copy of a remote website for backup or for local browsing.

How to Make Mirrors from a Site with wget command?

wget comes equipped with so many useful switches and features and one of them is to mirror stuff off the Internet, from online to offline. For example, you have a blogspot blog and you want a local copy of the entire blog files / web pages, including all those CSS style sheets, images and scripts. No problem, just go with

wget -mk http://yours.blogspot.com

Don’t worry, wget will create a new directory named by the domain, in this case, "yours.blogspot.com", to store all those files. After screens and screens of real-time logging, when it’s done, the local mirror should be ready to use.

The –m switch is short for –mirror and the –k switch translates all absolute URLs in the HTML into relative ones for totally offline browsing.

The problem is that wget works much better with sites using sanitized / clean URLs such as a WordPress blog. Though a blogspot blog is basically in static HTML, it contains quite some complicated URLs here and there that’s filled with parameters. The result doesn’t look too pretty.

However, after trying wget to recursively download and mirror a WordPress blog, it looks fantastic. Try it with a permalink enabled WordPress blog and it will surely amaze you.

It’s best used to generate mirrors of websites completely built in static files or sites with simple URL schemes.