Tag: wget

Make ‘wget’ Resume From An Interrupted Download

Today’s just going to be a quick article, an article where we learn how to make ‘wget’ resume from an interrupted download. This is a darned useful function you can add to a wget command, especially if you’re in an area with sketchy connectivity. To learn to make wget resume from an interrupted download, read on!

So many of my articles are written because of something I did recently. Many are still based on my copious notes (we’re well over 300 articles here on Linux Tips), but those will run out eventually. I’m often thinking of new ideas for articles and sometimes my day-to-day computing gives me an article idea that’s not from my notes. This is one of those…

Today, we’re going to cover yet another wget feature! We’ve had many wget articles. Here are a few of them:

Limit The Download Speed For ‘wget’
Rename A File Downloaded With ‘wget’
How To: Hide The Output From wget

And we’ve used wget in many articles. Go search for “wget”.

By now, many of my regular readers will be more than familiar with wget. So, what is wget? It’s a terminal-based tool that you use to download files. I use it often. You’re encouraged to check man wget for more information.

I use it outside of the browser, even if I found the download link via a browser. It’s just that handy and the throughput rate seems to be greater with wget (oftentimes). If you check the man page, wget describes itself as:

The non-interactive network downloader.

Which is exactly what it does. Which is nice…

How To: Make ‘wget’ Resume From An Interrupted Download:

You’ll kinda sorta maybe need an open terminal for this article. If you don’t know how to open the terminal, you can do so with your keyboard. Press CTRL + ALT + T and your default terminal should pop open.

With your terminal open, you just need a file to download… I’ll let you pick that. You also need to interrupt your download, so that you can practice this…

Wait, no… That’s just silly. Instead of practicing this, just learn from my usage and call it good. There’s no need to replicate this until you need it. Yeah, that’s the ticket!

So, imagine my surprise when I learned that Gentoo now has a live USB edition. (I was pretty surprised.) I immediately decided to download the file, though I’ve still not tried it. To download the .iso, I used the fantastic wget tool.

My terminal was already open. My present working directory was already the ‘Downloads’ directory. I had nothing to do except enter the wget command. The command I entered was this:

As you may know, my DSL provider made me angry and I’m now using a combination of a mobile hot spot and satellite. My mobile provider likes to disconnect me for 30 minutes at a time and does so at varied intervals.

I normally just switch to the satellite connection for 30 minutes but wget didn’t like that. I’d already downloaded half of the file while tethered to my phone and didn’t want to download it again. Downloading data I’d previously downloaded is just a pain in the butt and slows things down. So, I added the -c flag. The command I then used, once connectivity was restored, was this:

Sure enough, wget resumed from where it left off when the connection dropped out. I didn’t have to download that all over again. Sure, wget will automatically retry a few times (which you can modify) but it’s not going to keep trying for 30 minutes (by default) or longer. So, this is how you make wget resume from an interrupted download.

Closure:

See, it’s easy to make wget resume from an interrupted download. Was it worth writing an entire article for a single flag? I’d say yes. Well, of course, I would say yes. If I didn’t think it was worth an entire article, I wouldn’t have written an entire article about it!

Ah well…

And now you know…

You’re welcome…

Thanks for reading! If you want to help, or if the site has helped you, you can donate, register to help, write an article, or buy inexpensive hosting to start your site. If you scroll down, you can sign up for the newsletter, vote for the article, and comment.

Smash a button!
[Total: 9 Average: 5]

Limit The Download Speed For ‘wget’

In today’s article, we’re going to learn how to limit the download speed for ‘wget’. It will be a short article because I’m watching a race and didn’t bother writing an article ahead of time. It’ll also be suitable for someone new to ‘wget’. So, to learn how to limit the download speed for ‘wget’, read on!

We’ve had a number of ‘wget’ articles in the past. There’s quite a bit to it. If you don’t know, you use ‘wget’ in the terminal, as a way to transfer files across a network. The ‘wget’ command is often used to download files from the internet. If you’re unfamiliar with the ‘wget’ command, it defines itself as:

Wget – The non-interactive network downloader.

There are a number of existing ‘wget’ articles if you want to gain some familiarity with the command. Here’s some:

Rename A File Downloaded With ‘wget’
How To: Hide The Output From wget
Make wget Use IPv4 or IPv6
How To: Make ‘wget’ Ignore Certificate Errors

Plus, there are numerous articles that used wget to download files.

So, when you’ve taken a look at those articles and feel you have a solid grasp of what ‘wget’ is used for, you might as well take a look at the man page to get a better idea of the possibilities:

With that information properly learned or refreshed, let’s just skip right ahead to the main article…

Limit The Download Speed For ‘wget’:

It’s a bit different lately, as I’ve been using a different ISP. In the past, if I was downloading anything the whole network would bog down. I’d use things like ‘wondershaper‘ to get around that. Else, if I was using ‘wget’ to download a large file, I’d limit the download speed.

This isn’t much of an issue with my current connection, so I don’t worry about it. Still, you might have all sorts of reasons to limit the download speed for ‘wget’. We can do that! You will need an open terminal, however. 

So, open up your default terminal emulator. If you don’t know how to open the terminal, you can do so with your keyboard – just press CTRL + ALT + T and your default terminal should open.

If you don’t know what we’re looking for, we’re looking for the --limit-rate flag. It’s not all that complicated and looks like this:

For example:

Where you see 50k, you can change it to 100k, 500k, or whatever. The k obviously stands for kilobytes. You can also use m, as in --limit-rate=50m and that’s obviously megabytes. If you’ve got enough bandwidth for it to matter, you can even use a g in that space, and it’s obviously for gigabytes.

See? That’s actually all there is to it. I told you it was simple and quick! Tada! It was!

Closure:

And there’s another article. This one was hurried but hopefully is worth reading and sharing with your online friends. You’ll have learned that you can limit the download speed for ‘wget’, a potentially handy way to use the tool. You can combine it with the various other flags, so be sure to read the man page!

Thanks for reading! If you want to help, or if the site has helped you, you can donate, register to help, write an article, or buy inexpensive hosting to start your own site. If you scroll down, you can sign up for the newsletter, vote for the article, and comment.

Smash a button!
[Total: 4 Average: 5]

Rename A File Downloaded With ‘wget’

In today’s article, we’re going to learn how to rename a file downloaded with ‘wget’. This will not be a complicated article. It will also be a pretty short article. If you want to learn how to rename a file downloaded with ‘wget’, read on and you’ll see how!

Often, you’ll use ‘wget’ to download a file and it will have some sort of convoluted file name. You can download a file with ‘wget’ and rename it automatically. If this is something you’re interested in doing, we’ll learn how to do it in this article.

We’ve used ‘wget before. See:

How To: Hide The Output From wget
Make wget Use IPv4 or IPv6
How To: Make ‘wget’ Ignore Certificate Errors

We’ve used ‘wget’ in other articles, so feel free to use the search function to see the other articles about ‘wget’.

If you’re curious, you can check the man page. You’ll see ‘wget’ defines itself as:

Wget – The non-interactive network downloader.

That’s a good enough description, I suppose. Basically, you use ‘wget’ in the terminal to download files. It’s a pretty handy application and, importantly, a pretty universal application. If you’re using a GNU/Linux desktop (or server), you’ve almost certainly got ‘wget’ as an available tool.

So then…

Rename A File Downloaded With ‘wget’:

This article requires an open terminal, like many other articles on this site. If you don’t know how to open the terminal, you can do so with your keyboard – just press CTRL + ALT + T and your default terminal should open.

Let’s ensure you have ‘wget’ installed with:

If ‘wget’ is not installed, install it from your package manager. Once installed, you can take it on a test run:

Now, we’re going to make ‘wget’ keep trying in case the download has issues. We’ll use the -c (continue) flag:

You probably don’t need the -c flag for this file, but it’s a good habit to get into. If we add the -O flag (for output-document) to the command, it’d look like this:

Or, to try to make it more clear:

In the latter example, you’ll have downloaded the file and renamed it to numbers.txt instead of saving it as the original filename. See? Pretty simple!

Closure:

There you have it. You have a new article. This time, we’ve learned how to rename a file downloaded with wget. It’s a handy trick to keep in mind, as people tend to want to give their files names with things like version information and all that. If you want a simpler file name, this is a nice easy way to do it.

Thanks for reading! If you want to help, or if the site has helped you, you can donate, register to help, write an article, or buy inexpensive hosting to start your own site. If you scroll down, you can sign up for the newsletter, vote for the article, and comment.

Smash a button!
[Total: 6 Average: 5]

How To: Hide The Output From wget

In today’s article you’re going to learn that you actually can hide the output from wget. I am not one of folks want to do this, but some do. Or at least the option is there, so I assume they do. Either way, read on and you’ll know how to hide the output from wget! 

Goodwood Revival is this weekend, but you’ll still get an article. I am thinking about going in person next year, so I’ll have to write articles ahead of time. I probably should have done that even though I’m just streaming it.

Anyhow, there’s an option that will let you hide the output from wget and it’s in my notes. I might as well turn it into an article because I’m sure someone wants to do this. 

What this does, to be clear, is shows no wget output in the terminal once you enter the command. You’re not running blind, however. I’ll show you how to at least ensure the command gets completed. So, it does have uses – when  you just don’t need to see the clutter.

Lots of people do loads of useful work in the terminal and don’t really need to see clutter, so this is one way to avoid that terminal clutter. I actually prefer to see what’s going on, but I’m weird like that. If you do not prefer to see what’s going on with wget, this article is for you!

Hide The Output From wget:

This article requires an open terminal, like many other articles on this site. If you don’t know how to open the terminal, you can do so with your keyboard – just press CTRL + ALT + T and your default terminal should open.

It next requires that you use wget to get something. So, pick something and download it with wget. I don’t care what. You do you and download anything you want. To hide the output from wget, the command is:

That’s really it. However, you then have no idea if it it completed. Fortunately, you can make sure wget completes its task (within reason) with the -c flag. So then the command would look like:

See? Pretty simple. That command will not only hide the output from wget, it will ensure the download is completed. You’ll avoid cluttering up your terminal, or something…

Closure:

There you have it! You now know you can, and how you can, hide the output from wget. You can even be reasonably sure it completes behind the scenes. It’s not a very difficult article to follow today, so consider it an easy day. Now, back to my racing…

This article requires an open terminal, like many other articles on this site. If you don’t know how to open the terminal, you can do so with your keyboard – just press CTRL + ALT + T and your default terminal should open.

Smash a button!
[Total: 10 Average: 5]

Make wget Use IPv4 or IPv6

Today’s article is a fun one, an article where you learn how to make wget use IPv4 or IPv6. Not only is this useful, it’s easy! In fact, it’s easy enough for most anyone to figure out. Plus, this will be a fairly short article – I think. I mean, I haven’t written it yet, but it seems like a short one.

I’ve done a couple wget articles, with the most recent showing you how to make wget ignore certificate errors. In fact, in that article I showed you how to enable that permanently. I’ll have to add that to this article. That’ll come in handy for like 0.01% of you.

If you don’t know, wget is a tool for downloading content from servers. These days, we use it to grab stuff from web servers without having to go through a browser – more often than not. Sometimes we use it to scrape entire sites without actually visiting them in a browser! (Sometimes, doing that makes you a dick.) It’s a handy tool for that sort of stuff.

Once in a while, while using wget, you’ll come across a finicky download that will throw an error about wanting IPv6 (or IPv4 if you’re using IPv6). That’s when this article is going to come in handy. Sometimes, a server isn’t configured for, or will refuse connections from, one or the other. That’s when you’ll see errors and that’s what we’ll resolve in this article.

Hmm… Do I need to explain wget more? Nah, y’all know what it is. IPv4 vs IPv6? Well, how about you check out this link to learn the difference and why it matters. That’s a good link. Alright, moving on…

Make wget use IPv4 or IPv6:

This article requires an open terminal, like many other articles on this site. If you don’t know how to open the terminal, you can do so with your keyboard – just press CTRL + ALT + T and your default terminal should open. The wget command will throw an error, letting you know if you must use one or the other.

If you need to force IPv4, it’s nice and easy:

If you need to force IPv6, it’s also nice and easy:

If you find yourself doing this often, one way or the other, you can actually tell wget to do this on a permanent basis. Like in the previous wget article you can edit your .wgetrc file (create it if it doesn’t exist) to include either --prefer-family=IPv4 or --prefer-family=IPv6 and it’ll try one before trying the other if it’s available.

Closure:

See? Nice, neat, and simple. You’ve learned how to make wget use IPv4 or IPv6 – should you find yourself needing to do so. It’s a handy trick to have up your sleeve, ’cause you will eventually find a server that requires one or the other. Given enough use, it’s gonna happen.

Thanks for reading! If you want to help, or if the site has helped you, you can donate, register to help, write an article, or buy inexpensive hosting to start your own site. If you scroll down, you can sign up for the newsletter, vote for the article, and comment.

Smash a button!
[Total: 6 Average: 5]
Subscribe To Our Newsletter
Get notified when new articles are published! It's free and I won't send you any spam.
.
Linux Tips
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.