Find And Remove Duplicate Files With rdfind

In today’s article, we’re going to find and remove duplicate files with rdfind. We’ll try to make this as safe as possible. I’d suggest newer  users not actually worry about duplicate files. Allocate enough space to your OS and don’t worry about it. Disk space is cheap these days.

Warning: Blindly removing duplicate files can be a risky operation. It can break things. You have been warned. Exercise caution!

If you’re interested in removing duplicate files, then the rdfind application is one solution you can try. There are others, but we’ll be using rdfind. We may cover other choices in the future.

You don’t have to run rdfind with it automatically deleting the duplicate files and that’s what I’m going to suggest you do – at least at first. It’s good to see what’ll be deleted before it is actually deleted.

If you check the rdfind man page, you’ll see it’s described as:

rdfind – finds duplicate files

It does what it says on the tin. It finds duplicate files. You can run the command in a manner that automatically removes the found duplicates, but that’s not something to take lightly.

Again, and I can’t stress this enough, some duplicates are there for a reason – they belong there. So, don’t run this on the root directory and expect a good outcome. Running this on the root directory and automatically removing duplicates is going to break stuff. Feel free to do so, ’cause it’s your computer. Just don’t blame me when it breaks.

There… I feel you’re safely and properly informed! Let’s get this article started…

Install rdfind:

We’ll just use the terminal to install rdfind. To open your default terminal emulator, press CTRL + ALT + T and your default terminal should open. You might as well leave it open, as rdfind also runs in the terminal and you’ll need an open terminal in the next step.

Debian/Ubuntu:
Arch/Derivatives:
RHEL/CentOS:
Fedora/Derivatives:

Now that you have installed rdfind, you should probably consult the man page. That’s an easy command:

With that knowledge fresh in your memory and rdfind installed, we can just jump into the article!

Find And Remove Duplicate Files With rdfind:

Your terminal should still be open from the previous step. If not, go ahead and open it now. You’ll need a terminal open to find and remove duplicate files with rdfind. It is not a graphical application.

So, I suppose you can start with this command:

That may look dangerous, but it’s not. If you run that command, it simply finds the duplicate files and then creates a text file for you. You then review the text file and manually remove the duplicate files. This is probably for the best. It’s also the same thing if you do a dry run, like so:

You can actually delete the files and replace the first one found with hard links. While not recommended by me, it’s at least safer.

Finally, you can just go right ahead and just find and remove duplicate files! This is safer if you have both a recent backup and you’ve gone ahead and run one of the first two commands. Then, if you have run one of those two rdfind commands, you’ll know what’s going to be deleted.

Just don’t run rdfind on your root directory, and probably don’t run it directly on your home directory, and you should be more or less okay. Feel free to run it on your Downloads folder, on your Documents folder, or even your Pictures folder.

Running rdfind that way, on those types of directories, will be fine and at least should not break things. Rdfind pretty good at finding just duplicates, or I’d not recommend it. Be sure to backup first and make sure you give it a dry run before you start automatically removing stuff! Seriously, do not run this on your root directory.

Closure:

And there you have it… You have yet another article! This time, we’ve learned how to find and remove duplicate files with rdfind. You were given a clear warning, but you’re gonna do what you’re gonna do. Man, I really need to write that article about backing up properly!

Thanks for reading! If you want to help, or if the site has helped you, you can donate, register to help, write an article, or buy inexpensive hosting to start your own site. If you scroll down, you can sign up for the newsletter, vote for the article, and comment.

Smash a button!
[Total: 8 Average: 5]

How To: Find Multiple Filenames By Extension

Today’s article will show you how to find multiple filenames by extension, using the find command in the terminal. It’s a pretty handy skill to have for when you need to know where files of a certain extension reside on your file system.

If you got a new article notification yesterday, that’s because I’m an idiot. Instead of hitting the schedule button, I hit the publish button. I’m not sure what I was thinking. It was fairly early in the afternoon and I wasn’t even sipping wine at the time! Sorry for disturbing you unnecessarily. I almost sent out an ‘oops’ newsletter, but then I’d have just disturbed you twice.

Anyhow, this will be another article that makes use of the find command. The find command is a rather robust command and can be somewhat daunting for new people. I feel more comfortable writing articles that let you learn it in chunks, rather than trying to cover the entire thing. I do find it hard to explain, but I’ll do my best.

What’s this useful for? Well, let’s say you want to find .deb, .zip, and .iso files in you ~/Downloads directory. That’s what this command is going to do for you. You can find multiple filenames by extension in the terminal and it’s not overly complex once you understand the basics of the command.

Instead of making the intro needlessly longer, and to make up for today’s scheduling gaff, I’ll keep the intro short and we’ll just run straight into the article…

Find Multiple Filenames By Extension:

In the intro, I mentioned that this was going to be done in the terminal. As such, we’re obviously going to need an open terminal for this exercise. To do so, press CTRL + ALT + T and your default terminal should open. Tada!

Warning: I do not explain this one as well as I’d hoped. So, I tried to explain by way of demonstrating. I’m hopeful that works.

Now, here’s the command I just ran in my terminal:

Now, if you want to run it in the current directory, you can specify the directory or you can change ~/Downloads to a . (period).

If you want to find just one file, you’d stop after "*.deb" and leave the closing \).  If you want to add additional files, you would include -o -name "*.<extension>" and make sure to keep the closing ).

It might be easier to show you. For formatting reasons, I’ll use the . (period) instead of specifying a directory. It’ll fit on your screen better than a longer command. So, “How To:”…

Find One File By Extension:
Find Two Files By Extension:
Find Three Files By Extension:

So, hopefully you can see how this find command works. I can’t think of a better way to explain the command than to show it to you in examples. I hope that works for people. Feel free to comment in either direction, as I think it might work for some but be less effective for others.

In theory, you could find all sorts of files by extension, just remember to include the -o -name and file type and noting that the asterisk is a wildcard in this instance, meaning all files with that extension will be found. So, .gz files would be "*.gz". You can make the command as long as your heart desires!

Well, no… There’s bound to be an upper limit somewhere. (Wait, I looked it up, the maximum number of characters in the terminal is 4096 characters. And now we know…)

EDIT: You have no idea how much of a pain in the butt this article turned out to be. Holy crap. For safety reasons, WordPress eats the backslash \. I did not know this. Nobody knows this. The solution is to escape the backslash by using it twice. This article is full of backslashes. I think I got them all. It eats them every time I save the draft, so hopefully they show up in publication. I can never edit this article again, so it is what it is. Well, I could edit it again, but it’d be a pain in the butt.

Closure:

So, yeah… Today we’ve learned to find multiple filenames by extension. At least I hope we have. It’s not so easy to explain, but I figured if I explained it by showing examples then you’d be able to pick it up in context. If you do have any questions, just drop ’em into the comment box below and I’m usually pretty speedy at getting back to people. As always, the man page is probably helpful.

Again, sorry about the fake article notification. That doesn’t happen often, but it does sometimes happen. In an ideal world, I’d have an awesome editor and I would just save everything as a draft. If you’re interested in volunteering for that role, let me know! It’d make my life so much easier, I think… I mean, I don’t really know… It just seems like something that’d help.

Also, I’m pretty excited to write this month’s meta article. I’ll probably wait and schedule it for the holiday or a weekend day. They’re not important articles, but I find it interesting. The site’s growing steadily.

Thanks for reading! If you want to help, or if the site has helped you, you can donate, register to help, write an article, or buy inexpensive hosting to start your own site. If you scroll down, you can sign up for the newsletter, vote for the article, and comment.

Smash a button!
[Total: 7 Average: 5]

How To: Find Files Owned By A Specific User

In today’s article, we’ll be learning how to find files owned by a specific user. We’ll even use the ‘find’ command, as we find files owned by a specific user! That seems to be the best idea, and the best way to do it.

This should also be a fairly quick article. I don’t see any reason why I’d have to make it longer than it needs to be. So, it won’t take too much of your time today.

This article will be published on November 11. That’s a day known by a number of other names. It’s Veteran’s Day, Armistice Day, Remembrance Day, and probably a dozen more names that I don’t know. It was the day WWI ended, which was thought to be the war to end all wars ’cause it was just that horrific.

Well, as you can see, it was definitely not the last war – but we still choose this day to remember. In the US, veterans are celebrated today. Memorial Day is only for those who are no longer with us. Today is for the vets, as well as those who are no longer here.

It’s a holiday, which means it’s a fine day to have a nice and simple article. It’s a fine day to cherish your friends and family, instead of spending your time online reading Linux articles. (But thanks for doing so!)

Find Files Owned By A Specific User:

This article requires an open terminal, like so very many other articles. If you don’t know how to open the terminal, you can do so with your keyboard, just press CTRL + ALT + T and your default terminal should open.

With your terminal now open, you should probably navigate to a directory other than your home directory. If you run this command in your home directory, it’s gonna output a whole lot of text. So, let’s just try this first in your ~/Downloads directory:

The command we’re going to use is find, and the format is the find command, a dot to say the current directory, the -user flag, and then the username. So, your command would look something like:

(No brackets, of course.)

You probably don’t have any files owned by root in that directory, so a good test to make sure it’s working properly would be something like:

Now, you can mix things up a bit. Instead of using the dot to indicate the current directory, let’s find files owned by root in the /etc directory.

See? That’s not all that hard at all. It’s remarkably easy to find files owned by a specific user – and the command really isn’t that hard to memorize! You can run it in the folder you’re in, or you can use the directory path method.

Closure:

I told you that it’d be a quick article today. It’s a good day for just a quick tip and everyone can benefit from knowing how to find files owned by a specific user. Toss this tip into your growing list of tools in your Linux toolbox, because  you never know when this will come in handy.

Thanks for reading! If you want to help, or if the site has helped you, you can donate, register to help, write an article, or buy inexpensive hosting to start your own site. If you scroll down, you can sign up for the newsletter, vote for the article, and comment.

Smash a button!
[Total: 6 Average: 4.8]

How To: Compress And Decompress .bz2 Files

In today’s article, we’re going to discuss how to compress and decompress .bz2 files. This is something you may eventually need to know, so I’ll cover it here. I’ll just cover the basics, as most folks won’t need to know anything more than the basics. This should actually be a fairly short and direct article. There’s not a whole lot to it.

If you don’t already know, .bz2 files are bzip2 files. You’ll find that bzip2 is an opensource compression program that gets some regular usage, and you’ll sometimes find downloaded files that are compressed with this format. You may also, for compatibility reasons, want to compress files with bzip2 to share with other users who are already set on using the .bz2 format.

For the curious, the bzip2 man pages define this particular application as (and, as always, I highly encourage folks to read the man pages themselves – this one being a bit more complicated than others):

bzip2, bunzip2 – a block-sorting file compressor

Again, we’re going to just cover how to compress and decompress .bz2 files in this article. That’s all we’re going to do. You don’t tend to come across too many files compressed with bzip2, but they do show up from time to time.

Because of that, we’re going to cover how to compress and decompress those files in this article. It’s gotta get covered eventually, so it might as well be now. Read on!

Compress And Decompress .bz2 Files:

This article requires an open terminal, like many other articles on this site. If you don’t know how to open the terminal, you can do so with your keyboard – just press CTRL + ALT + T and your default terminal should open.

You might actually not have bzip2 installed. It’s not always installed by default. Fortunately, as far as I can tell, it shares the same name in every major distro, that is ‘bzip2‘. So, just go ahead and install it like you’d install any other software. For example, if you’re using Fedora, your command would look similar to:

See? I didn’t use Ubuntu as the default example! We’re mixing everything up today! (Use apt if you’re using a distro with apt, like Debian or Ubuntu and derivatives.)

At this point, you should probably have a .bz2 file to work on for the sake of the article. Seeing as I have no idea what you’ve already downloaded, we should probably start with you making one – just so you can see how to decompress it. 

To compress a file with bzip2, the command looks like (See the detailed warning below this command, do not use this command without reading the warning!):

That will create a file with the same filename but make a .bz2 file. However, this is a destructive act. If you use the above command, the original file will be deleted! If you wish to keep the original file, you need the -k (keep) flag. That looks like:

That command will not remove the original file, as would be the default. Obviously, the -z flag means ‘zip’.

If you want to decompress a file with bzip2, the command looks like:

This will extract the file(s) into the current directory. Of course, the -d means ‘decompress’. This is not a destructive operation. The original and extracted files will remain on your file system.

As you can see, it’s not all that difficult to compress and decompress .bz2 files. You might go years not seeing any files in that format, but you’re eventually going to bump into one and now you know  how to deal with it in the terminal – and how to respond in kind.

Closure:

And there you have it. You have yet another article. We’re getting close to 300 articles at this point, so it has been a long journey. If you feel like writing an article, let me know! Anyhow, you can now compress and decompress .bz2 files easily enough, and that was the point of the article.

Thanks for reading! If you want to help, or if the site has helped you, you can donate, register to help, write an article, or buy inexpensive hosting to start your own site. If you scroll down, you can sign up for the newsletter, vote for the article, and comment.

Smash a button!
[Total: 4 Average: 4.8]

Some Fun With Sorting The Output Of ‘ls’

Today’s article will be relatively short and easy, as we have some fun with sorting the output of ‘ls’. It’ll be a pretty short and simple article, but that doesn’t mean it’s not useful. Read on and learn about sorting the output of ‘ls’!

The last article I wrote was:

Some Fun With Formatting The Output From ‘ls’

This article will be remarkably similar, but isn’t about formatting the output of ‘ls’, it’s about sorting the output of ‘ls’. Yes, there is a difference! This is useful for a variety of reasons, but it’s mostly useful for viewing your files categorically. There are just a handful of sorting options, but we’ll cover them all.

Like always, it’s important to know that you should not to parse the output of ‘ls’. Parsing the output of ‘ls’ if full of risks, so you shouldn’t do it! The link explains it better than I can, and it tells you what to do instead of parsing the output of ‘ls’.

Obviously, and again, we’ll be using the ‘ls’ command. The ‘ls’ command defines itself fairly accurately as:

ls – list directory contents

So, just like the last article, there’s no real reason to make the intro longer than it needs to be. Let’s just jump right into the article.

Sorting The Output Of ‘ls’:

This article requires an open terminal, like many other articles on this site. If you don’t know how to open the terminal, you can do so with your keyboard – just press CTRL + ALT + T and your default terminal should open.

With your terminal now open, you can stay right there in your /home directory, or you can ‘cd‘ into a directory of your choice – preferably one with a variety of files in it.

When you’re done picking a directory, you have the following choices:

See? I even alphabetized them for you! The list of what they do should be fairly obvious, with ls --sort=version being the most unusual and only applying to versioned files.  Also, while included, there’s not a whole lot of reason to use ls --sort=none, as that’s just the default output.

I find sorting by time to be pretty handy, as well as sorting by size. I don’t have much of a reason to use the rest, but they’re there if you need them. I figured I’d include ’em all, rather than just those that I find useful.

Closure:

There you have it, another short article! It’s also another article about the ‘ls’ command, but this time we’re sorting the output of ‘ls’. I figured I’d do this article right after the other one, mostly so that I’d remember to actually cover it. 

Thanks for reading! If you want to help, or if the site has helped you, you can donate, register to help, write an article, or buy inexpensive hosting to start your own site. If you scroll down, you can sign up for the newsletter, vote for the article, and comment.

Smash a button!
[Total: 2 Average: 5]
Linux Tips
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.
Zoom to top!