Mastering the Linux Terminal Pipe Command

Well, if it’s not obvious by the title, it soon will be obvious that I’ve once again leaned on AI to write an article, this time about the pipe command. I decided to stick (mostly) to the title AI gave this article, but it was longer than it should be.

AI tried to title this:

“Mastering the Linux Terminal Pipe Command: A Comprehensive Guide”

Anyhow, this is one of those articles that I just can’t write. No matter what I write, it will not be adequate – even though the pipe is a simple enough concept. Much like a recent grep article, this is just one of those articles I won’t write well.

Also, I’m not sure that I should call it a command. It’s more an operator than a command, but the references I see refer to it as a command more frequently than as an operator. Perhaps the word would be ‘operand’? But, for convenience and convention’s sake, I will call it the pipe command.

No, this isn’t something you install. This is a command that you use with other commands. It’s a lot like the operators I’ve already written about. If you’re unfamiliar with the concept, read this article:

How To: Write Text To A File From The Terminal with “>” and “>>”

The short of it is that the pipe takes the output from one command and inserts it into another command. This lets you take the output from one command and parse it with another command. That’s all there is to it – which is why I’m unable to write this article.

Which is why I leaned on my good buddy AI for this article…

Mastering the Linux Terminal Pipe Command:

In the world of Linux, the terminal pipe command stands as a quintessential tool, offering a powerful and flexible means to manipulate data streams. Understanding and mastering the pipe command can significantly enhance your efficiency and productivity in the Linux environment. In this comprehensive guide, we delve into the intricacies of the pipe command, exploring its functionalities, use cases, and advanced techniques.

What is the Pipe Command?

At its core, the pipe command, represented by the symbol |, allows you to redirect the output of one command as input to another command. This seamless connection between commands enables the creation of complex data processing pipelines, facilitating the manipulation and transformation of data with remarkable ease.

Basic Usage:

The basic syntax of the pipe command is straightforward:

Here, the output generated by command1 is passed as input to command2. This chaining of commands enables the execution of multiple operations in a single line, streamlining workflows and reducing the need for intermediate files.

Practical Examples:

Let’s explore some practical examples to illustrate the utility of the pipe command:

Counting Words in a File:

This command displays the number of words in the file file.txt. The cat command outputs the contents of the file, which are then piped to wc -w, which counts the words.

Searching for a Pattern:

Here, grep is used to search for the specified pattern in the file file.txt. The output, which consists of lines containing the pattern, is then piped to wc -l, which counts the number of matching lines.

Sorting Data:

This command sorts the lines in the file file.txt alphabetically and removes duplicate lines using the uniq command.

Advanced Techniques:

While the basic usage of the pipe command is invaluable, mastering advanced techniques can unlock its full potential:

Chaining Multiple Commands:

You can chain multiple commands together to create sophisticated data processing pipelines. Each command in the pipeline operates on the output of the preceding command, enabling complex transformations with minimal effort.

Combining Filters:

Filters such as grep, sed, and awk can be combined to perform intricate text-processing tasks. By leveraging the unique capabilities of each filter, you can manipulate data in a myriad of ways, ranging from pattern matching to text substitution.

Redirecting Output:

In addition to chaining commands, you can redirect the output of a pipeline to a file using the > operator. This allows you to capture the results of your data processing pipeline for future reference or analysis.

Common Pitfalls:

While the pipe command offers immense flexibility, it’s essential to be mindful of potential pitfalls:

  1. Order of Operations:

    The order in which commands are chained together matters. Ensure that you sequence commands appropriately to achieve the desired outcome. A misplaced command in the pipeline can lead to unexpected results or errors.

  2. Input Compatibility:

    Ensure that the output of one command is compatible with the input requirements of the next command in the pipeline. Mismatched data formats or incompatible options can result in errors or unintended behavior.

  3. Performance Considerations:

    Complex pipelines involving multiple commands may incur performance overhead. Be mindful of resource utilization, especially when working with large datasets or computationally intensive operations.

Addendum:

The Linux terminal pipe command is a versatile tool that empowers users to streamline data processing tasks effectively. By mastering its usage and exploring advanced techniques, you can unlock the full potential of the Linux command-line environment. Whether you’re a novice user or a seasoned Linux enthusiast, the pipe command remains an indispensable asset in your toolkit, enabling you to easily conquer complex data manipulation challenges.

Closure:

So, that was AI doing my job for me. Like grep, I tried to write an article about pipe, which was a hot mess. I can usually salvage articles and publish something of my work, but I just didn’t do well with a couple of subjects. This is one of them.

The thing is, I refer to the pipe command with some regularity. I don’t have an article about pipes, so I can’t link to that. This leaves the reader with a search engine and I’d rather they have an excuse to open an additional link. It’s not just good SEO, it’s good hospitality. I’ll never explain everything, but I can explain some things and people won’t need to leave the site to learn those things.

Also, even AI had issues with this article. I told it to write 1200 words and it came up with maybe 600 words. I applaud those who can turn the pipe command into more than a blurb with a few examples that help people grasp the concept. Seriously, hats off to them. I don’t write nearly as well as my volume of articles would imply.

I don’t think I’ll need to use AI for any near-future articles. I’m doing two of them fairly close together because they’re things I feel need to be done. They are articles that need to be written. It is information that needs to be on the site. I did separate the two AI-written articles by some time, just to give folks a break between them. I know, they’re not preferred and they surely don’t match my writing style.

Thanks for indulging me, if nothing else. Amusingly, this isn’t much of a time-saver. The way ChatGPT formats stuff is not compatible with the editor used by my instance of WordPress. I spend a lot of time just formatting things.

Speaking of time invested…

Thanks for reading! If you want to help, or if the site has helped you, you can donate, register to help, write an article, or buy inexpensive hosting to start your site. If you scroll down, you can sign up for the newsletter, vote for the article, and comment.

Avoid Storing Commonly Used Commands In Your Bash History

This article won’t need to be all that long but it might be complicated as we discuss how to avoid storing commonly used commands in your Bash history. Yes, it’s a long title. 

This is also a bit contrary. It is one of those things that is easier done than said. It’s a very wordy thing, after all. I’ll do my best to describe what’s going on and why you might want to do this.

In this case, Bash stands for Bourne Again Shell. This article only applies to those who are using Bash. Bash is not the only shell available and people may opt to use other shells. If you’re one of those people, I don’t think this is going to work for you.

When you’re using the terminal, you’re using Bash. The commands you enter into the terminal are stored in ~/.bash_history, a hidden file in your home directory. We’ve discussed some of this before.

How To: Have Infinite Bash History
Playing With Your Bash History
How To: Not Save A Command To Bash History
How To: Reload Your .bash_profile

Well, you may type common commands, such as uptime. You may not want to store that command in your Bash history. Do you want to store every time you’ve typed the ls command?

You don’t have to. You have options!

What can you do? Well, you can tell Bash not to store certain commands in the ~/.bash_history file. This is actually a simple operation. To avoid storing commonly used commands in your Bash history, you need only to edit your ~/.bashrc file. I’ll show you how!

Man, this is going to impact the layout…

How To Avoid Storing Commonly Used Commands In Your Bash History:

Yeah, no amount of formatting is going to make that look good.

Anyhow, if this isn’t obvious, you’re going to learn how to do this in the terminal. You could edit your ~/.bashrc file with your favorite GUI editor but we’ll be doing this entire thing in the terminal.

As such, you should have an open terminal. More often than not, you can open your terminal by pressing CTRL + ALT + T. If that doesn’t work, you can find a shortcut to your terminal in your application menu. Should that not work, you’re probably already in the terminal!

So, first, we need to use Nano to edit the ~/.bashrc file. That’s an easy command:

Use your arrow button to navigate to the bottom of that file. Go to the absolute bottom and press enter to start a new line. You can press that button twice to provide some separation and to make it easier to read.

Now, let’s say we don’t want to store the ls, uptime, or touch commands in your Bash history file. We’ll use those as our examples. You should also probably leave a comment in your ~/.bashrc file so that you can easily identify what the code does and remember why you added it. That’s also useful if there are other users.

So, add the following lines:

Next, save that file. As we’re using Nano, you save the file by pressing CTRL + X, then Y, and then ENTER on your keyboard.

Next, you reload your ~/.bashrc file much like you reloaded your Bash profile (which was a link in the intro, should you wish to read it). You reload the ~/.bashrc file with this command:

That should reload the file. If it doesn’t, you can close all your terminal instances and open a new one. If that doesn’t work, you can log out and log back in again.

Anyhow…

Commands starting with :<command>: entries you used will not be stored in the ~/.bash_history file. If you type a command starting with those entries, it will be ignored, meaning they won’t clutter up your ~/.bash_history file with commands you’re already familiar with or commands that don’t need to be stored for things like auditing or security reasons.

It’s pretty simple to do, though it’s a bit of a pain in the butt to explain. This is how you avoid storing commonly used commands in your Bash history – something nobody is going to search for. (If you did find your way here via a search engine, be sure to leave a comment. I want to know who you are!)

Closure:

I realize that this is an awkward article and I’m okay with that. This isn’t something everyone is going to bother with, especially those people who don’t do much in the terminal. Still, it’s possible to avoid storing commonly used commands in your Bash history and now you know how.

Then, someday, someone’s going to search for this exact string of characters and, hopefully, they’ll find this article. I hope this satisfies their curiosity and helps them reach their Linux goals! If you did read this and find it valuable, you can always leave a comment.

Thanks for reading! If you want to help, or if the site has helped you, you can donate, register to help, write an article, or buy inexpensive hosting to start your site. If you scroll down, you can sign up for the newsletter, vote for the article, and comment.

Create A New User

Today’s article is going to be quick and easy as we simply discuss how you create a new user. This is a fairly basic task and shouldn’t take too long to cover. If you want to create a new user, read on!

If it’s not obvious,  you have a user account. You use this information even if you don’t realize it. Indeed, you use this information when you log into your computer to begin with. When you log in, you’re logging into your user account.

There are other users. You may have a root account or an account for MySQL. If you want to know how many different users are on your system, you can follow along with the following article:

How To: List All Users In Linux

One of the things that helps keep Linux secure is that it’s a true multi-user environment. You can only perform operations on the files you have access to. This is why you use sudo or root.

Managing users is a fundamental task in Linux. This article is going to cover how to create a new user and we’ll be doing so in the terminal. This should be fairly universal and you won’t need to install anything as user management tools will be included by default.

We will use a couple of tools, however. The first among them is:

useradd:

The useradd command is basic and, as the name implies, is used to add new users. There’s nothing complicated about it in today’s article and you can be certain that this is already a tool available to you.

If you’re curious about the command, check the man page:

If you do so, you’ll see that it’s described as this:

useradd – create a new user or update default new user information

So, that’s the correct tool for this job.

passwd:

The other tool we’ll be using is the passwd command. You can again tell by the name what the tool is going to do. Simply, it’s used as a password management tool. This too isn’t all that complicated and you can check the man page with this command:

If you do so, you’ll see that I wasn’t kidding and that this tool does what you think it does. It’s described like so:

passwd – change user password

This is the correct tool for the job. After we create a new user, we’ll assign them a password. If the user wishes, they can change that password on their own.

Create A New User:

As mentioned above, we’ll create a new user with terminal-based tools. This is a nice and universal way to do things. Sure, there are GUI tools out there but this is going to work on any Linux system you’re likely to engage with. You can crack open your favorite terminal, often by just pressing CTRL + ALT + T on your keyboard.

First, we’ll create a new user with the useradd command. The syntax is very simple:

For example:

Now, we’ll add a password. This is also a simple command:

For example:

You’ll be asked to enter the password a couple of times. This is to help ensure that you’ve not made any typographical errors while entering the password. It’s all basic stuff.

Next, you can verify that the new user account has been created. For this next step, we’ll simply use cat and grep.

Again, here’s an example:

The output should look a little something like this:

If you find your user, you’ve done this properly and you’ve learned how to create a new user. I told you that it wouldn’t be too complicated!

Closure:

So… This is an article about how to create a new user. It’s a pretty basic task but one you might just want to know about. You never know when you’ll need to create a new user but now you know where to look if you do need to. User management can be a pretty important task, especially for a server admin.

Thanks for reading! If you want to help, or if the site has helped you, you can donate, register to help, write an article, or buy inexpensive hosting to start your site. If you scroll down, you can sign up for the newsletter, vote for the article, and comment.

Let’s Learn About Grep

I’ve used the grep command many times but haven’t written an article to learn about grep. It seems like a good idea to do so so I can reference this article. That’s something that’s considered a good thing.

The thing is, I’ve tried to write this article before and it just came out terrible. That article never got published. It just wasn’t good enough. As you’ve seen the quality of some of these articles, tells you how bad that attempt went. 

You won’t need to install anything for this article. If you’re using a desktop or a server, you have grep. I’d expect to find grep in embedded systems because it’s just a useful tool. I suppose some companies might have ripped it out of their devices to save space and stop you from rooting around and messing with things.

So, I decided I’d do what I’d done before. I reached out to my buddy ChatGPT and asked them to write an article. Sure enough, ChatGPT did a fine job at it. This doesn’t save me much time. I still need to do all the formatting and that takes more time than you might expect.

So then, let’s get into the article…

About Grep:

The grep command in Linux is a powerful tool used for searching and manipulating text within files or standard input streams. It stands for “global regular expression print” and is primarily used to match patterns in text and display the lines that contain those patterns. grep is highly versatile and widely used in various scenarios, ranging from simple text searches to complex pattern matching and filtering tasks.

Basic Syntax:

The basic syntax for grep is as follows:

  • pattern: The pattern to search for. It can be a simple string or a complex regular expression.
  • file: Optional. The file(s) to search. If not specified, grep reads from standard input.
Common Options:
  • -i or --ignore-case: Ignore case distinctions.
  • -v or --invert-match: Invert the sense of matching, displaying non-matching lines.
  • -r or --recursive: Recursively search subdirectories.
  • -n or --line-number: Prefix each line of output with its line number.
  • -l or --files-with-matches: Display only the names of files containing matches.
  • -E or --extended-regexp: Interpret pattern as an extended regular expression (ERE).
  • -F or --fixed-strings: Interpret pattern as a list of fixed strings (not regular expressions).
Examples:

Basic Text Search:

This command searches for the occurrence of “keyword” in the file file.txt and displays all lines containing that keyword.

Case Insensitive Search:

This command performs a case-insensitive search for the pattern “pattern” in the file file.txt.

Invert Match:

This command displays all lines in file.txt that do not contain the word “exclude”.

Search in Multiple Files:

This command searches for the pattern in both file1.txt and file2.txt.

Recursive Search:

This command recursively searches for the pattern in all files within the specified directory and its subdirectories.

Display Line Numbers:

This command displays the line numbers along with the lines containing the pattern in file.txt.

Regular Expressions:

grep supports regular expressions, allowing for more advanced pattern matching. Regular expressions enable users to define complex search patterns, such as matching specific character sequences, ranges, or repetitions.

For example:

  • . matches any single character.
  • ^ matches the beginning of a line.
  • $ matches the end of a line.
  • [ ] specifies a character class.
  • * matches zero or more occurrences of the preceding element.
  • \ is used to escape special characters.
Additional Information:

The grep command is an essential tool for text processing and manipulation in the Linux terminal. Its versatility, combined with regular expressions, allows users to perform a wide range of tasks, including simple text searches, pattern matching, filtering, and data extraction. Whether it’s analyzing log files, searching for specific information in codebases, or performing system administration tasks, grep remains an indispensable utility for Linux users. Understanding its capabilities and various options can greatly enhance productivity and efficiency when working with text data in the terminal.

Closure:

So, there you have it. You have an AI-generated article about grep. It’s formatted quite differently than I’d normally format it, but it works. I dare say that AI did the job better than I had when I tried in the past.

People worry about AI but it’s just a tool. I am slowly learning when to make it useful to me. I’m not sure how many jobs AI is going to replace, but I can see lots of ways to use AI to make life easier. I can see ways to make AI more educational, such as in this article. It’s a nice overview, I think.

Thanks for reading! If you want to help, or if the site has helped you, you can donate, register to help, write an article, or buy inexpensive hosting to start your site. If you scroll down, you can sign up for the newsletter, vote for the article, and comment.

Find Files Created On A Specific Date

We sure do have a lot of file management articles and this is another one that will help you find files created on a specific date. This is a relatively easy task and you won’t even have to install anything new. If you want to find files created on a specific date, you might as well use this method!

Here’s a similar article.

Find Files Modified On A Specific Day

So, what is file management? To me, it’s manipulating files on your file system. It’s gathering information about the files on your file system. That’s what I mean when I say ‘file management’. I’ve covered a lot of articles with this as a subject.

Seriously… I tag articles with various terms. There are pages and pages of articles that have been tagged with file management.

Articles Tagged With File Management

Yup, that’s a lot of articles. Managing your files is something you do on a daily basis. You’re creating, editing, and moving your files around your file system. That’s file management and that’s okay!

Because this is something you do on a daily basis, even if you don’t pay attention to it, it’s something covered often. Even if you’re just browsing the internet, you’re adding files such as cached files. If you’re emailing with an email client, you have added and deleted files – if not more. File management is a pretty big thing.

Today, we’ll be using two basic commands – ls and grep. Let’s learn more!

The ls Command:

The ls command is an application that lets you list files. There are flags you can use, but it’s a basic application that, at its core, just lists files. You don’t need to install anything. You can verify that ls is available with this command:

The output should look like this:

Check the ls man page with this command:

If you do that, you’ll see that we’re on the right path if we want to find files created on a specific date. The ls command is briefly described as this:

ls – list directory contents

We’ll be using the ls command to output a list of files and we’ll then use the pipe operator to process that output with grep.

The grep Command:

The grep command is one of those commands that’s used in conjunction with another command. By itself, it’s not that useful. It is when you use grep with other commands that you realize how powerful it is.

The grep command is usually used with the | “pipe” operator. The pipe operator takes the output from one command and processes it with another application. I’ve not yet covered this in an article, nor have  I done an article about the grep command. I should do both.

Again, you won’t need to install anything. You can verify this with this command:

The output should match this:

Next, you can check the man page with this command:

grep, egrep, fgrep, rgrep – print lines that match patterns

You can see that there are options. We’ll be using just plain grep. And, as you can deduce, this is yet another process that will help us find files created on a specific date. We’ll take the output from one command and use it with the grep command. This should appear obvious shortly if you have not yet realized how we’ll be proceeding in this article.

Find Files Created On A Specific Date:

Both ls and grep are tools used in the terminal. As such, you can be sure that you’ll need an open terminal. Either open a terminal from your application menu or open a terminal by pressing CTRL + ALT + T on your keyboard. This article assumes that you don’t have year numbers in your file names, as you’ll see…

With your terminal open, we can begin to find files created on a specific date. Further, let’s stay right there in the ~/home/user directory. We’ll keep it simple.

Let’s start with this command:

That will list all your files. Now, let’s add hidden files – with the -a (all) flag with the following command:

Next up, let’s get more information from the ls command with the -l (long listing format) flag. That looks like this:

You should now see that there’s a time listed. We’re going through this just so that the example commands work. This isn’t strictly necessary, it’s just how it is being explained in this article.

If you want, you can sort that output by time. The following example command will show you the time the files were created, starting with the most recent example.

Again, that’s not strictly necessary.

Pay attention to the date section, because that’s what we’ll be using. 

More specifically, we’ll be piping the output from the ls command to the grep command. That’s done like this:

NOTE: You’ll be grepping a pattern. So, if you look carefully, the dates from 1 to 9 have two spaces! You’ll need to grep accordingly!

So, if you wanted to list articles created on the day this will be published, the command would look like this:

Notice that there are two spaces.

If you wanted to find files created on the 10th of February, your command would be different. You don’t add a 2nd space, as there is no second space in the pattern you’re looking for:

That will show you all the files (including the hidden files) that were created on the 10th of February. It does not care about the year and doesn’t show the year field. You’re simply finding files created on a specific date (ignoring the year, of course).

This is still useful.

This is also only true for files created within the past year. The ls command will happily show the date on files older than a year.

Here’s an example where the files are older than a year:

So, to find files including the year, your command now has more spaces to contend with. For this command, you would now add two spaces between the day and the year. Like so:

See the two spaces? That’s essential.

Again, don’t forget that you need two spaces when you’re working with days that are a single digit. If I wanted to find files from the 4th of November in 2022, that command would be this:

REMEMBER: The grep command is used to find patterns and will only find the specified patterns. It’s very specific! It is also very powerful when used properly.

You can do even more with this. If you want to find files older than a year and by the specific year of their creation, you could simply run a command that looks like this:

Of course, you can substitute ‘2020’ with any year you’d like and get results – so long as you have files that were created in that year.

Also, of course, that command is ONLY useful if you do not have year numbers or numbers similar to years in file names. After all, 2022 is 2022 and grep is going to find them all. That’s what grep does – it finds patterns. As such, it’s up to you to pick the patterns you’ve used and this does have a limited functionality if you’ve gone ahead and added dates to your file names.

If you do have numbers in your file names and you don’t have spaces, you can still grep for a pattern. For example, this might work assuming no space in your file name is in front of the year field – which there generally shouldn’t be:

I can’t be too specific because I don’t know if you’ve put numbers/years in your file names. If you have, grep will equally find those. This whole article assumes that you’ve done nothing of the sort.

This can be even handier and here’s a little bit of some bonus information.

Let’s say you have a directory where you’ve stored files for years. Further, you want to know how many files you created in the year 2022. This directory contains files containing many years and you don’t want to just count them.

Well, you can do something about that!

That command lists the files. The grep command then finds files created in 2022. You then pipe that output to the ‘wc’ command to count the lines.

Here’s an example output:

That means I added 8 files to that directory in the year 2023.

It’s pretty complicated. If you have years and spaces in your file names, this isn’t going to work that well for you. If you use patterns that match the output from the ls command, grep will find them. That’s what grep does.

While this does have a bit of a limited use case, it’s more of an exploration of what you can do with two simple commands. Unless you use matching patterns in your file names, this should work just famously for you. Otherwise, not so much – but you can have both fun trying AND you can look for other patterns that you can pass to grep.

Closure:

Well, that was kind of fun. We used the ls and grep commands to find files created on a specific date. There are always so many fun things to do with Linux. Many of these fun things can be done in the terminal. I like to think we both learn and have fun here on this site.

This can also be a handy tool. Let’s say you made some changes and then had some trouble later. You know you added files, but you can’t remember which files those were. Before restoring from a backup, you decide you want to troubleshoot. To do so, you start by finding files you created on the same day that you last changed your system.

Of course, this can be useful in a business setting. If you need to account for a problem, you can find the files created that day. Should there be something amiss, you can find files created on a specific date. If, for example, files were created on a date when no files should have been created… Well, you get the idea.

Perhaps more importantly, this shows what you can do with just two basic Linux commands. You’ll find ls and grep on every single desktop Linux (or server Linux) you touch. There’s no need for you to install something. You don’t have to worry about which package manager you use or building software from scratch because the developers didn’t release a package for your distro. You just use the commands that are already available.

Anyhow, this article is already too long. Sorry about that…

I kid. If I was sorry, I wouldn’t have done it in the first place! Ha! Take that!

Thanks for reading! If you want to help, or if the site has helped you, you can donate, register to help, write an article, or buy inexpensive hosting to start your site. If you scroll down, you can sign up for the newsletter, vote for the article, and comment.

Linux Tips
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.