Category: Command Line

Demystifying journalctl: A Comprehensive Guide to Linux System Logging

It was suggested that I write an article about journalctl, which seemed like a large topic. I decided that I’d let AI have a shot at it, so this article was written by ChatGPT.

It took a few prompts to get what I wanted – which turned out to be the first result. I gave the AI the chance to rework the article but the result was that I much preferred the initial offering. After all, I was only after a very light overview of the journalctl command. 

There’s a lot to the journalctl command. The journalctl command is far too much to cover in a single article. Heck, I don’t even know some aspects of the command. You can see this by checking the man page with the following command:

See? There’s a lot to the command. At the end of the day, AI did a good job of summing up what you really need from the command as an average user. So, I’m going to go ahead and publish that content. It did a better job than I’d have done!

Introduction To journalctl:

In the realm of Linux system administration, understanding and managing system logs is indispensable. Logs provide crucial insights into the health, performance, and security of a system. Among the plethora of tools available for log management, journalctl stands out as a powerful and versatile command for accessing and analyzing logs in systems utilizing systemd. In this comprehensive guide, we will delve into the intricacies of journalctl, exploring its features, functionalities, and practical applications.

Understanding Systemd Journal:

Systemd, the init system adopted by many modern Linux distributions, introduced the systemd journal as a replacement for traditional syslog. The journal, stored in binary format, offers numerous advantages over syslog, including structured logging, faster search capabilities, and enhanced metadata.

journalctl serves as the primary interface for querying and interacting with the systemd journal. It provides administrators with a rich set of options for filtering, displaying, and analyzing log entries, empowering them to effectively troubleshoot issues, monitor system activity, and extract valuable insights.

Basic Usage:

At its core, journalctl allows users to retrieve and view log entries from the systemd journal. The simplest invocation of journalctl displays the entire journal, starting with the most recent entries:

This command presents a paginated output of log entries, including timestamps, log levels, and message contents. By default, journalctl displays logs from the current boot session. However, it also supports options for querying logs from previous boots or specific time ranges.

Filtering and Querying:

One of the key strengths of journalctl lies in its ability to filter log entries based on various criteria. Administrators can narrow down the search results by specifying filters such as time range, log level, systemd unit, or specific fields within log messages.

For example, to display all log entries generated by the sshd service, the following command can be used:

Similarly, to retrieve logs pertaining to a particular time range, the --since and --until options can be utilized:

Additionally, journalctl supports advanced filtering using Boolean expressions, enabling complex queries to be constructed for precise log retrieval.

Output Formatting and Presentation:

journalctl offers flexible options for customizing the format and presentation of log entries. Administrators can choose from various output formats, including short, verbose, and JSON formats, depending on their preferences and requirements.

For instance, to display log entries in a more detailed and verbose format, the -o verbose option can be used:

Moreover, journalctl provides options for controlling the display of timestamps, including the ability to convert timestamps to the local time zone or display them in a human-readable format.

Real-time Monitoring and Follow Mode:

In addition to viewing historical log entries, journalctl can also be used for real-time monitoring of system logs. By invoking journalctl with the -f or --follow option, administrators can tail the journal and receive live updates as new log entries are added:

This feature is particularly useful for monitoring system activity in real time, diagnosing issues as they occur, and tracking the progress of system services during startup and shutdown sequences.

Integration with Other Tools:

journalctl seamlessly integrates with other Linux system administration tools, enabling administrators to combine its capabilities with those of other utilities for more comprehensive log analysis and management.

For example, grep can be used in conjunction with journalctl to perform pattern matching and further refine log queries:

Furthermore, administrators can leverage shell scripting and automation to automate log analysis tasks, extract meaningful insights, and trigger alerts based on specific log patterns or conditions.

Conclusion:

In conclusion, journalctl emerges as a powerful and indispensable tool for managing system logs in Linux environments utilizing systemd. Its rich feature set, flexible filtering options, and real-time monitoring capabilities make it an invaluable asset for system administrators, enabling them to effectively diagnose issues, monitor system activity, and ensure the stability and security of their systems.

By mastering the intricacies of journalctl, administrators can gain deeper insights into system behavior, streamline troubleshooting workflows, and maintain the optimal performance of their Linux infrastructure. Whether it’s debugging a system issue, auditing security events, or analyzing performance metrics, journalctl empowers administrators to navigate the complexities of system logging with confidence and efficiency.

Closure:

Well, I keep finding uses for AI. This is a better article than I’d have written. I think I’ll next use AI for a solid article about grep. That sounds like a reasonable subject and it’s an article that I can reference in many other articles. In fact, I should have done an article about grep already!

So, this is an article about journalctl. It’s an overwhelming command. It’s amazingly complicated and powerful, but you (as a regular user, as most of my readers are) will only need to know the basics. This is indeed the basics and they appear to be well-described.

Thanks for reading! If you want to help, or if the site has helped you, you can donate, register to help, write an article, or buy inexpensive hosting to start your site. If you scroll down, you can sign up for the newsletter, vote for the article, and comment.

Extract Text From Multiple File Types

Today we will have a fairly simple exercise as we’re going to just use a Python application to extract text from multiple file types. This is a pretty standard operation but will require some preparation.

Fortunately, I’m ahead of the game! You’re good to go if you follow along on the site and have already enabled PIP. Otherwise…

You will need to install PIP for this article. This is not complicated.

First, read this article:

Install Python’s PIP Part One

Technically, you could just do that. However, you should add the path so that you don’t have to specify the location of your Python applications and can easily use them from the terminal.

So, read this article:

Install Python’s PIP Part Two

Now that you’ve done those two things, you’re good to proceed. See? It was worth the time to write those articles! They’re useful and save a lot of time.

The tool we’re going to use is known as “Textract“. Don’t quote me on this, but I believe this could also apply to Windows users, though installing the dependencies for this would be a different process. I’m not a Windows user. If you are, feel free to comment and let us know how things work on your side of life.

Textract: 

While there is no built-in man page, the Textract application is described like this:

While several packages exist for extracting content from each of these formats on their own, this package provides a single interface for extracting content from any type of file, without any irrelevant markup.

It is a pretty handy application and claims to extract the text from more file types than I could reasonably expect to test. Here’s a list of files that you should be able to extract text from.

.csv via python builtins
.doc via antiword
.docx via python-docx2txt
.eml via python builtins
.epub via ebooklib
.gif via tesseract-ocr
.jpg and .jpeg via tesseract-ocr
.json via python builtins
.html and .htm via beautifulsoup4
.mp3 via sox, SpeechRecognition, and pocketsphinx
.msg via msg-extractor
.odt via python builtins
.ogg via sox, SpeechRecognition, and pocketsphinx
.pdf via pdftotext (default) or pdfminer.six
.png via tesseract-ocr
.pptx via python-pptx
.ps via ps2text
.rtf via unrtf
.tiff and .tif via tesseract-ocr
.txt via python builtins
.wav via SpeechRecognition and pocketsphinx
.xlsx via xlrd
.xls via xlrd

You may need to install specific packages for some of these file formats. Those packages can usually be found in your default repositories. It otherwise comes with quite a lot of functionality out of the box.

I did test some of those formats and it seemed to work okay. Your mileage may vary, of course. However, Textract was able to extract text from multiple file types.

Extract Text From Multiple File Types:

If you want to extract text from multiple file types with Textract (a fantastic name for an application) then you’ll first need to install it. I’ve yet to find a working GUI PIP installation tool, so that means you’re going to need an open terminal.

More often than not, you can open your terminal by simply pressing CTRL + ALT + T on your keyboard. If your distro doesn’t adhere to the norms, you can find a terminal in your application menu. If you don’t use an application menu, you already know how to open a terminal and you don’t need any help from me.

First, let’s install Textract:

Note the lack of sudo. You’re installing this for your user account and do not need elevated permissions for this. Python packages go right into your ~/ directory. See below, as you’ll want to install some dependencies for full functionality.

You may see an error or two during installation but that doesn’t seem to matter. It will take a minute to install and watching the installation chug along is good fun.

Using Textract:

With Textract installed, you can now extract text from a whole variety of file types. The syntax is as follows:

That sends the output to the standard output (your terminal). I suspect that most folks are going to want to save the output to a file. For that, you just need to add the -o flag and a file name. So, something like this:

That’s going to extract the text from some file types but not all of them.

Now, this is from a Lubuntu installation…

This isn’t going to work with all the listed file types at this time. You need some dependencies to be installed. For me, and it’s a long one, the command was:

That’s slightly different from the command they include on their page, but it appears to do the trick. You’ll have some of those installed by default but running the command will sort itself out. You’ll have to modify the command to suit your distro, but that should work with Debian, Ubuntu, Linux Mint, and other Debian-based distros.

With that installed, I can even grab the text from image files.

Here’s an example:

a simple picture with simple text
This is some simple text to test how well Textract really works.

Here’s the command:

Here’s the output:

I dare say that’s pretty good. I tried other pictures and it was good enough to get the gist of things. Complicated image files with many columns appear to be a bit of a stumbling block. But it’s not terrible.

It has no trouble at all with other file formats.

It can be a bit fussy to get Textract properly installed but it seems to do the trick once installed. If you want to extract text from multiple file types, Textract is a pretty good piece of software.

Closure:

If you want to extract text from multiple file types, this is definitely a good tool for the job. It certainly handles a lot of files and does a good job with them. It’s not perfect. None of these tools are. Complicated image files threw it off a bit, but Textract lives up to its name.

There was a reason I wrote those articles about PIP. Being able to install Python packages via a repository is a great thing. There’s some great Python software out there and we’ve barely touched the surface. Linux is great like that, that is offering great Python support.

Do you have a use for this in your daily activities? If so, leave a comment letting us know how you use Textract and what makes you pick it over other applications. You can even use a real email address. I never send spam. I never sell your information.

Thanks for reading! If you want to help, or if the site has helped you, you can donate, register to help, write an article, or buy inexpensive hosting to start your site. If you scroll down, you can sign up for the newsletter, vote for the article, and comment.

View Some Logs In The Terminal

Today we’ll have a relatively simple set of commands that will show you how to view some system logs in the terminal. We’re only going to cover a few of them that are similar in operation. There are far more things that get logged.

You generate logs as you use your computer. These will vary and there are quite a few logs kept. Logs aren’t just kept by the system. Some third-party applications create logs. We’ll only be covering a few system logs. I just want to keep things simple.

The tool we’re using for this exercise is the cat command. The cat application is one tool that lets you read text files in the terminal. It’s a frequently used tool at my house. Hopefully, you too will get comfortable using this command at your house.

It seems like that should be enough of an intro. If you have any questions, you can always leave a comment. Those get seen and addressed more quickly than when you try to contact me elsewhere. (Comments almost instantly send a notification to whatever computer I’m using. They’ll even ping my phone if I turn the notification sound on.)

Anyhow… Ask away, should you have any questions…

View Some System Logs In The Terminal:

We’ll be viewing a few different types of logs. As mentioned above and in the heading, this is something we do in the terminal. You can usually open a terminal by pressing CTRL + ALT + T. If that doesn’t work, you’ll almost certainly find a terminal in your application menu.

First, we’ll view the kernel log.

The kernel is the actual “Linux” in your Linux. The kernel deals with task scheduling, and running processes as needed. It’s an abstraction between you and the hardware.

If you want to view the current kernel log, use this command:

If you want to view the previous session’s kernel logs, use this command:

Both of those commands will flood your terminal. That’s okay, you can use the pipe operator and the less command like so:

Next, we’ll view the boot log.

When you boot your computer, the computer keeps track of what happens during that process. This is known as the boot log. It can be exceptionally handy when you have a problem booting. 

If you want to check the current boot log, use this command:

If you want to check the previous boot log, check it with this command:

Again, you can use a pipe and the less command to manage the flow of data.

Next, we’ll view the system log.

The system logs all sorts of additional information. If you’re not finding the information you want in the previous two logs, checking the system log is prudent. The system logs all sorts of things that are useful for resolving problems.

If you want to check the current system log, use this command:

If you want to check the previous system log, run this command:

Don’t forget that you can use a pipe and less in this command. This will give you a slower output you can manage with your arrow keys.

And that will show you your system log.

That’s all it takes to view the major system logs. There are other logs and we’ll have to cover them at some point.

Closure:

Well, this didn’t take too many words or too much time… That’s nice. If you want to start troubleshooting your own problems, learn to read the system logs. If you don’t want to ask for help, learn to read the system logs. Well, that and learn to read the man pages, but that last bit is outside the scope of this article.

I do hope you enjoyed this article. They’re fun to write but it does sometimes feel like work. I never wanted this to feel like it was work, but here we are. It probably would have been wiser to not set such a schedule and to allow some vacation time in there. This whole project started when I had far more time due to the pandemic.

We do have a special day coming up… You’ll see!

Thanks for reading! If you want to help, or if the site has helped you, you can donate, register to help, write an article, or buy inexpensive hosting to start your site. If you scroll down, you can sign up for the newsletter, vote for the article, and comment.

Short: Move A File To Multiple Directories

Today’s article should be a fairly short article, where I take the chance to show you how to move a file to multiple directories – at the same time. It’s a pretty simple process, albeit a bit weird. If you want to move files to multiple directories, this is the article for you!

I’ve done a lot of articles about file management. This is another one. We usually manage files in the terminal here on Linux Tips. We’ll be doing that again this time around. We’ll be managing files in the terminal!

The tool we’ll be using is the tee command. I covered that recently.

The task we’ll be doing is showing you how to move a file to multiple directories – at the same time and in the Linux terminal. It’s a pretty handy skill to have, though there’s some tee weirdness along the way.

Move A File To Multiple Directories:

You’ll need an open terminal for this, as the intro suggested. You can find a terminal application in your GUI file manager. You can often open your default terminal by pressing CTRL + ALT + T on your keyboard.

With your terminal open, let’s just run through some exercises to ensure we’re all on the same page and ready to move a file. 

First, let’s make a directory:

Next, let’s move to that directory:

Next, let’s make that file that we’ll copy to multiple directories:

I suppose we’ll need a few folders next, so let’s create them:

Now, let’s move the file foo to multiple directories:

The syntax is:

We throw the & in so that the tee command doesn’t hang, awaiting further input. You’ll get an extra message or two, but that’s fine and can be ignored.

Now, we can verify them:

The following command will show you that the file exists:

That should show the file, like so:

Now, you can check the other folders. 

And, of course:

Each of those should show you that the foo file exists in each directory.

Now, we can clean up after ourselves with this command:

Tada! Now there shouldn’t be any remains left behind and we’ve covered how to move a file to multiple directories at the same time and in the Linux terminal.

Pretty easy and pretty short!

Closure:

You never know when you’ll want to move a file to multiple directories but now you know how to do so. It isn’t a very difficult exercise, warranting only a short article. This seemed like good fodder for a short article and a handy tip to share with my readers.

Thanks for reading! If you want to help, or if the site has helped you, you can donate, register to help, write an article, or buy inexpensive hosting to start your site. If you scroll down, you can sign up for the newsletter, vote for the article, and comment.

Another Way To Show Mounted Filesystems

We’ve covered this topic before, but I want to show you another way that you can show mounted filesystems. As previously covered, you have multiple filesystems mounted at any one time. So, let’s examine them, shall we?

If you use Linux, you use multiple filesystems. There are real and virtual filesystems created and used by the operating system. A real filesystem would be something like a partition on your storage device (one filesystem per partition, of course). A virtual filesystem would be things like your temporary files or cached files, among other filesystems used by the system.

We’ve covered this before and this is just another way to show your mounted filesystems. This is good information to have, as it’s handy to know the path for those things so that you don’t do something silly like misfire a dd command and hosing your OS partition. (Ask me how easy that is.)

Show Your Filesystem In The Linux Terminal
Another Way To View Attached Storage Devices
Another Way To View Attached Storage Devices

(And there are more! This is an oft-covered subject on Linux-Tips!)

So, what will we be using?

findmnt:

You shouldn’t need to install anything for this article. The findmnt tool should be installed by default. You can verify that findmnt is installed by running the following command:

The output of that command should probably match this one:

You check the man page with this command:

There’s additional information available in this application. Run this command:

But, if you check the man page, you’ll see that this is indeed one of the correct tools for the job. This being a basic task, there are many ways to show mounted filesystems. It’s up to you to pick a favorite – or find the most useful of commands for your particular situation.

Anyhow, the man page describes it as:

findmnt – find a filesystem

That’s what we want to do. That means that this is the correct tool for the job.

Show Mounted Filesystems:

You’ve been here long enough to know that we’re likely to use the terminal. If your hunch was that we would be using the terminal, your hunch was correct. You’ll find that findmnt is a terminal-based application so open your default terminal emulator by finding it in your application menu or pressing CTRL + ALT + T on your keyboard.

With your terminal open, you can show mounted filesystems with this command:

You’ll see that it even has a nice tree output. You’ll find things like Snaps have their own filesystem if you’re using a system that uses Snaps. Other jailed applications may also use their own filesystem as a way to keep things separate from the system as a whole. 

Here’s an example truncated output from Lubuntu:

That’s not nearly as complicated as it looks, once you get used to it.

Now, you can trim all that down. You can opt to show only the ‘real’ filesystems. That’s done with the --real flag.

This is an example of that command on that same Lubuntu system used above (Snap applications are seen as ‘real’ filesystems as a part of their separation from the system):

Here’s an example output from a Linux Mint system that does not use Snaps:

You can do more with the findmnt command, such as select the columns you want in your output, so be sure to check the man page. This is just a quick overview because the vast majority of you will never need more than just the basic command.

As I said, it’s a useful command for when you want to verify the path to a filesystem. Beyond that, the usage is up to you. That’s what I use it for when I don’t bother with any one of several other commands.

Closure:

You might see no reason to add this to your list of commands, but it is useful when you want to see all the mounted filesystems on your system. If you need the path for a filesystem, this is a good command. It’s also useful for finding out some attributes of your mounted filesystems.

This is one of those commands you really could keep in your back pocket. You might first lean on a command like lsblk but that only lists block devices – that is, storage devices, and doesn’t include the many other mounted filesystems. If you don’t need this command today, that’s fine. You might need this command in the future, and now you know it exists.

Thanks for reading! If you want to help, or if the site has helped you, you can donate, register to help, write an article, or buy inexpensive hosting to start your site. If you scroll down, you can sign up for the newsletter, vote for the article, and comment.

Linux Tips
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.