Demystifying journalctl: A Comprehensive Guide to Linux System Logging

It was suggested that I write an article about journalctl, which seemed like a large topic. I decided that I’d let AI have a shot at it, so this article was written by ChatGPT.

It took a few prompts to get what I wanted – which turned out to be the first result. I gave the AI the chance to rework the article but the result was that I much preferred the initial offering. After all, I was only after a very light overview of the journalctl command. 

There’s a lot to the journalctl command. The journalctl command is far too much to cover in a single article. Heck, I don’t even know some aspects of the command. You can see this by checking the man page with the following command:

See? There’s a lot to the command. At the end of the day, AI did a good job of summing up what you really need from the command as an average user. So, I’m going to go ahead and publish that content. It did a better job than I’d have done!

Introduction To journalctl:

In the realm of Linux system administration, understanding and managing system logs is indispensable. Logs provide crucial insights into the health, performance, and security of a system. Among the plethora of tools available for log management, journalctl stands out as a powerful and versatile command for accessing and analyzing logs in systems utilizing systemd. In this comprehensive guide, we will delve into the intricacies of journalctl, exploring its features, functionalities, and practical applications.

Understanding Systemd Journal:

Systemd, the init system adopted by many modern Linux distributions, introduced the systemd journal as a replacement for traditional syslog. The journal, stored in binary format, offers numerous advantages over syslog, including structured logging, faster search capabilities, and enhanced metadata.

journalctl serves as the primary interface for querying and interacting with the systemd journal. It provides administrators with a rich set of options for filtering, displaying, and analyzing log entries, empowering them to effectively troubleshoot issues, monitor system activity, and extract valuable insights.

Basic Usage:

At its core, journalctl allows users to retrieve and view log entries from the systemd journal. The simplest invocation of journalctl displays the entire journal, starting with the most recent entries:

This command presents a paginated output of log entries, including timestamps, log levels, and message contents. By default, journalctl displays logs from the current boot session. However, it also supports options for querying logs from previous boots or specific time ranges.

Filtering and Querying:

One of the key strengths of journalctl lies in its ability to filter log entries based on various criteria. Administrators can narrow down the search results by specifying filters such as time range, log level, systemd unit, or specific fields within log messages.

For example, to display all log entries generated by the sshd service, the following command can be used:

Similarly, to retrieve logs pertaining to a particular time range, the --since and --until options can be utilized:

Additionally, journalctl supports advanced filtering using Boolean expressions, enabling complex queries to be constructed for precise log retrieval.

Output Formatting and Presentation:

journalctl offers flexible options for customizing the format and presentation of log entries. Administrators can choose from various output formats, including short, verbose, and JSON formats, depending on their preferences and requirements.

For instance, to display log entries in a more detailed and verbose format, the -o verbose option can be used:

Moreover, journalctl provides options for controlling the display of timestamps, including the ability to convert timestamps to the local time zone or display them in a human-readable format.

Real-time Monitoring and Follow Mode:

In addition to viewing historical log entries, journalctl can also be used for real-time monitoring of system logs. By invoking journalctl with the -f or --follow option, administrators can tail the journal and receive live updates as new log entries are added:

This feature is particularly useful for monitoring system activity in real time, diagnosing issues as they occur, and tracking the progress of system services during startup and shutdown sequences.

Integration with Other Tools:

journalctl seamlessly integrates with other Linux system administration tools, enabling administrators to combine its capabilities with those of other utilities for more comprehensive log analysis and management.

For example, grep can be used in conjunction with journalctl to perform pattern matching and further refine log queries:

Furthermore, administrators can leverage shell scripting and automation to automate log analysis tasks, extract meaningful insights, and trigger alerts based on specific log patterns or conditions.

Conclusion:

In conclusion, journalctl emerges as a powerful and indispensable tool for managing system logs in Linux environments utilizing systemd. Its rich feature set, flexible filtering options, and real-time monitoring capabilities make it an invaluable asset for system administrators, enabling them to effectively diagnose issues, monitor system activity, and ensure the stability and security of their systems.

By mastering the intricacies of journalctl, administrators can gain deeper insights into system behavior, streamline troubleshooting workflows, and maintain the optimal performance of their Linux infrastructure. Whether it’s debugging a system issue, auditing security events, or analyzing performance metrics, journalctl empowers administrators to navigate the complexities of system logging with confidence and efficiency.

Closure:

Well, I keep finding uses for AI. This is a better article than I’d have written. I think I’ll next use AI for a solid article about grep. That sounds like a reasonable subject and it’s an article that I can reference in many other articles. In fact, I should have done an article about grep already!

So, this is an article about journalctl. It’s an overwhelming command. It’s amazingly complicated and powerful, but you (as a regular user, as most of my readers are) will only need to know the basics. This is indeed the basics and they appear to be well-described.

Thanks for reading! If you want to help, or if the site has helped you, you can donate, register to help, write an article, or buy inexpensive hosting to start your site. If you scroll down, you can sign up for the newsletter, vote for the article, and comment.

Extract Text From Multiple File Types

Today we will have a fairly simple exercise as we’re going to just use a Python application to extract text from multiple file types. This is a pretty standard operation but will require some preparation.

Fortunately, I’m ahead of the game! You’re good to go if you follow along on the site and have already enabled PIP. Otherwise…

You will need to install PIP for this article. This is not complicated.

First, read this article:

Install Python’s PIP Part One

Technically, you could just do that. However, you should add the path so that you don’t have to specify the location of your Python applications and can easily use them from the terminal.

So, read this article:

Install Python’s PIP Part Two

Now that you’ve done those two things, you’re good to proceed. See? It was worth the time to write those articles! They’re useful and save a lot of time.

The tool we’re going to use is known as “Textract“. Don’t quote me on this, but I believe this could also apply to Windows users, though installing the dependencies for this would be a different process. I’m not a Windows user. If you are, feel free to comment and let us know how things work on your side of life.

Textract: 

While there is no built-in man page, the Textract application is described like this:

While several packages exist for extracting content from each of these formats on their own, this package provides a single interface for extracting content from any type of file, without any irrelevant markup.

It is a pretty handy application and claims to extract the text from more file types than I could reasonably expect to test. Here’s a list of files that you should be able to extract text from.

.csv via python builtins
.doc via antiword
.docx via python-docx2txt
.eml via python builtins
.epub via ebooklib
.gif via tesseract-ocr
.jpg and .jpeg via tesseract-ocr
.json via python builtins
.html and .htm via beautifulsoup4
.mp3 via sox, SpeechRecognition, and pocketsphinx
.msg via msg-extractor
.odt via python builtins
.ogg via sox, SpeechRecognition, and pocketsphinx
.pdf via pdftotext (default) or pdfminer.six
.png via tesseract-ocr
.pptx via python-pptx
.ps via ps2text
.rtf via unrtf
.tiff and .tif via tesseract-ocr
.txt via python builtins
.wav via SpeechRecognition and pocketsphinx
.xlsx via xlrd
.xls via xlrd

You may need to install specific packages for some of these file formats. Those packages can usually be found in your default repositories. It otherwise comes with quite a lot of functionality out of the box.

I did test some of those formats and it seemed to work okay. Your mileage may vary, of course. However, Textract was able to extract text from multiple file types.

Extract Text From Multiple File Types:

If you want to extract text from multiple file types with Textract (a fantastic name for an application) then you’ll first need to install it. I’ve yet to find a working GUI PIP installation tool, so that means you’re going to need an open terminal.

More often than not, you can open your terminal by simply pressing CTRL + ALT + T on your keyboard. If your distro doesn’t adhere to the norms, you can find a terminal in your application menu. If you don’t use an application menu, you already know how to open a terminal and you don’t need any help from me.

First, let’s install Textract:

Note the lack of sudo. You’re installing this for your user account and do not need elevated permissions for this. Python packages go right into your ~/ directory. See below, as you’ll want to install some dependencies for full functionality.

You may see an error or two during installation but that doesn’t seem to matter. It will take a minute to install and watching the installation chug along is good fun.

Using Textract:

With Textract installed, you can now extract text from a whole variety of file types. The syntax is as follows:

That sends the output to the standard output (your terminal). I suspect that most folks are going to want to save the output to a file. For that, you just need to add the -o flag and a file name. So, something like this:

That’s going to extract the text from some file types but not all of them.

Now, this is from a Lubuntu installation…

This isn’t going to work with all the listed file types at this time. You need some dependencies to be installed. For me, and it’s a long one, the command was:

That’s slightly different from the command they include on their page, but it appears to do the trick. You’ll have some of those installed by default but running the command will sort itself out. You’ll have to modify the command to suit your distro, but that should work with Debian, Ubuntu, Linux Mint, and other Debian-based distros.

With that installed, I can even grab the text from image files.

Here’s an example:

a simple picture with simple text
This is some simple text to test how well Textract really works.

Here’s the command:

Here’s the output:

I dare say that’s pretty good. I tried other pictures and it was good enough to get the gist of things. Complicated image files with many columns appear to be a bit of a stumbling block. But it’s not terrible.

It has no trouble at all with other file formats.

It can be a bit fussy to get Textract properly installed but it seems to do the trick once installed. If you want to extract text from multiple file types, Textract is a pretty good piece of software.

Closure:

If you want to extract text from multiple file types, this is definitely a good tool for the job. It certainly handles a lot of files and does a good job with them. It’s not perfect. None of these tools are. Complicated image files threw it off a bit, but Textract lives up to its name.

There was a reason I wrote those articles about PIP. Being able to install Python packages via a repository is a great thing. There’s some great Python software out there and we’ve barely touched the surface. Linux is great like that, that is offering great Python support.

Do you have a use for this in your daily activities? If so, leave a comment letting us know how you use Textract and what makes you pick it over other applications. You can even use a real email address. I never send spam. I never sell your information.

Thanks for reading! If you want to help, or if the site has helped you, you can donate, register to help, write an article, or buy inexpensive hosting to start your site. If you scroll down, you can sign up for the newsletter, vote for the article, and comment.

Meta: I’ve Been At This For Three Years!

I have not done a meta article lately. I don’t find them interesting to write even though people seem to show some interest in them. They’re a pain to write and I can write a regular article easier than a meta article.

After all, my normal meta article is just updating you on how much traffic the site gets and how much bandwidth the site uses. I also tend to toss in some stats about where the traffic comes from and what that traffic looks like.

That’s not all that fun and can be a pain to write.

So, let’s get that out of the way…

February wasn’t as great but January was awesome. Last month saw a new level reached – where this site averaged more than 1000 visitors per day. 

visit statistics
Yup… There are a lot of you these days…

I guess the next major threshold will be when I get 1000 unique visitors in a month. Quite a few of you visit more than once, those are the ‘visits’ in the chart. The unique visitors are just that, how many unique people visit the site.

Sure enough, my traffic still comes from Google – even though I no longer use AdSense. As far as search traffic, the next two most popular search engines (for this site) are DuckDuckGo and StartPage. 

The most popular operating systems are Linux, MacOS, and Windows. I suppose that makes some sense. Some of what’s on this site applies to MacOS as they’re a POSIX-compliant operating system.

The most popular browsers are Chrome (and those that identify as Chrome), Unknown, and Safari. Firefox isn’t well represented here. They’re 4th on the list with about 19% of my traffic.

My legitimate traffic comes from the United States, Germany, and the United Kingdom. China and Russia are in the stats, but they’re mostly bots. Finding accurate stats to pick between them can be difficult. So, the above is about what I can figure out.

Yeah, I regularly consume 50+ GB of traffic per month. Considering the site is pretty much pure text, that’s a lot of writing. There are millions of words on the site now and I’m still not out of ideas.

Anniversary!

This is our third anniversary! Three years!!! Whodathunkit?

This was the first article:

Welcome to Our New Home!

I’ve had some guest articles along the way (and at least another one coming when I can schedule it). Most of the articles, 99.999% at least, have been written by me. 

We don’t get a lot of donations but we get a couple here and there. That helps cover the costs, plus people can now advertise here on the site. That’s helpful. I do love my CDN though it (and hosting) are expenses.

This has taken a whole lot of my time. I value my time, but I guess I also value the site. Otherwise, I’d have stopped publishing articles. One of these days I’ll quit but I plan on keeping up with the schedule for the time being.

That schedule? Well, you get a new article every other day. Most of those articles don’t contain any major errors, which is nice. I can’t be perfect all of the time, but I do my best. You’re welcome. 

I’m working my way towards a million visits. That’s nice. 

Here are the most popular articles:

How To: Remove AppArmor From Ubuntu
Change Between CLI and GUI Mode
How To: Disable Sleep And Hibernation on Ubuntu Server

Once upon a time, I was stoked to see 20 visits in a day. So, I guess I’ve built something here. Pardon me, but I’m a wee bit proud of my accomplishment. It has come a long way.

As such, I’m just going to keep this short.

Thank you. Thank you for your readership and encouragement. Here’s to another year. I think I’ve got another year of this in me. We’ll have to see. I’m bound to miss a day eventually, but this is three years without doing so (technically).

Closure:

That’s all, folks. I appreciate you.

Thanks for reading! If you want to help, or if the site has helped you, you can donate, register to help, write an article, or buy inexpensive hosting to start your site. If you scroll down, you can sign up for the newsletter, vote for the article, and comment.

How To: Disable CPU Cores

This is not something everyone is interested in doing but you might as well learn how to disable CPU cores. This is something that’s easily done and easily reversed, so you might as well have an article on the subject.

Why would you do this? Well, maybe you have a low-power project and want to save energy. You may have more CPU horsepower than you need and you prioritize energy use. This might even be something you do to increase the battery life in your mobile devices. Maybe you want to lower the heat output from your CPUs.

There are valid reasons why you might want to disable CPU cores.

It’s just that this isn’t going to apply to most of you. I’d say that 99% of you aren’t going to want to disable your CPU cores.

What are cores? Well, you probably have one single CPU chip in your computer. That computer is then probably broken down into multiple cores. Each core looks like a single CPU to the hardware and operating system.

Of course, each core may also have multiple threads. Threads would also represent themselves as individual cores. If you disable a CPU core, you’ll also disable the threads on that core, if you use a multi-threaded CPU.

NOTE: Moden CPUs may have high-power and low-power CPUs. I have no clue which will be disabled if you follow this article. I don’t have a fancy CPU to test with. You can test and let us know the results! It’s easily reversed. You can add your cores back with no problems.

GRUB:

GRUB is the application we’ll be playing with. I should also mention that GRUB stands for Grand Unified Bootloader and is the default bootloader for most Linux distributions.

GRUB controls how your system boots and GRUB can also give you the option of which operating system to boot if you use a multi-boot system. It’s one of those things that you can modify pretty heavily. You can even modify your boot screen’s appearance when you’re using GRUB.

Trying to access the GRUB man page is interesting. It isn’t your traditional man page and isn’t accessed in the same way. If you want to check the man page, you use this command:

If you run that command you’ll see what looks like an index page. That’s because it is an index page. Use your arrow keys to navigate and use the enter button to open a category page. It’s pretty simple to navigate but there’s a whole lot of information there.

If you’re a newbie, you might just want to look and see that info page. Trying any of that out may result in a system you can’t boot. You have been warned. Tinkering blindly with GRUB will eventually break your system unless you know what you’re doing. Not even I understand everything – but that’s true about a lot of things.

This article assumes that you have GRUB and Nano installed.

So, now that you know what we’ll be using, let’s get into the article!

Disable CPU Cores:

If you didn’t figure it out from the ‘Nano’ mentioned above, this is one of those things you do in the terminal. More often than not, you can open a terminal by pressing CTRL + ALT + T on your keyboard. If that doesn’t work, look in your application menu to find your terminal and open it.

Next, you need to know how many CPUs you have. You should probably already have that information. If you don’t have that information, you can use something like ScreenFetch to find that information. (I used lscpu to find my information, but I’m trying to keep it simple for you.)

Just run this command (once you’ve installed ScreenFetch):

The output should tell you how many cores you have available. Like so:

screenfetch showing CPU cores
This should be easy enough for anyone to figure out! You can find this information easily.
The Exercise Itself:

Now that you know how many cores you have, you can modify GRUB.

Find the following line:

Change that line to something like this:

As you saw above, this system has 8 CPUs listed. Let’s say I wanted to cut that in half, perhaps to save battery life while I’m on a trip and not doing any computationally heavy tasks. I’d change that line to match this:

Now that you’ve made the edit, you need to save the file. As you’re using Nano for this, you save files by pressing CTRL + X, then Y, and then ENTER.

The next step is to update GRUB, telling GRUB to reconfigure itself because you’ve changed some of the information. This step is essential but simple.

Next, you reboot the system for the changes to take effect:

When you next open your system, run screenfetch again to confirm that you’re using fewer cores and that you’re using the number of cores you specified.

I did this in a virtual machine with 2 CPUs. This was done to confirm that it works and to provide you with some screenshots. When I did so, I did so with lscpu and not ScreenFetch. I hadn’t thought of ScreenFetch and I’d already taken the screenshots. So, that’s what you get.

The Results:

The original configuration:

using lsusb to show the number of CPU cores
As you can see, there are two cores in this instance. Core 0 and Core 1.

After the change:

using lscpu to show that there are now fewer cores loaded
Sure enough, there are now fewer cores in use than there were before. It’s just core 0.

As you can see from the screenshots (again, taken in a virtual machine), it’s easy enough to disable CPU cores in Linux. Well, assuming you’re using GRUB… I’m sure it’s possible with other bootloaders but I don’t have any real experience with them in the memorable past. I’ve used GRUB, more or less exclusively, for quite some time.

If you want to undo this change, or if you want to change the CPU cores to some other number, just remove or edit the added text. If you do that, be sure to run sudo update-grub again to ensure that your changes take effect. If you don’t remember to do that, any changes made will be lost. So, it’s important to update GRUB when you’ve changed any of the information – not just this information.

Closure:

Well, I’m not sure how many Linux users are going to want to change their CPU cores to a lower value. It was in my notes and seemed like a fine article to share with you. I can see doing this in a few instances, such as when you want to save power. Of course, modern operating systems are pretty good at saving power on their own, but it’s still something one might consider.

If you have any other reasons as to why one might want to disable CPU cores, leave a comment. I’m a bit curious. I’d probably do it if I needed longer battery life while on the road or something. I don’t do too many computationally heavy tasks these days. So, I could get away with it on some systems.

Also, this is from my notes. My notes predate the modern CPUs. Quite literally, I have no idea how this will work on systems that have cores that are different, such as high-power and low-power cores. Feel free to test and let me know what you learn. Again, leave a comment. I promise I won’t share your information with anyone and I’ll never send you an email you didn’t ask for.

Thanks for reading! If you want to help, or if the site has helped you, you can donate, register to help, write an article, or buy inexpensive hosting to start your site. If you scroll down, you can sign up for the newsletter, vote for the article, and comment.

Seeing as you’re here, why not visit our friends at Blunix?

“Blunix GmbH offers Debian and Ubuntu focussed Linux Emergency Support 24/7, Project-based Linux Consulting, FOSS-based Managed Hosting on your company IaaS provider account as well as custom Training and Workshops. We prioritize fair business partnerships and employee quality of life over profit. We focus on reasonable security by default and encourage your employees and developers to keep things secure enough to avoid 99% of security issues with automated attack vectors. We focus exclusively on this and do partner with other companies to refer customers that do not fit exactly what we do, or where we require additional help (for example we have a security-focused company that we partner with for higher risk factor hosting setups).”

Click to visit Blunix GmbH. 

View Some Logs In The Terminal

Today we’ll have a relatively simple set of commands that will show you how to view some system logs in the terminal. We’re only going to cover a few of them that are similar in operation. There are far more things that get logged.

You generate logs as you use your computer. These will vary and there are quite a few logs kept. Logs aren’t just kept by the system. Some third-party applications create logs. We’ll only be covering a few system logs. I just want to keep things simple.

The tool we’re using for this exercise is the cat command. The cat application is one tool that lets you read text files in the terminal. It’s a frequently used tool at my house. Hopefully, you too will get comfortable using this command at your house.

It seems like that should be enough of an intro. If you have any questions, you can always leave a comment. Those get seen and addressed more quickly than when you try to contact me elsewhere. (Comments almost instantly send a notification to whatever computer I’m using. They’ll even ping my phone if I turn the notification sound on.)

Anyhow… Ask away, should you have any questions…

View Some System Logs In The Terminal:

We’ll be viewing a few different types of logs. As mentioned above and in the heading, this is something we do in the terminal. You can usually open a terminal by pressing CTRL + ALT + T. If that doesn’t work, you’ll almost certainly find a terminal in your application menu.

First, we’ll view the kernel log.

The kernel is the actual “Linux” in your Linux. The kernel deals with task scheduling, and running processes as needed. It’s an abstraction between you and the hardware.

If you want to view the current kernel log, use this command:

If you want to view the previous session’s kernel logs, use this command:

Both of those commands will flood your terminal. That’s okay, you can use the pipe operator and the less command like so:

Next, we’ll view the boot log.

When you boot your computer, the computer keeps track of what happens during that process. This is known as the boot log. It can be exceptionally handy when you have a problem booting. 

If you want to check the current boot log, use this command:

If you want to check the previous boot log, check it with this command:

Again, you can use a pipe and the less command to manage the flow of data.

Next, we’ll view the system log.

The system logs all sorts of additional information. If you’re not finding the information you want in the previous two logs, checking the system log is prudent. The system logs all sorts of things that are useful for resolving problems.

If you want to check the current system log, use this command:

If you want to check the previous system log, run this command:

Don’t forget that you can use a pipe and less in this command. This will give you a slower output you can manage with your arrow keys.

And that will show you your system log.

That’s all it takes to view the major system logs. There are other logs and we’ll have to cover them at some point.

Closure:

Well, this didn’t take too many words or too much time… That’s nice. If you want to start troubleshooting your own problems, learn to read the system logs. If you don’t want to ask for help, learn to read the system logs. Well, that and learn to read the man pages, but that last bit is outside the scope of this article.

I do hope you enjoyed this article. They’re fun to write but it does sometimes feel like work. I never wanted this to feel like it was work, but here we are. It probably would have been wiser to not set such a schedule and to allow some vacation time in there. This whole project started when I had far more time due to the pandemic.

We do have a special day coming up… You’ll see!

Thanks for reading! If you want to help, or if the site has helped you, you can donate, register to help, write an article, or buy inexpensive hosting to start your site. If you scroll down, you can sign up for the newsletter, vote for the article, and comment.

Subscribe To Our Newsletter
Get notified when new articles are published! It's free and I won't send you any spam.
.
Linux Tips
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.