Category: Software

Update Python Packages (PIP)

We’ve had a run of Python packages recently and you can tell that I’m a fan because today we will discuss how to update Python packages that were installed via PIP. This should be a pretty easy article to follow along with.

Before diving into the world of installing Python packages from a centralized repository (via PIP), you should probably be familiar with the entire process. So, read these two articles before proceeding:

Install Python’s PIP Part One

And then follow up with this article:

Install Python’s PIP Part Two

It’s important to upgrade the packages you’ve installed with PIP. All software requires updates. Bugs are fixed with newer software but, more importantly, security issues are addressed with updates. This doesn’t just apply to Python. It applies to your whole computer. Software gets updated and you need to apply those updates.

So, today we’re going to do some maintenance and we’re going to update Python packages. Rather than waste time with a long intro, let’s get started!

Update Python Packages:

Just so you know, Python packages are installed in the terminal. So, it stands to reason that updates are also done in the terminal. To follow along in this article, you will need an open terminal. So, if you want to update Python packages you should start by opening a terminal. You can usually just CTRL + ALT + T to open your default terminal emulator.

With your terminal open, let’s first ensure PIP is installed with this command:

Next, make sure PIP is updated to the newest version:

With PIP upgraded to the most current version, you can check to see which Python packages you have previously installed. That’s done like this:

Now, you can see which packages can be updated to newer packages:

That will give you an output similar to this:

Now, you can update the packages, like so:

You can also do multiple packages at the same time:

By doing this, you can update your Python packages, at least those installed via PIP. That is indeed pretty easy.

However, I have a command that I certainly didn’t come up with. This is a command I found in my notes and I do not see a reference URL – or I’d cite the source. Doing some searching, I saw that this command is referenced at multiple sites. So, finding the source is problematic for me.

If you want to upgrade all the Python packages at once, try this command:

I tested this and it appears to work well enough. PIP does love to throw errors in the terminal but generally works okay. That command should update all the packages you’ve installed with PIP – including any Python dependencies that were installed at the same time.

See? It’s pretty easy to update Python packages…

Closure:

Well, now you know how to update Python packages. I figured that this was an important article to write. If you’re going to use PIP to install Python packages, you might as well know how to keep yourself secure and how to keep yourself updated. That seemed reasonable.

However, my Python skills aren’t that great. I can do a Hello World program and that’s about it. I haven’t even done that in a while. So, don’t go asking me detailed Python questions! I probably won’t have an answer. My use is pretty limited to things I can trivially install with PIP.

Also, you may not want to ask me questions. While I’ll be polite, my time is constrained these days. I’m just as likely to refer you to a forum or two. You can ask questions. If they’re good, I’ll maybe answer them in an article. I’m just pointing out that you shouldn’t expect too much from me.

“If you don’t expect too much from me, you might not be let down.”

Thanks for reading! If you want to help, or if the site has helped you, you can donate, register to help, write an article, or buy inexpensive hosting to start your site. If you scroll down, you can sign up for the newsletter, vote for the article, and comment.

Demystifying journalctl: A Comprehensive Guide to Linux System Logging

It was suggested that I write an article about journalctl, which seemed like a large topic. I decided that I’d let AI have a shot at it, so this article was written by ChatGPT.

It took a few prompts to get what I wanted – which turned out to be the first result. I gave the AI the chance to rework the article but the result was that I much preferred the initial offering. After all, I was only after a very light overview of the journalctl command. 

There’s a lot to the journalctl command. The journalctl command is far too much to cover in a single article. Heck, I don’t even know some aspects of the command. You can see this by checking the man page with the following command:

See? There’s a lot to the command. At the end of the day, AI did a good job of summing up what you really need from the command as an average user. So, I’m going to go ahead and publish that content. It did a better job than I’d have done!

Introduction To journalctl:

In the realm of Linux system administration, understanding and managing system logs is indispensable. Logs provide crucial insights into the health, performance, and security of a system. Among the plethora of tools available for log management, journalctl stands out as a powerful and versatile command for accessing and analyzing logs in systems utilizing systemd. In this comprehensive guide, we will delve into the intricacies of journalctl, exploring its features, functionalities, and practical applications.

Understanding Systemd Journal:

Systemd, the init system adopted by many modern Linux distributions, introduced the systemd journal as a replacement for traditional syslog. The journal, stored in binary format, offers numerous advantages over syslog, including structured logging, faster search capabilities, and enhanced metadata.

journalctl serves as the primary interface for querying and interacting with the systemd journal. It provides administrators with a rich set of options for filtering, displaying, and analyzing log entries, empowering them to effectively troubleshoot issues, monitor system activity, and extract valuable insights.

Basic Usage:

At its core, journalctl allows users to retrieve and view log entries from the systemd journal. The simplest invocation of journalctl displays the entire journal, starting with the most recent entries:

This command presents a paginated output of log entries, including timestamps, log levels, and message contents. By default, journalctl displays logs from the current boot session. However, it also supports options for querying logs from previous boots or specific time ranges.

Filtering and Querying:

One of the key strengths of journalctl lies in its ability to filter log entries based on various criteria. Administrators can narrow down the search results by specifying filters such as time range, log level, systemd unit, or specific fields within log messages.

For example, to display all log entries generated by the sshd service, the following command can be used:

Similarly, to retrieve logs pertaining to a particular time range, the --since and --until options can be utilized:

Additionally, journalctl supports advanced filtering using Boolean expressions, enabling complex queries to be constructed for precise log retrieval.

Output Formatting and Presentation:

journalctl offers flexible options for customizing the format and presentation of log entries. Administrators can choose from various output formats, including short, verbose, and JSON formats, depending on their preferences and requirements.

For instance, to display log entries in a more detailed and verbose format, the -o verbose option can be used:

Moreover, journalctl provides options for controlling the display of timestamps, including the ability to convert timestamps to the local time zone or display them in a human-readable format.

Real-time Monitoring and Follow Mode:

In addition to viewing historical log entries, journalctl can also be used for real-time monitoring of system logs. By invoking journalctl with the -f or --follow option, administrators can tail the journal and receive live updates as new log entries are added:

This feature is particularly useful for monitoring system activity in real time, diagnosing issues as they occur, and tracking the progress of system services during startup and shutdown sequences.

Integration with Other Tools:

journalctl seamlessly integrates with other Linux system administration tools, enabling administrators to combine its capabilities with those of other utilities for more comprehensive log analysis and management.

For example, grep can be used in conjunction with journalctl to perform pattern matching and further refine log queries:

Furthermore, administrators can leverage shell scripting and automation to automate log analysis tasks, extract meaningful insights, and trigger alerts based on specific log patterns or conditions.

Conclusion:

In conclusion, journalctl emerges as a powerful and indispensable tool for managing system logs in Linux environments utilizing systemd. Its rich feature set, flexible filtering options, and real-time monitoring capabilities make it an invaluable asset for system administrators, enabling them to effectively diagnose issues, monitor system activity, and ensure the stability and security of their systems.

By mastering the intricacies of journalctl, administrators can gain deeper insights into system behavior, streamline troubleshooting workflows, and maintain the optimal performance of their Linux infrastructure. Whether it’s debugging a system issue, auditing security events, or analyzing performance metrics, journalctl empowers administrators to navigate the complexities of system logging with confidence and efficiency.

Closure:

Well, I keep finding uses for AI. This is a better article than I’d have written. I think I’ll next use AI for a solid article about grep. That sounds like a reasonable subject and it’s an article that I can reference in many other articles. In fact, I should have done an article about grep already!

So, this is an article about journalctl. It’s an overwhelming command. It’s amazingly complicated and powerful, but you (as a regular user, as most of my readers are) will only need to know the basics. This is indeed the basics and they appear to be well-described.

Thanks for reading! If you want to help, or if the site has helped you, you can donate, register to help, write an article, or buy inexpensive hosting to start your site. If you scroll down, you can sign up for the newsletter, vote for the article, and comment.

Extract Text From Multiple File Types

Today we will have a fairly simple exercise as we’re going to just use a Python application to extract text from multiple file types. This is a pretty standard operation but will require some preparation.

Fortunately, I’m ahead of the game! You’re good to go if you follow along on the site and have already enabled PIP. Otherwise…

You will need to install PIP for this article. This is not complicated.

First, read this article:

Install Python’s PIP Part One

Technically, you could just do that. However, you should add the path so that you don’t have to specify the location of your Python applications and can easily use them from the terminal.

So, read this article:

Install Python’s PIP Part Two

Now that you’ve done those two things, you’re good to proceed. See? It was worth the time to write those articles! They’re useful and save a lot of time.

The tool we’re going to use is known as “Textract“. Don’t quote me on this, but I believe this could also apply to Windows users, though installing the dependencies for this would be a different process. I’m not a Windows user. If you are, feel free to comment and let us know how things work on your side of life.

Textract: 

While there is no built-in man page, the Textract application is described like this:

While several packages exist for extracting content from each of these formats on their own, this package provides a single interface for extracting content from any type of file, without any irrelevant markup.

It is a pretty handy application and claims to extract the text from more file types than I could reasonably expect to test. Here’s a list of files that you should be able to extract text from.

.csv via python builtins
.doc via antiword
.docx via python-docx2txt
.eml via python builtins
.epub via ebooklib
.gif via tesseract-ocr
.jpg and .jpeg via tesseract-ocr
.json via python builtins
.html and .htm via beautifulsoup4
.mp3 via sox, SpeechRecognition, and pocketsphinx
.msg via msg-extractor
.odt via python builtins
.ogg via sox, SpeechRecognition, and pocketsphinx
.pdf via pdftotext (default) or pdfminer.six
.png via tesseract-ocr
.pptx via python-pptx
.ps via ps2text
.rtf via unrtf
.tiff and .tif via tesseract-ocr
.txt via python builtins
.wav via SpeechRecognition and pocketsphinx
.xlsx via xlrd
.xls via xlrd

You may need to install specific packages for some of these file formats. Those packages can usually be found in your default repositories. It otherwise comes with quite a lot of functionality out of the box.

I did test some of those formats and it seemed to work okay. Your mileage may vary, of course. However, Textract was able to extract text from multiple file types.

Extract Text From Multiple File Types:

If you want to extract text from multiple file types with Textract (a fantastic name for an application) then you’ll first need to install it. I’ve yet to find a working GUI PIP installation tool, so that means you’re going to need an open terminal.

More often than not, you can open your terminal by simply pressing CTRL + ALT + T on your keyboard. If your distro doesn’t adhere to the norms, you can find a terminal in your application menu. If you don’t use an application menu, you already know how to open a terminal and you don’t need any help from me.

First, let’s install Textract:

Note the lack of sudo. You’re installing this for your user account and do not need elevated permissions for this. Python packages go right into your ~/ directory. See below, as you’ll want to install some dependencies for full functionality.

You may see an error or two during installation but that doesn’t seem to matter. It will take a minute to install and watching the installation chug along is good fun.

Using Textract:

With Textract installed, you can now extract text from a whole variety of file types. The syntax is as follows:

That sends the output to the standard output (your terminal). I suspect that most folks are going to want to save the output to a file. For that, you just need to add the -o flag and a file name. So, something like this:

That’s going to extract the text from some file types but not all of them.

Now, this is from a Lubuntu installation…

This isn’t going to work with all the listed file types at this time. You need some dependencies to be installed. For me, and it’s a long one, the command was:

That’s slightly different from the command they include on their page, but it appears to do the trick. You’ll have some of those installed by default but running the command will sort itself out. You’ll have to modify the command to suit your distro, but that should work with Debian, Ubuntu, Linux Mint, and other Debian-based distros.

With that installed, I can even grab the text from image files.

Here’s an example:

a simple picture with simple text
This is some simple text to test how well Textract really works.

Here’s the command:

Here’s the output:

I dare say that’s pretty good. I tried other pictures and it was good enough to get the gist of things. Complicated image files with many columns appear to be a bit of a stumbling block. But it’s not terrible.

It has no trouble at all with other file formats.

It can be a bit fussy to get Textract properly installed but it seems to do the trick once installed. If you want to extract text from multiple file types, Textract is a pretty good piece of software.

Closure:

If you want to extract text from multiple file types, this is definitely a good tool for the job. It certainly handles a lot of files and does a good job with them. It’s not perfect. None of these tools are. Complicated image files threw it off a bit, but Textract lives up to its name.

There was a reason I wrote those articles about PIP. Being able to install Python packages via a repository is a great thing. There’s some great Python software out there and we’ve barely touched the surface. Linux is great like that, that is offering great Python support.

Do you have a use for this in your daily activities? If so, leave a comment letting us know how you use Textract and what makes you pick it over other applications. You can even use a real email address. I never send spam. I never sell your information.

Thanks for reading! If you want to help, or if the site has helped you, you can donate, register to help, write an article, or buy inexpensive hosting to start your site. If you scroll down, you can sign up for the newsletter, vote for the article, and comment.

Another Way To Show Mounted Filesystems

We’ve covered this topic before, but I want to show you another way that you can show mounted filesystems. As previously covered, you have multiple filesystems mounted at any one time. So, let’s examine them, shall we?

If you use Linux, you use multiple filesystems. There are real and virtual filesystems created and used by the operating system. A real filesystem would be something like a partition on your storage device (one filesystem per partition, of course). A virtual filesystem would be things like your temporary files or cached files, among other filesystems used by the system.

We’ve covered this before and this is just another way to show your mounted filesystems. This is good information to have, as it’s handy to know the path for those things so that you don’t do something silly like misfire a dd command and hosing your OS partition. (Ask me how easy that is.)

Show Your Filesystem In The Linux Terminal
Another Way To View Attached Storage Devices
Another Way To View Attached Storage Devices

(And there are more! This is an oft-covered subject on Linux-Tips!)

So, what will we be using?

findmnt:

You shouldn’t need to install anything for this article. The findmnt tool should be installed by default. You can verify that findmnt is installed by running the following command:

The output of that command should probably match this one:

You check the man page with this command:

There’s additional information available in this application. Run this command:

But, if you check the man page, you’ll see that this is indeed one of the correct tools for the job. This being a basic task, there are many ways to show mounted filesystems. It’s up to you to pick a favorite – or find the most useful of commands for your particular situation.

Anyhow, the man page describes it as:

findmnt – find a filesystem

That’s what we want to do. That means that this is the correct tool for the job.

Show Mounted Filesystems:

You’ve been here long enough to know that we’re likely to use the terminal. If your hunch was that we would be using the terminal, your hunch was correct. You’ll find that findmnt is a terminal-based application so open your default terminal emulator by finding it in your application menu or pressing CTRL + ALT + T on your keyboard.

With your terminal open, you can show mounted filesystems with this command:

You’ll see that it even has a nice tree output. You’ll find things like Snaps have their own filesystem if you’re using a system that uses Snaps. Other jailed applications may also use their own filesystem as a way to keep things separate from the system as a whole. 

Here’s an example truncated output from Lubuntu:

That’s not nearly as complicated as it looks, once you get used to it.

Now, you can trim all that down. You can opt to show only the ‘real’ filesystems. That’s done with the --real flag.

This is an example of that command on that same Lubuntu system used above (Snap applications are seen as ‘real’ filesystems as a part of their separation from the system):

Here’s an example output from a Linux Mint system that does not use Snaps:

You can do more with the findmnt command, such as select the columns you want in your output, so be sure to check the man page. This is just a quick overview because the vast majority of you will never need more than just the basic command.

As I said, it’s a useful command for when you want to verify the path to a filesystem. Beyond that, the usage is up to you. That’s what I use it for when I don’t bother with any one of several other commands.

Closure:

You might see no reason to add this to your list of commands, but it is useful when you want to see all the mounted filesystems on your system. If you need the path for a filesystem, this is a good command. It’s also useful for finding out some attributes of your mounted filesystems.

This is one of those commands you really could keep in your back pocket. You might first lean on a command like lsblk but that only lists block devices – that is, storage devices, and doesn’t include the many other mounted filesystems. If you don’t need this command today, that’s fine. You might need this command in the future, and now you know it exists.

Thanks for reading! If you want to help, or if the site has helped you, you can donate, register to help, write an article, or buy inexpensive hosting to start your site. If you scroll down, you can sign up for the newsletter, vote for the article, and comment.

PDF Exif Data

Yes, this is another PDF article, but this time it involves a bit of privacy or security because there’s such a thing as PDF Exif data. This shouldn’t take all that long and it shouldn’t be all that complicated.

Files have Exif data. Well, many files do. You’ll find Exif data in your image files, document files, and even PDF files.

Exif data is metadata. It can contain information about when the file was created, how it was created, and when it was modified, among other things. This is useful data but you may still want to know it exists. Additionally, you may want to change it.

I’ve covered Exif data in the past. Exif stands for Exchangeable image file format and is used in many different ways.

Here are some other Exif articles:

How To: Sanitize Exif Data From Your Digital Images For Privacy Sake
Short: Show File Creation, Modification, and Access Times
Review: MetaClean (Clean Exif/meta Data From Email Attachments Automatically)

So, this isn’t a foreign concept. There are other Exif data articles, but this one is specifically for PDF files. We will need some software for this and that software is ‘ExifTool’…

ExifTool:

The tool we’ll be using in this article is known as ‘ExifTool’ and it’s a widely available application. The ExifTool application is used to view the metadata from a wide variety of file types.

If you check the man page, it should describe ExifTool as such:

exiftool – Read and write meta information in files

You should check the man page once you have ExifTool installed. You do that with the following command:

The list of things ExifTool works with is why you should check the man page. It’s rather extensive and it may surprise you how much metadata is kept in a file. It’s a lot and this is the correct tool for the job.

Let’s see if I can fit the man page into this article to show you how many file types contain meta information that you may not be aware of…

These are the types of metadata that ExifTool can interact with:

That’s not as well-formatted as I’d like, but it will have to do. It’s a lot of information and I’m not going to invest more effort than I have already invested in this. It’s just a code block, which should work well enough.

So, ExifTool is quite extensive and we only need it for PDF Exif Data!

Seriously, this is quite a tool. It’s something you may want to be familiar with.

PDF Exif Data:

This ExifTool is a terminal-based application. That means you’re going to need an open terminal and a PDF file to work with. You can start by pressing CTRL + ALT + T to open your terminal.

You’ll need to install ExifTool. The command will vary based on your distribution’s package manager. It should be available for the major distros and I’ll cover some of those installation instructions.

Debian/Ubuntu/etc:

RHEL/CentOS/etc:

Arch/Manjaro/etc:

You can also use your GUI package installation tools to install ExifTool. If the above commands don’t work for you, be sure to leave a comment.

You can download a test PDF: stacer.pdf (which may indicate a future article.)

If you’ve downloaded that file, you need to navigate to the correct directory. Unless you changed something, you likely default to the ~/Downloads directory. You can navigate to that directory with cd ~/Downloads and then use the ls command to ensure the stacer.pdf file exists.

So, the syntax to get started is just this:

In our example, it’d look like this:

You’ll get an output similar to this:

There’s quite a bit of information available and that file doesn’t even have all the information that can be found in your PDF’s metadata.

This PDF isn’t a good one, but you can at least change the title. You can blank the title with this command:

You can blank the title with this command:

Other PDF documents will contain other fields. You can wipe all the fields, resetting the creation and modification times, with this command:

If you’re interested in changing that for privacy reasons, that’s a nice and simple command that will reset all of that metadata. It’s a pretty handy command and ExifTool is a pretty handy tool.

Closure:

So, there’s some ExifTool information and some information about PDF Exif data. That seems like a reasonable article. A new Linux user may be interested in PDF Exif data and this should make it simple enough for anyone to follow along with. You should read the man page because this is an expansive application.

Thanks for reading! If you want to help, or if the site has helped you, you can donate, register to help, write an article, or buy inexpensive hosting to start your site. If you scroll down, you can sign up for the newsletter, vote for the article, and comment.

Linux Tips
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.