Technology Tales

Keeping a graphical eye on CPU temperature and power consumption on the Linux command line

Following my main workstation upgrade in January, some extra monitoring has been needed. This follows on from the experience with building its predecessor more than three years ago.

Being able to do this in a terminal session keeps things lightweight, and I have done that with text displays like what you see below using a combination of sensors and nvidia-smi in the following command:

watch -n 2 "sensors | grep -i 'k10'; sensors | grep -i 'tdie'; sensors | grep -i 'tctl'; echo "" | tee /dev/fd/2; nvidia-smi"

Everything is done within a watch command that refreshes the display every two seconds. Then, the panels are built up by a succession of commands separated with semicolons, one for each portion of the display. The grep command is used to pick out the desired output of the sensors command that is piped to it; doing that twice gets us two lines. The next command, echo "" | tee /dev/fd/2, adds an extra line by sending a space to STDERR output before the output of nvidia-smi is displayed. The result can be seen in the screenshot below.

However, I also came across a more graphical way to do things using commands like turbostat or sensors along with AWK programming and ttyplot. Using the temperature output from the above and converting that needs the following:

while true; do sensors | grep -i 'tctl' | awk '{ printf("%.2f\n", $2); fflush(); }'; sleep 2; done | ttyplot -s 100 -t "CPU Temperature (Tctl)" -u "°C"

This is done in an infinite while loop to keep things refreshing; the watch command does not work for piping output from the sensors command to both the awk and ttyplot commands in sequence and on a repeating, periodic basis. The awk command takes the second field from the input text, formats it to two places of decimals and prints it before flushing the output buffer afterwards. The ttyplot command then plots those numbers on the plot seen below in the screenshot with a y-axis scaled to a maximum of 100 (-s), units of °C (-u) and a title of CPU Temperature (Tctl) (-t).

A similar thing can be done for the CPU wattage, which is how I learned of the graphical display possibilities in the first place. The command follows:

sudo turbostat --Summary --quiet --show PkgWatt --interval 1 | sudo awk '{ printf("%.2f\n", $1); fflush(); }' | sudo ttyplot -s 200 -t "Turbostat - CPU Power (watts)" -u "watts"

Handily, the turbostat can be made to update every so often (every second in the command above), avoiding the need for any infinite while loop. Since only a summary is needed for the wattage, all other output can be suppressed, though everything needs to work using superuser privileges, unlike the sensors command earlier. Then, awk is used like before to process the wattage for plotting; the first field is what is being picked out here. After that, ttyplot displays the plot seen in the screenshot below with appropriate title, units and scaling. All works with output from one command acting as input to another using pipes.

All of this offers a lightweight way to keep an eye on system load, with the top command showing the impact of different processes if required. While there are graphical tools for some things, command line possibilities cannot be overlooked either.

Upgrading a web server from Debian 11 to Debian 12

While Debian 12 may be with us since the middle of 2023 and Debian 13 is due in the middle of next year, it has taken me until now to upgrade one of my web servers. The tardiness may have something to do with a mishap on another system that resulted in a rebuild, something to avoid it at all possible.

Nevertheless, I went and had a go with the aforementioned web server after doing some advance research. Thus, I can relate the process that you find here in the knowledge that it worked for me. Also, I will have it on file for everyone's future reference. The first step is to ensure that the system is up-to-date by executing the following commands:

sudo apt update
sudo apt upgrade
sudo apt dist-upgrade

Next, it is best to remove extraneous packages using these commands:

sudo apt --purge autoremove
sudo apt autoclean

Once you have backed up important data and configuration files, you can move to the first step of the upgrade process. This involves changing the repository locations from what is there for bullseye (Debian 11) to those for bookworm (Debian 12). Issuing the following commands will accomplish this:

sudo sed -i 's/bullseye/bookworm/g' /etc/apt/sources.list
sudo sed -i 's/bullseye/bookworm/g' /etc/apt/sources.list.d/*

In my case, I found the second of these to be extraneous since everything was included in the single file. Also, Debian 12 has added a new non-free repository called non-free-firmware. This can be added at this stage by manual editing of the above. In my case, I did it later because the warning message only began to appear at that stage.

Once the repository locations, it is time to update the package information using the following command:

sudo apt update

Then, it is time to first perform a minimal upgrade using the following command, that takes a conservative approach by updating existing packages without installing any new ones:

sudo apt upgrade --without-new-pkgs

Once that has completed, one needs to issue the following command to install new packages if needed for dependencies and even remove incompatible or unnecessary ones, as well as performing kernel upgrades:

sudo apt full-upgrade

Given all the changes, the completion of the foregoing commands' execution necessitates a system restart, which can be the most nerve-wracking part of the process when you are dealing with a remote server accessed using SSH. While, there are a few options for accomplishing this, here is one that is compatible with the upgrade cycle:

sudo systemctl reboot

Once you can log back into the system again, there is one more piece of housekeeping needed. This step not only removes redundant packages that were automatically installed, but also does the same for their configuration files, an act that really cleans up things. The command to execute is as follows:

sudo apt --purge autoremove

For added reassurance that the upgrade has completed, issuing the following command will show details like the operating system's distributor ID, description, release version and codename:

lsb_release -a

If you run the above commands as root, the sudo prefix is not needed, yet it is perhaps safer to execute them under a less privileged account anyway. The process needs the paying of attention to any prompts and questions about configuration files and service restarts if they arise. Nothing like that came up in my case, possibly because this web server serves flat files created using Hugo, avoiding the use of scripting and databases, which would add to the system complexity. Such a simple situation makes the use of scripting more of a possibility. The exercise was speedy enough for me too, though patience is of the essence should a 30–60 minute completion time be your lot, depending on your system and internet speed.

Manually updating Let's Encrypt certificates

Normally, Let's Encrypt certificates get renewed automatically. Thus, it came as a surprise to me to receive an email telling me that one of my websites had a certificate that was about to expire. The next step was to renew the certificate manually.

That sent me onto the command line in an SSH session to the Ubuntu server in question. Once there, I used the following command to check on my certificates to confirm that the email alert was correct:

sudo certbot certificates

Then, I issued this command to do a test run of the update:

sudo certbot renew --dry-run

In the knowledge that nothing of concern came up in the dry run, then it was time to do the update for real using this command:

sudo certbot renew

Rerunning sudo certbot certificates checked that all was in order. All that did what should have happened automatically; adding a cron job should address that, though, and adding the --quiet switch should cut down on any system emails too.

Saving yourself a reboot: remounting any overlooked volumes in Linux

Recently, I got things a little out of order when starting up my main Linux system after an absence. Usually, I start up my NAS first so that the volumes get mounted when I start my Linux machine. However, it happened that I near enough started them together. Thus, my workstation completed it startup without having the NAS volumes mounted. A reboot would have sorted this, but there was another way: issuing the command that you see below:

sudo mount -a

This looked in my /etc/fstab file and mounted anything that was missing as long as the noauto option was not set. Because this was executed after the NAS had completed its own boot process, it volumes were not mounted on my system and fully available for what I needed to do next. If I had wanted to see what had been mounted, then I needed to issue the following command instead:

sudo mount -av

In addition to the a switch that triggers the mounting of missing volumes, there is now a v (for verbose) one for telling you what has happened. Needless to say, all this happens only if your /etc/fstab file is set up properly. If you are adding a new volume, and I was not, it does no harm to mount it manually before updating the configuration file. That should catch any errors first.

Getting rid of the "Get more security upgrades through Ubuntu Pro with 'esm-apps' enabled" message when performing a system update

Not so long ago, I got the above message while running sudo apt upgrade on an Ubuntu Server system. This was not the first time that this kind of thing happened to me, so I started searching the web for a solution. You do get to see complaints about advertising, but these are never useful.

Accordingly, here are some possible ways of remediating the situation:

• Execute the following commands to disable the responsible services, renaming the configuration file to prevent it from being used (deleting or editing the configuration file to remove the unwanted content are other options):

  sudo systemctl mask apt-news.service

  sudo systemctl mask esm-cache.service

  sudo mv /etc/apt/apt.conf.d/20apt-esm-hook.conf
/etc/apt/apt.conf.d/20apt-esm-hook.conf.disabled

• Alternatively, simply remove the ubuntu-advantage-tools package, which contains the /etc/apt/apt.conf.d/20apt-esm-hook.conf file.
• Another option is to remove the ubuntu-pro-client package.
• Lastly, there also is the possibility of enabling ESM, though that was not desirable for me.

In my case, it may have been the penultimate option on the list that I chose. In any case, I was rid of the unwanted message.

Changing the Ansible Vault editor from Vi to Nano

Recently, I got to experiment with Ansible after reading about the orchestration tool in a copy of Admin magazine. It came in handy for updating a few web servers that I have, as well as updating my main Linux workstation. For the former, automated entry of SSH passwords sufficed, but the same did not apply for sudo usage on my local machine. This meant that I needed to use Ansible Vault to store the administrator password, and doing so opened up a file in the Vi editor. Since I am not familiar with Vi and wanted to get things sorted quickly, I fancied using something more user-friendly like Nano.

Doing this meant adding the following line to .bashrc:

export EDITOR=nano

Saving and closing the file followed by reloading the session set me up for what was needed.

When a hard drive is unrecognised by the Linux hddtemp command

One should not do a new PC build in the middle of a heatwave if you do not want to be concerned about how fast fans are spinning and how hot things are getting. Yet, that is what I did last month after delaying the act for numerous months.

My efforts mean that I have a system built around an AMD Ryzen 9 5950X CPU and a Gigabyte X570 Aorus Pro with 64 GB of memory, and things are settling down after the initial upheaval. That also meant some adjustments to the CPU fan profile in the BIOS for quieter running while the use of Be Quiet! Dark Rock 4 cooler also helps, as does a Be Quiet! Silent Wings 3 case fan. All are components from trusted brands, though I wonder how much abuse they got during their installation and subsequent running in.

Fan noise is a non-quantitative indicator of heat levels as much as touch, so more quantitative means are in order. Aside from using a thermocouple device, there are in-built sensors too. My using Linux Mint means that I have the sensors command from the lm-sensors package for checking on CPU and other temperatures, though hddtemp is what you need for checking on the same for hard drives. The latter can be used as follows:

sudo hddtemp /dev/sda /dev/sdb

This has to happen using administrator access and a list of drives needs to be provided because it cannot find them by itself. In my case, I have no mechanical hard drives installed in non-NAS systems and I even got to replace a 6 TB Western Digital Green disk with an 8 TB SSD, but I got the following when I tried checking on things with hddtemp:

WARNING: Drive /dev/sda doesn't seem to have a temperature sensor.
WARNING: This doesn't mean it hasn't got one.
WARNING: If you are sure it has one, please contact me (hddtemp@guzu.net).
WARNING: See --help, --debug and --drivebase options.
/dev/sda: Samsung SSD 870 QVO 8TB: no sensor

The cause of the message for me was that there is no entry for Samsung SSD 870 QVO 8TB in /etc/hddtemp.db so that needed to be added there. Before that could be rectified, I had to get some additional information using smartmontools and these had to be installed using the following command:

sudo apt-get install smartmontools

What I had to do was check the drive's SMART data output for extra information, and that was achieved using the following command:

sudo smartctl /dev/sda -a | grep -i Temp

What this does is to look for the temperature information from smartctl output using the grep command, with output from the first being passed to the second through a pipe. This yielded the following:

190 Airflow_Temperature_Cel 0x0032 072 050 000 Old_age Always - 28

The first number in the above (190) is the thermal sensor's attribute identifier, and that was needed in what got added to /etc/hddtemp.db. The following command added the necessary data to the aforementioned file:

echo \"Samsung SSD 870 QVO 8TB\" 190 C \"Samsung SSD 870 QVO 8TB\" | sudo tee -a /etc/hddtemp.db

Here, the output of the echo command was passed to the tee command for adding to the end of the file. In the echo command output, the first part is the name of the drive, the second is the heat sensor identifier, the third is the temperature scale (C for Celsius or F for Fahrenheit) and the last part is the label (it can be anything that you like, but I kept it the same as the name). On re-running the hddtemp command, I got output like the following, so all was as I needed it to be.

/dev/sda: Samsung SSD 870 QVO 8TB: 28°C

Since then, temperatures may have cooled and the weather become more like what we usually get, yet I am still keeping an eye on things, especially when the system is put under load using Perl, R, Python or SAS. There may be further modifications such as changing the case or even adding water cooling, not least to have a cooler power supply unit, but nothing is being rushed as I monitor things to my satisfaction.

Shared folders not automounting on an Ubuntu 18.04 guest in a VirtualBox virtual machine

Over the weekend, I finally got to resolve a problem that has affected Ubuntu 18.04 virtual machine for quite a while. The usual checks on Guest Additions installation and vboxsf group access assignment were performed but were not causing the issue. Also, no other VM (Windows (7 & 10) and Linux Mint Debian Edition) on the same Linux Mint 19.2 machine was experiencing the same issue. The latter observation made the problem intrinsic to the Ubuntu VM itself.

Because I install the Guest Additions software from the included virtual CD, I executed the following command to open the relevant file for editing:

sudo systemctl edit --full vboxadd-service

If I had installed virtualbox-guest-dkms and virtualbox-guest-utils from the Ubuntu repositories instead, then this would have been the command that I needed to execute instead of the above.

sudo systemctl edit --full virtualbox-guest-utils

Whichever configuration gets opened, the line that needs attention is the one beginning with "Conflicts" (line 6 in the file on my system). The required edit removes systemd-timesync.service from the list following the equals sign. It is worth checking that file paths include the correct version number for the Guest Additions software that is installed, in case this is not how things are. The only change that was needed on my Ubuntu VM was to the Conflicts line, and rebooting it got the Shared Folder automatically mounted under the /media directory as expected.

Running cron jobs using the www-data system account

When you set up your own web server or use a private server (virtual or physical), you will find that web servers run using the www-data account. That means that website files need to be accessible to that system account if not owned by it. The latter is mandatory if you want WordPress to be able to update itself with needing FTP details.

It also means that you probably need scheduled jobs to be executed using the privileges possessed by the www-data account. For instance, I use WP-CLI to automate spam removal and updates to plugins, themes and WordPress itself. Spam removal can be done without the www-data account, but the updates need file access and cannot be completed without this. Therefore, I got interested in setting up cron jobs to run under that account and the following command helps to address this:

sudo -u www-data crontab -e

For that to work, your own account needs to be listed in /etc/sudoers or be assigned to the sudo group in /etc/group. If it is either of those, then entering your own password will open the cron file for www-data, and it can be edited as for any other account. Closing and saving the session will update cron with the new job details.

In fact, the same approach can be taken for a variety of commands where files only can be accessed using www-data. This includes copying, pasting and deleting files as well as executing WP-CLI commands. The latter issues a striking message if you run a command using the root account, a pervasive temptation given what it allows. Any alternative to the latter has to be better from a security standpoint.

Sorting out sluggish start-up and shutdown times in Linux Mint 19

The Linux Mint team never forces users to upgrade to the latest version of their distribution, but curiosity often provides a strong enough impulse for me to do so. When I encounter rough edges, the wisdom of leaving things unchanged becomes apparent. Nevertheless, the process brings learning opportunities, which I am sharing in this post. It also allows me to collect various useful titbits that might help others.

Again, I went with the in-situ upgrade option, though the addition of the Timeshift backup tool means that it is less frowned upon than once would have been the case. It worked well too, apart from slow start-up and shutdown times, so I set about tracking down the causes on the two machines that I have running Linux Mint. As it happens, the cause was different on each machine.

On one PC, it was networking that holding up things. The cause was my specifying a fixed IP address in /etc/network/interfaces instead of using the Network Settings GUI tool. Resetting the configuration file back to its defaults and using the Cinnamon settings interface took away the delays. It was inspecting /var/log/boot.log that highlighted problem, so that is worth checking if I ever encounter slow start times again.

As I mentioned earlier, the second PC had a very different problem, though it also involved a configuration file. What had happened was that /etc/initramfs-tools/conf.d/resume contained the wrong UUID for my system's swap drive, so I was seeing messages like the following:

W: initramfs-tools configuration sets RESUME=UUID=<specified UUID for swap partition>
W: but no matching swap device is available.
I: The initramfs will attempt to resume from <specified file system location>
I: (UUID=<specified UUID for swap partition>)
I: Set the RESUME variable to override this.

Correcting the file and executing the following command resolved the issue by updating the affected initramfs image for all installed kernels and speeded up PC start-up times:

sudo update-initramfs -u -k all

Though it was not a cause of system sluggishness, I also sorted another message that I kept seeing during kernel updates and removals on both machines. This has been there for a while and causes warning messages about my system locale not being recognised. The problem has been described elsewhere as follows: /usr/share/initramfs-tools/hooks/root_locale is expecting to see individual locale directories in /usr/lib/locale, but locale-gen is configured to generate an archive file by default.  Issuing the following command sorted that:

sudo locale-gen --purge --no-archive

Following these, my new Linux Mint 19 installations have stabilised with more speedy start-up and shutdown times. That allows me to look at what is on Flathub to see what applications and if they get updated to the latest version on an ongoing basis. That may be a topic for another entry on here, but the applications that I have tried work well so far.