Technology Tales

Converting QEMU disk images to VirtualBox images on Linux Mint 21

Recently, VirtualBox gained fuller support for Windows 11 and I successively set up a new Windows 11 virtual machine that, I hope, will supplant a Windows 10 counterpart in time. While the setup itself was streamlined, I ran into such stability issues that I set the new VM aside until a new version of VirtualBox got released. That has happened with the appearance of version 7.0.2, but Windows 11 remains prone to freezing on my Linux Mint machine. Thankfully, that now is much less frequent, yet the need for added stability remains outstanding.

While I was thinking about trying out VirtualBox 7.0.0, I remembered a QEMU machine that I had running Windows 11. Though QEMU proved more limited than VirtualBox when it came to having easy availability of functionality like moving data in and out of the virtual machine or support for sound, there was no problem with TPM support or system stability. Since it did contain some useful data, I wondered about converting its virtual hard disk to VirtualBox format, and it is easy to do. First, you need to install qemu-img and other utilities as follows:

sudo apt-get install qemu-utils

With that in place, executing a command like the following performs the required conversion. Here, the -O switch specifies the required file type of vdi in this case.

qemu-img convert -O vdi [virtual hard disk].qcow2 [virtual hard disk].vdi

While I have yet to mount it on the new VirtualBox Windows 11 virtual machine, it is good to have the old virtual hard disk available for doing so. The thought of using it as a boot drive in VirtualBox did enter my mind, but the required change of drivers and other incompatibilities dissuaded me from doing so.

Changing the UUID of a VirtualBox Virtual Disk Image in Linux

Recent experimentation centring around getting my hands on a test version of Windows 11 had me duplicating virtual machines and virtual disk images, though VirtualBox still is not ready for the next Windows version; it has no TPM capability at the moment. Nevertheless, I was able to get something after a fresh installation that removed whatever files were on the disk image. That meant that I needed to mount the old version to get at those files again.

While renaming partially helped with this, what I really needed to do was change the UUID, so VirtualBox would not report a collision between two disk images with the same UUID. To avoid this, the UUID of one of the disk images had to be changed and a command like the following was used to accomplish this:

VBoxManage internalcommands sethduuid [Virtual Disk Image Name].vdi

Because I was doing this on Linux Mint, I could call VBoxManage without need to tell the system where it was, as would be the case on Windows. Otherwise, it is the sethduuid portion that changes the UUID as required. Another way around this is to clone the VDI file using the following command, but I had not realised that at the time:

VBoxManage clonevdi [old virtual disk image].vdi [new virtual disk image].vdi

It appears that there can be more than one way to do things in VirtualBox at times, so the second way will remain on reference for the future.

Getting rid of the Windows Resizing message from a Manjaro VirtualBox guest

Like Fedora, Manjaro also installs a package for VirtualBox Guest Additions when you install the Linux distro in a VirtualBox virtual machine. However, it does have certain expectations when doing this. On many systems, and my own is one of these, Linux guests are forced to use the VMSVGA virtual graphics controller while Windows guests are allowed to use the VBoxSVGA one. It is the latter that Manjaro expects, so you get a message like the following appearing when the desktop environment has loaded:

Windows Resizing
Set your VirtualBox Graphics Controller to enable windows resizing

After ensuring that gcc, make, perl and kernel headers are installed, I usually install VirtualBox Guest Additions myself from the included ISO image, and so I did the same with Manjaro. Doing that and restarting the virtual machine got me extra functionality like screen resizing and being able to copy and paste between the VM and elsewhere after choosing the Bidirectional setting in the menus under Devices > Shared Clipboard.

That still left an unwanted message popping up on startup. To get rid of that, I just needed to remove /etc/xdg/autostart/mhwd-vmsvga-alert.desktop. While it can be deleted, I just moved it somewhere else and a restart proved that the message was gone as needed. Now everything is working as I wanted.

Contents not displaying for Shared Folders on a Fedora 32 guest instance in VirtualBox

While some Linux distros like Fedora install VirtualBox drivers during installation time, I prefer to install the VirtualBox Guest Additions themselves. Before doing this, it is best to remove the virtualbox-guest-additions package from Fedora to avoid conflicts. After that, execute the following command to ensure that all prerequisites for the VirtualBox Guest Additions are in place before mounting the VirtualBox Guest Additions ISO image and installing from there:

sudo dnf -y install gcc automake make kernel-headers dkms bzip2 libxcrypt-compat kernel-devel perl

During the installation, you may encounter a message like the following:

ValueError: File context for /opt/VBoxGuestAdditions-<VERSION>/other/mount.vboxsf already defined

This is generated by SELinux, so the following commands need to be executed before repeating the installation of VirtualBox Guest Additions:

sudo semanage fcontext -d /opt/VBoxGuestAdditions-<VERSION>/other/mount.vboxsf
sudo restorecon /opt/VBoxGuestAdditions-<VERSION>/other/mount.vboxsf

Without doing the above step and fixing the preceding error message, I had an issue with mounting of Shared Folders whereby the mount point was set up, but no folder contents were displayed. This happened even when my user account was added to the vboxsf group, and it proved to be the SELinux context issue that was the cause.

Shared folders not automounting on an Ubuntu 18.04 guest in a VirtualBox virtual machine

Over the weekend, I finally got to resolve a problem that has affected Ubuntu 18.04 virtual machine for quite a while. The usual checks on Guest Additions installation and vboxsf group access assignment were performed but were not causing the issue. Also, no other VM (Windows (7 & 10) and Linux Mint Debian Edition) on the same Linux Mint 19.2 machine was experiencing the same issue. The latter observation made the problem intrinsic to the Ubuntu VM itself.

Because I install the Guest Additions software from the included virtual CD, I executed the following command to open the relevant file for editing:

sudo systemctl edit --full vboxadd-service

If I had installed virtualbox-guest-dkms and virtualbox-guest-utils from the Ubuntu repositories instead, then this would have been the command that I needed to execute instead of the above.

sudo systemctl edit --full virtualbox-guest-utils

Whichever configuration gets opened, the line that needs attention is the one beginning with "Conflicts" (line 6 in the file on my system). The required edit removes systemd-timesync.service from the list following the equals sign. It is worth checking that file paths include the correct version number for the Guest Additions software that is installed, in case this is not how things are. The only change that was needed on my Ubuntu VM was to the Conflicts line, and rebooting it got the Shared Folder automatically mounted under the /media directory as expected.

Enlarging a VirtualBox VDI virtual disk

It is remarkable how the Windows folder manages to grow on your C drive, and one in a Windows 7 installation was the cause of my needing to expand the VirtualBox virtual machine VDI disk on which it was installed. After trying various ways to cut down the size, an enlargement could not be avoided. In all of this, it was handy that I had a recent backup for restoration after any damage.

The same thing meant that I could resort to enlarging the VDI file with more peace of mind than otherwise might have been the case. This needed use of the command line once the VM was shut down. The form of the command that I used was the following:

VBoxManage modifyhd <filepath/filename>.vdi --resize 102400

It appears that this also would work on a Windows host, but mine was Linux, and it did what I needed. The next step was to attach it to an Ubuntu VM and use GParted to expand the main partition to fill the newly available space. That does not mean that it takes up 100 GiB on my system just yet because these things can be left to grow over time and there is a way to shrink them too if you ever need to do just that. As ever, having a backup made before any such operation may have its uses if anything goes awry.

Compressing a VirtualBox VDI file for a Linux guest

In a previous posting, I talked about compressing a virtual hard disk for a Windows guest system running in VirtualBox on a Linux system. Since then, I have needed to do the same for a Linux guest following some housekeeping. Because the Linux distribution used is Debian, the instructions are relevant to that and maybe its derivatives such as Ubuntu, Linux Mint and their like.

While there are other alternatives like dd, I am going to stick with a utility named zerofree to overwrite the newly freed up disk space with zeroes to aid compression later on in the process for this and the first step is to install it using the following command:

apt-get install zerofree

Once that has been completed, the next step is to unmount the relevant disk partition. Luckily for me, what I needed to compress was an area that I reserved for synchronisation with Dropbox. If it was the root area where the operating system files are kept, a live distro would be needed instead. In any event, the required command takes the following form, with the mount point being whatever it is on your system (/home, for instance):

sudo umount [mount point]

With the disk partition unmounted, zerofree can be run by issuing a command that looks like this:

zerofree -v /dev/sdxN

Above, the -v switch tells zerofree to display its progress and a continually updating percentage count tells you how it is going. The /dev/sdxN piece is generic with the x corresponding to the letter assigned to the disk on which the partition resides (a, b, c or whatever) and the N is the partition number (1, 2, 3 or whatever; before GPT, the maximum was 4). Putting all this together, we get an example like /dev/sdb2.

Once, that had completed, the next step is to shut down the VM and execute a command like the following on the host Linux system ([file location/file name] needs to be replaced with whatever applies on your system):

VBoxManage modifyhd [file location/file name].vdi --compact

With the zero filling in place, there was a lot of space released when I tried this. While it would be nice for dynamic virtual disks to reduce in size automatically, I accept that there may be data integrity risks with those, so the manual process will suffice for now. It has not been needed that often anyway.

Migrating a virtual machine from VirtualBox to VMware Player on Linux

The progress of Windows 10 is something that I have been watching. Early signs have been promising, and the most recent live event certainly contained its share of excitement. The subsequent build that was released was another step in the journey, though the new Start Menu appears more of a work in progress than it did in previous builds. Keeping up with these advances sometimes steps ahead of VirtualBox support for them, and I discovered that again in the last few days. VMware Player seems unaffected, so I thought that I'd try a migration of the VirtualBox VM with Windows 10 onto there. In the past, I did something similar with a 32-bit instance of Windows 7 that subsequently got upgraded all the way up to 8.1, but that may not have been as slick as the latest effort, so I thought that I would share it here.

The first step was to export the virtual machine as an OVF appliance, and I used File > Export Appliance... only to make a foolish choice regarding the version of OVF. The one that I picked was 2.0 only to subsequently discover that 1.0 was the better option. The equivalent command line would look like the following (there are two dashes before the ovf10 option below):

VboxManage export [name of VM] -o [name of file].ova --ovf10

VMware has a tool for extracting virtual machines from OVF files that will generate a set of files that will work with Player and other similar products of theirs. It goes under the unsurprising name of OVF Tool and usefully works from a command line session. When I first tried it with an OVF 2.0 files, I got the following error, and it stopped doing anything as a result:

Line 2: Incorrect namespace http://schemas.dmtf.org/ovf/envelope/2 found.

The only solution was to create a version 1.0 file and use a command like the following:

ovftool --lax [name of file].ova [directory location of VM files]/[name of file].vmx

The --lax option is needed to ensure successful execution, even with an OVF 1.0 file as the input. Once I had done this on my Ubuntu GNOME system, the virtual machine could be opened up on VMware Player and I could use the latest build of Windows 10 at full screen, something that was not possible with VirtualBox. This may be how I survey the various builds of the operating that appear before its final edition is launched later this year.

Dropping back to a full screen terminal session from a desktop one in Linux

There are times when you might need to access a full screen terminal from a Linux graphical desktop. For example, I have needed this when installing Nvidia's graphics drivers on Ubuntu or Linux Mint. Another instance occurred on Arch Linux when a Cinnamon desktop update prevented me from opening a terminal window. The full screen command let me install an alternative terminal emulator, with Tech Drive-in's list proving helpful. Similar issues might need fixing on FreeBSD installations. These latter examples happened within VirtualBox, which has special requirements for accessing full screen command line sessions, which I'll explain later.

When running Linux on a physical PC, press CTRL + ALT + F1 to enter a full screen terminal and CTRL + ALT + F7 to return to the graphical desktop. In a Linux VirtualBox guest with a Linux host, these shortcuts affect the host instead. For the guest OS, use [Host Key] + F1 to enter a full screen terminal and [Host Key] + F7 to return to the graphical desktop. The default Host Key is the right CTRL key, unless you've changed it.

X sessions in GNOME and Cinnamon desktop environments support this functionality, but I can't confirm it works with alternatives like Wayland. Hopefully, this feature extends to other setups, as terminal sessions are occasionally needed for system recovery. Such mishaps are thankfully rare and should be virtually non-existent for most users.

Creating a test web server using Ubuntu Server 13.04 and VirtualBox

Having seen Linux Format cover tools like Vagrant and Puppet that manage virtual machines, I have been attracted by the prospect of a virtual web server running on my own PC. Certainly, having the LAMP software stack in a VM means that the corresponding tools don't need to be added to a host system should its operating system need a fresh installation.

As intriguing as tools like Vagrant may be, I decided that I needed to learn a bit more about getting server instances set up in VirtualBox anyway. Thus, I went and downloaded the latest version of Ubuntu Server and gave that a go. One lesson that I learned was that Bridged Networking needs to be added to the VM before installation of the operating system, unless you fancy overcoming the challenge of getting Ubuntu Server to recognise an altered or additional network interface. In my case, I added an extra adapter for the Bridged Networking and left the original in place as NAT. The reason for having Bridged Networking set up is that it allows access to the virtual web server from the host once you know the IP address and that information can be obtained by executing the ifconfig command on the virtual machine.

With the networking sorted, the next step was to install the 64-bit edition of Ubuntu Server. Unlike its desktop counterpart, this is all driven by text menus but remains fairly intuitive and there is hardly anything there that you wouldn't see with another Linux distribution. A useful addition is the addition of a menu to select the types of server services that you'd like to see installed. From this, I chose the web server and SSH options, and I seem to remember that there was a database server option too. If there was an FTP server option, I would have chosen that too, but it was no ordeal to add Proftpd later on anyway.

All of this set was done through the VirtualBox GUI, just to keep life more straightforward. Even so, I only selected 12 MB of video memory and was tempted to cut the overall memory back from 512 MB but leaving things be for now. However, what I have begun to do is start and stop the virtual machine from the command line, since servers are headless operations anyway. With SSH enabled, there is little need to have the VirtualBox GUI going. The command for starting the server is below:

VBoxManage startvm "Ubuntu Server" --type=headless

There is a VBoxHeadless command for the same end too, but VBoxManage does what I need. The startvm option is what tells VBoxManage to start the server, and the virtual machine's name is enclosed in quotes. The --type=headless ensures that no window pops up. To stop the virtual web server cleanly, a command like the following is needed:

VBoxManage controlvm "Ubuntu Server" acpipowerbutton

Again, the VBoxManage command gets used and the acpipowerbutton option ensures that a clean shutdown is performed. Not doing so results in the server not fully starting up, according to my experiences thus far. Getting the virtual web server to start and stop with the host machine itself starting and stopping, but this looks more complex, so I plan to leave things a while before trying that experiment.