Cloud infrastructure | Technology Tales

Converting QEMU disk images to VirtualBox images on Linux Mint 21

30^th October 2022

Recently, VirtualBox gained fuller support for Windows 11 and I successively set up a new Windows 11 virtual machine that, I hope, will supplant a Windows 10 counterpart in time. While the setup itself was streamlined, I ran into such stability issues that I set the new VM aside until a new version of VirtualBox got released. That has happened with the appearance of version 7.0.2, but Windows 11 remains prone to freezing on my Linux Mint machine. Thankfully, that now is much less frequent, yet the need for added stability remains outstanding.

While I was thinking about trying out VirtualBox 7.0.0, I remembered a QEMU machine that I had running Windows 11. Though QEMU proved more limited than VirtualBox when it came to having easy availability of functionality like moving data in and out of the virtual machine or support for sound, there was no problem with TPM support or system stability. Since it did contain some useful data, I wondered about converting its virtual hard disk to VirtualBox format, and it is easy to do. First, you need to install qemu-img and other utilities as follows:

sudo apt-get install qemu-utils

With that in place, executing a command like the following performs the required conversion. Here, the -O switch specifies the required file type of vdi in this case.

qemu-img convert -O vdi [virtual hard disk].qcow2 [virtual hard disk].vdi

While I have yet to mount it on the new VirtualBox Windows 11 virtual machine, it is good to have the old virtual hard disk available for doing so. The thought of using it as a boot drive in VirtualBox did enter my mind, but the required change of drivers and other incompatibilities dissuaded me from doing so.

Changing the UUID of a VirtualBox Virtual Disk Image in Linux

11^th July 2021

Recent experimentation centring around getting my hands on a test version of Windows 11 had me duplicating virtual machines and virtual disk images, though VirtualBox still is not ready for the next Windows version; it has no TPM capability at the moment. Nevertheless, I was able to get something after a fresh installation that removed whatever files were on the disk image. That meant that I needed to mount the old version to get at those files again.

While renaming partially helped with this, what I really needed to do was change the UUID, so VirtualBox would not report a collision between two disk images with the same UUID. To avoid this, the UUID of one of the disk images had to be changed and a command like the following was used to accomplish this:

VBoxManage internalcommands sethduuid [Virtual Disk Image Name].vdi

Because I was doing this on Linux Mint, I could call VBoxManage without need to tell the system where it was, as would be the case on Windows. Otherwise, it is the sethduuid portion that changes the UUID as required. Another way around this is to clone the VDI file using the following command, but I had not realised that at the time:

VBoxManage clonevdi [old virtual disk image].vdi [new virtual disk image].vdi

It appears that there can be more than one way to do things in VirtualBox at times, so the second way will remain on reference for the future.

Getting rid of the Windows Resizing message from a Manjaro VirtualBox guest

27^th July 2020

Like Fedora, Manjaro also installs a package for VirtualBox Guest Additions when you install the Linux distro in a VirtualBox virtual machine. However, it does have certain expectations when doing this. On many systems, and my own is one of these, Linux guests are forced to use the VMSVGA virtual graphics controller while Windows guests are allowed to use the VBoxSVGA one. It is the latter that Manjaro expects, so you get a message like the following appearing when the desktop environment has loaded:

Windows Resizing Set your VirtualBox Graphics Controller to enable windows resizing

After ensuring that gcc, make, perl and kernel headers are installed, I usually install VirtualBox Guest Additions myself from the included ISO image, and so I did the same with Manjaro. Doing that and restarting the virtual machine got me extra functionality like screen resizing and being able to copy and paste between the VM and elsewhere after choosing the Bidirectional setting in the menus under Devices > Shared Clipboard.

That still left an unwanted message popping up on startup. To get rid of that, I just needed to remove /etc/xdg/autostart/mhwd-vmsvga-alert.desktop. While it can be deleted, I just moved it somewhere else and a restart proved that the message was gone as needed. Now everything is working as I wanted.

Contents not displaying for Shared Folders on a Fedora 32 guest instance in VirtualBox

26^th July 2020

While some Linux distros like Fedora install VirtualBox drivers during installation time, I prefer to install the VirtualBox Guest Additions themselves. Before doing this, it is best to remove the virtualbox-guest-additions package from Fedora to avoid conflicts. After that, execute the following command to ensure that all prerequisites for the VirtualBox Guest Additions are in place before mounting the VirtualBox Guest Additions ISO image and installing from there:

sudo dnf -y install gcc automake make kernel-headers dkms bzip2 libxcrypt-compat kernel-devel perl

During the installation, you may encounter a message like the following:

ValueError: File context for /opt/VBoxGuestAdditions-<VERSION>/other/mount.vboxsf already defined

This is generated by SELinux, so the following commands need to be executed before repeating the installation of VirtualBox Guest Additions:

sudo semanage fcontext -d /opt/VBoxGuestAdditions-<VERSION>/other/mount.vboxsf sudo restorecon /opt/VBoxGuestAdditions-<VERSION>/other/mount.vboxsf

Without doing the above step and fixing the preceding error message, I had an issue with mounting of Shared Folders whereby the mount point was set up, but no folder contents were displayed. This happened even when my user account was added to the vboxsf group, and it proved to be the SELinux context issue that was the cause.

Shared folders not automounting on an Ubuntu 18.04 guest in a VirtualBox virtual machine

1^st October 2019

Over the weekend, I finally got to resolve a problem that has affected Ubuntu 18.04 virtual machine for quite a while. The usual checks on Guest Additions installation and vboxsf group access assignment were performed but were not causing the issue. Also, no other VM (Windows (7 & 10) and Linux Mint Debian Edition) on the same Linux Mint 19.2 machine was experiencing the same issue. The latter observation made the problem intrinsic to the Ubuntu VM itself.

Because I install the Guest Additions software from the included virtual CD, I executed the following command to open the relevant file for editing:

sudo systemctl edit --full vboxadd-service

If I had installed virtualbox-guest-dkms and virtualbox-guest-utils from the Ubuntu repositories instead, then this would have been the command that I needed to execute instead of the above.

sudo systemctl edit --full virtualbox-guest-utils

Whichever configuration gets opened, the line that needs attention is the one beginning with "Conflicts" (line 6 in the file on my system). The required edit removes systemd-timesync.service from the list following the equals sign. It is worth checking that file paths include the correct version number for the Guest Additions software that is installed, in case this is not how things are. The only change that was needed on my Ubuntu VM was to the Conflicts line, and rebooting it got the Shared Folder automatically mounted under the /media directory as expected.

Enlarging a VirtualBox VDI virtual disk

20^th December 2018

It is remarkable how the Windows folder manages to grow on your C drive, and one in a Windows 7 installation was the cause of my needing to expand the VirtualBox virtual machine VDI disk on which it was installed. After trying various ways to cut down the size, an enlargement could not be avoided. In all of this, it was handy that I had a recent backup for restoration after any damage.

The same thing meant that I could resort to enlarging the VDI file with more peace of mind than otherwise might have been the case. This needed use of the command line once the VM was shut down. The form of the command that I used was the following:

VBoxManage modifyhd <filepath/filename>.vdi --resize 102400

It appears that this also would work on a Windows host, but mine was Linux, and it did what I needed. The next step was to attach it to an Ubuntu VM and use GParted to expand the main partition to fill the newly available space. That does not mean that it takes up 100 GiB on my system just yet because these things can be left to grow over time and there is a way to shrink them too if you ever need to do just that. As ever, having a backup made before any such operation may have its uses if anything goes awry.

Compressing a VirtualBox VDI file for a Linux guest

6^th June 2016

In a previous posting, I talked about compressing a virtual hard disk for a Windows guest system running in VirtualBox on a Linux system. Since then, I have needed to do the same for a Linux guest following some housekeeping. Because the Linux distribution used is Debian, the instructions are relevant to that and maybe its derivatives such as Ubuntu, Linux Mint and their like.

While there are other alternatives like dd, I am going to stick with a utility named zerofree to overwrite the newly freed up disk space with zeroes to aid compression later on in the process for this and the first step is to install it using the following command:

apt-get install zerofree

Once that has been completed, the next step is to unmount the relevant disk partition. Luckily for me, what I needed to compress was an area that I reserved for synchronisation with Dropbox. If it was the root area where the operating system files are kept, a live distro would be needed instead. In any event, the required command takes the following form, with the mount point being whatever it is on your system (/home, for instance):

sudo umount [mount point]

With the disk partition unmounted, zerofree can be run by issuing a command that looks like this:

zerofree -v /dev/sdxN

Above, the -v switch tells zerofree to display its progress and a continually updating percentage count tells you how it is going. The /dev/sdxN piece is generic with the x corresponding to the letter assigned to the disk on which the partition resides (a, b, c or whatever) and the N is the partition number (1, 2, 3 or whatever; before GPT, the maximum was 4). Putting all this together, we get an example like /dev/sdb2.

Once, that had completed, the next step is to shut down the VM and execute a command like the following on the host Linux system ([file location/file name] needs to be replaced with whatever applies on your system):

VBoxManage modifyhd [file location/file name].vdi --compact

With the zero filling in place, there was a lot of space released when I tried this. While it would be nice for dynamic virtual disks to reduce in size automatically, I accept that there may be data integrity risks with those, so the manual process will suffice for now. It has not been needed that often anyway.

Migrating a virtual machine from VirtualBox to VMware Player on Linux

1^st February 2015

The progress of Windows 10 is something that I have been watching. Early signs have been promising, and the most recent live event certainly contained its share of excitement. The subsequent build that was released was another step in the journey, though the new Start Menu appears more of a work in progress than it did in previous builds. Keeping up with these advances sometimes steps ahead of VirtualBox support for them, and I discovered that again in the last few days. VMware Player seems unaffected, so I thought that I'd try a migration of the VirtualBox VM with Windows 10 onto there. In the past, I did something similar with a 32-bit instance of Windows 7 that subsequently got upgraded all the way up to 8.1, but that may not have been as slick as the latest effort, so I thought that I would share it here.

The first step was to export the virtual machine as an OVF appliance, and I used File > Export Appliance... only to make a foolish choice regarding the version of OVF. The one that I picked was 2.0 only to subsequently discover that 1.0 was the better option. The equivalent command line would look like the following (there are two dashes before the ovf10 option below):

VboxManage export [name of VM] -o [name of file].ova --ovf10

VMware has a tool for extracting virtual machines from OVF files that will generate a set of files that will work with Player and other similar products of theirs. It goes under the unsurprising name of OVF Tool and usefully works from a command line session. When I first tried it with an OVF 2.0 files, I got the following error, and it stopped doing anything as a result:

Line 2: Incorrect namespace http://schemas.dmtf.org/ovf/envelope/2 found.

The only solution was to create a version 1.0 file and use a command like the following:

ovftool --lax [name of file].ova [directory location of VM files]/[name of file].vmx

The --lax option is needed to ensure successful execution, even with an OVF 1.0 file as the input. Once I had done this on my Ubuntu GNOME system, the virtual machine could be opened up on VMware Player and I could use the latest build of Windows 10 at full screen, something that was not possible with VirtualBox. This may be how I survey the various builds of the operating that appear before its final edition is launched later this year.

Dropping back to a full screen terminal session from a desktop one in Linux

29^th May 2014

There are times when you might need to access a full screen terminal from a Linux graphical desktop. For example, I have needed this when installing Nvidia's graphics drivers on Ubuntu or Linux Mint. Another instance occurred on Arch Linux when a Cinnamon desktop update prevented me from opening a terminal window. The full screen command let me install an alternative terminal emulator, with Tech Drive-in's list proving helpful. Similar issues might need fixing on FreeBSD installations. These latter examples happened within VirtualBox, which has special requirements for accessing full screen command line sessions, which I'll explain later.

When running Linux on a physical PC, press CTRL + ALT + F1 to enter a full screen terminal and CTRL + ALT + F7 to return to the graphical desktop. In a Linux VirtualBox guest with a Linux host, these shortcuts affect the host instead. For the guest OS, use [Host Key] + F1 to enter a full screen terminal and [Host Key] + F7 to return to the graphical desktop. The default Host Key is the right CTRL key, unless you've changed it.

X sessions in GNOME and Cinnamon desktop environments support this functionality, but I can't confirm it works with alternatives like Wayland. Hopefully, this feature extends to other setups, as terminal sessions are occasionally needed for system recovery. Such mishaps are thankfully rare and should be virtually non-existent for most users.

Installing FreeBSD in a VirtualBox Virtual Machine

2^nd March 2014

With UNIX being the basis of Linux, I have a soft spot for trying out any UNIX that can be installed on a PC. For a while, I had OpenSolaris on the go and even vaguely recall having a look at one of the BSD's. However, any recent attempt to install one of the latter, and there are quite a few around now, got stymied by some sort of kernel panic caused by using AMD CPU's. With the return to the Intel fold arising from the upgrade of my main home PC last year, it perhaps was time to try again.

The recent release of FreeBSD 10.0 was the cue and I downloaded a DVD image for a test installation in a VirtualBox virtual machine with 4 GB of memory and a 32 GB virtual hard drive attached (expanding storage was chosen so not all the allocated space has been taken so far). The variant of FreeBSD chosen was the 64-bit x86 one, and I set to installing it in there. Though not as pretty in appearance as those in various Linux distros, the installer was not that user unfriendly to me. Mind you, I have experience of installing Arch Linux, which might have acclimatised me somewhat.

Those installation screens ask about the keyboard mapping that you want, and I successfully chose one of the UK options. There was limited opportunity for adding extras, though there was a short list of a few from which I made some selections. Given that user account set up also was on offer, I would have been better off knowing what groups to assign for my personal user account to have to avoid needing to log in as root so often following system start up later. Otherwise, all the default options were sufficient.

When the installation process was complete, it was time to boot into the new system and all that was on offer was a command line log in session. After logging in as root, it was time to press pkg into service to get a desktop environment in place. The first step was to install X:

pkg install xorg

Then, it was time to install a desktop environment. While using XFCE or KDE were alternatives, I chose GNOME 2 due to familiarity and more extensive instructions on the corresponding FreeBSD handbook page. Issuing the following command added GNOME and all its helper applications:

pkg install gnome2

So that GNOME starts up at the next reboot, some extra steps are needed. The first of these is to add the following line into /etc/fstab:

proc /proc procfs rw 0 0

Then, two lines were needed in /etc/rc.conf:

gdm_enable="YES" gnome_enable="YES"

The first enables the GNOME display manager, while the second activates other GNOME programs that are needed for a desktop session to start. With each of these in place, I got a graphical login screen at the next boot time.

With FreeBSD being a VirtualBox Guest, it was time to consult the relevant FreeBSD manual page. Here, there are sections for a number of virtual machine tools, so a search was needed to find the one for VirtualBox. VirtualBox support for FreeBSD is incomplete in that there is no installation media for BSD systems, while Linux and Solaris are supported along with Windows. Therefore, it is over to the FreeBSD repositories for the required software:

pkg install virtualbox-ose-additions

Aside from the virtual machine session not capturing and releasing the mouse pointer automatically, that did everything that was needed, even if it was the open source edition of the drivers and their proprietary equivalents. To resolve the mouse pointer issue, I needed to temporarily disable the GNOME desktop session in /etc/rc.conf to drop to a console only session where xorg.conf could be generated using the following commands:

Xorg -configure cp xorg.conf.new /etc/xorg.conf

In the new xorg.conf file, the mouse section needs to be as follows:

Section "InputDevice" Identifier "Mouse0" Driver "vboxmouse" EndSection

If it doesn't look like the above, and it wasn't the case for me, then it needs changing. Also, any extra lines from the default set up also need removing, or the mouse will not function as it should. The ALT+F1 (for accessing GNOME menus) and ALT+F2 (for running commands) keyboard shortcuts then become crucial when your mouse is not working as it should and could avert a panic too; knowing that adjusting a single configuration file will resolve a problem when doing so is less accessible is not a good feeling as I discovered to my own cost. The graphics settings were fine by default, but here's what you should have in case it isn't for you:

Section "Device" ### Available Driver options are:- ### Values: <i>: integer, <f>: float, <bool>: "True"/"False", ### <string>: "String", <freq>: "<f> Hz/kHz/MHz" ### [arg]: arg optional Identifier "Card0" Driver "vboxvideo" VendorName "InnoTek Systemberatung GmbH" BoardName "VirtualBox Graphics Adapter" BusID "PCI:0:2:0" EndSection

The next step is to ensure that your HAL settings are as they should. I needed to create a file in /usr/local/etc/hal/fdi/policy called 90-vboxguest.fdi that contains the following:

<?xml version="1.0" encoding="utf-8"?>  <deviceinfo version="0.2"> <device> <match key="info.subsystem" string="pci"> <match key="info.product" string="VirtualBox guest Service"> <append key="info.capabilities" type="strlist">input</append> <append key="info.capabilities" type="strlist">input.mouse</append> <merge key="input.x11_driver" type="string">vboxmouse</merge> <merge key="input.device" type="string">/dev/vboxguest</merge> </match> </match> </device> </deviceinfo>

With all that set, it is time to ensure that the custom user account is added to the wheel and operator groups using this command:

pw user mod [user name] -G wheel operator

Executing the above as root means that the custom account can run the su command so that logging in as root at the start of a desktop session no longer is needed. That is what being in the wheel group allows, so anyone in the operator group can shut down or restart the system. Since both are facilities readily available on Linux, so I fancied having them in FreeBSD too.

Being able to switch to root in a terminal session meant that I could go on to add software like Firefox, LibreOffice, GIMP, EMACS, Geany, NetBeans, Banshee and so on. Though there may be a line of opinion that FreeBSD is a server operating system, all of these make it more than passable for serving as a desktop one too. There may be no package management GUI as such and the ones that come with GNOME do not work either, yet anyone familiar with command line working will get around that.

While FreeBSD may be conservative, that has its place too, and being able to build up a system one item at a time teaches far more than getting everything already sorted in one hit. So far, there is enough documentation to get me going, leaving me to see where else things go too. So far, the OS hasn't been that intimidating, which is good to see.

« Older Entries «