Technology Tales

Tidying dynamic URL’s

A few years back, I came across a very nice article discussing how you would make a dynamic URL more palatable to a search engine and I made good use of its content for my online photo gallery. The premise was that URL’s that look like that below are no help to search engines indexing a website. Though this is received wisdom in some quarters, it doesn’t seem to have done much to stall the rise of WordPress as a blogging platform.

http://www.mywebsite.com/serversidescript.php?id=394

That said, WordPress does offer a friendlier URL display option too and you can see this in use on this blog; they look a little like the example URL that you see below, and the approach is equally valid for both Perl and PHP. I have been using the same approach for the Perl scripts powering my online phone gallery and now want to apply the same thinking to a gallery written in PHP:

http://www.mywebsite.com/serversidescript.pl/id/394

The way that both expressions work is that a web server will chop pieces from a URL until it reaches a physical file. For a query URL, the extra information after the question mark is retained in its QUERY_STRING variable while extraneous directory path information is passed in the variable PATH_INFO. For both Perl and PHP, these are extracted from the entries in an array; for Perl, this array is called is $ENV and $_SERVER is the PHP equivalent. Thus, $ENV{QUERY_STRING} and $_SERVER{‘QUERY_STRING’} traps what comes after the “?” while $ENV{PATH_INFO} and $_SERVER{‘PATH_INFO’} picks up the extra information following the file name (the “/id/394/” in the example). From there on, the usual rules apply regarding cleaning of any input but changing from one to another should be too arduous.

The wonders of mod_rewrite

When I wrote about tidying dynamic URL’s a little while back, I had no inkling that that would be a second part to the tale. My discovery of mod_rewrite, an Apache module that facilitates URL translation. The effect is that one URL is presented to the user in the browser address bar, and the exact same URL is also seen by search engines, while another is passed to the server for processing. It might sound like subterfuge but it works very well once you manage to get it set up properly. The web host for my hillwalking blog/photo gallery has everything configured such it is ready to go but the same did not apply to the offline Apache 2.2.x server that I have going on my own Windows XP box. There were two parts to getting it working there:

1. Activating mod-rewrite on the server: this is as easy as uncommenting a line in the httpd.conf file for the site (the line in question is: LoadModule rewrite_module modules/mod_rewrite.so).
2. Ensuring that the .htaccess file in the root of the web server directory is active. You need to set the values of the AllowOverride directives for the server root and CGI directories to All so that .htaccess is active. Not doing it for the latter will result in the an error beginning with the following: Options FollowSymLinks or SymLinksIfOwnerMatch is off which implies that RewriteRule directive is forbidden. Having Allow from All set for the required directories is another option to consider when you see errors like that.

Once you have got the above sorted, adding this line to .htaccess: RewriteEngine On. Preceding it with an Options directive to ensure that FollowSymLinks and SymLinksIfOwnerMatch are switched on does no harm at all and may even be needed to get things running. That done, you can set about putting mod_write to work with lines like this:

RewriteRule ^pages/(.*)/?$ pages.php?query=$1

The effect of this is to take http://www.website.com/pages/input and convert it into a form for action by the server; in this case, that is http://www.website.com/pages.php?query=input. Anything contained by a bracket is assigned to the value of a system-named variable. If you have several bracketed sections, they are assigned to sequentially numbered variables as follows: $1 for the first, $2 for the second and so on. It’s all good stuff when you get it going and not only does it make things look much neater but it also possesses an advantage when it comes to future-proofing too. Web addresses can be kept constant over time, even if things change behind the scenes. It means that returning visitors will find what they saw the last time that they visited and surely must ensure good karma in eyes of those all important search engines.

A web development toolbox

Something to watch?

Apparently, PayPal does recurring payments. This one passed me by until I notice that a certain (reputable) software vendor had set up just such a thing on me. As it happens, I have no recollection of being told about it but it is a little while ago so I may have forgotten. Regardless of the history, I shall be a keeping an eagle eye out for such things from this point forward. You cannot be too careful with your money this days so it’s best to learn as you live…

Creating a test web server using Ubuntu Server 13.04 and VirtualBox

Having seen Linux Format cover tools like Vagrant and Puppet that manage virtual machines, I have been attracted by the prospect of a virtual web server running on my own PC. Certainly, having the LAMP software stack in a VM means that the corresponding tools don’t need to be added to a host system should its operating system need a fresh installation.

As intriguing as tools like Vagrant may be, I decided that I needed to learn a bit more about getting server instances set up in VirtualBox anyway. Thus, I went and downloaded the latest version of Ubuntu Server and gave that a go. One lesson that I learned was that Bridged Networking needs to be added to the VM before installation of the operating system unless you fancy overcoming the challenge of getting Ubuntu Server to recognise an altered or additional network interface. In my case, I added an extra adapter for the Bridged Networking and left the original in place as NAT. The reason for having Bridged Networking set up is that it allows access to the virtual web server from the host once you know the IP address and that information can be obtained by executing the ifconfig command on the virtual machine.

With the networking sorted, the next step was to install the 64-bit edition of Ubuntu Server. Unlike its desktop counterpart, this is all driven by text menus but remains fairly intuitive and there is hardly anything there that you wouldn’t see with another Linux distribution. A useful addition is the addition of a menu to select the type of server services that you’d like to see installed. From this, I chose the web server and SSH options and I seem to remember that there was a database server option too. If there was an FTP server option, I would have chosen that too but it was no ordeal to add ProFTPd later on anyway.

All of this set was done through the VirtualBox GUI just to keep life more straightforward. Even so, I only selected 12 MB of video memory and was tempted to cut the overall memory back from 512 MB but leaving things be for now. However, what I have begun to do is start and stop the virtual machine from the command line since servers are headless operations anyway. With SSH enabled, there is little need to have the VirtualBox GUI going. The command for starting the server is below:

VBoxManage startvm "Ubuntu Server" --type=headless

There is a VBoxHeadless command for the same end too but VBoxManage does what I need. The startvm option is what tells VBoxManage to start the server and the virtual machine’s name is enclosed in quotes. The --type=headless ensures that no window pops up. To stop the virtual web server cleanly, a command like the following is needed:

VBoxManage controlvm "Ubuntu Server" acpipowerbutton

Again, the VBoxManage command gets used and the acpipowerbutton option ensures that a clean shutdown is performed. Not doing so results in the server not fully starting up according to my experiences thus far. Getting the virtual web server to start and stop with the host machine itself starting and stopping but this looks more complex so I plan to leave things a while before trying that experiment.

Adding a new domain or subdomain to an SSL certificate using Certbot

On checking the Site Health page of a WordPress blog, I saw errors that pointed to a problem with its SSL set up. The www subdomain was not included in the site’s certificate and was causing PHP errors as a result though they had no major effect on what visitors saw. Still, it was best to get rid of them so I needed to update the certificate as needed. Execution of a command like the following did the job:

sudo certbot --expand -d existing.com,www.example.com

Using a Let’s Encrypt certificate meant that I could use the certbot command since that already was installed on the server. The --expand and -d switches ensured that the listed domains were added to the certificate to sort out the observed problem. In the above, a dummy domain name is used but this was replaced by the real one to produce the desired effect and make things as they should have been.

Disk corruption can be virtual too

It’s the sort of sight that causes you to fear the worst, an unchanging black screen with a flashing cursor. That was what started greet me in recent days when I tried to fire up a Windows XP guest in VMware Workstation. The cause was the corruption of a virtual disk, an ominous thing that affected a supplemental virtual hard drive that I had added to the virtual machine rather than its main one. There might have been some data on there but it was nothing that I didn’t have elsewhere anyway. Removing the broken disk from the XP VM returned the situation to fuller health and I simply tried creating a new one again. So far, this seems to be working fine but I’ll be keeping a watchful eye on things.

I have no idea why the corruption happened but the broken disk files were created without a VMDK extension so that might indicate that something was amiss with the process that created them. It would be better if VMware highlighted the state of its virtual disks but it was when I tried opening the trashed disk with VirtualBox that a warning was given and VMware did the same when I tried I tried it with that afterwards (opening VMware virtual disks with VirtualBox is possible but you need to ensure that no attempt at importation is made or you could end up with a breakage). I may have discovered the fault in a roundabout manner but it’s better to know what’s gone wrong than not to know at all.

Shared folders not automounting on an Ubuntu 18.04 guest in a VirtualBox virtual machine

Over the weekend, I finally got to fixing a problem that has affected Ubuntu 18.04 virtual machine for quite a while. The usual checks on Guest Additions installation and vboxsf group access assignment were performed but were not causing the issue. Also, no other VM (Windows (7 & 10) and Linux Mint Debian Edition) on the same Linux Mint 19.2 machine was experiencing the same issue. The latter observation made the problem intrinsic to the Ubuntu VM itself.

Because I install the Guest Additions software from the included virtual CD, I executed the following command to open the relevant file for editing:

sudo systemctl edit --full vboxadd-service

If I had installed installed virtualbox-guest-dkms and virtualbox-guest-utils from the Ubuntu repositories instead, then this would have been the command that I needed to execute instead of the above.

sudo systemctl edit --full virtualbox-guest-utils

Whichever configuration gets opened, the line that needs attention is the one beginning with Conflicts (line 6 in the file on my system). The required edit removes systemd-timesync.service from the list following the equals sign. It is worth checking that file paths include the correct version number for the Guest Additions software that is installed in case this was not the case. The only change that was needed on my Ubuntu VM was to the Conflicts line and rebooting it got the Shared Folder automatically mounted under the /media directory as expected.

Lessons learned on managing Windows Taskbar and Start Menu colouring in VirtualBox virtual machines

In the last few weeks, I have had a few occasions when the colouration of the Windows 10 taskbar and its Star Menu has departed from my expectations. At times, this happened in VirtualBox virtual machine installations and both the legacy 5.2.x versions and the current 6.x ones have thrown up issues.

The first one actually happened with a Windows 10 installation in VirtualBox 5.2.x when the taskbar changed colour to light grey and there was no way to get it to pick up the colour of the desktop image to become blue instead. The solution was to change the Windows from Light to Dark in order for the desired colouration to be applied and the settings above are taken from the screen that appears on going to Settings > Personalisation > Colours.

The second issue appeared in Windows 10 Professional installation in VirtualBox 6.0.x when the taskbar and Start Menu turned transparent after an updated. This virtual machine is used to see what is coming in the slow ring of Windows Insider so some rough edges could be expected. The solution here was to turn off 3D acceleration in the Display pane of the VM settings after shutting it down. Starting it again showed that all was back as expected.

Both resolutions took a share of time to find and there was a deal of experimentation needed too. Once identified, they addressed the issues as desired so I am recording here for use by others as much as future reference for myself.

Using .htaccess to control hotlinking

There are times when blogs cease to exist and the only place to find the content is on the Wayback Machine. Even then, it is in danger of being lost completely. One such example is the subject of this post.

Though this website makes use of the facilities of Cloudflare for various functions that include the blocking of image hotlinking, the same outcome can be achieved using .htaccess files on Apache web servers. It may work on Nginx to a point too but there are other configuration files that ought to be updated instead of using a .htaccess when some frown upon the approach. In any case, the lines that need adding to .htaccess are listed below though the web address needs to include your own domain in place of the dummy example provided:

RewriteEngine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?yourdomain.com(/)?.*$ [NC]
RewriteRule .*\.(gif|jpe?g|png|bmp)$ [F,NC]

The first line turns on the mod_rewrite engine and you may have that done anyway. Of course, the module needs enabling in your Apache configuration for this to work and you have to be allowed to perform the required action as well. This means changing the Apache configuration files. The next pair of lines look at the HTTP referer strings and the third one only allows images to be served from your own web domain and not others. To add more, you need to copy the third line and change the web address accordingly. Any new lines need to precede the last line that defines the file extensions that are to be blocked to other web addresses.

RewriteEngine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?yourdomain.com(/)?.*$ [NC]
RewriteRule \.(gif|jpe?g|png|bmp)$ /images/image.gif [L,NC]

Another variant of the previous code involves changing the last line to display a default image showing others what is happening. That may not reduce the bandwidth usage as much as complete blocking but it may be useful for telling others what is happening.