Technology Tales

Self-hosted web analytics tracking

It amazes me now to think how little tracking I used to do on my various web “experiments” only a few short years ago. However, there was a time when a mere web counter, perhaps displayed on web pages themselves, was enough to yield some level of satisfaction, or dissatisfaction in many a case. Things have come a long way since then and we now seem to have analytics packages all around us. In fact, we don’t even have to dig into our pockets to get our hands on the means to peruse this sort of information either.

At this point, I need to admit that I am known to make use of a few simultaneously but thoughts about reducing their number are coming to mind but there’ll be more on that later. Given that this site is hosted using WordPress software, it should come as no surprise that Automattic’s own plugin has been set into action to see how things are going. The main focus is on the total number of visits by day, week and month with a breakdown showing what pages are doing well as well as an indication of how people came to the site and what links they followed while there. Don’t go expecting details of your visitors like the software that they are using and the country where they are accessing the site with this minimalist option and satisfaction should head your way.

There is next to no way of discussing the subject of website analytics without mentioning Google’s comprehensive offering in the area. You have to admit that it’s comprehensive with perhaps the only bugbear being the lack of live tracking. That need has been addressed very effectively by Woopra, even if its WordPress plugin will not work with IE6. Otherwise, you need the desktop application (being written in Java, it’s a cross-platform affair and I have had it going in both Windows and Linux) but that works well too. Apart maybe from the lack of campaigns, Woopra supplies as good as all of the information that its main competitor provides. It certainly doe what I would need from it.

However, while they can be free as in beer, there are a some costs associated with using using external services like Google Analytics and Woopra. Their means of tracking your web pages for you is by executing a piece of JavaScript that needs to be added to every page. If you have everything set to use a common header or footer page, that shouldn’t be too laborious and there are plugins for publishing platforms like WordPress too. This way of working means that if anyone has JavaScript disabled or decides not to enable JavaScript for the requisite hosts while using the NoScript extension with Firefox, then your numbers are scuppered. Saying that, the same concerns probably any JavaScript code that you may want to execute but there’s another cost again: the calls to external websites can, even with the best attention in the world, slow down the loading of your own pages. Not only is additional JavaScript being run but there also is the latency caused by servers having to communicate across the web.

A self-hosted analytics package would avoid the latter and I found one recently through Lifehacker. Amazingly, it has been around for a while and I hadn’t known about it but I can’t say that I was actively looking for it either. Piwik, formerly known as PHPMyVisites, is the name of my discovery and it seems not too immature either. In fact, I’d venture that it does next to everything that Google Analytics does. While I’d prefer that it used PHP, JavaScript is its means of tracking web pages too. Nevertheless, page loading is still faster than with Google Analytics and/or Woopra and Firefox/NoScript users would only have to allow JavaScript for one site too. If you have had experience with installing PHP/MySQL powered publishing platforms like WordPress, Textpattern and such like, then putting Piwik in place is no ordeal. You may find yourself changing folder access but uploading of the required files, the specification of database credentials and adding an administration user is all fairly standard stuff. I have the thing tracking this edifice as well as my outdoor activities (hillwalking/cycling/photography) web presence and I cannot say that I have any complaints so we’ll see how it goes from here.

Suffering from neglect?

There have been several recorded instances of Google acquiring something and then not developing it to its full potential. FeedBurner is yet another acquisition where this sort of thing has been suspected. Changeovers by monolithic edict and lack of responsiveness from support fora are the sorts of things that breed resentment in some that share opinions on the web. Within the last month, I found that my FeedBurner feeds were not being updated as they should have been and it would not accept a new blog feed when I tried adding it. The result of both these was that I got to deactivating the FeedBurner FeedSmith plugin to take FeedBurner out of the way for my feed subscribers; those regulars on my hillwalking blog were greeted by a splurge of activity following something of a hiatus. There are alternatives such as RapidFeed and Pheedo but I will stay away from the likes of these for a little while and take advantage of the newly added FeedStats plugin to keep tabs on how many come to see the feeds. The downside to this is that IE6 users will see the pure XML rather than a version with a more friendly formatting.

Episodes of poor performance

Over the last few days, I have been noticing from various that this blog isn’t performing as I would want it. The first hint was a comment on a tuxmachines.org mention for a recent entry (thanks for the support, by the way) that the link wasn’t working as it should have been. Add to this various emails from Are My Sites Up? saying that the site seemed to be down. By all accounts, this free service that I found through Lifehacker would appear to be doing its job and without the annoying advertising emails that Internetseer used to send me in addition to its weekly report when I used its free service. In fact, that you get alert emails several times a day is a factor in favour of the newcomer.

With one exception, these problems would appear to be intermittent. The exception was when I went using the WP Super Cache WordPress plugin and it seemed to result in breakage of the site so it got disabled even if it is meant to be helpful during episodes of heavy load. Otherwise, the outages would seem to be general flakiness of the service provided by my hosting provider. I have a site with them on an older server and that seems to fare far better than the one playing host to this blog. This sort of thing does make me wonder if we are getting real progress or whether it’s a case of one step forward and two steps back. Nevertheless, I’ll continue keeping an eye on things and, if there is too much deterioration, a move might be in order but that’s a good bit away yet.

Work locally, update remotely

Here’s a trick that might have its uses: using a local WordPress instance to update your online blog (yes, there are plenty of applications that promise to edit your online blog but these need file permissions to the likes of xmlrpc.php to be opened up). Along with the right database access credentials and the ability to log in remotely, adding the following two lines to wp-config.php does the trick:

define('WP_SITEURL', 'http://localhost/blog');

define('WP_HOME', 'http://localhost/blog');

These two constants override what is in the database and allow to update the online database from your own PC using WordPress running on a local web server (Apache or otherwise). One thing to remember here is that both online and offline directory structures are similar. For example, if your online WordPress files are in blog in the root of the online web server file system (typically htdocs for Linux), then they need to be contained in the same directory in the root of the offline server too. Otherwise, things could get confusing and perhaps messy. Another thing to consider is that you are modifying your online blog so the usual rules about care and attention apply, particularly with respect to using the same version of WordPress both locally and remotely. This is especially a concern if you, like me, run development versions of WordPress to see if there are any upheavals ahead of us like the overhaul that is coming in with WordPress 2.7.

An alternative use of this same trick is to keep a local copy of your online database in case of any problems while using a local WordPress instance to work with it. I used to have to edit the database backup directly (on my main Ubuntu system), first with GEdit but then using a sed command like the following:

sed -e s/www\.onlinewebsite\.com/localhost/g backup.sql > backup_l.sql

The -e switch uses regular expression substitution that follows it to edit the input with the output being directed to a new file. It’s slicker than the interactive GEdit route but has been made redundant by defining constants for a local WordPress installation as described above.

Remove Revisions 2.2

There is already a post on here devoted to version 1.0 of this plugin and that very much tells you what it does. The new version will work with the forthcoming WordPress 2.7 (itself a release that’s had a development cycle with such upheavals that it would make you want to watch from the relative safety of the sidelines) and has been made to be a little more user friendly in its actions; in fact, it behaves more like any other plugin now.

Download Remove Revisions 2.2

Dean’s FCKEditor for WordPress plugin, a wish list

I must admit that I have a liking for FCKEditor over and above what comes as standard with WordPress and the FCKEditor for WordPress plugin has been addressing my preference for a while now. However, its most recent release dates from last April and its integration with WordPress 2.6.x has been leaving a lot to be desired. In that vein, I have decided to collect a few of them here:

1. Automatic saves: the idea behind this feature of WordPress is that you aren’t hitting the save button that often at all. In fact, given that hitting save creates a revision and an extra record in your database, it really isn’t something that you should be doing very often anyway. Unless, you don’t mind a bloated database, it’s probably best to avoid that habit of saving every few minutes like I do when using Word.
2. Word count: this doesn’t update without saving a revision while it should update periodically in a manner akin to the automatic saves.
3. Insertion of media such as images: this is just broken and it takes away the possibility of having galleries and captions without manual work.

What I have above are the major inconsistencies but there have always been annoyances like the adding unwanted   entities allover the place, probably a habit of FCKEditor itself anyway. Nevertheless, it’s the integration work that really shows the lack of attention. Maybe, it’s time to move Dean Lee’s labour of love over to a fully community-maintained course of development. I know that it’s hard to see your “baby” leave you and take flight but I am inclined that it’s the best way forward when you considered how rapidly WordPress has been changing over the last year. Some moves have been made towards this but they really do need to go further.

A radical new look on the way for WordPress’ administration area

One thing that you can never say about Automattic is that it stands still for very long. That may generate adverse commentary from some but it’s bad to leave things stagnate too. In fact, resting on laurels also generates flak so you can’t please everyone all the time.

Earlier on this year, the WordPress administration screens went through something of an overall for version 2.5. In hindsight, it wasn’t terribly dramatic, but the prospect had me checking out what was happening with the development version and contributing to the project in my own small way. Now, it seems that a bigger upheaval is in prospect with wireframes and what not being brought into play on the design side.

The first change that anyone will notice is that the navigation has moved from the top to the side and that some things have been moved around and renamed. Another thing that you’ll see soon enough is that there is a QuickPress section added to the dashboard screen for those really quick and short postings. Sticking with the subject of content creation, alterations to post and page editing screens and the link creation screen are obvious too. What really comes to mind here is the level of customisation on offer so you can make yourself feel right at home: some of the screen furniture can be moved about and you can remove what you feel is nothing but useless clutter. Content generates comments, so the addition of keyboard shortcuts for comment moderation should be a boon for those with very active readers.

Some functionality currently added by plugins is getting incorporated in the main application. An example is automatic upgrades of WordPress itself. In the development version, it installs the latest nightly build but I am certain that it can be made to point to the latest stable release. That makes it more convenient for keeping a backseat eye on things rather than getting stuck into the hurly-burly of checking on what’s in Subversion.

There are some pieces that remain incompletely functional at present, such as the Inbox and Media added to this post pieces, but I have seen nothing that used to work to be broken. So, while the development version of WordPress is more of a work in progress than I have seen for a while, it will do what you need it to do. You always need to be cautious and I’d advise you to keep away until it’s ready for the big time until you have the knowledge to put things back should the undesirable befall your blog. I had an automatic WordPress update kick in maintenance mode without turning it off again. Nothing happened to the database, so a manual re-installation was all that was required to restore order. Otherwise, the development installation remains relatively stable though there are display problems in IE6 that do not afflict either Firefox, Opera or even IE8 Beta2.

All in all, these changes will make the next major WordPress a very big one and I have little doubt that the new administration interface will make for many comments. I must admit that I wasn’t too taken with it when I first glimpsed it in the crazyhorse branch but it now seems to be growing on me. The biggest change on the aesthetic front is that there’s a lot less whitespace about which may or may not float your boat. In any event, you can always change the colours like I have to make things more amenable. Even so, I reckon that WordPress 2.7 will be a major step forward when it comes and I think that I could like it. Saying that, I reckon that the release date is a good while away yet and the fact that 2.6.2 is being readied for release is telling in itself. Leaving plenty of time to remove any roughness is never a bad thing and especially so when you see the changes that are being made. In the meantime, I am certain that most people can wait.

Another way to look through WordPress code

At times in the past, I have complained about there not being sufficient documentation of the actual code used in WordPress itself. True, there is the venerable Codex but that can be incomplete and any extra help is always useful. Peter Westwood (a.k.a westi) has generated documentation from the code itself and put it up on the web for all to peruse. While we are on the subject of information on the more advanced aspects of WordPress, blogs by Mark Jaquith and Andrew Ozz are also worth a look. As the cliché goes, every little helps…

WordPress plugin for removing post revisions from database

WordPress 2.6 added post revisions as a new feature that is turned on by default. In an earlier post, I described how you could control this by editing wp-config.php and there are a number of plugins that purport to provide the same level of control through the administration screens. Even so, I decided to look at things from the housekeeping side of things and create my own plugin for clearing the database of revisions at one swoop. Currently, it takes out all revisions but I am thinking of adding the facility for selecting which revision to keep and which to delete. It goes without saying that you should back up your database first in case anything might go wrong.

Download Remove Revisions 1.0

Controlling the post revision feature in WordPress 2.6

This may seem esoteric for some but I like to be in control of the technology that I use. So, when Automattic included post revision retention to WordPress 2.6, I had my reservations about how much it would clutter my database with things that I didn’t need. Thankfully, there is a way to control the feature, but you won’t find the option in the administration screens (they seem to view this as an advanced setting and so don’t want to be adding clutter to the interface for the sake of something that only a few might ever use); you have to edit wp-config.php yourself to add it. Here are the lines that can be added and the effects that they have:

Code: define('WP_POST_REVISIONS','0');

Effect: turns off post revision retention

Code: define('WP_POST_REVISIONS','-1');

Effect: turns it on (the default setting)

Code: define('WP_POST_REVISIONS','2');

Effect: only retains two previous versions of a post (the number can be whatever you want so long as it’s an integer with a value more than zero).

Update (2008-07-23):

There is now a plugin from Dion Hulse that does the above for you and more.