TOPIC: WORLD WIDE WEB
An unseen arsenal: How web developers can use specialised tools to build better websites
4th March 2026
Modern web development takes place within an ecosystem of tools so precisely suited to individual tasks that they often go unnoticed by anyone outside the profession. These utilities, spanning performance analysers, security checkers and colour palette generators, form the backbone of a workflow that must balance speed, security and visual consistency. For an industry where user experience and technical efficiency are inseparable priorities, such tools are far from optional luxuries.
Performance Testing and Page Speed Analysis
The first hurdle most developers encounter is performance measurement, and several tools have established themselves as essential in this space. GTmetrix, Google PageSpeed Insights and WebPageTest each draw on Google's open-source Lighthouse framework to varying degrees, though each approaches the task differently.
A performance grade alongside separate scores for page speed and structural quality is what GTmetrix produces for any URL submitted to it. It measures Core Web Vitals, including Largest Contentful Paint (LCP), Total Blocking Time (TBT) and Cumulative Layout Shift (CLS), which are the same metrics Google uses as ranking signals in search. The tool can run tests from multiple global server locations and simulates a real browser loading your page, producing a waterfall chart and a video replay of the load process, so developers can identify precisely which elements are causing delays.
Maintained directly by Google, PageSpeed Insights analyses pages against both laboratory data generated through Lighthouse and real-world field data drawn from the Chrome User Experience Report (CrUX). It provides separate performance scores for mobile and desktop, which is significant given that Google confirmed page speed as a ranking factor for mobile searches in July 2018. Both GTmetrix and PageSpeed Insights go well beyond raw figures, mapping out a prioritised list of optimisations so that developers can address the most impactful issues first.
A different position in the toolkit is occupied by WebPageTest, originally created by Patrick Meenan and open-sourced in 2008, and acquired by Catchpoint in 2020. Rather than returning a simple score, it runs tests from a choice of locations across the globe using real browsers at actual connection speeds, and produces detailed waterfall charts that break down every individual network request. This makes it the tool of choice when the question is not just how fast a page is, but precisely why a particular element is slow.
One of the longer-established names in website speed testing, Pingdom offers a free tool that remains widely used for its accessible reporting. Tests can be run from seven global server locations, and results are presented in four sections: a waterfall breakdown, a performance grade, a page analysis and a historical record of previous tests. The page analysis breaks down asset sizes by domain and content type, which is useful for comparing the weight of CDN-served assets against those served directly. Pingdom is based on the YSlow open-source project and does not currently measure the Core Web Vitals metrics that Google uses as ranking signals, so it is best treated as a quick and readable first pass rather than a definitive audit.
Security and Infrastructure Diagnostics
Performance alone cannot sustain a trustworthy website, as a misconfigured certificate, an insecure resource or a flagged IP address can each undermine user confidence and search visibility. One of the most frustrating post-migration problems is the disappearance of the HTTPS padlock despite an SSL certificate being in place, and Why No Padlock? exists specifically to address it. The cause is almost always mixed content, where a page served over HTTPS loads at least one resource (an image, a script or a stylesheet) over plain HTTP. Why No Padlock? scans any HTTPS URL and returns a list of every insecure resource found, along with the HTML element responsible, making it straightforward to trace and resolve the problem. Google has used HTTPS as a ranking signal since 2014, so unresolved mixed content issues carry an SEO cost as well as a security one.
For traffic-level threats, AbuseIPDB operates as a community-maintained IP blacklist. Managed by Marathon Studios Inc., the project allows system administrators and webmasters to report IP addresses involved in malicious behaviour, including hacking attempts, spam campaigns, DDoS attacks and phishing, and to check any IP address against the database before acting on traffic from it. A free API is available for integration with server tools such as Fail2Ban, enabling automatic reporting and real-time checks.
Bot traffic and automated form submissions are a persistent nuisance for any site that accepts user input, and hCaptcha addresses this by presenting challenges that are straightforward for human visitors but reliably difficult for automated scripts. Operated by Intuition Machines, it positions itself as a privacy-focused alternative to reCAPTCHA, collecting minimal data and retaining no personally identifiable information beyond what is necessary to complete a challenge. It is compliant with GDPR, CCPA and several other international privacy frameworks, and holds both ISO 27001 and SOC 2 Type II certifications. A free tier is available, with a Pro plan covering 100,000 evaluations per month, and an Enterprise tier offering additional controls including data localisation and zero-PII processing modes.
Red Sift offers two distinct products that address different aspects of infrastructure security, both relevant to the day-to-day operation of a website. Red Sift OnDMARC automates the configuration and monitoring of DMARC, SPF, DKIM, BIMI and MTA-STS, which are the protocols that collectively prevent attackers from sending spoofed emails that appear to originate from a legitimate domain. This is the basis for most phishing and business email compromise (BEC) attacks, and OnDMARC guides teams to full enforcement typically within six to eight weeks. Red Sift Certificates Lite addresses a separate but equally critical concern, monitoring SSL/TLS certificates for upcoming expiry and alerting administrators seven days ahead of time. It is free for up to 250 certificates and has been formally recommended by Let's Encrypt as its preferred monitoring service, following the retirement of Let's Encrypt's own expiry notification emails. The product was built on the foundation of Hardenize, which Red Sift acquired in 2022, a company founded by Ivan Ristić, creator of SSL Labs.
Colour Management and Visual Design
A website's visual coherence depends heavily on colour consistency, and the distance between a palette sketched on paper and one that functions in code can be significant. With over two million active users, Coolors is a fast and intuitive palette generator built around a simple interaction: pressing the space bar produces a new five-colour palette derived from colour theory algorithms. The platform includes an accessibility checker that calculates contrast ratios against WCAG standards and a colour extractor that derives palettes from uploaded photographs. It also offers interoperability with Figma, Adobe Creative Suite and the Chrome browser. A free tier is available, with a Pro plan at approximately $3 per month for unlimited saving and export options.
A quite different approach is taken by Colormind, which uses a deep learning model based on Generative Adversarial Networks (GANs) to generate harmonious colour schemes. The model is trained on datasets drawn from photographs, films, popular art and website designs, and is updated daily with fresh material. A particularly useful feature allows users to preview how a generated palette would look applied to a website layout, which is a more direct test of practicality than viewing swatches in isolation. A REST API is available for personal and non-commercial use. For converting between colour formats, tools such as Color-Hex, RGBtoHex and the WebFX Hex to RGB converter bridge the gap between design decisions and code implementation, translating colour values in both directions between the hexadecimal and RGB formats that CSS requires.
Optimisation and Code Utilities
Lean, efficient code is a direct contributor to load speed, and unused CSS is a surprisingly common source of unnecessary page weight that PurifyCSS Online addresses by scanning a website's HTML and JavaScript source against its stylesheets to identify selectors that are never used. CSS frameworks such as Bootstrap or Tailwind ship with many utility classes, and most websites use only a small fraction of them. Removing the unused rules can reduce stylesheet file size substantially, which in turn shortens the time a browser spends processing styles before rendering a page. The online version requires no build pipeline or command-line tools, making it accessible to developers at any workflow stage.
Image compression is equally important, as unoptimised images are among the most common causes of slow load times. ImageCompressor handles JPEG, PNG, WebP, GIF and SVG files in the browser, applying lossy or lossless algorithms with adjustable quality settings to reduce file sizes without visible degradation, and processes everything locally, which means that no images are uploaded to an external server. Contact forms and directory listings on websites are a persistent target for spam harvesters, and Email Obfuscator encodes email addresses into a format that is readable by browsers but opaque to most automated scrapers, generating both a plain HTML entity version and a JavaScript-dependent alternative for stronger protection.
For websites that publish mathematical or scientific content, QuickLaTeX provides a practical solution to embedding equations in web pages without a local LaTeX installation. Authors write standard LaTeX expressions directly in their content, and the service renders them as high-quality images that are cached and returned via URL for embedding. Its companion WordPress plugin, WP QuickLaTeX, handles this process automatically within the editor, supporting inline formulas, numbered displayed equations and TikZ graphics.
Server Response and Infrastructure Monitoring
Infrastructure performance sits beneath the layer that most visitors ever see, yet it determines how quickly any content reaches a browser at all, and the Time to First Byte (TTFB) is the metric that captures this most directly. It measures the interval between a browser sending an HTTP request and receiving the first byte of data from the server, and ByteCheck exists solely to measure it. This metric captures the combined effect of DNS resolution time, TCP connection time, SSL negotiation time and server processing time. Google considers a TTFB of 200ms or below to be good, and Byte Check breaks the total down into each constituent step, so developers can identify precisely where delays are occurring. Slow TTFB is often a server-side issue, such as inadequate caching, an overloaded database or a lack of a content delivery network (CDN).
Analytics and Content Evaluation
The final layer of tooling concerns understanding what content a site serves and how it performs in context. Dandelion is a natural language processing API developed by SpazioDati that can extract entities, classify text and analyse the semantic content of web pages, which has applications in content tagging, SEO auditing and editorial quality control. A free tier, covering up to 1,000 API units per day, is available without a credit card, making it accessible for developers who need semantic analysis at low to moderate volume.
Quiet Workhorses of the Web
Individually, each of these tools addresses a specific and well-defined problem. Taken together, they form a coherent toolkit that covers the full lifecycle of a web project, from initial performance diagnosis through to deployment of a secure, efficiently coded and visually consistent site. They do not replace professional judgement but extend it, handling time-consuming checks and conversions that would otherwise consume the attention needed for more complex work. As websites grow in complexity and user expectations continue to rise, familiarity with this kind of specialist tooling becomes a practical necessity rather than an optional extra.
Fixing Crontab editor permissions for www-data
3rd September 2025
There are times when I set jobs to run using the web server account www-data for various website maintenance tasks. To ensure that I am doing this for the right account, I issue the following command:
sudo -u www-data crontab -e
However, doing so on this server yielded the following:
touch: cannot touch '/var/www/.selected_editor': Permission denied
Unable to create directory
/var/www/.local/share/nano/: No such file or directory
It is required for saving/loading search history or cursor positions.
No modification made
While things otherwise worked as they should with nano as the editor, I felt it best to avoid such output if I could. Thus, I modified the command like this:
sudo -u www-data HOME=/tmp crontab -e
This sets the home directory for www-data as /tmp to allow the setting of an editor, at least on an ephemeral basis. The root cause of the messages is that www-data is not a user account like others and does not get a home area. Thus, the above workaround gets around that, without the artificiality of creating a www-data folder in the /home directory. Some might get around the whole business using the Vi editor, but nano suits me better.
Running cron jobs using the www-data system account
22nd December 2018
When you set up your own web server or use a private server (virtual or physical), you will find that web servers run using the www-data account. That means that website files need to be accessible to that system account if not owned by it. The latter is mandatory if you want WordPress to be able to update itself with needing FTP details.
It also means that you probably need scheduled jobs to be executed using the privileges possessed by the www-data account. For instance, I use WP-CLI to automate spam removal and updates to plugins, themes and WordPress itself. Spam removal can be done without the www-data account, but the updates need file access and cannot be completed without this. Therefore, I got interested in setting up cron jobs to run under that account and the following command helps to address this:
sudo -u www-data crontab -e
For that to work, your own account needs to be listed in /etc/sudoers or be assigned to the sudo group in /etc/group. If it is either of those, then entering your own password will open the cron file for www-data, and it can be edited as for any other account. Closing and saving the session will update cron with the new job details.
In fact, the same approach can be taken for a variety of commands where files only can be accessed using www-data. This includes copying, pasting and deleting files as well as executing WP-CLI commands. The latter issues a striking message if you run a command using the root account, a pervasive temptation given what it allows. Any alternative to the latter has to be better from a security standpoint.
More on mod_rewrite
25th June 2007
Today, I caught sight of an article on anti-plagiarism tools at The Blog Herald, and among the tricks was to use mod-rewrite to stop people "borrowing" both your images and your bandwidth. The gist is that you set up one or more conditions that exclude websites from the application of a rule forbidding access to images; the logic is that if the website referencing an image is not one of the websites listed in the conditions, then it doesn't get to display any of your images.
RewriteCond %{HTTP_REFERER} !^http://(www\.)?awebsite.com(/)?.*$ [NC]
RewriteRule .*\.(gif|jpe?g|png|bmp)$ [F,NC]
The wonders of mod_rewrite
24th June 2007
When I wrote about tidying dynamic URL's a little while back, I had no inkling that that would be a second part to the tale. My discovery of mod_rewrite, an Apache module that facilitates URL translation. The effect is that one URL is presented to the user in the browser address bar, and the very same URL is also seen by search engines, while another is passed to the server for processing. Though it might sound like subterfuge, it works very well once you manage to get it set up properly. While the web host for my hillwalking blog/photo gallery has everything configured such that it is ready to go, the same did not apply to the offline Apache 2.2.x server that I have going on my own Windows XP box. There were two parts to getting it working there:
- Activating mod-rewrite on the server: this is as easy as uncommenting a line in the
httpd.conffile for the site (the line in question is:LoadModule rewrite_module modules/mod_rewrite.so). - Ensuring that the
.htaccessfile in the root of the web server directory is active. You need to set the values of theAllowOverridedirectives for the server root and CGI directories toAllso that.htaccessis active. Not doing it for the latter will result in an error beginning with the following:Options FollowSymLinks or SymLinksIfOwnerMatch is off which implies that. HavingRewriteRuledirective is forbiddenAllow from Allset for the required directories is another option to consider when you see errors like that.
Once you have got the above sorted, add this line to .htaccess: RewriteEngine On. Preceding it with an Options directive to ensure that FollowSymLinks and SymLinksIfOwnerMatch are switched on does no harm at all and may even be needed to get things running. That done, you can set about putting mod_write to work with lines like this:
RewriteRule ^pages/(.*)/?$ pages.php?query=$1
The effect of this is to take http://www.website.com/pages/input and convert it into a form for action by the server; in this case, that is http://www.website.com/pages.php?query=input. Anything contained by a bracket is assigned to the value of a system-named variable. If you have several bracketed sections, they are assigned to sequentially numbered variables as follows: $1 for the first, $2 for the second and so on. It's all good stuff when you get it going, and not only does it make things look much neater, but it also possesses an advantage when it comes to future-proofing too. Web addresses can be kept constant over time, even if things change behind the scenes. It means that any returning visitors will find what they saw the last time that they visited and surely must ensure good karma in the eyes of those all important search engines.