TOPIC: SEARCH ENGINE OPTIMIZATION
An unseen arsenal: How web developers can use specialised tools to build better websites
4th March 2026
Modern web development takes place within an ecosystem of tools so precisely suited to individual tasks that they often go unnoticed by anyone outside the profession. These utilities, spanning performance analysers, security checkers and colour palette generators, form the backbone of a workflow that must balance speed, security and visual consistency. For an industry where user experience and technical efficiency are inseparable priorities, such tools are far from optional luxuries.
Performance Testing and Page Speed Analysis
The first hurdle most developers encounter is performance measurement, and several tools have established themselves as essential in this space. GTmetrix, Google PageSpeed Insights and WebPageTest each draw on Google's open-source Lighthouse framework to varying degrees, though each approaches the task differently.
A performance grade alongside separate scores for page speed and structural quality is what GTmetrix produces for any URL submitted to it. It measures Core Web Vitals, including Largest Contentful Paint (LCP), Total Blocking Time (TBT) and Cumulative Layout Shift (CLS), which are the same metrics Google uses as ranking signals in search. The tool can run tests from multiple global server locations and simulates a real browser loading your page, producing a waterfall chart and a video replay of the load process, so developers can identify precisely which elements are causing delays.
Maintained directly by Google, PageSpeed Insights analyses pages against both laboratory data generated through Lighthouse and real-world field data drawn from the Chrome User Experience Report (CrUX). It provides separate performance scores for mobile and desktop, which is significant given that Google confirmed page speed as a ranking factor for mobile searches in July 2018. Both GTmetrix and PageSpeed Insights go well beyond raw figures, mapping out a prioritised list of optimisations so that developers can address the most impactful issues first.
A different position in the toolkit is occupied by WebPageTest, originally created by Patrick Meenan and open-sourced in 2008, and acquired by Catchpoint in 2020. Rather than returning a simple score, it runs tests from a choice of locations across the globe using real browsers at actual connection speeds, and produces detailed waterfall charts that break down every individual network request. This makes it the tool of choice when the question is not just how fast a page is, but precisely why a particular element is slow.
One of the longer-established names in website speed testing, Pingdom offers a free tool that remains widely used for its accessible reporting. Tests can be run from seven global server locations, and results are presented in four sections: a waterfall breakdown, a performance grade, a page analysis and a historical record of previous tests. The page analysis breaks down asset sizes by domain and content type, which is useful for comparing the weight of CDN-served assets against those served directly. Pingdom is based on the YSlow open-source project and does not currently measure the Core Web Vitals metrics that Google uses as ranking signals, so it is best treated as a quick and readable first pass rather than a definitive audit.
Security and Infrastructure Diagnostics
Performance alone cannot sustain a trustworthy website, as a misconfigured certificate, an insecure resource or a flagged IP address can each undermine user confidence and search visibility. One of the most frustrating post-migration problems is the disappearance of the HTTPS padlock despite an SSL certificate being in place, and Why No Padlock? exists specifically to address it. The cause is almost always mixed content, where a page served over HTTPS loads at least one resource (an image, a script or a stylesheet) over plain HTTP. Why No Padlock? scans any HTTPS URL and returns a list of every insecure resource found, along with the HTML element responsible, making it straightforward to trace and resolve the problem. Google has used HTTPS as a ranking signal since 2014, so unresolved mixed content issues carry an SEO cost as well as a security one.
For traffic-level threats, AbuseIPDB operates as a community-maintained IP blacklist. Managed by Marathon Studios Inc., the project allows system administrators and webmasters to report IP addresses involved in malicious behaviour, including hacking attempts, spam campaigns, DDoS attacks and phishing, and to check any IP address against the database before acting on traffic from it. A free API is available for integration with server tools such as Fail2Ban, enabling automatic reporting and real-time checks.
Bot traffic and automated form submissions are a persistent nuisance for any site that accepts user input, and hCaptcha addresses this by presenting challenges that are straightforward for human visitors but reliably difficult for automated scripts. Operated by Intuition Machines, it positions itself as a privacy-focused alternative to reCAPTCHA, collecting minimal data and retaining no personally identifiable information beyond what is necessary to complete a challenge. It is compliant with GDPR, CCPA and several other international privacy frameworks, and holds both ISO 27001 and SOC 2 Type II certifications. A free tier is available, with a Pro plan covering 100,000 evaluations per month, and an Enterprise tier offering additional controls including data localisation and zero-PII processing modes.
Red Sift offers two distinct products that address different aspects of infrastructure security, both relevant to the day-to-day operation of a website. Red Sift OnDMARC automates the configuration and monitoring of DMARC, SPF, DKIM, BIMI and MTA-STS, which are the protocols that collectively prevent attackers from sending spoofed emails that appear to originate from a legitimate domain. This is the basis for most phishing and business email compromise (BEC) attacks, and OnDMARC guides teams to full enforcement typically within six to eight weeks. Red Sift Certificates Lite addresses a separate but equally critical concern, monitoring SSL/TLS certificates for upcoming expiry and alerting administrators seven days ahead of time. It is free for up to 250 certificates and has been formally recommended by Let's Encrypt as its preferred monitoring service, following the retirement of Let's Encrypt's own expiry notification emails. The product was built on the foundation of Hardenize, which Red Sift acquired in 2022, a company founded by Ivan Ristić, creator of SSL Labs.
Colour Management and Visual Design
A website's visual coherence depends heavily on colour consistency, and the distance between a palette sketched on paper and one that functions in code can be significant. With over two million active users, Coolors is a fast and intuitive palette generator built around a simple interaction: pressing the space bar produces a new five-colour palette derived from colour theory algorithms. The platform includes an accessibility checker that calculates contrast ratios against WCAG standards and a colour extractor that derives palettes from uploaded photographs. It also offers interoperability with Figma, Adobe Creative Suite and the Chrome browser. A free tier is available, with a Pro plan at approximately $3 per month for unlimited saving and export options.
A quite different approach is taken by Colormind, which uses a deep learning model based on Generative Adversarial Networks (GANs) to generate harmonious colour schemes. The model is trained on datasets drawn from photographs, films, popular art and website designs, and is updated daily with fresh material. A particularly useful feature allows users to preview how a generated palette would look applied to a website layout, which is a more direct test of practicality than viewing swatches in isolation. A REST API is available for personal and non-commercial use. For converting between colour formats, tools such as Color-Hex, RGBtoHex and the WebFX Hex to RGB converter bridge the gap between design decisions and code implementation, translating colour values in both directions between the hexadecimal and RGB formats that CSS requires.
Optimisation and Code Utilities
Lean, efficient code is a direct contributor to load speed, and unused CSS is a surprisingly common source of unnecessary page weight that PurifyCSS Online addresses by scanning a website's HTML and JavaScript source against its stylesheets to identify selectors that are never used. CSS frameworks such as Bootstrap or Tailwind ship with many utility classes, and most websites use only a small fraction of them. Removing the unused rules can reduce stylesheet file size substantially, which in turn shortens the time a browser spends processing styles before rendering a page. The online version requires no build pipeline or command-line tools, making it accessible to developers at any workflow stage.
Image compression is equally important, as unoptimised images are among the most common causes of slow load times. ImageCompressor handles JPEG, PNG, WebP, GIF and SVG files in the browser, applying lossy or lossless algorithms with adjustable quality settings to reduce file sizes without visible degradation, and processes everything locally, which means that no images are uploaded to an external server. Contact forms and directory listings on websites are a persistent target for spam harvesters, and Email Obfuscator encodes email addresses into a format that is readable by browsers but opaque to most automated scrapers, generating both a plain HTML entity version and a JavaScript-dependent alternative for stronger protection.
For websites that publish mathematical or scientific content, QuickLaTeX provides a practical solution to embedding equations in web pages without a local LaTeX installation. Authors write standard LaTeX expressions directly in their content, and the service renders them as high-quality images that are cached and returned via URL for embedding. Its companion WordPress plugin, WP QuickLaTeX, handles this process automatically within the editor, supporting inline formulas, numbered displayed equations and TikZ graphics.
Server Response and Infrastructure Monitoring
Infrastructure performance sits beneath the layer that most visitors ever see, yet it determines how quickly any content reaches a browser at all, and the Time to First Byte (TTFB) is the metric that captures this most directly. It measures the interval between a browser sending an HTTP request and receiving the first byte of data from the server, and ByteCheck exists solely to measure it. This metric captures the combined effect of DNS resolution time, TCP connection time, SSL negotiation time and server processing time. Google considers a TTFB of 200ms or below to be good, and Byte Check breaks the total down into each constituent step, so developers can identify precisely where delays are occurring. Slow TTFB is often a server-side issue, such as inadequate caching, an overloaded database or a lack of a content delivery network (CDN).
Analytics and Content Evaluation
The final layer of tooling concerns understanding what content a site serves and how it performs in context. Dandelion is a natural language processing API developed by SpazioDati that can extract entities, classify text and analyse the semantic content of web pages, which has applications in content tagging, SEO auditing and editorial quality control. A free tier, covering up to 1,000 API units per day, is available without a credit card, making it accessible for developers who need semantic analysis at low to moderate volume.
Quiet Workhorses of the Web
Individually, each of these tools addresses a specific and well-defined problem. Taken together, they form a coherent toolkit that covers the full lifecycle of a web project, from initial performance diagnosis through to deployment of a secure, efficiently coded and visually consistent site. They do not replace professional judgement but extend it, handling time-consuming checks and conversions that would otherwise consume the attention needed for more complex work. As websites grow in complexity and user expectations continue to rise, familiarity with this kind of specialist tooling becomes a practical necessity rather than an optional extra.
An AI email newsletter roundup: Cutting through the noise
23rd August 2025
This time last year, I felt out of the loop on all things AI. That was put to rights during the autumn when I experimented a lot with GenAI while enhancing travel content on another portal. In addition, I subscribed to enough email newsletters that I feel the need to cull them at this point. Maybe I should use a service like Kill the Newsletter to consolidate things into an RSS feed instead; that sounds like an interesting option for dealing with any overload.
So much is happening in this area that it is too easy to feel overwhelmed by what is happening. That sense got me compiling the state of things in a previous post using some help from GenAI, though I was making the decisions about what was being consolidated and how it was being done. The whole process took a few hours, an effort clearly beyond a single button push.
This survey is somewhat eclectic in its scope; two of the newsletters are hefty items, while others include brevity as part of their offer. Regarding the latter, I found strident criticism of some of them (The Rundown and Superhuman are two that are mentioned) in an article published in the Financial Times, which is behind a paywall. Their content has been called slop, with the phrase slopaganda being coined and used to describe this. That cannot be applied everywhere, though. Any brevity cannot cloak differences in tone and content choices can help with developing a more rounded view of what is going on with AI.
This newsletter came to my notice because I attended SAS Innovate on Tour 2025 in London last June. Oliver Patel, who authors this and serves as Enterprise AI Governance Lead at AstraZeneca as well as contributing to various international organisations including the OECD Expert Group on AI Risk and Accountability, was a speaker with the theme of his talk naturally being AI governance as well as participating in an earlier panel on the day. Unsurprisingly, the newsletter also got a mention.
It provides in-depth practical guidance on artificial intelligence governance and risk management for professionals working in enterprise environments, though not without a focus on scaling governance frameworks across organisations. Actionable insights are emphasised in place of theoretical concepts, covering areas such as governance maturity models that progress from nascent stages through to transformative governance, implementation strategies and leadership approaches needed to drive effective AI governance within companies.
Patel brings experience from roles spanning policy work, academia and privacy sectors, including positions with the UK government and University College London, which informs his practical approach to helping organisations develop robust AI governance structures. The newsletter targets AI governance professionals, risk managers and executives who need clear, scalable solutions for real-world implementation challenges, and all content remains freely accessible to subscribers.
Unlike other newsletters featured here, this is a seven-day publication that delivers a five‑minute digest on AI industry happenings each day that combines news, productivity tips, polls and AI‑generated art. It was launched in June 2023 by Matt Village and Adam Biddlecombe, using of beehiiv’s content‑focused platform that was acquired by HubSpot in March 2025, placing it within the HubSpot Media Network.
Created by Zain Kahn and based in Toronto, weekday issues of this newsletter typically follow a structured format featuring three AI tools for productivity enhancement, two significant AI developments and one quick tutorial to develop practical skills. On Saturdays, there is a round-up on what is happening in robotics, while the Sunday issue centres on developments in science. Everything is crafted to be brief, possibly allowing a three-minute survey of latest developments.
The Artificially Intelligent Enterprise
My interest in the world of DevOps led me to find out about Mark Hinkle, the solopreneur behind Peripety Labs and his in-depth weekly newsletter published every Friday that features comprehensive deep dives into strategic trends and emerging technologies. This has been complemented by a shorter how-to version which focusses on concrete AI lessons and implementation tips and comes out every Tuesday, taking forward a newsletter acquired from elsewhere. The idea is that we should concentrate on concrete AI lessons and implementation tips in place of hype, particularly in business settings. These forms part of The AIE Network alongside complementary publications including AI Tangle, AI CIO and AI Marketing Advantage.
Found though my following the Artificially Intelligent Enterprise, this daily newsletter delivers artificial intelligence developments and insights within approximately five minutes of reading time per issue. Published by Rowan Cheung, it covers key AI developments, practical guides and tool recommendations, with some articles spanning technology and robotics categories. Beyond the core newsletter, the platform operates AI University, which provides certificate courses, implementation guides, expert-led workshops and community networking opportunities for early adopters.
Building a sitemap in XML
24th November 2022
While there are many tools that will build XML site maps, there is some satisfaction to be had in creating your own. This is despite there being a multitude of search engine optimisation plugins for content management systems like WordPress or what is built into static site generators like Hugo. Sometimes, building your own allows for added simplicity, and that is shared with recent efforts in WordPress theme development.
The sitemap XML protocol is simple enough to offer a short coding project. The basis was what Hugo generates, and I used Python to create the XML files. The only libraries that I needed were configparser, SQLAlchemy and pandas. The first two of these allowed databases to be queried, and the last on the list was used for data processing. Otherwise, it was a case of using what is built into the Python language, like file writing and looping.
Once the scripts were ready, they could be uploaded to web servers and executed by scheduled jobs using CRON to keep things up to date. Along the way, I also uncovered a way to publicise the locations of the sitemap files to search engine bots using robots.txt. The structure of the instruction is the following:
User-agent: *
Sitemap: sitemap.xml
This means that it announces to all bots the location of the sitemap file. In my case, I always included the full URL for the XML file, and that clearly varies by website location.
The wonders of mod_rewrite
24th June 2007
When I wrote about tidying dynamic URL's a little while back, I had no inkling that that would be a second part to the tale. My discovery of mod_rewrite, an Apache module that facilitates URL translation. The effect is that one URL is presented to the user in the browser address bar, and the very same URL is also seen by search engines, while another is passed to the server for processing. Though it might sound like subterfuge, it works very well once you manage to get it set up properly. While the web host for my hillwalking blog/photo gallery has everything configured such that it is ready to go, the same did not apply to the offline Apache 2.2.x server that I have going on my own Windows XP box. There were two parts to getting it working there:
- Activating mod-rewrite on the server: this is as easy as uncommenting a line in the
httpd.conffile for the site (the line in question is:LoadModule rewrite_module modules/mod_rewrite.so). - Ensuring that the
.htaccessfile in the root of the web server directory is active. You need to set the values of theAllowOverridedirectives for the server root and CGI directories toAllso that.htaccessis active. Not doing it for the latter will result in an error beginning with the following:Options FollowSymLinks or SymLinksIfOwnerMatch is off which implies that. HavingRewriteRuledirective is forbiddenAllow from Allset for the required directories is another option to consider when you see errors like that.
Once you have got the above sorted, add this line to .htaccess: RewriteEngine On. Preceding it with an Options directive to ensure that FollowSymLinks and SymLinksIfOwnerMatch are switched on does no harm at all and may even be needed to get things running. That done, you can set about putting mod_write to work with lines like this:
RewriteRule ^pages/(.*)/?$ pages.php?query=$1
The effect of this is to take http://www.website.com/pages/input and convert it into a form for action by the server; in this case, that is http://www.website.com/pages.php?query=input. Anything contained by a bracket is assigned to the value of a system-named variable. If you have several bracketed sections, they are assigned to sequentially numbered variables as follows: $1 for the first, $2 for the second and so on. It's all good stuff when you get it going, and not only does it make things look much neater, but it also possesses an advantage when it comes to future-proofing too. Web addresses can be kept constant over time, even if things change behind the scenes. It means that any returning visitors will find what they saw the last time that they visited and surely must ensure good karma in the eyes of those all important search engines.