TOPIC: COMPUTING
Open Source Tools for Pharmaceutical Clinical Data Reporting, Analysis & Regulatory Submissions
25th March 2026
There was a time when SAS was the predominant technology for clinical data reporting, analysis and submission work in the pharmaceutical industry. Within the last decade, open-source alternatives have gained a lot of traction, and the {pharmaverse} initiative has arisen from this. The range of packages ranges from dataset creation (SDTM and ADaM) to output production, with utilities for test data and submission activities. All in all, there is quite a range here. The effort also is a marked change from each company working by itself to sharing and collaborating with others. Here then is the outcome of their endeavours.
{admiral}
Designed as an open-source, modular R toolbox, the {admiral} package assists in the creation of ADaM datasets through reusable functions and utilities tailored for pharmaceutical data analysis. Core packages handle general ADaM derivations whilst therapeutic area-specific extensions address more specialised needs, with a structured release schedule divided into two phases. Usability, simplicity and readability are central priorities, supported by comprehensive documentation, vignettes and example scripts. Community contributions and collaboration are actively encouraged, with the aim of fostering a shared, industry-wide approach to ADaM development in R. Related packages for test data and metadata manipulation complement the main toolkit, alongside a commitment to consistent coding practices and accessible code.
{aNCA}
Maintained by contributors from F. Hoffmann-La Roche AG, {aNCA} is an open-source R Shiny application that makes Non-Compartmental Analysis (NCA) accessible to scientists working with clinical and pre-clinical pharmacokinetic datasets. Users can upload their own data, apply pre-processing filters and run NCA with configurable options including half-life calculation rules, manual slope selection and user-defined AUC intervals. Results are explorable through interactive box plots, scatter plots and summary statistics tables, and can be exported in `PP` and `ADPP` dataset domains alongside a reproducible R script. Analysis settings can be saved and reloaded for continuity across sessions. Installation is available from CRAN via a standard install command, from GitHub using the `pak` package manager, or by cloning the repository directly for those wishing to contribute.
{autoslider.core}
The {autoslider.core} package generates standard table templates commonly used in Study Results Endorsement Plans. Its principal purpose is to reduce duplicated effort between statisticians and programmers when creating slides. Available on CRAN, the package can be installed either through the standard installation method or directly from GitHub for the latest development version.
{cards}
Supporting the CDISC Analysis Results Standard, the {cards} package facilitates the creation of analysis results data sets that enhance automation, reproducibility and consistency in clinical research. Structured data sets for statistical summaries are generated to enable tasks such as quality control, pre-calculating statistics for reports and combining results across studies. Tools for creating, modifying and analysing these data sets are provided, with the {cardx} extension offering additional functions for statistical tests and models. Installation is available through CRAN or GitHub, with resources including documentation and community contributions.
{cardx}
Extending the {cards} package, {cardx} facilitates the creation of Analysis Results Data Objects (ARD's) in R by leveraging utility functions from {cards} and statistical methods from packages such as {stats} and {emmeans}. These ARD's enable the generation of tables and visualisations for regulatory submissions, support quality control checks by storing both results and parameters, and allow for reproducible analyses through the inclusion of function inputs. Installation options include CRAN and GitHub, with examples demonstrating its use in t-tests and regression models. External statistical library dependencies are not enforced by the package, requiring explicit references in code for tools like {renv} to track them.
{chevron}
A collection of high-level functions for generating standardised outputs in clinical trials reporting, {chevron} covers a broad range of output types including tables for safety summaries, adverse events, demographics, ECG results, laboratory findings, medical history, response data, time-to-event analyses and vital signs, as well as listings and graphs such as Kaplan-Meier and mean plots. Straightforward implementation with limited parameterisation is a defining characteristic of the package. It is available on CRAN, with a development version accessible via GitHub, and those requiring greater flexibility are directed to the related {tern} package and its associated catalogue.
{clinify}
Built on the {flextable} and {officer} packages, {clinify} streamlines the creation of clinical tables, listings and figures whilst addressing challenges such as adherence to organisational reporting standards, the need for flexibility across different clients and the importance of reusable configurations. Compatibility with existing tools is a key priority, ensuring that its features do not interfere with the core functionalities of {flextable} or {officer}, whilst enabling tasks like dynamic page breaks, grouped headers and customisable formatting. Complex documents such as Word files with consistent layouts and tailored elements like footnotes and titles can be produced with reduced effort by building on these established frameworks.
{connector}
Offering a unified interface for establishing connections to various data sources, the {connector} package covers file systems and databases through a central configuration file that maintains consistent references across project scripts and facilitates switching between data sources. Functions such as connector_fs() for file system access and connector_dbi() for database connections are provided, with additional expansion packages enabling integration with specific platforms like Databricks and SharePoint. Installation is available via CRAN or GitHub, and usage involves defining a YAML configuration file to specify connection details that can then be initialised and utilised to interact with data sources. Operations including reading, writing and listing content are supported, with methods for managing connections and handling data in formats like parquet.
{covtracer}
Linking test traces to package code and documentation using coverage data from {covr}, the {covtracer} package enables the creation of a traceability matrix that maps tests to specific documented functions. Installation is via remotes from GitHub with specific dependencies, and configuration of {covr} is required to record tests alongside coverage traces. Untested behaviours can be identified and the direct testing of functions assessed, providing insights into test coverage and software validation. The example workflow demonstrates generating a matrix to show which tests evaluate code related to documented behaviours, highlighting gaps in test coverage.
{datacutr}
An open-source solution for applying data cuts to SDTM datasets within R, the {datacutr} package is designed to support pharmaceutical data analysis workflows. Available via CRAN or GitHub, it offers options for different types of cuts tailored to specific SDTM domains. Supplemental qualifiers are assumed to be merged with their parent domain before processing, allowing users flexibility in defining cut types such as patient, date, or domain-specific cuts. Documentation, contribution guidelines and community support through platforms like Slack and GitHub provide further assistance.
{datasetjson}
Facilitating the creation and manipulation of CDISC Dataset JSON formatted datasets, the {datasetjson} R package enables users to generate structured data files by applying metadata attributes to data frames. Metadata such as file paths, study identifiers and system details can be incorporated into dataset objects and written to disk or returned as JSON text. Reading JSON files back into data frames is also supported, with metadata preserved as attributes for use in analysis. The package currently supports version 1.1.0 of the Dataset JSON standard and is available via CRAN or GitHub.
{dataviewR}
An interactive data viewer for R, {dataviewR} enhances data exploration through a Shiny-based interface that enables users to examine data frames and tibbles with tools for filtering, column selection and generating reproducible {dplyr} code. Viewing multiple datasets simultaneously is supported, and the tool provides metadata insights alongside features for importing and exporting data, all within a responsive and user-friendly design. By combining intuitive navigation with automated code generation, the package aims to streamline data analysis workflows and improve the efficiency of dataset manipulation and documentation.
{docorator}
Generating formatted documents by adding headers, footers and page numbers to displays such as tables and figures, {docorator} exports outputs as PDF or RTF files. Accepted inputs include tables created with the {gt} package, figures generated using {ggplot2}, or paths to existing PNG files, and users can customise document elements like titles and footers. The package can be installed from CRAN or via GitHub, and its use involves creating a display object with specified formatting options before rendering the output. LaTeX libraries are required for PDF generation.
{envsetup}
Providing a configuration system for managing R project environments, the {envsetup} package enables adaptation to different deployment stages such as development, testing and production without altering code. YAML files are used to define paths for data and output directories, and R scripts are automatically sourced from specified locations to reduce the need for manual configuration changes. This approach supports consistent code usage across environments whilst allowing flexibility in environment-specific settings, streamlining workflows for projects requiring multiple deployment contexts.
{ggsurvfit}
Simplifying the creation of survival analysis visualisations using {ggplot2}, the {ggsurvfit} package offers tools to generate publication-ready figures with features such as confidence intervals, risk tables and quantile markers. Seamless integration with {ggplot2} functions allows for extensive customisation of plot elements whilst maintaining alignment between graphical components and annotations. Competing risks analysis is supported through `ggcuminc()`, and specific functions such as Surv_CNSR() handle CDISC ADaM `ADTTE` data by adjusting event coding conventions to prevent errors. Installation options are available via CRAN or GitHub, with examples and further resources accessible through its documentation and community links.
{gridify}
Addressing challenges in creating consistent and customisable graphical arrangements for figures and tables, the {gridify} package leverages the base {grid} package to facilitate the addition of headers, footers, captions and other contextual elements through predefined or custom layouts. Multiple input types are supported, including {ggplot2}, {flextable} and base R plots, and the workflow involves generating an object, selecting a layout and using functions to populate text elements before rendering the final output. Installation options include CRAN and GitHub, with examples demonstrating its application in enhancing tables with metadata and formatting. Uniformity across different projects is promoted, reducing manual adjustments and aligning visual elements consistently.
{gtsummary}
Offering a streamlined approach to generating publication-quality analytical and summary tables in R, the {gtsummary} package enables users to summarise datasets, regression models and other statistical outputs with minimal code. Variable types are identified automatically, relevant descriptive statistics computed and measures of data incompleteness included, whilst customisation of table formatting such as adjusting labels, adding p-values or merging tables for comparative analysis is also supported. Integration with packages like {broom} and {gt} facilitates the creation of visually appealing tables, and results can be exported to multiple formats including HTML, Word and LaTeX, making the package suitable for reproducible reporting in academic and professional contexts.
{logrx}
Supporting logging in clinical programming environments, the {logrx} package generates detailed logs for R scripts, ensuring code execution is traceable and reproducible. An overview of script execution and the associated environment is provided, enabling users to recreate conditions for verification or further analysis. Available on CRAN, installation is possible via standard methods or from its development repository, offering flexibility for both file-based and scripted usage. Structured logging tailored to the specific requirements of clinical applications is the defining characteristic of the package, with simplicity and minimal intrusion in coding workflows maintained throughout.
{metacore}
Providing a standardised framework for managing metadata within R sessions, the {metacore} package is particularly suited to clinical trial data analysis. Metadata is organised into six interconnected tables covering dataset specifications, variable details, value definitions, derivations, code lists and supplemental information, ensuring consistency and ease of access. By centralising metadata in a structured, immutable format, the package facilitates the development of tools that can leverage this information across different workflows, reducing the need for redundant data structures. Reading metadata from various sources, including Define-XML 2.0, is also supported.
{metatools}
Working with {metacore} objects, {metatools} enables users to build datasets, enhance columns in existing datasets and validate data against metadata specifications. Installation is available from CRAN or via GitHub. Core functionality includes pulling columns from existing datasets, creating new categorical variables, converting columns to factors and running checks to verify that data conforms to control terminology and that all expected variables are present.
{pharmaRTF}
Developed to address gaps in RTF output capabilities within R, {pharmaRTF} is a package for pharmaceutical industry programmers who produce RTF documents for clinical trial data analysis. Whilst the {huxtable} package offers extensive RTF styling and formatting options, it lacks the ability to set document properties such as page size and orientation, repeat column headers across pages, or create multi-level titles and footnotes within document headers and footers. These limitations are resolved by {pharmaRTF}, which wraps around {huxtable} tables to provide document property controls, proper multipage display and title and footnote management within headers and footers. Two core objects form the basis of the package: rtf_doc for document-wide attributes and hf_line for creating individual title and footnote lines, each carrying formatting properties such as alignment, font and bold or italic styling. Default output files use Courier New at 12-point size, Letter page dimensions in landscape orientation with one-inch margins, though all of these can be adjusted through property functions. The package is available on CRAN and supports both a {tidyverse} piping style and a more traditional assignment-based coding approach.
{pharmaverseadam}
Serving as a repository for ADaM test datasets generated by executing templates from related packages such as {admiral} and its extensions, the {pharmaverseadam} package automates dataset creation through a script that installs required packages, runs templates and saves results. Metadata is managed centrally in an XLSX file to ensure consistency in documentation, and updates occur regularly or ad-hoc when templates change. Documentation is generated automatically from metadata and saved as `.R` files, and the package includes contributions from multiple developers with examples provided for each dataset. Preparing metadata, updating configuration files for new therapeutic areas and executing a script to generate datasets and documentation ensures alignment with the latest versions of dependent packages. Installation is available via CRAN or GitHub.
{pharmaverseraw}
Providing raw datasets to support the creation of SDTM datasets, the {pharmaverseraw} package includes examples that are independent of specific electronic data capture systems or data standards such as CDASH. Datasets are named using SDTM domain identifiers with the suffix _raw, and installation options include CRAN or direct GitHub access. Updates involve contributing via GitHub issues, generating new or modified datasets through standalone R scripts stored in the data-raw folder, and ensuring generated files are saved in the data folder as .rda files with consistent naming. Documentation is maintained in R/*.R files, and changes require updating `NAMESPACE` and `.Rd` files using devtools::document.
{pharmaversesdtm}
A collection of test datasets formatted according to the SDTM standard, the {pharmaversesdtm} package is designed for use within the pharmaverse family of packages. Datasets applicable across therapeutic areas, such as `DM` and VS, are included alongside those specific to particular areas, like `RS` and `OE`. Available via CRAN and GitHub, the package provides installation instructions for both stable and development versions, with test data sourced from the CDISC pilot project and ad-hoc datasets generated by the {admiral} team. Naming conventions distinguish between general and therapeutic area-specific categories, with examples such as dm for general use and rs_onco for oncology-specific data. Updates involve creating or modifying R scripts in the data-raw folder, generating `.rda` files and updating metadata in a central JSON file to automate documentation and maintain consistency, including specifying dataset details like labels, descriptions and therapeutic areas.
{pkglite} (R)
Converting R package source code into text files and reconstructing package structures from those files, {pkglite} enables the exchange and management of R packages as plain text. Single or multiple packages can be processed through functions that collate, pack and unpack files, with installation options available via CRAN or GitHub. The tool adheres to a defined format for text files and includes documentation for generating specifications and managing file collections.
{pkglite} (Python)
An open-source framework licensed under the MIT licence, {pkglite} for Python, allows source projects written in any programming language to be packed into portable files and restored to their original directory structure. Installation is available via PyPI or as a development version cloned from GitHub, and the package can also be run without installation using `uvx`. A command line interface is provided in addition to the Python API, which can be installed globally using `pipx`.
{rhino}
Streamlining the development of high-quality, enterprise-grade Shiny applications, {rhino} integrates software engineering best practices, modular code structures and robust testing frameworks. Scalable architecture is supported through modularisation, code quality is enhanced with unit and end-to-end testing, and automation is facilitated via tools for project setup, continuous integration and dependency management. Comprehensive documentation is divided into tutorials, explanations and guides, with examples and resources available for learning.
{risk.assessr}
Evaluating the reliability and security of R packages during validation, the {risk.assessr} package analyses maintenance, documentation and dependencies through metrics such as R CMD check results, unit test coverage and dependency assessments. A traceability matrix linking functions to tests is generated, and risk profiles are based on predefined thresholds including documentation completeness, licence type and code coverage. The tool supports installation from GitHub or CRAN, processes local package files or `renv.lock` dependencies and offers detailed outputs such as risk analysis, dependency lists and reverse dependency information. Advanced features include identifying potential issues in suggested package dependencies and generating HTML reports for risk evaluation, with applications in clinical trial workflows and package validation processes.
{riskassessment}
Built on the {riskmetric} framework, the {riskassessment} application offers a user-friendly interface for evaluating the risk of using R packages within regulated industries, assessing development practices, documentation and sustainability. Non-technical users can review {riskmetric} outputs, add personalised comments, categorise packages into risk levels, generate reports and store assessments securely, with features such as user authentication and role-based access. Alignment with validation principles outlined by the R Validation Hub supports decision-making in regulated settings, though deeper software inspection may be required in some cases. Deployment is possible using tools like Shiny Server or Posit Connect, with installation options including GitHub and local configuration via {renv}.
{riskmetric}
Providing a framework for evaluating the quality of R packages, the {riskmetric} package assesses development practices, documentation, community engagement and sustainability through a series of metrics. Currently operating in a maintenance-only phase, further development is focused on a new tool called {val.metre}. The workflow involves retrieving package information, assessing it against predefined criteria and generating a risk score, with installation available from CRAN or GitHub. An associated application, {riskassessment}, offers a user interface for organisations to review and manage package risk assessments, store metrics and apply organisational rules.
{rlistings}
Designed to create and display formatted listings with a focus on ASCII rendering for tables and regulatory-ready outputs, the {rlistings} R package relies on the {formatters} package for formatting infrastructure. Requirements such as flexible pagination, multiple output formats and repeated key columns informed its development. Available on CRAN and GitHub, the package is under active development and includes features such as adjustable column widths, alignment and support for titles and footnotes.
{rtables}
Tailored for generating submission-ready tables for health authority review, the {rtables} R package creates and displays complex tables with advanced formatting and output options that support regulatory requirements for clinical trial data presentation. Separation of data values from their visualisation is enabled, multiple values can be included within cells, and flexible tabulation and formatting capabilities are provided, including cell spans, rounding and alignment. Output formats include HTML, ASCII, LaTeX, PDF and PowerPoint, with additional formats under development. Also, the package incorporates features such as pagination, distinction between data names and labels for CDISC standards and support for titles and footnotes. Installation is available via CRAN or GitHub, with ongoing community support and training resources.
{rtflite}
A lightweight Python library focused on precise formatting of production-quality tables and figures, {rtflite} is designed for composing RTF documents. Installation is available via PyPI or directly from its GitHub repository, with optional dependencies available to enable DOCX assembly support and RTF-to-PDF or RTF-to-DOCX conversion via LibreOffice.
{sdtm.oak}
Offering a modular, open-source solution for generating CDISC SDTM datasets, the {sdtm.oak} R package is designed to work across different electronic data capture systems and data standards. Industry challenges related to inconsistent raw data structures and varying data collection practices are addressed through reusable algorithms that map raw datasets to SDTM domains, with current capabilities covering Findings, Events and Intervention classes. Future developments aim to expand domain support, introduce metadata-driven code generation and enhance automation potential, though sponsor-specific metadata management tasks are not yet handled by the package. Available on CRAN and GitHub, development is ongoing with refinements based on user feedback and evolving SDTM requirements.
{sdtmchecks}
Providing functions to detect common data issues in SDTM datasets, the {sdtmchecks} package is designed to be broadly applicable and useful for analysis. Installation is available from CRAN or via GitHub, with development versions accessible through specific repositories, and users are not required to specify SDTM versions. A range of data check functions stored as R scripts is included, and contributions are encouraged that maintain flexibility across different data standards.
{siera}
Facilitating the generation of Analysis Results Datasets (ARD's) by processing Analysis Results Standard (ARS) metadata, the {siera} package works with parameters such as analysis sets, groupings, data subsets and methods. Metadata is typically provided in JSON format and used to create R scripts automatically that, when executed with corresponding ADaM datasets, produce ARD's in a structured format. The package can be installed from CRAN or GitHub, and its primary function, `readARS`, requires an ARS file, an output directory and access to relevant ADaM data. The CDISC Analysis Results Standard underpins this process, promoting automation and consistency in analysis outcomes.
{teal}
An open-source, Shiny-based interactive framework for exploratory data analysis, {teal} is developed as part of the pharmaverse ecosystem and maintained by F. Hoffmann-La Roche AG alongside a broad community of contributors. Analytical applications are built by combining supported data types, including CDISC clinical trial data, independent or relational datasets and `MultiAssayExperiment` objects, with modular analytical components known as teal modules. These modules can be drawn from dedicated packages covering general data exploration, clinical reporting and multi-omics analysis and define the specific analyses presented within an application. A suite of companion packages handles logging, reproducibility, data loading, filtering, reporting and transformation. The package is available on CRAN and is under active development, with community support provided through the {pharmaverse} Slack workspace.
{tern}
Supporting clinical trial reporting through a broad range of analysis functions, the {tern} R package offers data visualisation capabilities including line plots, Kaplan-Meier plots, forest plots, waterfall plots and Bland-Altman plots. Statistical model fit summaries for logistic and Cox regression are also provided, along with numerous analysis and summary table functions. Many of these outputs can be integrated into interactive Teal Shiny applications via the {teal.modules.clinical} package.
{tfrmt}
Offering a structured approach to defining and applying formatting rules for data displays in clinical trials, the {tfrmt} package streamlines the creation of mock displays, aligns with industry-standard Analysis Results Data (ARD) formats and integrates formatting tasks into the programming workflow to reduce manual effort and rework. Metadata is leveraged to automate styling and layout, enabling standardised formatting with minimal code, supporting quality control before final output and facilitating the reuse of datasets across different table types. Built on the {gt} package, the tool provides a flexible interface for generating tables and mock-ups, allowing users to focus on data interpretation rather than repetitive formatting tasks.
{tfrmtbuilder}
A tool for defining display-related metadata to streamline the creation and modification of table formats, the {tfrmtbuilder} package supports workflows such as generating tables from scratch, using templates or editing existing ones. Features include a toggle to switch between mock and real data, options to load or create datasets, tools for mapping and formatting data and the ability to export results as JSON, HTML or PNG. Designed for use in study planning and analysis phases, the package allows users to manage table structures efficiently.
{tidyCDISC}
An open-source R Shiny application, {tidyCDISC} is designed to help clinical personnel explore and analyse ADaM-standard data sets without writing any code. Customised clinical tables can be generated through a point-and-click interface, trends across patient populations examined using dynamic figures and individual patient profiles explored in detail. A broad range of users is served, from clinical heads with no programming background to statisticians and statistical programmers, with reported time savings of around 95% for routine trial analysis tasks. The app accepts only `sas7bdat` files conforming to CDISC ADaM standards and includes a feature to export reproducible R scripts from its table generator. A demo version is available without installation using CDISC pilot data, whilst uploading study data requires installing the package from CRAN or via GitHub.
{tidytlg}
Facilitating the creation of tables, listings and graphs using the {tidyverse} framework, the {tidytlg} package offers two approaches: a functional method involving custom scripts for each output and a metadata-driven method that leverages column and table metadata to generate results automatically. Tools for data analysis, including frequency tables and univariate statistics, are included alongside support for exporting outputs to formatted documents.
{Tplyr}
Simplifying the creation of clinical data summaries by breaking down complex tables into reusable layers, {Tplyr} allows users to focus on presentation rather than repetitive data processing. The conceptual approach of {dplyr} is mirrored but applied to common clinical table types, such as counting event-based variables, generating descriptive statistics for continuous data and categorising numerical ranges. Metadata is included with each summary produced to ensure traceability from raw data to final output, and user-acceptance testing documentation is provided to support its use in regulated environments. Installation options are available via CRAN or GitHub, accompanied by detailed vignettes covering features like layer templates, metadata extension and styled table outputs.
{valtools}
Streamlining the validation of R packages used in clinical research and drug development, {valtools} offers templates and functions to support tasks such as setting up validation frameworks, managing requirements and test cases and generating reports. Developed by the R Package Validation Framework PHUSE Working Group, the package integrates with standard development tools and provides functions prefixed with `vt` to facilitate structured validation processes including infrastructure setup, documentation creation and automated checks. Generating validation reports, scraping metadata from validation configurations and executing validation workflows through temporary installations or existing packages are all supported.
{whirl}
Facilitating the execution of scripts in batch mode whilst generating detailed logs that meet regulatory requirements, the {whirl} package produces logs including script status, execution timestamps, environment details, package versions and environmental variables, presented in a structured HTML format. Individual or multiple scripts can be run simultaneously, with parallel processing enabled through specified worker counts. A configuration file allows scripts to be executed in sequential steps, ensuring dependencies are respected, and the package produces individual logs for each script alongside a summary log and a tibble summarising execution outcomes. Installation options include CRAN and GitHub, with documentation available for customisation and advanced usage.
{xportr}
Assisting clinical programmers in preparing CDISC compliant XPT files for clinical data sets, the {xportr} package associates metadata with R data frames, performs validation checks and converts data into transportable SAS v5 XPT format. Tools are included to define variable types, set appropriate lengths, apply labels, format data, reorder variables and assign dataset labels, ensuring adherence to standards such as variable naming conventions, character length limits and the absence of non-ASCII characters. A practical example demonstrates how to use a specification file to apply these transformations to an ADSL dataset, ultimately generating a compliant XPT file.
Some R packages to explore as you find your feet with the language
24th March 2026
Here are some commonly used R packages and other tools that are pervasive, along with others that I have encountered while getting started with the language, itself becoming pervasive in my line of business. The collection grew organically as my explorations proceeded, and reflects what I was trying out during my acclimatisation.
General
Here are two general packages to get things started, with one of them being unavoidable in the R world. The other is more advanced, possibly offering more to package developers.
You cannot use R without knowing about this collection of packages. In many ways, they form a mini-language of their own, drawing some criticism from those who reckon that base R functionality covers a sufficient gamut anyway. Nevertheless, there is so much here that will get you going with data wrangling and visualisation that it is worth knowing what is possible. The complaints may come from your not needing to use anything else for these purposes.
This R package enables developers to convert existing R functions into web API endpoints by adding roxygen2-like comment annotations to their code. Once annotated, functions can handle HTTP GET and POST requests, accept query string or JSON parameters and return outputs such as plain values or rendered plots. The package is available on CRAN as a stable release, with a development version hosted on GitHub. For deployment, it integrates with DigitalOcean through a companion package called {plumberDeploy}, and also supports Posit Connect, PM2 and Docker as hosting options. Related projects in the same space include OpenCPU, which is designed for hosting R APIs in scientific research contexts, and the now-discontinued jug package, which took a more programmatic approach to API construction.
Data Preparation
You simply cannot avoid working with data during any analysis or reporting work. While there is a learning curve if you are used to other languages, there is little doubt that R is well-endowed when it comes to performing these tasks. Here are some packages that extend base R capabilities and might even add some extra user-friendliness along the way.
The {forcats} package in R provides functions to manage categorical variables by reordering factor levels, collapsing infrequent values and adjusting their sequence based on frequency or other variables. It includes tools such as reordering by another variable, grouping rare categories into 'other' and modifying level order manually, which are useful for data analysis and visualisation workflows. Designed as part of the tidyverse, it integrates with other packages to streamline tasks like counting and plotting categorical data, enhancing clarity and efficiency in handling factors within R.
Around this time last year, I remember completing a LinkedIn course on a set of good practices known as tidy data, where each variable occupies a column, each observation a row and each value a single cell. This package is designed to help users restructure data so it follows those rules. It provides tools for reshaping data between long and wide formats, handling nested lists, splitting or combining columns, managing missing values and layering or flattening grouped data.
Installation options include the {tidyverse} collection, standalone installation, or the development version from GitHub. The package succeeds earlier reshaping tools like {reshape2} and {reshape}, offering a focused approach to tidying data rather than general reshaping or aggregation.
Having a long track record of working with SAS, {haven} with its abilities to read and write data files from statistical software such as SAS, SPSS and Stata, leveraging the ReadStat library, arouses my interest. Handily, it supports a range of file formats, including SAS transport and data files, SPSS system and older portable files and Stata data files up to version 15, converting these into tibbles with enhanced printing capabilities. Value labels are preserved as a labelled class, allowing conversion to factors, while dates and times are transformed into standard R classes.
While there are other approaches to working with databases using R, {RMariaDB} provides a database interface and driver for MariaDB, designed to fully comply with the DBI specification and serve as a replacement for the older {RMySQL} package. It supports connecting to databases using configuration files, executing queries, reading and writing data tables and managing results in chunks. Installation options include binary packages from CRAN or development versions from GitHub, with additional dependencies such as MariaDB Connector/C or libmysqlclient required for Linux and macOS systems. Configuration is typically handled through a MariaDB-specific file, and the package includes acknowledgments for contributions from various developers and organisations.
For many people, the pandemic may be a fading memory, yet it offered its chances for learning R, not least because there was a use case with more than a hint of personal interest about it. Here is a library making it easier to get hold of the data, with some added pre-processing too. Memories of how I needed to wrangle what was published by various sources make me appreciate just how vital it is to have harmonised data for analysis work.
Table Production
While many appear to graphical presentation of results to their tabular display, R does have its options here too. In recent times, the options have improved, particularly of the pharmaverse initiative. Here is a selection of what I found during my explorations.
Part of the {officeverse} along with {officedown}, {Flextable}, {Rvg} and {mschart}, the {officer} R package enables users to create and modify Word and PowerPoint documents directly from R, allowing the insertion of images, tables and formatted content, as well as the import of document content into data frames. It supports the generation of RTF files and integrates with other packages for advanced features such as vector graphics and native office charts. Installation options include CRAN and GitHub, with community resources available for assistance and contributions. The package facilitates the manipulation of document elements like paragraphs, tables and section breaks and provides tools for exporting and importing content between R and office formats, alongside functions for managing slide layouts and embedded objects in presentations.
If you work in clinical research like I do, the need to produce data tabulations is a non-negotiable requirement. That is how this package came to be developed and the pharmaverse of which it is part has numerous other options, should you need to look at using one of those. The flavour of RTF produced here is the Microsoft Word variety, which did not look as well in LibreOffice Writer when I last looked at the results with that open-source alternative. Otherwise, the results look well to many eyes.
Here is a package that enhances data presentation by applying customisable formatting to vectors and data frames, supporting formats such as percentages, currency and accounting. Available on GitHub and CRAN, it integrates with dynamic document tools like {knitr} and {rmarkdown} to produce visually distinct tables, with features including gradient colour scales, conditional styling and icon-based representations. It automatically converts to {htmlwidgets} in interactive environments and is licensed under MIT, enabling flexible use in both static and interactive data displays.
The {reactable} package for R provides interactive data tables built on the React Table library, offering features such as sorting, filtering, pagination, grouping with aggregation, virtual scrolling for large datasets and support for custom rendering through R or JavaScript. It integrates seamlessly into R Markdown documents and Shiny applications, enabling the use of HTML widgets and conditional styling. Installation options include CRAN and GitHub, with examples demonstrating its application across various datasets and scenarios. The package supports major web browsers and is licensed under MIT, designed for developers seeking dynamic data presentation tools within the R ecosystem.
Particularly useful in dynamic web applications like Shiny, the {DT} package in R provides a means of rendering interactive HTML tables by building on the DataTables JavaScript library. It supports features including sorting, searching, pagination and advanced filtering, with numeric, date and time columns using range-based sliders whilst factor and character columns rely on search boxes or dropdowns. Filtering operates on the client side by default, though server-side processing is also available. JavaScript callbacks can be injected after initialisation to manipulate table behaviour, such as enabling automatic page navigation or adding child rows to display additional detail. HTML content is escaped by default as a safeguard against cross-site scripting attacks, with the option to adjust this on a per-column basis. Whilst the package integrates with Shiny applications, attention is needed around scrolling and slider positioning to prevent layout problems. Overall, the package is well suited to exploratory data analysis and the building of interactive dashboards.
The {gt} package in R enables users to create well-structured tables with a variety of formatting options, starting from data frames or tibbles and incorporating elements such as headers, footers and customised column labels. It supports output in HTML, LaTeX and RTF formats and includes example datasets for experimentation. The package prioritises simplicity for common tasks while offering advanced functions for detailed customisation, with installation available via CRAN or GitHub. Users can access resources like documentation, community forums and example projects to explore its capabilities, and it is supported by a range of related packages that extend its functionality.
Enabling users to produce publication-ready outputs with minimal code, the {gtsummary} package offers a streamlined approach to generating analytical and summary tables in R. It automates the summarisation of data frames, regression models and other datasets, identifying variable types and calculating relevant statistics, including measures of data incompleteness. Customisation options allow for formatting, merging and styling tables to suit specific needs, while integration with packages such as {broom} and {gt} facilitates seamless incorporation into R Markdown workflows. The package supports the creation of side-by-side regression tables and provides tools for exporting results as images, HTML, Word, or LaTeX files, enhancing flexibility for reporting and sharing findings.
Here is an R package designed to generate LaTeX and HTML tables with a modern, user-friendly interface, offering extensive control over styling, formatting, alignment and layout. It supports features such as custom borders, padding, background colours and cell spanning across rows or columns, with tables modifiable using standard R subsetting or dplyr functions. Examples demonstrate its use for creating simple tables, applying conditional formatting and producing regression output with statistical details. The package also facilitates quick export to formats like PDF, DOCX, HTML and XLSX. Installation options include CRAN, R-Universe and GitHub, while the name reflects its origins as an enhanced version of the {xtable} package. The logo was generated using the package itself, and the background design draws inspiration from Piet Mondrian’s artwork.
Figure Generation
R has such a reputation for graphical presentations that it is cited as a strong reason to explore what the ecosystem has to offer. While base R itself is not shabby when it comes to creating graphs and charts, these packages will extend things by quite a way. In fact, the first on this list is near enough pervasive.
Though its default formatting does not appeal to me, the myriad of options makes this a very flexible tool, albeit at the expense of some code verbosity. Multi-panel plots are not among its strengths, which may send you elsewhere for that need.
Focusing on features not included in the core library, the {ggforce} package extends {ggplot2} by offering additional tools to enhance data visualisation. Designed to complement the primary role of {ggplot2} in exploratory data analysis, it provides a range of geoms, stats and other components that are well-documented and implemented, aiming to support more complex and custom plot compositions. Available for installation via CRAN or GitHub, the package includes a variety of functionalities described in detail on its associated website, though specific examples are not included here.
Developed by Claus O. Wilke for internal use in his lab, {cowplot} is an R package designed to help with the creation of publication-quality figures built on top of {ggplot2}. It provides a set of themes, tools for aligning and arranging plots into compound figures and functions for annotating plots or combining them with images. The package can be installed directly from CRAN or as a development version via GitHub, and it has seen widespread use in the book Fundamentals of Data Visualisation.
The {sjPlot} package provides a range of tools for visualising data and statistical results commonly used in social science research, including frequency tables, histograms, box plots, regression models, mixed effects models, PCA, correlation matrices and cluster analyses. It supports installation via CRAN for stable releases or through GitHub for development versions, with documentation and examples available online. The package is licensed under GPL-3 and developed by Daniel Lüdecke, offering functions to create visualisations such as scatter plots, Likert scales and interaction effect plots, along with tools for constructing index variables and presenting statistical outputs in tabular formats.
By offering a centralised approach to theming and enabling automatic adaptation of plot styles within Shiny applications, the {thematic} package simplifies the styling of R graphics, including {ggplot2}, {lattice} and base R plots, R Markdown documents and RStudio. It allows users to apply consistent visual themes across different plotting systems, with auto-theming in Shiny and R Markdown relying on CSS and {bslib} themes, respectively. Installation requires specific versions of dependent packages such as {shiny} and {rmarkdown}, while custom fonts benefit from {showtext} or {ragg}. Users can set global defaults for background, foreground and accent colours, as well as fonts, which can be overridden with plot-specific theme adjustments. The package also defines default colour scales for qualitative and sequential data and integrates with tools like bslib to import Google Fonts, enhancing visual consistency across different environments and user interfaces.
Publishing Tools
The R ecosystem goes beyond mere graphical and tabular display production to offer means for taking things much further, often offering platforms for publishing your work. These can be used locally too, so there is no need to entrust everything to a third-party provider. The uses are endless for what is available, and it appears that Posit has used this to help with building documentation and training too.
What you have here is one of those distinguishing facilities of the R ecosystem, particularly for those wanting to share their analysis work with more than a hint of reproducibility. The tool combines narrative text and code to generate various outputs, supporting multiple programming languages and formats such as HTML, PDF and dashboards. It enables users to produce reports, presentations and interactive applications, with options for publishing and scheduling through platforms like RStudio Connect, facilitating collaboration and distribution of results in professional settings.
Distill for R Markdown is a tool designed to streamline the creation of technical documents, offering features such as code folding, syntax highlighting and theming. It builds on existing frameworks like Pandoc, MathJax and D3, enabling the production of dynamic, interactive content. Users can customise the appearance with CSS and incorporate appendices for supplementary information. The tool acknowledges the contributions of developers who created foundational libraries, ensuring accessibility and functionality for a wide audience. Its design prioritises clarity, allowing authors to focus on presenting results rather than underlying code, while maintaining flexibility for those who wish to include detailed explanations.
For a while, this was one of R's unique selling points, and remains as compelling a reason to use the language even when Python has got its own version of the package. Enabling the creation of interactive web applications for data analysis without requiring web development expertise allows users to build interfaces that let others explore data through dynamic visualisations and filters. Here is a simple example: an app that generates scatter plots with adjustable variables, species filters and marginal plots, hosted either on personal servers or through a dedicated hosting service.
The {bslib} R package offers a modern user interface toolkit for Shiny and R Markdown applications, leveraging Bootstrap to enable the creation of customisable dashboards and interactive theming. It supports the use of updated Bootstrap and Bootswatch versions while maintaining compatibility with existing defaults, and provides tools for real-time visual adjustments. Installation is available through CRAN, with example previews demonstrating its capabilities.
Enabling users to manipulate and validate data within a spreadsheet-like interface, the {rhandsontable} package introduces an interactive data grid for R. It supports features such as custom cell rendering, validation rules and integration with Shiny applications. When used in Shiny, the widget requires explicit conversion of data using the hot_to_r function, as updates may not be immediately reflected in reactive contexts. Examples demonstrate its application in various scenarios, including date editing, financial calculations and dynamic visualisations linked to charts. The package also accommodates bookmarks in Shiny apps with specific handling. Users are encouraged to report issues or contribute improvements, with guidance provided for those seeking to expand its functionality. The development team welcomes feedback to refine the tool further, ensuring it aligns with evolving user needs.
{xaringanExtra} offers a range of enhancements and extensions for creating and presenting slides with xaringan, enabling features such as adding an overview tile view, making slides editable, broadcasting in real time, incorporating animations, embedding live video feeds and applying custom styles. It allows users to selectively activate individual tools or load multiple features simultaneously through a single function call, supporting tasks like adding banners, enabling code copying, fitting slides to screen dimensions and integrating utility toolkits. The package is available for installation via CRAN or GitHub, providing flexibility for developers and presenters seeking to expand the functionality of their slides.
From summary statistics to published reports with R, LaTeX and TinyTeX
19th March 2026
For anyone working across LaTeX, R Markdown and data analysis in R, there comes a point where separate tools begin to converge. Data has to be summarised, those summaries have to be turned into presentable tables and the finished result has to compile into a report that looks appropriate for its audience rather than a console dump. These notes follow that sequence, moving from the practical business of summarising data in R through to tabulation and then on to the publishing infrastructure that makes clean PDF and Word output possible.
Summarising Data with {dplyr}
The starting point for many analyses is a quick exploration of the data at hand. One useful example uses the anorexia dataset from the {MASS} package together with {dplyr}. The dataset contains weight change data for young female anorexia patients, divided into three treatment groups: Cont for the control group, CBT for cognitive behavioural treatment and FT for family treatment.
The basic manipulation starts by loading {MASS} and {dplyr}, then using filter() to create separate subsets for each treatment group. From there, mutate() adds a wtDelta column defined as Postwt - Prewt, giving the weight change for each patient. group_by(Treat) prepares the data for grouped summaries, and arrange(wtDelta) sorts within treatment groups. The notes then show how {dplyr}'s pipe operator, %>%, makes the workflow more readable by chaining these operations. The final summary table uses summarize() to compute the number of observations, the mean weight change and the standard deviation within each treatment group. The reported values are count 29, average weight change 3.006897 and standard deviation 7.308504 for CBT, count 26, average weight change -0.450000 and standard deviation 7.988705 for Cont and count 17, average weight change 7.264706 and standard deviation 7.157421 for FT.
That example is not presented as a complete statistical analysis. Instead, it serves as a quick exploratory route into the data, with the wording remaining appropriately cautious and noting that this is only a glance and not a rigorous analysis.
Choosing an R Package for Descriptive Summaries
The question of how best to summarise data opens up a broader comparison of R packages for descriptive statistics. A useful review sets out a common set of needs: a count of observations, the number and types of fields, transparent handling of missing data and sensible statistics that depend on the data type. Numeric variables call for measures such as mean, median, range and standard deviation, perhaps with percentiles. Categorical variables call for counts of levels and some sense of which categories dominate.
Base R's summary() does some of this reasonably well. It distinguishes categorical from numeric variables and reports distributions or numeric summaries accordingly, while also highlighting missing values. Yet, it does not show an overall record count, lacks standard deviation and is not especially tidy or ready for tools such as kable. Several contributed packages aim to improve on that. Hmisc::describe() gives counts of variables and observations, handles both categorical and numerical data and reports missing values clearly, showing the highest and lowest five values for numeric data instead of a simple range. pastecs::stat.desc() is more focused on numeric variables and provides confidence intervals, standard errors and optional normality tests. psych::describe() includes categorical variables but converts them to numeric codes by default before describing them, which the package documentation itself advises should be interpreted cautiously. psych::describeBy() extends this approach to grouped summaries and can return a matrix form with mat = TRUE.
Among the packages reviewed, {skimr} receives especially strong attention for balancing readability and downstream usefulness. skim() reports record and variable counts clearly, separates variables by type and includes missing data and standard summaries in an accessible layout. It also works with group_by() from {dplyr}, making grouped summaries straightforward to produce. More importantly for analytical workflows, the skim output can be treated as a tidy data frame in which each combination of variable and statistic is represented in long form, meaning the results can be filtered, transformed and plotted with standard tidyverse tools such as {ggplot2}.
{summarytools} is presented as another strong option, though with a distinction between its functions. descr() handles numeric variables and can be converted to a data frame for use with kable, while dfSummary() works across entire data frames and produces an especially polished summary. At the time of the original notes, dfSummary() was considered slow. The package author subsequently traced the issue, as documented in the same review, to an excessive number of histogram breaks being generated for variables with large values, imposing a limit to resolve it. The package also supports output through view(dfSummary(data)), which yields an attractive HTML-style summary.
Grouped Summary Table Packages
Once the data has been summarised, the next step is turning those summaries into formal tables. A detailed comparison covers a number of packages specifically designed for this purpose: {arsenal}, {qwraps2}, {Amisc}, {table1}, {tangram}, {furniture}, {tableone}, {compareGroups} and {Gmisc}. {arsenal} is described as highly functional and flexible, with tableby() able to create grouped tables in only a few lines and then be customised through control objects that specify tests, display statistics, labels and missing value treatment. {qwraps2} offers a lot of flexibility through nested lists of summary specifications, though at the cost of more code. {Amisc} can produce grouped tables and works with pander::pandoc.table(), but is noted as not being on CRAN. {table1} creates attractive tables with minimal code, though its treatment of missing values may not suit every use case. {tangram} produces visually appealing HTML output and allows custom rows such as missing counts to be inserted manually, although only HTML output is supported. {furniture} and {tableone} both support grouped table creation, but {tableone} in particular is notable because it is widely used in biomedical research for baseline characteristics tables.
The {tableone} package deserves separate mention because it is designed to summarise continuous and categorical variables in one table, a common need in medical papers. As the package introduction explains, CreateTableOne() can be used on an entire dataset or on a selected subset of variables, with factorVars specifying variables that are coded numerically but should be treated as categorical. The package can display all levels for categorical variables, report missing values via summary() and switch selected continuous variables to non-normal summaries using medians and interquartile ranges instead of means and standard deviations. For grouped comparisons, it prints p-values by default and can switch to non-parametric tests or Fisher's exact test where needed. Standardised mean differences can also be shown. Output can be captured as a matrix and written to CSV for editing in Excel or Word.
Styling and Exporting Tables
With tables constructed, the focus shifts to how they are presented and exported. As Hao Zhu's conference slides explain, the {kableExtra} package builds on knitr::kable() and provides a grammar-like approach to adding styling layers, importing the pipe %>% symbol from {magrittr} so that formatting functions can be added in the same way that layers are added in {ggplot2}. It supports themes such as kable_paper, kable_classic, kable_minimal and kable_material, as well as options for striping, hover effects, condensed layouts, fixed headers, grouped rows and columns, footnotes, scroll boxes and inline plots.
Table output is often the visible end of an analysis, and a broader review of R table packages covers a range of approaches that go well beyond the default output. In R Markdown, packages such as {gt}, {kableExtra}, {formattable}, {DT}, {reactable}, {reactablefmtr} and {flextable} all offer richer possibilities. Some are aimed mainly at HTML output, others at Word. {DT} in particular supports highly customised interactive tables with searching, filtering and cell styling through more advanced R and HTML code. {flextable} is highlighted as the strongest option when knitting to Word, given that the other packages are primarily designed for HTML.
For users working in Word-heavy settings, older but still practical workflows remain relevant too. One approach is simply to write tables to comma-separated text files and then paste and convert the content into a Word table. Another route is through {arsenal}'s write2 functions, designed as an alternative to SAS ODS. The convenience functions write2word(), write2html() and write2pdf() accept a wide range of objects: tableby, modelsum, freqlist and comparedf from {arsenal} itself, as well as knitr::kable(), xtable::xtable() and pander::pander_return() output. One notable constraint is that {xtable} is incompatible with write2word(). Beyond single tables, the functions accept a list of objects so that multiple tables, headers, paragraphs and even raw HTML or LaTeX can all be combined into a single output document. A yaml() helper adds a YAML header to the output, and a code.chunk() helper embeds executable R code chunks, while the generic write2() function handles formats beyond the three convenience wrappers, such as RTF.
The Publishing Infrastructure: CTAN and Its Mirrors
Producing PDF output from R Markdown depends on a working LaTeX installation, and the backbone of that ecosystem is CTAN, the Comprehensive TeX Archive Network. CTAN is the main archive for TeX and LaTeX packages and is supported by a large collection of mirrors spread around the world. The purpose of this distributed system is straightforward: users are encouraged to fetch files from a site that is close to them in network terms, which reduces load and tends to improve speed.
That global spread is extensive. The CTAN mirror list organises sites alphabetically by continent and then by country, with active sites listed across Africa, Asia, Europe, North America, Oceania and South America. Africa includes mirrors in South Africa and Morocco. Asia has particularly wide coverage, with many mirrors in China as well as sites in Korea, Hong Kong, India, Indonesia, Japan, Singapore, Taiwan, Saudi Arabia and Thailand. Europe is especially rich in mirrors, with hosts in Denmark, Germany, Spain, France, Italy, the Netherlands, Norway, Poland, Portugal, Romania, Switzerland, Finland, Sweden, the United Kingdom, Austria, Greece, Bulgaria and Russia. North America includes Canada, Costa Rica and the United States, while Oceania covers Australia and South America includes Brazil and Chile.
The details matter because different mirrors expose different protocols. While many support HTTPS, some also offer HTTP, FTP or rsync. CTAN provides a mirror multiplexer to make the common case simpler: pointing a browser to https://mirrors.ctan.org/ results in automatic redirection to a mirror in or near the user's country. There is one caveat. The multiplexer always redirects to an HTTPS mirror, so anyone intending to use another protocol needs to select manually from the mirror list. That is why the full listings still include non-HTTPS URLs alongside secure ones.
There is also an operational side to the network that is easy to overlook when things are working well. CTAN monitors mirrors to ensure they are current, and if one falls behind, then mirrors.ctan.org will not redirect users there. Updates to the mirror list can be sent to ctan@ctan.org. The master host of CTAN is ftp.dante.de in Cologne, Germany, with rsync access available at rsync://rsync.dante.ctan.org/CTAN/ and web access on https://ctan.org/. For those who want to contribute infrastructure rather than simply use it, CTAN also invites volunteers to become mirrors.
TinyTeX: A Lightweight LaTeX Distribution
This infrastructure becomes much more tangible when looking at a lightweight TeX distribution such as TinyTeX. TinyTeX is a lightweight, cross-platform, portable and easy-to-maintain LaTeX distribution based on TeX Live. It is small in size but intended to function well in most situations, especially for R users. Its appeal lies in not requiring users to install thousands of packages they will never use, installing them as needed instead. This also means installation can be done without administrator privileges, which removes one of the more familiar barriers around traditional TeX setups. TinyTeX can even be run from a flash drive.
For R users, TinyTeX is closely tied to the {tinytex} R package. The distinction is important: tinytex in lower case refers to the R package, while TinyTeX refers to the LaTeX distribution. Installation is intentionally direct. After installing the R package with install.packages('tinytex'), a user can run tinytex::install_tinytex(). Uninstallation is equally simple with tinytex::uninstall_tinytex(). For the average R Markdown user, that is often enough. Once TinyTeX is in place, PDF compilation usually requires no further manual package management.
There is slightly more to know if the aim is to compile standalone LaTeX documents from R. The {tinytex} package provides wrappers such as pdflatex(), xelatex() and lualatex(). These functions detect required LaTeX packages that are missing and install them automatically by default. In practical terms, that means a small example document can be written to a file and compiled with tinytex::pdflatex('test.tex') without much concern about whether every dependency has already been installed. For R users, this largely removes the old pattern of cryptic missing-package errors followed by manual searching through TeX repositories.
Developers may want more than the basics, and TinyTeX has a path for that as well. A helper such as tinytex:::install_yihui_pkgs() installs a collection of packages needed for building the PDF vignettes of many CRAN packages. That is a specific convenience rather than a universal requirement, but it illustrates the design philosophy behind TinyTeX: keep the initial footprint light and offer ways to add what is commonly needed later.
Using TinyTeX Outside R
For users outside R, TinyTeX still works, but the focus shifts to the command-line utility tlmgr. The documentation is direct in its assumptions: if command-line work is unwelcome, another LaTeX distribution may be a better fit. The central command is tlmgr, and much of TinyTeX maintenance can be expressed through it.
On Linux, installation places TinyTeX in $HOME/.TinyTeX and creates symlinks for executables such as pdflatex under $HOME/bin or $HOME/.local/bin if it exists. The installation script is fetched with wget and piped to sh, after first checking that Perl is correctly installed. On macOS, TinyTeX lives in ~/Library/TinyTeX, and users without write permission to /usr/local/bin may need to change ownership of that directory before installation. Windows users can run a batch file, install-bin-windows.bat, and the default installation directory is %APPDATA%/TinyTeX unless APPDATA contains spaces or non-ASCII characters, in which case %ProgramData% is used instead. PowerShell version 3.0 or higher is required on Windows.
Uninstallation follows the same self-contained logic. On Linux and macOS, tlmgr path remove is followed by deleting the TinyTeX folder. On Windows, tlmgr path remove is followed by removing the installation directory. This simplicity is a deliberate contrast with larger LaTeX distributions, which are considerably more involved to remove cleanly.
Maintenance and Package Management
Maintenance is where TinyTeX's relationship to CTAN and TeX Live becomes especially visible. If a document fails with an error such as File 'times.sty' not found, the fix is to search for the package containing that file with tlmgr search --global --file "/times.sty". In the example given, that identifies the psnfss package, which can then be installed with tlmgr install psnfss. If the package includes executables, tlmgr path add may also be needed. An alternative route is to upload the error log to the yihui/latex-pass GitHub repository, where package searching is carried out remotely.
If the problem is less obvious, a full update cycle is suggested: tlmgr update --self --all, then tlmgr path add and fmtutil-sys --all. R users have wrappers for these tasks too, including tlmgr_search(), tlmgr_install() and tlmgr_update(). Some situations still require a full reinstallation. If TeX Live reports Remote repository newer than local, TinyTeX should be reinstalled manually, which for R users can be done with tinytex::reinstall_tinytex(). Similarly, when a TeX Live release is frozen in preparation for a new one, the advice is simply to wait and then reinstall when the next release is ready.
The motivation behind TinyTeX is laid out with unusual clarity. Traditional LaTeX distributions often present a choice between a small basic installation that soon proves incomplete and a very large full installation containing thousands of packages that will never be used. TinyTeX is framed as a way around those frustrations by building on TeX Live's portability and cross-platform design while stripping away unnecessary size and complexity. The acknowledgements also underline that TinyTeX depends on the work of the TeX Live team.
Connecting the R Workflow to a Finished Report
Taken together, these notes show how closely summarisation, tabulation and publishing are linked. {dplyr} and related tools make it easy to summarise data quickly, while a wide range of R packages then turn those summaries into tables that are not only statistically useful but also presentable. CTAN and its mirrors keep the TeX ecosystem available and current across the world, and TinyTeX builds on that ecosystem to make LaTeX more manageable, especially for R users. What begins with a grouped summary in the console can end with a polished report table in HTML, PDF or Word, and understanding the chain between those stages makes the whole workflow feel considerably less mysterious.
Some R functions for working with dates, strings and data frames
18th March 2026
Working with data in R often comes down to a handful of recurring tasks: combining text, converting dates and times, reshaping tables and creating summaries that are easier to interpret. This article brings together several strands of base R and tidyverse-style practice, with a particular focus on string handling, date parsing, subsetting and simple time series smoothing. Taken together, these functions form part of the everyday toolkit for data cleaning and analysis, especially when imported data arrive in inconsistent formats.
String Building
At the simplest end of this toolkit is paste(), a base R function for concatenating character vectors. Its purpose is straightforward: it converts one or more R objects to character vectors and joins them together, separating terms with the string supplied in sep, which defaults to a space. If the inputs are vectors, concatenation happens term by term, so paste("A", 1:6, sep = "") yields "A1" through "A6", while paste(1:12) behaves much like as.character(1:12). There is also a collapse argument, which takes the resulting vector and combines its elements into a single string separated by the chosen delimiter, making paste() useful both for constructing values row by row and for creating one final display string from many parts.
That basic string-building role becomes more important when dates and times are involved because imported date-time data often arrive as text split across multiple columns. A common example is having one column for a date and another for a time, then joining them with paste(dates, times) before parsing the result. In that sense, the paste() function often acts as a bridge between messy raw input and structured date-time objects. It is simple, but it appears repeatedly in data preparation pipelines.
Date-Time Conversion
For date-time conversion, base R provides strptime(), strftime() and format() methods for POSIXlt and POSIXct objects. These functions convert between character representations and R date-time classes, and they are central to understanding how R reads and prints times. strptime() takes character input and converts it to an object of class "POSIXlt", while strftime() and format() move in the other direction, turning date-time objects into character strings. The as.character() method for "POSIXt" classes fits into the same family, and the essential idea is that the date-time value and its textual representation are separate things, with the format string defining how R should interpret or display that representation.
Format strings rely on conversion specifications introduced with %, and many of these are standard across systems. %Y means a four-digit year with century, %y means a two-digit year, %m is a month, %d is the day of a month and %H:%M:%S captures hours, minutes and seconds in 24-hour time. %F is equivalent to %Y-%m-%d, which is the ISO 8601 date format. %b and %B represent abbreviated and complete month names, while %a and %A do the same for weekdays. Locale matters here because month names, weekday names, AM/PM indicators and some separators depend on the LC_TIME locale, meaning a date string like "1jan1960" may parse correctly in one locale and return NA in another unless the locale is set appropriately.
R's defaults generally follow ISO 8601 rules, so dates print as "2001-02-28" and times as "14:01:02", though R inserts a space between date and time by default. Several details matter in practice. strptime() processes input strings only as far as needed for the specified format, so trailing characters are ignored. Unspecified hours, minutes and seconds default to zero, and if no year, month or day is supplied then the current values are assumed, though if a month is given, the day must also be valid for that month. Invalid calendar dates such as "2010-02-30 08:00" produce results whose components are all NA.
Time Zones and Daylight Saving
Time zones add another layer of complexity. The tz argument specifies the time zone to use for conversion, with "" meaning the current time zone and "GMT" meaning UTC. Invalid values are often treated as UTC, though behaviour can be system-specific. The usetz argument controls whether a time zone abbreviation is appended to output, which is generally more reliable than %Z. %z represents a signed UTC offset such as -0800, and R supports it for input on all platforms. Even so, time zones can be awkward because daylight saving transitions create times that do not occur at all, or occur twice, and strptime() itself does not validate those cases, though conversion through as.POSIXct may do so.
Two-Digit Years
Two-digit years are a notable source of confusion for analysts working with historical data. As described in the R date formats guide on R-bloggers, %y maps values 00 to 68 to the years 2000 to 2068 and 69 to 99 to 1969 to 1999, following the POSIX standard. A value such as "08/17/20" may therefore be interpreted as 2020 when the intended year is 1920. One practical workaround is to identify any parsed dates lying in the future and then rebuild them with a 19 prefix using format() and ifelse(). This approach is explicit and practical, though it depends on the assumptions of the data at hand.
Plain Dates
For plain dates, rather than full date-times, as.Date() is usually the entry point. Character dates can be imported by specifying the current format, such as %m/%d/%y for "05/27/84" or %B %d %Y for "May 27 1984". If no format is supplied, as.Date() first tries %Y-%m-%d and then %Y/%m/%d. Numeric dates are common when data come from Excel, and here the crucial issue is the origin date: Windows Excel uses an origin of "1899-12-30" for dates after 1900 because Excel incorrectly treated 1900 as a leap year (an error originally copied from Lotus 1-2-3 for compatibility), while Mac Excel traditionally uses "1904-01-01". Once the correct origin is supplied, as.Date() converts the serial numbers into standard R dates.
After import, format() can display dates in other ways without changing their underlying class. For example, format(betterDates, "%a %b %d") might yield values like "Sun May 27" and "Thu Jul 07". This distinction between storage and display is important because once R recognises values as dates, they can participate in date-aware operations such as mean(), min() and max(), and a vector of dates can have a meaningful mean date with the minimum and maximum identifying the earliest and latest observations.
Extracting Columns and Manipulating Lists
These ideas about correct types and structure carry over into table manipulation. A data frame column often needs to be extracted as a vector before further processing, and there are several standard ways to do this, as covered in this guide from Statistics Globe. In base R, the $ operator gives a direct route, as in data$x1. Subsetting with data[, "x1"] yields the same result for a single column, and in the tidyverse, dplyr::pull(data, x1) serves the same purpose. All three approaches convert a column of a data frame into a standalone vector, and each is useful depending on the surrounding code style.
List manipulation has similar patterns, detailed in this Statistics Globe tutorial on removing list elements. Removing elements from a list can be done by position with negative indexing, as in my_list[-2], or by assigning NULL to the relevant component, for example my_list_2[2] <- NULL. If names are more meaningful than positions, then subsetting with names(my_list) != "b" or names(my_list) %in% "b" == FALSE removes the named element instead. The same logic extends to multiple elements, whether by positions such as -c(2, 3) or names such as %in% c("b", "c") == FALSE. These are simple techniques, but they matter because lists are a common structure in R, especially when working with nested results.
Subsetting, Renaming and Reordering Data Frames
Data frames themselves can be subset in several ways, and the choice often depends on readability, as the five-method overview on R-bloggers demonstrates clearly. The bracket form example[x, y] remains the foundation, whether selecting rows and columns directly or omitting unwanted ones with negative indices. More expressive alternatives include which() together with %in%, the base subset() function and tidyverse verbs like filter() and select(). The point is not that one method is universally best, but that R offers both low-level precision and higher-level readability, depending on the task.
Column names and column order also need regular attention. Renaming can be done with dplyr::rename(), as explained in this lesson from Datanovia, for instance changing Sepal.Length to sepal_length and Sepal.Width to sepal_width. In base R, the same effect comes from modifying names() or colnames(), either by matching specific names or by position. Reordering columns is just as direct, with a data frame rearranged by column indices such as my_data[, c(5, 4, 1, 2, 3)] or by an explicit character vector of names, as the STHDA guide on reordering columns illustrates. Both approaches are useful when preparing data for presentation or for functions that expect variables in a certain order.
Sorting and Cumulative Calculations
Sorting and cumulative calculations fit naturally into this same preparatory workflow. To sort a data frame in base R, the DataCamp sorting reference demonstrates that order() is the key function: mtcars[order(mpg), ] sorts ascending by mpg, while mtcars[order(mpg, -cyl), ] sorts by mpg ascending and cyl descending. For cumulative totals, cumsum() provides a running sum, as in calculating cumulative air miles from the airmiles dataset, an example covered in the Data Cornering guide to cumulative calculations. Within grouped data, dplyr::group_by() and mutate() can apply cumsum() separately to each group, and a related idea is cumulative count, which can be built by summing a column of ones within groups, or with data.table::rowid() to create a group index.
Time Series Smoothing
Time series smoothing introduces one further pattern: replacing noisy raw values with moving averages. As the Storybench rolling averages guide explains, the zoo::rollmean() function calculates rolling means over a window of width k, and examples using 3, 5, 7, 15 and 21-day windows on pandemic deaths and confirmed cases by state demonstrate the approach clearly. After arranging and grouping by state, mutate() adds variables such as death_03da, death_05da and death_07da. Because rollmean() is centred by default, the resulting values are symmetrical around the observation of interest and produce NA values at the start and end where there are not enough surrounding observations, which is why odd values of k are usually preferred as they make the smoothing window balanced.
The arithmetic is uncomplicated, but the interpretation is useful. A 3-day moving average for a given date is the mean of that day, the previous day and the following day, while a 7-day moving average uses three observations on either side. As the window widens, the line becomes smoother, but more short-term variation is lost. This trade-off is visible when comparing 3-day and 21-day averages: a shorter average tracks recent changes more closely, while a longer one suppresses noise and makes broader trends stand out. If a trailing rather than centred calculation is needed, rollmeanr() shifts the window to the right-hand end.
The same grouped workflow can be used to derive new daily values before smoothing. In the pandemic example, daily new confirmed cases are calculated from cumulative confirmed counts using dplyr::lag(), with each day's new cases equal to the current cumulative total minus the previous day's total. Grouping by state and date, summing confirmed counts and then subtracting the lagged value produces new_confirmed_cases, which can then be smoothed with rollmean() in the same way as deaths. Once these measures are available, reshaping with pivot_longer() allows raw values and rolling averages to be plotted together in ggplot2, making it easier to compare volatility against trend.
How These R Data Manipulation Techniques Fit Together
What links all of these techniques is not just that they are common in R, but that they solve the mundane, essential problems of analysis. Data arrive as text when they should be dates, as cumulative counts when daily changes are needed, as broad tables when only a few columns matter, or as inconsistent names that get in the way of clear code. Functions such as paste(), strptime(), as.Date(), order(), cumsum(), rollmean(), rename(), select() and simple bracket subsetting are therefore less like isolated tricks and more like pieces of a coherent working practice. Knowing how they fit together makes it easier to move from raw input to reliable analysis, with fewer surprises along the way.
Making sense of parallel and asynchronous execution in Python
16th March 2026
Parallel processing in Python is often presented as a straightforward route to faster programs, though the reality is rather more nuanced. At its core, parallel processing means executing parts of a task simultaneously across multiple processors or cores on the same machine, with the intention of reducing the total time needed to complete the work. Any honest explanation must include an important caveat because parallelism brings overhead of its own: processes need to be created, scheduled and coordinated, and data often has to be passed between them. For small or lightweight tasks, that overhead can outweigh any gain, and two tasks that each take five seconds may still require around eight seconds when parallelised, rather than the ideal five.
The Multiprocessing Module
One of the standard ways to work with parallel execution in Python is the multiprocessing module This module creates subprocesses rather than threads, which matters because each process has its own memory space. On both Unix-like systems and Windows, this arrangement allows Python code to use multiple processors more effectively for independent work, and it sidesteps some of the limitations commonly associated with threads in CPython, particularly for CPU-bound tasks. Threads still have an important role, especially for workloads that are heavy on input/output operations, but multiprocessing is often the better fit when the work involves substantial computation.
Understanding the Global Interpreter Lock
The reason threads are less effective for CPU-bound work in CPython relates directly to the Global Interpreter Lock (GIL). The GIL is a mutex that allows only one thread to hold control of the Python interpreter at any one time, meaning that even in a multithreaded programme, only one thread can execute Python bytecode at a given moment. When a thread is waiting for an external input/output operation it releases the GIL, allowing other threads to run, which is why threading remains a reasonable choice for I/O-bound workloads. Multiprocessing sidesteps the GIL entirely by spawning separate processes, each with its own Python interpreter, allowing genuine parallel execution across cores.
How Many Processes Can Run in Parallel?
Before using multiprocessing, it helps to understand the practical ceiling on how many processes can run in parallel. The upper bound is usually tied to the number of logical processors or cores available on the machine, and Python exposes this through multiprocessing.cpu_count(), which returns the number of processors detected. That figure is a useful starting point rather than an absolute rule. In real applications, the best number of worker processes can vary according to available memory, the nature of the task and what else the machine is doing at the time.
Synchronous and Asynchronous Execution
Another foundation worth clarifying is the difference between synchronous and asynchronous execution. In synchronous execution, tasks are coordinated so that results are typically gathered in the same order in which they were started, and the main programme effectively waits for those tasks to finish. In asynchronous execution, by contrast, tasks can complete in any order and the results may not correspond to the original input sequence, which often improves throughput but requires the programmer to be more deliberate about collecting and arranging results.
Pool and Process: The Two Main Abstractions
The multiprocessing module offers two main abstractions for parallel work: Pool and Process. For most practical tasks, Pool is the easier and more convenient option. It manages a collection of worker processes and provides methods such as apply(), map() and starmap() for synchronous execution, alongside apply_async(), map_async() and starmap_async() for asynchronous execution. The lower-level Process class offers more control and suits more specialised cases, but for many data-processing jobs Pool is sufficient and considerably easier to reason about.
An Example: Counting Values in a Range
A useful way to see these ideas in action is through a concrete example. Suppose there is a two-dimensional list, or matrix, where each row contains a small set of integers, and the task is to count how many values in each row fall within a given range. In the example, the data are generated with NumPy using np.random.randint(0, 10, size=[200000, 5]) and then converted to a plain list of lists with tolist(). A simple function, howmany_within_range(row, minimum, maximum), loops through each number in a row and increments a counter whenever the number falls between the supplied minimum and maximum values.
Without any parallelism, this task is handled with a straightforward loop in which each row is passed to the function in turn and the returned counts are appended to a results list. This serial approach is simple, easy to read and often good enough as a baseline, and it provides an important benchmark because parallel processing should not be adopted merely because it is available but should address an actual performance problem.
Pool.apply()
To parallelise the same function, the first step is to create a process pool, typically with mp.Pool(mp.cpu_count()). The simplest method to understand is Pool.apply(), which runs a function in a worker process using the arguments supplied through args. In the range-counting example, each row is submitted with the same minimum and maximum values. The resulting code is concise, but there is an important detail to note: when apply() is used inside a list comprehension, each call still blocks until it completes. It is parallel in terms of the workers available, but it is not always the most efficient pattern for distributing a large iterable of similar tasks.
Pool.map()
That is where Pool.map() can be more suitable. The map() method accepts a single iterable and applies the target function to each element. Because the original howmany_within_range() function expects more than one argument, the example adapts it by defining howmany_within_range_rowonly(row, minimum=4, maximum=8), giving default values to the range bounds so that only the row must be supplied. This is not always the cleanest design, but it illustrates the central constraint of map(): it expects one iterable of inputs rather than multiple arguments per call. In return, it is often a good fit for simple, repeated operations over a dataset.
Pool.starmap()
When a function genuinely needs multiple arguments and one wants the convenience of map-like behaviour, Pool.starmap() is usually the better choice. Like map(), it takes a single iterable, but each element of that iterable is itself another iterable containing the arguments for one function call. In the example, the input becomes [(row, 4, 8) for row in data], with each tuple unpacked into howmany_within_range(). This tends to be clearer than altering function signatures purely to satisfy the constraints of map().
Asynchronous Variants
The asynchronous equivalents follow the same broad pattern but differ in one crucial respect: they do not force the main process to wait for each task in order. With Pool.apply_async(), tasks are submitted, and the programme can continue while workers process them in the background. The example demonstrates this by redefining the counting function as howmany_within_range2(i, row, minimum, maximum), which returns both the original index and the count, a distinction that matters because asynchronous execution may alter the order of results. A callback function appends each completed result to a shared list and, after all tasks finish, that list is sorted by index so that the final output matches the original row order.
There is also an alternative form of apply_async() that avoids callbacks by returning ApplyResult objects, which can later be resolved with .get() to retrieve the actual result. This approach can be easier to follow when callbacks feel too indirect, though it still requires care to ensure that the pool is properly closed and joined so that all processes complete. The use of pool.join() is particularly important here because it prevents subsequent lines of code from running until the queued work is finished. Asynchronous mapping methods are available too, including Pool.starmap_async(), which mirrors starmap() but returns an asynchronous result object whose data can be fetched with .get().
Parallelising Pandas DataFrames
Parallelism in Python is not restricted to plain lists. In data analysis and machine learning work, it is often more relevant to process pandas DataFrames, and there are several levels at which this can happen: a function can operate on one row, one column or an entire DataFrame. The first two can be managed with the standard multiprocessing module alone, while whole-DataFrame parallelism often needs more flexible serialisation support than the standard library provides.
Row-wise and Column-wise Parallelism
For row-wise work, one approach is to iterate over df.itertuples(name=False) so that each row is presented as a simple tuple. A hypotenuse(row) function can compute the square root of the sum of squares of two values from each row, with a pool of four worker processes handling the rows through pool.imap(). This resembles pd.apply() conceptually, but the work is spread across processes rather than performed in a single interpreter thread.
Column-wise parallelism follows the same idea but uses df.items() to iterate over columns (it is worth noting that df.iteritems(), which older examples may reference, was deprecated in pandas 1.5.0 and has since been removed, with df.items() being the correct modern equivalent). A sum_of_squares(column) function receives each column as a pair containing the column label and the series itself, and pool.imap() distributes this work across multiple processes. This pattern is useful when independent operations need to be applied to separate columns.
Whole-DataFrame Parallelism with Pathos
Parallelising functions that accept an entire DataFrame or similarly complex object is more difficult with the standard multiprocessing machinery because of serialisation constraints, since the standard library uses pickle internally and pickle has well-known limitations with certain object types. The pathos package addresses this by using dill internally, which supports serialising and deserialising almost all Python types. A DataFrame is split into chunks with np.array_split(df, cores, axis=0), and a ProcessingPool from pathos.multiprocessing maps a function across those chunks, with the results combined using np.vstack(). This extends the same Pool > Map > Close > Join pattern, though the pool is also cleared afterwards with pool.clear().
Lower-level Process Control and Queues
There are broader ways to think about parallel execution beyond multiprocessing.Pool. Lower-level process management with multiprocessing.Process gives explicit control over individual processes, and this can be paired with queues managed through multiprocessing.Manager() for inter-process communication. In such designs, one queue can hold tasks and another can collect results, with worker processes repeatedly fetching tasks, processing them and placing outputs in the result queue, terminating when they receive a sentinel value such as -1. This approach is more verbose than using a pool, but it can be valuable when workflows are dynamic or when processes need long-lived coordination.
Threads, Executors and External Commands
Python also offers other concurrency models worth knowing. Threads, available through the threading module or concurrent.futures.ThreadPoolExecutor, are often well suited to I/O-bound work such as downloading files or waiting on network responses. Because of the GIL in CPython, threads are less effective for CPU-bound pure Python code, though they can still provide concurrency when much of the time is spent waiting. Process-based approaches, including ProcessPoolExecutor, are generally more effective for CPU-heavy work because they achieve genuine parallel execution across cores.
External process execution forms another category entirely. The os.system() method can launch shell commands, potentially in the background, though it is relatively crude. The subprocess module is more robust, providing better control over arguments, output capture and return codes. These tools are useful when the work is best handled by external programmes rather than Python functions, though they are conceptually distinct from in-Python data parallelism.
Choosing the Right Approach for Parallel Processing in Python
What emerges from all of this is that parallel processing in Python is less about memorising one trick and more about matching the method to the problem at hand. For simple data transformations over independent records, Pool.map() or Pool.starmap() can be effective, while asynchronous methods come into play when result order is not guaranteed or when responsiveness matters. When working with pandas, row-wise and column-wise strategies fit naturally into the standard multiprocessing model, whereas whole-object processing may call for a package such as pathos. Lower-level process control, thread pools, external commands and task queues each have their place too.
It is also worth remembering that parallelism is not free. Process creation, serialisation, memory usage and coordination all introduce cost, and the right question is not whether code can be parallelised but whether the effort and overhead make sense for the workload in question. Python provides several mature tools for splitting work across processes and threads, and the multiprocessing module remains one of the most practical for CPU-bound tasks on a single machine, with the Pool interface offering the clearest path from serial code to parallel execution for many everyday applications.
Sending email reliably from Linux web servers and WordPress websites
6th March 2026
Setting up a new VPS brought with it the need to get email services working at the operating system level, and that particular endeavour proved stubbornly resistant to resolution. What eventually broke the deadlock was stepping back from the OS entirely and turning to the Post SMTP plugin for WordPress, which handled the complexity of Gmail authentication cleanly and got things moving. Since then, things have moved on again: Proton Mail, with a subscription upgrade that adds a custom domain and an associated address, now handles outgoing mail seamlessly.
Sending email from a server involves block lists, authentication requirements and cloud provider restrictions that have a habit of turning a simple task into an extended troubleshooting session. That even applies with well-established approaches using Postfix or Sendmail, relaying through Gmail or specialist SMTP providers, continue to serve administrators reliably with minimal overhead. The list of things to do and manage is not a short one at the server level.
What follows draws together practical guidance from the Linode documentation on configuring Postfix with external SMTP servers, the Linode guide on relaying through Gmail specifically, the Cloudways walkthrough for Post SMTP on WordPress and the linuxconfig.org guide to Sendmail with Gmail, keeping to fundamentals that have changed little even as the surrounding tooling has moved on. In some ways, the sources are disparate yet complementary, something that is reflected in the rest of what you find below. The whole intent is that all this is in file for everyone.
Starting with the Environment
A sensible first step is to examine the environment in which the server runs. Some cloud platforms, including Akamai Cloud's Linode Compute Instances for certain new accounts, block outbound SMTP ports 25, 465 and 587 by default to combat abuse. This blocking prevents applications from sending any email until the restrictions are lifted. The remedy is procedural rather than technical: platform documentation explains how to request removal of these restrictions after acknowledging the provider's email policies. Tackling this early avoids fruitless troubleshooting later on.
Alongside the port restriction check, it is worth setting a proper fully qualified domain name (FQDN) on the host and applying all available system updates. A correctly configured hostname reduces delays during mailer start-up and helps ensure that headers and logs appear coherent to downstream systems. Basic checks, such as confirming that you can log in to the mail account you intend to use as a relay, will also spare time later.
Configuring Postfix on Debian and Ubuntu
On Debian or Ubuntu, Postfix offers a straightforward route to sending mail via a relay. Installing the required packages begins with apt-get update followed by apt-get install of libsasl2-modules and postfix. The installer will prompt for a general type of mail configuration, and choosing "Internet Site" is appropriate in this scenario. The System Mail Name should then be set to the domain through which you intend to send.
After installation, verify that the myhostname parameter in /etc/postfix/main.cf reflects the server's FQDN, for example:
myhostname = fqdn.example.comThis setting anchors Postfix's identity and helps downstream receivers interpret messages correctly. The myhostname value is also used in outgoing SMTP greetings, so accuracy matters.
Relaying through Gmail
Relaying through Gmail or Google Workspace adds a layer of security considerations that are worth understanding before proceeding. Google retired its "less secure apps" feature in 2024, which had previously allowed basic username-and-password authentication over SMTP. All third-party SMTP connections now require either OAuth or an app password, and traditional password-based authentication is no longer accepted. Google is also pushing towards passkeys as a replacement for passwords at the account sign-in level, though their practical applicability to server-level SMTP relay remains limited for now. App passwords, whilst still available, are presented by Google as a transitional measure rather than a long-term solution, so OAuth is the more future-proof path where it is supported.
Where two-step verification is enabled on a Google account, the recommended approach for Postfix relay is to generate an app password. Within the Google Account security settings, enabling two-step verification unlocks the ability to create app passwords under the "How you sign in to Google" section. Choosing a descriptive name such as "Postfix" keeps records intelligible, and the resulting 16-character password should be stored securely since it will not be displayed again. This app password is then used in place of your regular account password throughout the Postfix configuration.
Storing SMTP Credentials
With credentials in hand, Postfix needs to know how to authenticate to the relay. Depending on the guide you follow, credentials may be stored at /etc/postfix/sasl/sasl_passwd or at /etc/postfix/sasl_passwd. Either location works as long as the corresponding path is referenced correctly in main.cf. In the credentials file, the entry for Gmail using STARTTLS on port 587 takes this form:
[smtp.gmail.com]:587 username@gmail.com:app-passwordThe square brackets around the hostname instruct Postfix not to perform MX lookups for that host, ensuring it connects directly to the submission server. After saving the file, create the hash database with postmap, using whichever path you chose:
sudo postmap /etc/postfix/sasl/sasl_passwdThis produces a .db file that Postfix consults at run-time. Because both the plain-text file and the .db file contain credentials, you should tighten permissions so that only root can read or write them:
sudo chown root:root /etc/postfix/sasl/sasl_passwd /etc/postfix/sasl/sasl_passwd.db
sudo chmod 0600 /etc/postfix/sasl/sasl_passwd /etc/postfix/sasl/sasl_passwd.dbConfiguring the Gmail Relay
The relay configuration in /etc/postfix/main.cf forms the core of the setup. Setting relayhost to [smtp.gmail.com]:587 instructs Postfix to deliver all mail via Google's submission server. At the end of the file, the following block enables SASL authentication and STARTTLS, points to the hash file created earlier, disallows anonymous mechanisms and specifies the CA bundle for TLS verification:
relayhost = [smtp.gmail.com]:587
smtp_sasl_auth_enable = yes
smtp_sasl_security_options = noanonymous
smtp_sasl_password_maps = hash:/etc/postfix/sasl/sasl_passwd
smtp_tls_security_level = encrypt
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crtRestarting Postfix applies the changes:
sudo systemctl restart postfixA simple test uses Postfix's built-in sendmail implementation. Invoking sendmail recipient@elsewhere.com, then entering From: and Subject: headers followed by a message body and a single dot on a line by itself, is sufficient to trigger a delivery attempt. Watching sudo tail -f /var/log/syslog (or /var/log/mail.log on some distributions) while testing helps confirm that authentication and delivery are succeeding, and the way back to the shell from sendmail is Ctrl+C.
Using Third-Party SMTP Providers
The same relay pattern works with third-party SMTP providers that specialise in transactional delivery. Service-specific details differ only in hostnames and credentials, while the underlying mechanism remains identical.
For Mandrill (Mailchimp's transactional email service), the credentials file would contain:
[smtp.mandrillapp.com]:587 USERNAME:API_KEYThe relayhost line in main.cf becomes [smtp.mandrillapp.com]:587. Note that the password field takes an API key, not your Mailchimp account password. Running postmap on the credentials file and restarting Postfix completes the switch.
For SendGrid, the entry for credentials is:
[smtp.sendgrid.net]:587 apikey:YOUR_API_KEYHere, the username is the literal string apikey (not your account name), and the password is the API key generated within your SendGrid account. The relayhost becomes [smtp.sendgrid.net]:587, followed by the same postmap and restart sequence.
One practical point worth noting: some guides place the credentials file directly under /etc/postfix/sasl_passwd, whilst the Linode Gmail relay guide uses the subdirectory path /etc/postfix/sasl/sasl_passwd. Either location is valid, but the path set in smtp_sasl_password_maps within main.cf must match whichever you choose. A mismatch produces unhelpful "file not found" errors at send time that can take some effort to diagnose.
Configuring Sendmail as an Alternative
Some administrators prefer Sendmail, particularly on distributions where it remains the default. Relaying through Gmail with Sendmail follows its own clear sequence. Installing the required packages varies by distribution: on Ubuntu, Debian and Linux Mint the command is apt install sendmail mailutils sendmail-bin, whilst on CentOS, Fedora, AlmaLinux and Red Hat systems, dnf install sendmail is used instead.
Authentication details live in an authinfo map under /etc/mail/authinfo. Creating that directory with restricted permissions and then adding a file such as gmail-auth allows the following entry to be stored:
AuthInfo: "U:root" "I:YOUR_GMAIL_EMAIL_ADDRESS" "P:YOUR_APP_PASSWORD"The quotes here are significant: the P: is a literal prefix for the password field, not part of the password itself. Building the database with makemap hash gmail-auth < gmail-auth produces gmail-auth.db in the same directory, which Sendmail will consult when connecting to the smart host.
Sendmail's configuration is macro-driven and centred on /etc/mail/sendmail.mc. Placing the relay and authentication directives just above the first MAILER definition ensures they are processed correctly when sendmail.cf is rebuilt. The key definitions set SMART_HOST to [smtp.gmail.com], force the submission port by defining RELAY_MAILER_ARGS and ESMTP_MAILER_ARGS as TCP $h 587, enable authentication with confAUTH_OPTIONS set to A p, and wire in the authinfo map with:
FEATURE('authinfo', 'hash -o /etc/mail/authinfo/gmail-auth.db')After saving those changes, running make -C /etc/mail regenerates sendmail.cf and systemctl restart sendmail brings the service up with the new configuration. Hosts without a resolvable FQDN may pause briefly at start-up, but the service typically continues after a short delay.
WordPress and the Post SMTP Plugin
Web applications introduce different constraints, particularly where user authentication is delegated to a third party. For WordPress sites, the Post SMTP plugin (originally forked from Postman SMTP) modernises the classic approach and integrates OAuth 2.0 so that Gmail and Google Workspace can be used without storing a mailbox password. With Google having retired basic password authentication for SMTP, an OAuth-based approach is now the standard requirement rather than an optional convenience.
The process begins with installation and activation of the plugin, after which its setup wizard auto-detects smtp.gmail.com and recommends SMTP-STARTTLS with OAuth 2.0 authentication on port 587. At this point, the wizard asks for a Client ID and Client Secret, which are obtained from the Google Cloud Console rather than the Gmail settings page. Creating a project in the console, enabling the Gmail API, and completing the OAuth consent screen with basic application information lays the necessary groundwork. Selecting "Web application" as the application type then allows you to enter the Authorised JavaScript origins and Authorised redirect URIs that the plugin displays during its setup step. Completing this creation reveals the Client ID and Client Secret, which are pasted back into the plugin wizard to proceed.
Before the plugin can authorise fully, the publishing status of the OAuth consent screen must usually be changed from "Testing" to "Production" (or "In production"). This step matters more than it might appear: whilst an app remains in "Testing" status, Google's authorisation tokens expire after just seven days, which means the connection will silently stop working and require reauthorisation on a weekly basis. Moving to "In production" removes this expiry, and refresh tokens then remain valid indefinitely unless revoked. The Google console provides a "Publish App" option on the OAuth consent screen page to make this change. Once published, returning to the WordPress dashboard and clicking "Grant permission with Google" allows you to select the desired account and accept the requested permissions. The plugin's status view then confirms that authorisation has succeeded. A test email through the plugin's own action validates that messages are leaving the site as expected.
This OAuth-based arrangement aligns with Google's current security model, avoids the need for app passwords, and reduces the risk of unauthorised access if a site is compromised. General security hardening of the WordPress installation itself remains essential regardless.
The Underlying Protocols
Underpinning all of these approaches are protocols that remain stable and well understood. SMTP still carries the mail, STARTTLS upgrades plaintext connections to encrypted channels either opportunistically or by policy, and DNS resolves relay hostnames to IP addresses. The role of DNS here is easy to overlook, but it is foundational: as The TCP/IP Guide sets out in its coverage of SMTP and related protocols, correct name resolution underpins every step of message delivery. If a relay hostname cannot be resolved, nothing else will proceed. If the certificate bundle pointed to by smtp_tls_CAfile is missing or outdated, STARTTLS negotiation will fail. Logs record these conditions at the time they occur, which is precisely why watching syslog during tests is more informative than simply checking whether a message arrives in an inbox.
A few operational considerations round out a dependable setup. Permission hygiene on credential files protects against accidental disclosure during audits or backups, and commands that manipulate maps (such as postmap and makemap) must be re-run after any edit to their corresponding source files. Consistency between the port specified in the credentials file and the one set in main.cf's relayhost parameter also matters: mismatches lead to confusing connection attempts. Postfix's postconf command lists all current configuration values, making it a convenient way to verify that paths and flags are set as expected.
On Reflection
Reliable email from servers involves the installation of supporting right component software, authentication in the way the provider expects, encrypting the submission channel, keeping credentials safe and testing with your eyes on the logs. This list makes it sound like the complex endeavour that it is. Thus, If your remit extends to a WordPress dashboard, it is better to use a plugin that speaks OAuth 2.0 and complete the corresponding setup in the Google Cloud Console so that everything flows cleanly.
Getting to know Jira, its workflows, test management capabilities and the need for governance
3rd March 2026
Developed by Atlassian and first released in 2002 as a straightforward bug and issue tracker aimed at software developers, Jira has since grown into a platform used for project management across a wide range of industries and disciplines. The name itself is a truncation of Gojira, the Japanese word for Godzilla, originating as an internal nickname used by Atlassian developers for Bugzilla, the bug-tracking tool they had previously relied upon.
A Family of Products, Each With a Purpose
The Jira ecosystem has expanded well beyond its original single offering, and it is worth understanding what each product is designed to do. Jira (formerly marketed as Jira Software, now unified with Jira Work Management) remains the flagship, built around agile project management with Scrum and Kanban boards at its core. Jira Service Management serves IT operations and service desk teams, handling ticketing and customer support workflows; it originated as Jira Service Desk in 2013, following Atlassian's discovery that nearly 40 per cent of their customers had already adapted the base product for service requests, and it was rebranded in 2020. At the enterprise level, Jira Align connects team delivery to strategic business goals, while Jira Product Discovery helps product teams capture feedback, prioritise ideas and build roadmaps. Together, these products span the full organisational hierarchy, from individual contributors up to executive portfolio management.
Core Features
Agile Boards and Backlog Management
Jira supports a range of agile methodologies, with two primary project templates available to teams. The Scrum template is designed for teams that deliver work in time-boxed sprints, providing backlog management, sprint planning and capacity tracking in a single view. The Kanban template, by contrast, is built around a continuous flow of work, helping teams visualise tasks as they move through each stage of a process without the constraint of fixed iterations. Both templates support custom configurations for teams whose ways of working do not map neatly to either model.
Reporting and Analytics
Jira's reporting suite provides visibility into project progress through various charts and metrics. The Burndown chart tracks remaining story points against the time left in a sprint, offering an indication of whether the team is on course to complete its committed work. The Burnup chart takes a complementary view, tracking how much work has been completed over time and making it straightforward to compare planned scope against actual delivery. These tools are useful for identifying patterns in team performance, though they are most informative when used consistently over several sprints rather than in isolation.
Custom Workflows
Teams can design workflows that reflect their own processes, defining the states an issue passes through and the transitions between them. Automation rules can be applied to handle repetitive steps without manual intervention, reducing administrative overhead on routine tasks. This flexibility is one of the more frequently cited reasons for adopting Jira, though it does require ongoing governance to prevent workflows from becoming inconsistent or unwieldy as teams and processes evolve.
Jira Query Language
Jira Query Language (JQL) provides a structured way to search and filter issues across projects, enabling teams to construct precise queries based on any combination of fields, statuses, assignees, dates and custom attributes. For organisations that invest time in learning it, JQL is a practical tool for building custom reports and dashboards. It is also the underlying mechanism for many of Jira's more advanced filtering and automation features.
Integration Options
Jira connects with a range of tools both within and outside the Atlassian ecosystem. Confluence handles documentation, Bitbucket manages code repositories and links commits directly to Jira issues, and Loom, acquired by Atlassian in 2023, adds asynchronous video communication. Third-party integrations, including Zoom and a broad catalogue of tools available through the Atlassian Marketplace, extend this further for teams with specific requirements.
Test Management With Xray
Jira does not include dedicated test management functionality by default, and teams that need to manage structured test cases alongside their development work typically turn to the Xray plugin, one of the most widely used additions in the Atlassian Marketplace. Xray operates as a native Jira application, meaning it adds new issue types directly to the Jira instance rather than sitting as a separate external tool. The issue types it introduces include Test, Test Set, Test Plan and Test Execution, all of which behave like standard Jira issues and can be searched, filtered and reported on using JQL.
A key capability is requirements traceability: Xray links test cases directly to the user stories and requirements they cover, and connects those in turn to any defects raised during execution. This gives teams a clear picture of test coverage and release readiness without having to leave Jira or reconcile data from separate systems. Test executions can be manual or automated, and Xray integrates with CI/CD toolchains (including Jenkins and Robot Framework) via a REST API, allowing automated test results to be published back into Jira and associated with the relevant requirements.
Xray also supports Behaviour-Driven Development (BDD), enabling teams to write tests in Gherkin syntax and manage them alongside their other Jira work. For organisations already using Jira as their central project management tool, Xray offers a practical route to bringing QA activities into the same workflow rather than maintaining a separate test management system.
Who is Jira Best Suited For?
Jira is generally considered most suitable for larger teams that require detailed control over workflows, reporting and resource allocation, and that have the capacity to dedicate administrative effort to the platform. Smaller teams or those without a dedicated Jira administrator may find the learning curve significant, particularly when configuring custom workflows or working with more advanced reporting features. Pricing is subscription-based, with tiers determined by user count and deployment model (cloud-hosted or self-managed), which means costs can increase substantially as an organisation grows.
Project Types: Tailoring Access to Needs
Jira divides its project spaces into two categories that serve different audiences. Team-managed projects offer simplified configuration for smaller, autonomous teams that want to get started without involving a Jira administrator. Company-managed projects grant administrators full control over customisation, permissions and settings, making them more appropriate for enterprises with complex requirements and multiple teams operating within the same instance. The two types can coexist within the same deployment, giving organisations the option to apply different governance models to different teams as their needs dictate.
Strengths and Limitations
Jira's scalability is one of its more consistent strengths, in terms of both the size of the user base it can support and the complexity of workflows it can accommodate. Its query functions give teams a precise way to interrogate project data, and its breadth of integrations means it can be connected to most standard development and collaboration toolchains.
A significant consideration for any Jira deployment is the degree of upfront decision-making it requires. Because the platform places few constraints on how it is configured, teams must establish their own conventions around workflow design, issue hierarchy, naming and permissions before adoption begins in earnest. Without that groundwork, it is straightforward for individual teams to configure Jira in incompatible ways, making cross-team reporting difficult and creating inconsistencies that become harder to unpick over time. Organisations that treat Jira as something to be governed, rather than simply installed, tend to get considerably more out of it.
The principal technical limitation is its dependence on the wider Atlassian ecosystem. Advanced portfolio planning, capacity forecasting and cross-programme dependency management typically require either a higher-tier plan or additional tooling. Advanced Roadmaps (now called Plans) are available natively within Jira Premium and Enterprise, providing cross-team timeline planning and scenario modelling. For capacity planning, budget tracking and timesheet management, many organisations turn to third-party Marketplace tools such as Tempo. Teams evaluating Jira should factor in both the cost of the appropriate licence tier and any supplementary tooling they are likely to need.
Where to Go From Here
Jira has grown considerably from the issue tracker it was when first released in 2002, and is now used by over 300,000 organisations worldwide. Its capabilities are broad, and its configurability makes it adaptable to a wide range of team structures and workflows. That same configurability, however, means the platform rewards investment in setup and ongoing administration, and organisations should assess whether they have the resources to realise that potential before committing. For those looking to explore further, Atlassian's official guides, its wider documentation, the support portal, the Atlassian Community and the developer documentation are useful starting points, and there are courses from an independent provider too.
Hardening WordPress on Ubuntu and Apache: A practical layered approach
1st March 2026
Protecting a WordPress site rarely depends on a single control. Practical hardening layers network filtering, a web application firewall (WAF), careful browser-side restrictions and sensible log-driven tuning. What follows brings together several well-tested techniques and the precise commands needed to get them working, while also calling out caveats and known changes that can catch administrators out. The focus is on Ubuntu and Apache with ModSecurity and the OWASP Core Rule Set for WordPress, but complementary measures round out a cohesive approach. These include a strict Content Security Policy, Cloudflare or Nginx rules for form spam, firewall housekeeping for UFW and Docker, targeted network blocks and automated abuse reporting with Fail2Ban. Where solutions have moved on, that is noted so you do not pursue dead ends.
The Web Application Firewall
ModSecurity and the OWASP Core Rule Set
ModSecurity remains the most widespread open-source web application firewall and has been under the custodianship of the OWASP Foundation since January 2024, having previously been stewarded by Trustwave for over a decade. It integrates closely with the OWASP Core Rule Set (CRS), which aims to shield web applications from a wide range of attacks including the OWASP Top Ten, while keeping false alerts to a minimum. There are two actively maintained engines: 2.9.x is the classic Apache module and 3.x is the newer, cross-platform variant. Whichever engine you pick, the rule set is the essential companion. One important update is worth stating at the outset: CRS 4 replaces exclusion lists with plugins, so older instructions that toggle CRS 3's exclusions no longer apply as written.
Installing ModSecurity on Ubuntu
On Ubuntu 24.04 LTS, installing the Apache module is straightforward. The universe repository ships libapache2-mod-security2 at version 2.9.7, which meets the 2.9.6 minimum required by CRS 4.x, so no third-party repository is needed. You can fetch and enable ModSecurity with the following commands:
sudo apt install libapache2-mod-security2
sudo a2enmod security2
sudo systemctl restart apache2It is worth confirming the module is loaded before you proceed:
apache2ctl -M | grep securityThe default configuration runs in detection-only mode, which does not block anything. Copy the recommended file into place and then edit it so that SecRuleEngine On replaces SecRuleEngine DetectionOnly:
sudo cp /etc/modsecurity/modsecurity.conf-recommended /etc/modsecurity/modsecurity.confOpen /etc/modsecurity/modsecurity.conf and make the change, then restart Apache once more to apply it.
Pulling in the Core Rule Set
The next step is to pull in the latest Core Rule Set and wire it up. A typical approach is to clone the upstream repository, move the example setup into place and move the directory named rules into /etc/modsecurity:
cd
git clone https://github.com/coreruleset/coreruleset.git
cd coreruleset
sudo mv crs-setup.conf.example /etc/modsecurity/crs-setup.conf
sudo mv rules/ /etc/modsecurity/Now adjust the Apache ModSecurity include so that the new crs-setup.conf and all files in /etc/modsecurity/rules are loaded. On Ubuntu, that is governed by /etc/apache2/mods-enabled/security2.conf. Edit this file to reference the new paths, remove any older CRS include lines that might conflict, and then run:
sudo systemctl restart apache2On Ubuntu 26.04 (due for release in April 2026), the default installation includes a pre-existing CRS configuration at /etc/modsecurity/crs/crs-setup.conf. If this is left in place alongside your own cloned CRS, Apache will fail to start with a Found another rule with the same id error. Remove it before restarting:
sudo rm -f /etc/modsecurity/crs/crs-setup.confWordPress-Specific Allowances in CRS 3
WordPress tends to work far better with CRS when its application-specific allowances are enabled. With CRS 3, a variable named tx.crs_exclusions_wordpress can be set in crs-setup.conf to activate those allowances. The commented "exclusions" block in that file includes a template SecAction with ID 900130 that sets application exclusions. Uncomment it and reduce it to the single line that enables the WordPress flag:
SecAction
"id:900130,
phase:1,
nolog,
pass,
t:none,
setvar:tx.crs_exclusions_wordpress=1"Reload Apache afterwards with sudo service apache2 reload. If you are on CRS 4, do not use this older mechanism. The project has replaced exclusions with a dedicated WordPress rule exclusions plugin, so follow the CRS 4 plugin documentation instead. The WPSec guide to ModSecurity and CRS covers both the CRS 3 and CRS 4 approaches side by side if you need a reference that bridges the two versions.
Log Retention and WAF Tuning
Once the WAF is enforcing, logs become central to tuning. Retention is important for forensics as well as for understanding false positives over time, so do not settle for the default two weeks. On Ubuntu, you can extend Apache's logrotate configuration at /etc/logrotate.d/apache2 to keep weekly logs for 52 weeks, giving you a year of history to hand.
If you see Execution error – PCRE limits exceeded (-8) in the ModSecurity log, increase the following in /etc/modsecurity/modsecurity.conf to give the regular expression engine more headroom:
SecPcreMatchLimit 1000000
SecPcreMatchLimitRecursion 1000000File uploads can generate an Access denied with code 403 (phase 2). Match of "eq 0" against "MULTIPART_UNMATCHED_BOUNDARY" required error. One remedy used in practice is to comment out the offending check around line 86 of modsecurity.conf and then reload. The built-in Theme Editor can trigger Request body no files data length is larger than the configured limit. Bumping SecRequestBodyLimit to 6000000 addresses that, again followed by a reload of Apache.
Whitelisting Rule IDs for Specific Endpoints
There are occasions where whitelisting specific rule IDs for specific WordPress endpoints is the most pragmatic way to remove false positives without weakening protection elsewhere. Creating a per-site or server-wide include works well; on Ubuntu, a common location is /etc/apache2/conf-enabled/whitelist.conf. For the Theme Editor, adding a LocationMatch block for /wp-admin/theme-editor.php that removes a small set of well-known noisy IDs can help:
<LocationMatch "/wp-admin/theme-editor.php">
SecRuleRemoveById 300015 300016 300017 950907 950005 950006 960008 960011 960904 959006 980130
</LocationMatch>For AJAX requests handled at /wp-admin/admin-ajax.php, the same set with 981173 added is often used. This style of targeted exclusion mirrors long-standing community advice: find the rule ID in logs, remove it only where it is truly safe to do so, and never disable ModSecurity outright. If you need help finding noisy rules, the following command (also documented by InMotion Hosting) summarises IDs, hostnames and URIs seen in errors:
grep ModSecurity /usr/local/apache/logs/error_log | grep "[id" |
sed -E -e 's#^.*[id "([0-9]*).*hostname "([a-z0-9-_.]*)"].*uri "(.*?)".*"#1 2 3#' |
cut -d" -f1 | sort -n | uniq -c | sort -nAdd a matching SecRuleRemoveById line in your include and restart Apache.
Browser-Side Controls: Content Security Policy
Beyond the WAF, browser-side controls significantly reduce the harm from injected content and cross-site scripting. A Content Security Policy (CSP) is both simple to begin and very effective when tightened. An easy starting point is a report-only header that blocks nothing but shows you what would have been stopped. Adding the following to your site lets you open the browser's developer console and watch violations scroll by as you navigate:
Content-Security-Policy-Report-Only: default-src 'self'; font-src 'self'; img-src 'self'; script-src 'self'; style-src 'self'From there, iteratively allowlist the external origins your site legitimately uses and prefer strict matches. If a script is loaded from a CDN such as cdnjs.cloudflare.com, referencing the exact file or at least the specific directory, rather than the whole domain, reduces exposure to unrelated content hosted there. Inline code is best moved to external files. If that is not possible, hashes can allowlist specific inline blocks and nonces can authorise dynamically generated ones, though the latter must be unpredictable and unique per request. The 'unsafe-inline' escape hatch exists but undermines much of CSP's value and is best avoided.
Once the console is clean, you can add real-time reporting to a service such as URIports (their guide to building a solid CSP is also worth reading) by extending the header:
Content-Security-Policy-Report-Only: default-src 'self'; ...; report-uri https://example.uriports.com/reports/report; report-to defaultPair this with a Report-To header so that you can monitor and prioritise violations at scale. When you are satisfied, switch the key from Content-Security-Policy-Report-Only to Content-Security-Policy to enforce the policy, at which point browsers will block non-compliant content.
Server Fingerprints and Security Headers
While working on HTTPS and header hardening, it is useful to trim server fingerprints and raise other browser defences, and this Apache security headers walkthrough covers the rationale behind each directive clearly. Apache's ServerTokens directive can be set in /etc/apache2/apache.conf to mask version details. Options range from Full to Prod, with the latter sending only Server: Apache. Unsetting X-Powered-By in /etc/apache2/httpd.conf removes PHP version leakage. Adding the following headers in the same configuration file keeps responses out of hostile frames, asks browsers to block detected XSS and prevents MIME type sniffing:
X-Frame-Options SAMEORIGIN
X-XSS-Protection 1;mode=block
X-Content-Type-Options nosniffThese are not replacements for fixes in application code, but they do give the browser more to work with. If you are behind antivirus products or corporate HTTPS interception, bear in mind that these can cause certificate errors such as SEC_ERROR_UNKNOWN_ISSUER or MOZILLA_PKIX_ERROR_MITM_DETECTED in Firefox. Disabling encrypted traffic scanning in products like Avast, Bitdefender or Kaspersky, or ensuring enterprise interception certificates are correctly installed in Firefox's trust store, resolves those issues. Some errors cannot be bypassed when HSTS is used or when policies disable bypasses, which is the intended behaviour for high-value sites.
Contact Form Spam
Contact form spam is a different but common headache. Analysing access logs often reveals that many automated submissions arrive over HTTP/1.1 while legitimate traffic uses HTTP/2 with modern browser stacks, and this GridPane analysis of a real spam campaign confirms the pattern in detail. That difference gives you something to work with.
Filtering by Protocol in Cloudflare
You can block or challenge HTTP/1.x access to contact pages at the edge with Cloudflare's WAF by crafting an expression that matches both the old protocol and a target URI, while exempting major crawlers. A representative filter looks like this:
(http.request.version in {"HTTP/1.0" "HTTP/1.1" "HTTP/1.2"}
and http.request.uri eq "/contact/"
and not http.user_agent contains "Googlebot"
and not http.user_agent contains "Bingbot"
and not http.user_agent contains "DuckDuckBot"
and not http.user_agent contains "facebot"
and not http.user_agent contains "Slurp"
and not http.user_agent contains "Alexa")Set the action to block or to a managed challenge as appropriate.
Blocking Direct POST Requests Without a Valid Referrer
Another useful approach is to cut off direct POST requests to /wp-admin/admin-ajax.php and /wp-comments-post.php when the Referer does not contain your domain. In Cloudflare, this becomes:
(http.request.uri contains "/wp-admin/admin-ajax.php"
and http.request.method eq "POST"
and not http.referer contains "yourwebsitehere.com")
or
(http.request.uri contains "/wp-comments-post.php"
and http.request.method eq "POST"
and not http.referer contains "yourwebsitehere.com")The same logic can be applied in Nginx with small site includes that set variables based on $server_protocol and $http_user_agent, then return 403 if a combination such as HTTP/1.1 on /contact/ by a non-whitelisted bot is met. It is sensible to verify with Google Search Console or similar that legitimate crawlers are not impeded once rules are live.
Complementary Mitigations Inside WordPress
Three complementary tools work well alongside the server-side measures already covered. The first is WP Armour, a free honeypot anti-spam plugin that adds a hidden field to comment forms, contact forms and registration pages using JavaScript. Because spambots cannot execute JavaScript, the field is never present in a genuine submission, and any bot that attempts to fill it is rejected silently. No CAPTCHA, API key or subscription is required, and the plugin is GDPR-compliant with no external server calls.
The second measure is entirely native to WordPress. Navigate to Settings, then Discussion and tick "Automatically close comments on articles older than X days." Spammers disproportionately target older content because it tends to be less actively monitored, so setting this to 180 days significantly reduces spam without affecting newer posts where discussion is still active. The value can be adjusted to suit the publishing cadence of the site.
The third layer is Akismet, developed by Automattic. Akismet passes each comment through its cloud-based filter and marks likely spam before it ever appears in the moderation queue. It is free for personal sites and requires an API key obtained from the Akismet website. Used alongside WP Armour, the two cover different vectors: WP Armour stops most bot submissions before they are processed at all, while Akismet catches those that reach the comment pipeline regardless of origin. Complementing both, reCAPTCHA v3 or hCaptcha (where privacy demands it) and simple "bot test" questions remain useful additions, though any solution that adds heavy database load warrants testing before large-scale deployment.
Host-Level Firewalls: UFW and Docker
Host-level firewalls remain important, particularly when Docker is in the mix. Ubuntu's UFW is convenient, but Docker's default iptables rules can bypass UFW and expose published ports to the public network even when ufw deny appears to be in place. One maintained solution uses the kernel's DOCKER-USER chain, so UFW regains control without disabling Docker's iptables management.
Appending a short block to /etc/ufw/after.rules that defines ufw-user-forward, a ufw-docker-logging-deny target and a DOCKER-USER chain, then jumps from DOCKER-USER into ufw-user-forward, allows UFW to govern forwarded traffic. Returning early for RELATED,ESTABLISHED connections, dropping invalid ones, accepting docker0-to-docker0 traffic and returning for RFC 1918 source ranges preserves internal communications. New connection attempts from public networks destined for private address ranges are logged and dropped, with a final RETURN handing off to Docker's own rules for permitted flows.
Restart UFW to activate the change:
sudo systemctl restart ufw
# or
sudo ufw reloadFrom that point, you can allow external access to a container's service port:
ufw route allow proto tcp from any to any port 80Or scope to a specific container IP if needed:
ufw route allow proto tcp from any to 172.17.0.2 port 80UDP rules follow the same pattern. If you prefer not to edit by hand, the UFW-docker helper script can install, check and manage these rules for you. It supports options to auto-detect Docker subnets, supports IPv6 by enabling ip6tables and a ULA (Unique Local Address) range in /etc/docker/daemon.json and can manage Swarm service exposure from manager nodes.
Should you instead use Firewalld, note that it provides a dynamically managed firewall with zones, a D-Bus API and runtime versus permanent configuration separation. It is the default in distributions such as RHEL, CentOS, Fedora and SUSE, and it also works with Docker's iptables backend, though the interaction model differs from UFW's.
Keeping Firewall Rules Tidy
Keeping firewall rules tidy is a small but useful habit. UFW can show verbose and numbered views of its state, as Linuxize's UFW rules guide explains in full:
sudo ufw status verbose
sudo ufw status numberedDelete rules safely by number or by specification:
sudo ufw delete 4
sudo ufw delete allow 80/tcpIf you are scripting changes, the --force flag suppresses the interactive prompt. Take care never to remove your SSH allow rule when connected remotely, and remember that rule numbers change after deletions, so it is best to list again before removing the next one.
Logging Abusers with Fail2Ban and AbuseIPDB
Logging abusers and reporting them can reduce repeat visits. Fail2Ban watches logs for repeated failures and bans IPs by updating firewall rules for a set period. It can also report to AbuseIPDB via an action that was introduced in v0.10.0 (January 2017), which many installations have today.
Confirm that /etc/fail2ban/action.d/abuseipdb.conf exists and that your /etc/fail2ban/jail.local defines action_abuseipdb = abuseipdb. Within each jail that you want reported, add the following alongside your normal ban action, using categories that match the jail's purpose, such as SSH brute forcing:
%(action_abuseipdb)s[abuseipdb_apikey="my-api-key", abuseipdb_category="18,22"]Reload with fail2ban-client reload and watch your AbuseIPDB reported IPs page to confirm submissions are flowing. If reports do not arrive, check /var/log/fail2ban.log for cURL errors and ensure your API key is correct, bearing in mind default API limits and throttling. Newer Fail2Ban versions (0.9.0 and above) use a persistent database, so re-reported IPs after restart are less of a concern. If you run older releases, a wrapper script can avoid duplicates by checking ban times before calling the API.
Blocking Provider Ranges
Occasionally, administrators choose to block traffic from entire provider ranges that are persistent sources of scanning or abuse. There are scripts such as the AWS-blocker tool that fetch the official AWS IPv4 and IPv6 ranges and insert iptables rules to block them all, and community posts such as this rundown of poneytelecom.eu ranges that shares specific ranges associated with problematic hosts for people who have seen repeated attacks from those networks. Measures like these are blunt instruments that can have side effects, so they warrant careful consideration and ongoing maintenance if used at all. Where possible, it is preferable to block based on behaviour, authentication failures and reputation rather than on broad ownership alone.
Final ModSecurity Notes: Chasing False Positives
Two final ModSecurity notes help when chasing false positives. First, WordPress comments and posting endpoints can trip generic SQL injection protections such as rule 300016 when text includes patterns that appear dangerous to a naive filter, a well-documented occurrence that catches many administrators out. Watching /etc/httpd/logs/modsec_audit.log or the Apache error log immediately after triggering the offending behaviour, and then scoping SecRuleRemoveById lines to the relevant WordPress locations such as /wp-comments-post.php and /wp-admin/post.php, clears real-world issues without turning off protections globally. Second, when very large responses are legitimately expected in parts of wp-admin, increasing SecResponseBodyLimit in an Apache or Nginx ModSecurity context can be more proportionate than whitelisting many checks at once. Always restart or reload Apache after changes so that your edits take effect.
Defence in Depth
Taken together, these layers complement each other well. ModSecurity with CRS gives you broad, configurable protection at the HTTP layer. CSP and security headers narrow the browser's attack surface and put guardrails in place for any client-side content issues. Targeted edge and server rules dampen automated spam without hindering real users or crawlers. Firewalls remain the bedrock, but modern container tooling means integrating UFW or Firewalld with Docker requires a small amount of extra care. Logs feed both your WAF tuning and your ban lists, and when you report abusers you contribute to a wider pool of threat intelligence. None of this removes the need to keep WordPress core, themes and plugins up to date, but it does mean the same attacks are far less likely to succeed or even to reach your application in the first place.
A survey of commenting systems for static websites
25th February 2026
This piece grew out of a practical problem. When building a Hugo website, I went looking for a way to add reader comments. The remotely hosted options I found were either subscription-based or visually intrusive in ways that clashed with the site design. Moving to the self-hosted alternatives brought a different set of difficulties: setup proved neither straightforward nor reliably successful, and after some time I concluded that going without comments was the more sensible outcome.
That experience is, it turns out, a common one. The commenting problem for static sites has no clean solution, and the landscape of available tools is wide enough to be disorienting. What follows is a survey of what is currently out there, covering federated, hosted and self-hosted approaches, so that others facing the same decision can at least make an informed choice about where to invest their time.
Federated Options
At one end of the spectrum sit the federated solutions, which take the most principled approach to data ownership. Federated systems such as Cactus Comments stand out by building on the Matrix open standard, a decentralised protocol for real-time communication governed by the Matrix.org Foundation. Because comments exist as rooms on the Matrix network, they are not siloed within any single server, and users can engage with discussions using an existing Matrix account on any compatible home server, or follow threads using any Matrix client of their choosing. Site owners, meanwhile, retain the flexibility to rely on the public Cactus Comments service or to run their own Matrix home server, avoiding third-party tracking and centralised control alike. The web client is LGPLv3 licensed and the backend service is AGPLv3 licensed, making the entire stack free and open source.
Solutions for Publishers and Media Outlets
For publishers and media organisations, Coral by Vox Media offers a well-established and feature-rich alternative. Originally founded in 2014 as a collaboration between the Mozilla Foundation, The New York Times and The Washington Post, with funding from the Knight Foundation, it moved to Vox Media in 2019 and was released as open-source software. It provides advanced moderation tools supported by AI technology, real-time comment alerts and in-depth customisation through its GraphQL API. Its capacity to integrate with existing user authentication systems makes it a compelling choice for organisations that wish to maintain editorial control without sacrificing community engagement. Coral is currently deployed across 30 countries and in 23 languages, a breadth of adoption that reflects its standing among publishers of all sizes. The team has recently expanded the product to include a live Q&A tool alongside the core commenting experience, and the open-source codebase means that organisations with the technical resources can self-host the entire platform.
A strong alternative for publishers who handle large discussion volumes is GraphComment, a hosted platform developed by the French company Semiologic. It takes a social-network-inspired approach, offering threaded discussions with real-time updates, relevance-based sorting, a reputation-based voting system that enables the community to assist with moderation, and a proprietary Bubble Flow interface that makes individual threads indexable by search engines. All data are stored on servers based in France, which will appeal to publishers with European data-residency requirements. Its client list includes Le Monde, France Info and Les Echos, giving it considerable credibility in the media sector.
Hosted Solutions: Ease of Setup and Performance
Hosted solutions cater to those who prioritise simplicity and page performance above all else. ReplyBox exemplifies this approach, describing itself as 15 times lighter than Disqus, with a design focused on clean aesthetics and fast page loads. It supports Markdown formatting, nested replies, comment upvotes, email notifications and social login via Google, and it comes with spam filtering through Akismet. A 14-day free trial is available with no payment required, and a WordPress plugin is offered for those already on that platform.
Remarkbox takes a similarly restrained approach. Founded in 2014 by Russell Ballestrini after he moved his own blog to a static site and found existing solutions too slow or ad-laden, it is open source, carries no advertising and performs no user tracking. Readers can leave comments without creating an account, using email verification to confirm their identity, and the platform operates on a pay-what-you-can basis that keeps it accessible to smaller sites. It supports Markdown with real-time comment previews and deeply nested replies, and its developer notes that comments that are served through the platform contribute to SEO by making user-generated content indexable by search engines.
The choice between hosted and self-hosted systems often hinges on the trade-off between convenience and control. Staticman was a notable option in this space, acting as a Node.js bridge that committed comment submissions as data files directly to a GitHub or GitLab repository. However, its website is no longer accessible, and the project has been effectively abandoned since around 2020, with its maintainers publicly confirming in early 2024 that neither they nor the original author have been active on it for some time and that no volunteer has stepped forward to take it over. Those with a need for similar functionality are directed by the project's own contributors towards Cloudflare Workers-based alternatives. Utterances remains a viable option in this category, using GitHub Issues as its backend so that all comment data stays within a repository the site owner already controls. It requires some technical setup, but rewards that effort with complete data ownership and no external dependencies.
Open-Source, Self-Hosted Options
For developers who value privacy and data sovereignty above the convenience of a hosted service, open-source and self-hosted options present a natural fit. Remark42 is an actively maintained project that supports threaded comments, social login, moderation tools and Telegram or email notifications. Written in Python and backed by a SQLite database, Isso has been available since 2013 and offers a straightforward deployment with a small resource footprint, together with anonymous commenting that requires no third-party authentication. Both projects reflect a broader preference among privacy-conscious developers for keeping comment data entirely under their own roof.
The Case of Disqus
Valued for its ease of integration and its social features, Disqus remains one of the most widely recognised hosted commenting platform. However, it comes with well-documented drawbacks. Disqus operates as both a commenting service and a marketing and data company, collecting browsing data via tracking scripts and sharing it with third-party advertising partners. In 2021, the Norwegian Data Protection Authority notified Disqus of its intention to issue an administrative fine of approximately 2.5 million euros for processing user data without valid consent under the General Data Protection Regulation. However, following Disqus's response, the authority's final decision in 2024 was to issue a formal reprimand rather than impose the financial penalty. The proceedings nonetheless drew renewed attention to the privacy implications of relying on the platform. Site owners who prefer the convenience of a hosted service without those trade-offs may find more suitable alternatives in Hyvor Talk or CommentBox, both of which are designed around privacy-first principles and minimal setup.
Bridging the Gap: Talkyard and Discourse
Functioning as both a commenting system and a full community forum, Talkyard occupies an interesting position in the landscape. It can be embedded on a blog in the same manner as a traditional commenting widget, yet it also supports standalone discussion boards, making it a viable option for content creators who anticipate their audience outgrowing a simple comment section.
It also happens that Discourse operates on a similar principle but at greater scale, providing a fully featured forum platform that can be embedded as a comment section on external pages. Co-founded by Jeff Atwood (also a co-founder of Stack Overflow), Robin Ward and Sam Saffron, it is an open-source project whose server side is built on Ruby on Rails with a PostgreSQL database and Redis cache, while the client side uses Ember.js. Both Talkyard and Discourse are available as hosted services or as self-hosted installations, and both carry open-source codebases for those who wish to inspect or extend them.
Self-Hosting Discourse With Cloudflare CDN
For those who wish to take the self-hosted route, Discourse distributes an official Docker image that considerably simplifies deployment. The process begins by cloning the official repository into /var/discourse and running the bundled setup tool, which prompts for a hostname, administrator email address and SMTP credentials. A Linux server with at least 2 GB of memory is required, and a SWAP partition should be enabled on machines with only 1 GB.
Pairing a self-hosted instance with Cloudflare as a global CDN is a practical choice, as Cloudflare provides CDN acceleration, DNS management and DDoS mitigation, with a free tier that suits most community deployments. When configuring SSL, the recommended approach is to select Full mode in the Cloudflare SSL/TLS dashboard and generate an origin certificate using the RSA key type for maximum compatibility. That certificate is then placed in /var/discourse/shared/standalone/ssl/, and the relevant Cloudflare and SSL templates are introduced into Discourse's app.yml configuration file.
One important point during initial DNS setup is to leave the Cloudflare proxy status set to DNS only until the Discourse configuration is complete and verified, switching it to Proxied only afterwards to avoid redirect errors during first deployment. Email setup is among the more demanding aspects of running Discourse, as the platform depends on it for user authentication and notifications. The notification_email setting and the disable_emails option both require attention after a fresh install or a migration restore. Once configuration is finalised, running ./launcher rebuild app from the /var/discourse directory completes the build, typically within ten minutes.
Plugins can be added at any time by specifying their Git repository URLs in the hooks section of app.yml and triggering a rebuild. Discourse creates weekly backups automatically, storing them locally under /var/discourse/shared/standalone/backups, and these can be synchronised offsite via rsync or uploaded automatically to Amazon S3 if credentials are configured in the admin panel.
At a Glance
| Solution | Type | Best For |
|---|---|---|
| Cactus Comments | Federated, open source | Privacy-centric sites |
| Coral | Open source, hosted or self-hosted | Publishers and newsrooms |
| GraphComment | Hosted | Enhanced engagement and SEO |
| ReplyBox | Hosted | Simple static sites |
| Remarkbox | Hosted, optional self-host | Speed and simplicity |
| Utterances | Repository-backed | Developer-owned data |
| Remark42 | Self-hosted, open source | Privacy and control |
| Isso | Self-hosted, open source | Minimal footprint |
| Hyvor Talk | Hosted | Privacy-focused ease of use |
| CommentBox | Hosted | Clean design, minimal setup |
| Talkyard | Hosted or self-hosted | Comments and forums combined |
| Discourse | Hosted or self-hosted | Rich discussion communities |
| Disqus | Hosted | Ease of integration (privacy caveats apply) |
Closing Thoughts
None of the options surveyed here is without compromise. The hosted services ask you to accept some degree of cost, design constraint or data trade-off. The self-hosted and repository-backed tools demand technical time that can outweigh the benefit for a small or personal site. The federated approach is principled but asks readers to have, or create, a Matrix account before they can participate. It is entirely reasonable to weigh all of that and, as I did, conclude that going without comments is the right call for now. The landscape does shift, and a solution that is cumbersome today may become more accessible as these projects mature. In the meantime, knowing what exists and where the friction lies is a reasonable place to start.
The Open Worldwide Application Security Project: A cornerstone of digital safety in an age of evolving cybersecurity threats
24th February 2026
When Mark Curphey registered the owasp.org domain and announced the project on a security mailing list on the 9th of September 2001, there was no particular reason to expect that it would become one of the defining frameworks in the world of application security. Yet, OWASP, originally the Open Web Application Security Project, has done exactly that, growing from an informal community into a globally recognised nonprofit foundation that shapes how developers, security professionals and businesses think about the security of software. In February 2023, the board voted to update the name to the Open Worldwide Application Security Project, a change that better reflects its modern scope, which now extends beyond web applications to cover IoT, APIs and software security more broadly.
At its heart, OWASP operates on a straightforward principle: knowledge about software security should be free and openly accessible to everyone. The foundation became incorporated as a United States 501(c)(3) nonprofit charity on the 21st of April 2004, when Jeff Williams and Dave Wichers formalised the legal structure in Delaware. What began as an informal mailing list community grew into one of the most trusted independent voices in application security, underpinned by a community-driven model in which volunteers and corporate supporters alike contribute to a shared vision.
The OWASP Top 10
Of all OWASP's contributions, the OWASP Top 10 remains its most widely cited publication. First released in 2003, it is a standard awareness document representing broad consensus among security experts about the most critical risks facing web applications. The list is updated periodically, with a 2025 edition now published, following the 2021 edition.
The 2021 edition reorganised a number of longstanding categories to reflect how the threat landscape has shifted. Broken access control rose to the top position, reflecting its presence in 94 per cent of tested applications, while injection (which encompasses SQL injection and cross-site scripting, among others) fell to third place. Cryptographic failures, previously listed as sensitive data exposure, took second place. By organising risks into categories rather than exhaustive lists of individual vulnerabilities, the Top 10 provides a practical starting point for prioritising security efforts, and it is widely referenced in compliance frameworks and security policies as a baseline. It is, however, designed to be the beginning of a conversation about security rather than the final word.
Projects and Tools
Beyond the Top 10, OWASP maintains a substantial portfolio of open-source projects spanning tools, documentation and standards. Among the most widely used is OWASP ZAP (Zed Attack Proxy), a dynamic application security testing tool that helps developers and security professionals identify vulnerabilities in web applications. Originally created in 2010 by Simon Bennetts, ZAP operates as a proxy between a tester's browser and the target application, allowing it to intercept, inspect and manipulate HTTP traffic. It supports both passive scanning, which observes traffic without modifying it, and active scanning, which simulates real attacks against targets for which the tester has explicit authorisation.
The OWASP Testing Guide is another widely consulted resource, offering a comprehensive methodology for penetration testing web applications. The OWASP API Security Project addresses the distinct risks that face APIs, which have become an increasingly prominent attack surface, and OWASP also maintains a curated directory of API security tools for those working in this area. For teams managing web application firewalls, the OWASP ModSecurity Core Rule Set provides guidance on handling false positives, which is one of the more practically demanding aspects of deploying rule-based defences. OWASP SEDATED, a more specialised project, focuses on preventing sensitive data from being committed to source code repositories, addressing a problem that continues to affect development teams of all sizes. Projects are categorised by their maturity and quality, allowing users to distinguish between stable, production-ready tools and those that are still in active development, and this tiered approach helps organisations make informed decisions about which tools are appropriate for their needs.
Influence on Industry Practice
The reach of OWASP's guidance is considerable. Security teams use its materials to structure risk assessments and threat modelling exercises, while developers integrate its recommendations into code reviews and secure coding training. Auditors and regulators frequently reference OWASP standards during compliance checks, creating a shared vocabulary that helps bridge the gap between technical staff and leadership. This alignment has done much to normalise application security as a core part of the software development lifecycle, rather than a task bolted on after the fact.
OWASP's influence also extends into regulatory and standards environments. Frameworks such as PCI DSS reference the Top 10 as part of their requirements for web application security, lending it a degree of formal weight that few community-produced documents achieve. That said, OWASP is not a regulatory body and has no enforcement powers of its own.
Education and Community
Education remains a central part of OWASP's mission. The foundation runs hundreds of local chapters across the globe, providing forums for knowledge exchange at a local level, as well as global conferences such as Global AppSec that bring together practitioners from across the industry. All of OWASP's projects, tools, documentation and chapter activities are free and open to anyone with an interest in improving application security. This open model lowers barriers for those starting out in the field and fosters collaboration across academia, industry and open-source communities, creating an environment where expertise circulates freely and innovation is encouraged.
Limitations and Appropriate Use
OWASP is not without its limitations, and it is worth acknowledging these clearly. Because it is not a regulatory body, it cannot enforce compliance, and the quality of individual projects can vary considerably. The Top 10, in particular, is sometimes misread as a comprehensive checklist that, once ticked off, certifies an application as secure. It is not. It is an awareness document designed to highlight the most prevalent categories of risk, not to enumerate every possible vulnerability. Treating it as a complete audit framework rather than a starting point for more in-depth analysis is one of the most common mistakes organisations make when engaging with OWASP materials.
The OWASP Top 10 for Large Language Model Applications
As artificial intelligence has moved from research curiosity to production deployment at scale, OWASP has responded with a dedicated framework for the security risks unique to large language models. The OWASP Top 10 for Large Language Model Applications, maintained under the broader OWASP GenAI Security Project, was first published in 2023 as a community-driven effort to document vulnerabilities specific to LLM-powered applications. A 2025 edition has since been released, reflecting how quickly both the technology and the associated threat landscape have evolved.
The list shares the same philosophy as the web application Top 10, using categories to frame risk rather than enumerating every individual attack variant. Its 2025 edition identifies prompt injection as the leading concern, a class of vulnerability in which crafted inputs cause a model to behave in unintended ways, whether by ignoring instructions, leaking sensitive information or performing unauthorised actions. Other entries cover sensitive information disclosure, supply chain risks (including vulnerable or malicious components sourced from model repositories), data and model poisoning, improper output handling, excessive agency (where an LLM is granted more autonomy or permissions than its task requires) and unbounded consumption, which addresses the risk of uncontrolled resource usage leading to service disruption or unexpected cost. Two categories introduced in the 2025 edition, system prompt leakage and vector and embedding weaknesses, reflect lessons learned from real-world RAG deployments, where retrieval-augmented pipelines have introduced new attack surfaces that did not exist in earlier LLM architectures.
The LLM Top 10 is distinct from the web application Top 10 in an important respect: because the threat landscape for AI applications is evolving considerably faster than that of traditional web software, the list is updated more frequently and carries a higher degree of uncertainty about what constitutes best practice. It is best treated as a living reference rather than a settled standard, and organisations deploying LLM-powered applications would do well to monitor the GenAI Security Project's ongoing work on agentic AI security, which addresses the additional risks that arise when models are given the ability to take real-world actions autonomously.
An Ongoing Work
In an era defined by rapid technological change and an ever-expanding threat landscape, OWASP continues to occupy a distinctive and valuable position in the world of application security. Its freely available standards, practical tools and community-driven approach have made it an indispensable reference point for organisations and individuals working to build safer software. The foundation's work is a practical demonstration that security need not be a competitive advantage hoarded by a few, but a collective responsibility shared across the entire industry.
For developers, security engineers and organisations navigating the challenges of modern software development, OWASP represents both a toolkit and a philosophy: that improving the security of software is work best done together, openly and without barriers.