Technology Tales

A Compendium of Clean Graphs in R

Written by Eric-Jan Wagenmakers and Quentin F. Gronau, this compendium provides a practical guide to producing clean, publication-ready graphs using R, built around the philosophy that a good graph should be as simple as possible without sacrificing necessary information. It covers a broad range of graph types, including correlations, histograms, line plots, bar plots, density plots, time series and multi-panel figures, with each example accompanied by reproducible R code that readers can copy and adapt for their own purposes. The authors emphasise several core principles throughout, namely investing sufficient time and effort, removing unnecessary graphical elements, ensuring visual balance, using large font sizes and moving beyond R's default settings.

The majority of examples are produced in base R, though some use ggplot2, and many of the included graphs were contributed by colleagues and collaborators. A dedicated section covers graphs produced by JASP, a free statistical analysis programme developed at the University of Amsterdam, while a miscellaneous section features more specialist visualisations such as funnel plots, network graphs, forest plots and heatmaps. The compendium is intended as a hands-on reference rather than a conceptual guide to choosing graph types, with the authors estimating that applying its lessons will be roughly 80 per cent copying existing code and 20 per cent adapting it to suit individual needs.

Creating PDFs with fpdf2 and Python

The fpdf2 library offers a straightforward approach to generating PDF documents using Python. It supports fundamental tasks such as adding text, images and basic formatting, making it suitable for simple reports or forms.

While it lacks advanced features like complex layouts or interactive elements, it provides limited HTML support through the HTMLMixin class, enabling the conversion of basic HTML structures into PDFs. This capability allows for the inclusion of tables, lists and styled text, though the results may not match the precision of more specialised tools. The library also integrates with frameworks like Web2Py, facilitating the creation of reports within web applications.

However, its limitations become apparent when dealing with intricate designs or dynamic content, where alternatives such as ReportLab might be more appropriate. Despite these constraints, fpdf2 remains a viable option for users requiring minimalistic PDF generation without the complexity of more feature-rich libraries.

Convert dataframe into PDF report in Python

Converting a Pandas dataframe into a PDF report in Python can be achieved using the pdfkit library, which renders HTML into PDF format and supports various image formats and complex printable documents. The process requires installing pdfkit via pip and also downloading wkhtmltopdf, which handles images and other complex rendering tasks.

Once the necessary tools are in place, data is imported into a dataframe, in this case from an Excel file containing salary information, and then converted into HTML using Pandas' built-in to_html method, which writes the output to an HTML file. That HTML file is then passed to pdfkit, which converts it into a finished PDF document using the wkhtmltopdf software installed on the local machine.

Dataframe Styling using Pandas

Pandas, the popular Python data analysis library, offers built-in styling functionality that allows analysts to transform plain dataframes into visually informative, presentation-ready tables. By using the Styler object, users can apply conditional formatting to colour-code values, such as highlighting positive figures in green and negative ones in red, as well as format how values are displayed, for example rendering decimals as percentages.

Beyond that, custom CSS can be applied to control font size, alignment and background colour for individual table elements, while additional options such as colour gradient backgrounds, maximum value highlighting and custom captions further enhance clarity. The process typically involves first wrangling data using SQL, then preparing and enriching a dataframe in Python before applying the desired visual styling, enabling data consumers to read underlying figures while also benefiting from visual cues that make insight quicker and easier to extract.

Creating PDF Reports with Pandas, Jinja and WeasyPrint

The Pandas data manipulation library can output data in various formats, but combining multiple pieces of data into a single document requires additional tooling. One effective approach involves using Jinja templates to build an HTML document from multiple pandas DataFrames and summary statistics, then converting that HTML into a PDF using a rendering library.

The process begins by reading in sales data, generating pivot tables and descriptive statistics, and passing those outputs as variables into a Jinja template, which supports features such as loops, includes and filters for formatting values. Individual sections of the report, such as per-manager breakdowns and national summary statistics, can be structured across multiple pages using a CSS page-break directive. Once the template is rendered into an HTML string, it can be converted to a styled PDF document by applying a simple stylesheet, producing a readable, multipage report from what would otherwise require considerable manual effort to assemble.

Python List sort()

Python's built-in sort() method arranges the elements of a list in ascending order by default, though it accepts two optional keyword arguments that extend its functionality. The reverse argument, which defaults to False, can be set to True to sort elements in descending order. The key argument accepts a function that determines how comparisons are made during sorting, such as using the built-in len function to sort strings by their length rather than alphabetically. When sorting strings without a key argument, the method arranges them in dictionary order, and this behaviour is reversed when the reverse argument is set to True.

Embarrassingly parallel for loops in Python

Joblib is a Python library that simplifies the process of writing parallel for loops using multiprocessing, allowing computationally intensive tasks to be distributed across multiple CPUs by expressing code as generator expressions. It supports multiple parallelisation backends, with the default being the loky backend which runs tasks in separate worker processes, though a thread-based threading backend is available for code that releases the Python Global Interpreter Lock.

The parallel_config() context manager allows users to configure backend settings such as the number of jobs, verbosity and memory mapping behaviour without hardcoding these choices into library code. Serialisation of Python objects between processes is handled by cloudpickle, which supports a broader range of objects than the standard pickle module but can be slower for large data structures, and alternative serialisation options are available for performance-sensitive use cases.

For large numerical arrays, joblib can automatically convert data to memory-mapped files to allow worker processes to share memory rather than duplicating it, reducing memory overhead significantly. To prevent over-subscription of CPU resources, joblib limits the number of threads that third-party libraries such as OpenBLAS and MKL can use within worker processes, and this limit can be adjusted programmatically using the inner_max_num_threads argument. Results from parallel calls can be returned as a list, an ordered generator or an unordered generator, offering flexibility in how and when output data are consumed and aggregated.

5 Tips for Writing Clean R Code – Leave Your Code Reviewer Comment-less

Writing clean, readable code is essential for effective collaboration and long-term project quality, particularly in R development. Developers should use comments sparingly and purposefully, ensuring they provide context rather than simply restating what the code already makes clear, and any code marked for future revision should be flagged with a TODO note that identifies the author and explains the reason. String concatenation is more readable when using glue or sprintf rather than overusing the paste function, and debugging print statements should always be removed before submitting code for review. Loops require careful consideration, as vectorised alternatives through packages such as dplyr or purrr are often more efficient, and functions like seq_along are safer than relying on length-based iteration. When sharing code, absolute file paths should be avoided in favour of relative project paths, and sensitive credentials should be stored as environmental variables rather than committed to a repository. Finally, good general programming habits make a significant difference, including using descriptive variable names in a consistent style, avoiding abbreviations, writing logical comparisons correctly, maintaining consistent spacing and following an agreed style guide, keeping code DRY by avoiding unnecessary repetition and using a linter to catch errors automatically.

Executing Shell Commands with Python

Python offers developers and system administrators a more scalable and maintainable alternative to shell scripts for automating routine tasks, providing several methods for executing shell commands. The simplest of these is the os.system() function, which runs a command stored in a string, prints any output to the console and returns an exit code indicating success or failure.

For greater control over input and output, the subprocess module is the recommended approach, with subprocess.run() accepting commands as a list of strings and offering options to suppress output, pass input directly to a command and raise exceptions automatically when errors occur. For situations requiring a programme to continue working while a shell command is still executing, subprocess.Popen provides additional flexibility, using methods such as poll() to check completion status and communicate() to handle input and output. The choice between these three methods depends on the complexity of the task at hand, with os.system() suiting simple commands, subprocess.run() offering finer control and subprocess.Popen being best suited to non-blocking operations.

Group and Aggregate by One or More Columns in Pandas

Pandas, the Python data analysis library, offers SQL-like aggregation capabilities that allow users to group and summarise data by one or more columns. Using a sample dataset of baseball players containing their team, position and age, the groupby function can be applied to a single column, such as team name, to calculate aggregate statistics like mean, minimum and maximum values.

The agg function accepts a dictionary specifying which columns to aggregate and which functions to apply, though using multiple aggregation functions on a single column produces a multi-index structure that is best resolved by renaming the resulting columns and resetting the index. Grouping by multiple columns simultaneously is achieved by passing a list of column names to groupby rather than a single string, enabling more granular analysis such as breaking down player ages by both team and position at once.