Technology Tales

GxP Compliance in Pharma Made Easier: Good Documentation Practices with R Markdown and {officedown}

In regulated pharmaceutical industries, maintaining rigorous documentation standards is essential for ensuring consumer safety and product reliability. GxP, a globally recognised framework covering practices such as Good Clinical Practice and Good Manufacturing Practice, places significant emphasis on Good Documentation Practices, requiring that records be traceable, accountable and data-integrity compliant. Meeting these standards manually is time-consuming and error-prone, which is where programmatic tools offer a practical advantage. R Markdown enables the creation of automated, reproducible and testable regulatory documents in multiple formats, reducing the burden of repetitive manual reporting.

However, its formatting flexibility is limited when precise structural or stylistic requirements must be met. The R package officedown addresses these shortcomings by extending R Markdown with capabilities more suited to generating Microsoft Word and PowerPoint documents, allowing users to control document structure with greater precision, apply custom styling to paragraphs and tables, use Word-based templates for consistent formatting, merge multiple documents while preserving references and numbering, and switch selected pages to landscape orientation. Together, these features make it considerably more straightforward for pharmaceutical teams to produce documentation that satisfies the detailed requirements of regulatory bodies such as the FDA and the European Medicines Agency.

Metaprogramming workshop at JuliaCon 2021

This workshop on metaprogramming in Julia, held during JuliaCon 2021, provided an in-depth exploration of techniques for writing programs that manipulate or generate other programs, highlighting Julia's strengths in this area due to its flexible code structure. Led by David P. Sanders, the session covered foundational concepts such as symbols, expressions and abstract syntax trees, alongside practical applications like macros, code generation using eval and generated functions, which enable the creation of optimised, type-specific code.

Participants worked with Jupyter notebooks offering step-by-step exercises, including examples of substituting values in expressions and building domain-specific language features. The workshop emphasised the balance between leveraging metaprogramming for efficiency and avoiding excessive complexity, underscoring its role in enhancing Julia's performance and expressiveness for scientific and computational tasks.

Jedi SAS Tricks: Explicit SQL Pass-through in DS2

DS2, a SAS programming procedure, offers significant advantages over traditional DATA step processing through its tight integration with SQL and its ability to handle data retrieval and manipulation more efficiently. Unlike a conventional data step, which requires data to be pre-sorted or indexed before using a BY statement, DS2 retrieves data using an implicit SQL query, making pre-sorting unnecessary regardless of whether the data resides in SAS or in a relational database management system such as Oracle.

A particularly powerful feature of DS2 is its support for explicit SQL pass-through queries within the SET statement, allowing programmers to leverage database-specific functions, such as Oracle's DECODE function, while still applying DS2 logic to the results. Ordering of results can be handled either within the database using an ORDER BY clause or within DS2 using a BY statement, though the latter is generally more reliable across different processing environments, particularly in distributed or multithreaded configurations such as SAS Cloud Analytic Services. It is worth noting that row ordering behaviour in DS2 can vary depending on the platform and whether code is executed in-database or through threaded processing, so programmers are advised to consult the relevant SAS documentation to better understand these nuances before relying on any particular ordering approach.

SAS FILENAME Statement: EMAIL (SMTP) Access Method

The FILENAME statement's EMAIL access method in SAS allows users to send electronic mail programmatically via SMTP, and it supports a wide range of options for customising messages, including specifying recipients, carbon copy and blind carbon copy addresses, subject lines, message priority, sensitivity levels, expiration dates and file attachments. When SAS is operating in a locked-down state, the feature is unavailable unless re-enabled by a server administrator.

Users can incorporate conditional logic within a data step to control which recipients receive which messages and can direct procedure output, images and HTML content through email. PUT statement directives provide a way to override or modify message attributes at runtime, such as changing the recipient address, subject or attached files, and they also allow actions like sending a message mid-step or clearing existing message attributes. Secure communication with SMTP servers is supported through Transport Layer Security, though message-level encryption and digital signing are not currently available. Additional system options allow users to adjust the server response wait time and manage UTC offset settings for messages sent across time zones.

How to send email using SAS

Sending emails via SAS presents several challenges, from configuration issues to security considerations. Users often encounter problems such as connection refusals, authentication failures, or emails not appearing in sent folders. These issues typically stem from incorrect SMTP settings, authentication methods, or limitations in how SAS interacts with email servers. For instance, using SMTP does not allow emails to be logged in sent folders, unlike MAPI, which is less reliable for automation.

Security is another concern, as the FROM address can be spoofed, though ISPs and email services may block such attempts. Specific examples include configuring SendGrid with SAS, where proper authentication (like using the correct API key format) is crucial. Solutions often involve verifying SMTP server details, ensuring correct port numbers and using appropriate authentication types. Additionally, restricting the FROM address to a fixed value or the user's account requires careful configuration, as SAS does not enforce this by default.

Using SYSTASK and SAS macro loops for massively parallel processing

As data volumes continue to grow at a rapid pace, sequential processing increasingly falls short of meeting the demands of timely data analysis, making parallel processing a valuable alternative. A practical approach to parallelisation in SAS environments without SAS/CONNECT combines a shell script, a main SAS programme and individual thread programmes to handle a monthly data ingestion scenario.

The shell script launches the main SAS programme in the background, passing a year-month parameter to control which data are processed. The main programme is divided into three stages: pre-processing, which captures input parameters and calculates the number of days in the relevant month; parallel processing, which uses a SAS macro loop to generate a series of SYSTASK statements that spawn separate SAS sessions simultaneously, each responsible for ingesting a single day's CSV file; and post-processing, which consolidates the resulting daily data tables into a single monthly table.

The WAITFOR statement ensures that the main session pauses until all parallel threads have completed before the final consolidation step runs. Each thread programme writes its output directly to the WORK library of the main SAS session, making the data readily available for that final step. An alternative to the macro loop approach is to use CALL EXECUTE within a data step, which produces a comparable outcome by generating and executing SYSTASK statements sequentially whilst still allowing them to run in parallel at the operating system level.

SASPy Examples

This GitHub repository provides sample Python notebooks demonstrating the functionality of SASPy, a tool for interacting with SAS software. It includes examples created by SAS, user-submitted contributions and notebooks addressing specific issues or their resolutions. The repository outlines guidelines for contributing, specifies an Apache Licence 2.0 for use, and links to external resources such as the Python website and SASPy documentation.

SASPy Documentation

This Python module facilitates interaction with SAS systems by offering application programming interfaces that allow users to initiate SAS sessions, execute analytical procedures and transfer data between SAS datasets and Pandas dataframes, alongside exchanging values with SAS macro variables. It supports connections to SAS on the same or remote hosts, provides methods for data exploration such as describe and head, and integrates additional functionalities like machine learning and econometrics through dedicated Python classes. The module requires Python 3.4 or higher, SAS 9.4 or later and Java 7 or higher for specific connection methods, enabling compatibility across various SAS platforms.

Using the Hash Object to Store and Retrieve Data in SAS

The hash object in SAS is an in-memory mechanism for efficient data storage and retrieval, using lookup keys to locate specific values. To use it, a developer must declare and instantiate the object, then define keys and data variables using dot notation method calls, specifically the DEFINEKEY, DEFINEDATA and DEFINEDONE methods.

By default, each key must be unique, though the MULTIDATA argument tag allows multiple data values to be associated with a single key, which can then be traversed using methods such as FIND, FIND_NEXT, FIND_PREV, HAS_NEXT and HAS_PREV.

Data are stored using the ADD method and retrieved using the FIND method, while the REF method can combine both operations into a single call. The SUMINC argument tag enables a running numerical summary to be maintained for each key, updated automatically whenever certain methods are called.

Two attributes, NUM_ITEMS and ITEM_SIZE, allow developers to retrieve the number of stored items and the approximate memory being consumed by the object. The hash object can also be loaded directly from an existing data set using the dataset argument tag, and where duplicate keys are present, the DUPLICATE argument tag controls how they are handled.

Linear Regression in Python

Linear regression serves as a foundational technique in statistics and machine learning, offering a method to model relationships between variables. In Python, this approach can be implemented using libraries such as NumPy, scikit-learn and statsmodels.

Each tool brings distinct advantages: NumPy handles numerical computations and array manipulations, scikit-learn provides a streamlined interface for building and evaluating models and statsmodels offers detailed statistical insights. The process typically involves importing necessary modules, preparing and transforming data, fitting a model to the data, assessing its performance and using it to make predictions.

Whether the goal is to understand relationships between variables or to forecast outcomes, linear regression remains a versatile and widely applicable method. Its implementation in Python allows for both simplicity and depth, depending on the tools chosen and the level of analysis required.