Technology Tales

Using Pandas with Python Decimal for accurate currency arithmetic

When working with currency or other figures requiring precise decimal representation, Python's default float type and Pandas' NumPy float64 can introduce subtle inaccuracies due to their base 2 internal representation, meaning comparisons and calculations may produce unexpected results.

Two practical approaches exist for maintaining accuracy. The first involves integer arithmetic, where decimal separators are stripped and values are converted to integers before calculation, though this requires all values to share the same number of decimal places. The second, more flexible approach uses Python's Decimal module, which maintains a base 10 representation and produces exact results.

When using Decimal within Pandas, care is needed because standard aggregation methods such as sum() and mean() will silently convert Decimal objects back to float64. To preserve Decimal types, any sum operations should use apply() with a lambda function, and mean values should be calculated by combining apply() with a manual division by the length of the dataframe rather than relying on the built-in mean() method.

Julia 1.8 Highlights

This significant update to the Julia programming language brings several enhancements aimed at improving performance, usability and package management. The new version introduces a tool that provides detailed insights into how long dependencies take to load, helping developers identify bottlenecks in package initialisation. This is particularly useful for large projects where understanding load times can lead to optimisations.

The package manager now includes an indicator that shows whether a package is on its latest version and whether other packages are preventing it from updating. This helps users avoid confusion when encountering bugs or discrepancies between documentation and the code they are running. A new flag allows users to see which packages are outdated and what constraints are holding them back.

Support for creating sysimages with precompiled packages has been improved, allowing for faster load times. However, this comes with the caveat that versions of packages included in sysimages are fixed, meaning updates to those packages will not be automatically applied.

Pre-compilation has been enhanced to retain more type-inferred code, reducing the need for repeated inference during initial runs. This can significantly cut down on latency for workloads with predictable types. The changes also make pre-compilation more reliable, ensuring that methods linked to a package are cached effectively.

Apple Silicon is now a Tier 2 supported platform, with improvements to the linker reducing crashes that were common in earlier versions. This update ensures better stability on macOS devices using ARM processors. These changes collectively aim to make Julia more efficient, easier to manage and more compatible with modern hardware, addressing both developer and user needs in a rapidly evolving ecosystem.

Compiling a Custom Sysimage in Julia

Creating a custom sysimage in Julia can significantly reduce pre-compilation delays, particularly for frequently used packages like Plots, by precompiling essential functions and dependencies into a standalone file. This involves setting up a new project, defining a representative workflow in a script, compiling the sysimage using PackageCompiler.jl and then configuring Julia to load the custom image instead of the default one. The result is a faster startup time for Julia sessions, with minimal delay when accessing precompiled packages, though some steps still necessitate direct interaction with the terminal for full control over the compilation process. While this process can be initiated through terminal commands, support within VSCode remains experimental, requiring manual configuration to ensure the sysimage is recognised and used automatically when launching a REPL.

DevOps with VS Code, GIT, and SASjs

The integration of SAS development with DevOps practices through tools like SASjs and Visual Studio Code streamlines workflows for coding, testing and deploying SAS applications. Developers can configure multiple targets for isolated builds and deployments, ensuring collaboration without conflicts.

Automated linting enforces coding standards, while unit tests generate coverage reports and results in JSON or CSV formats. Documentation is automatically generated from SAS program headers, and data lineage can be tracked by defining inputs and outputs within jobs. The process supports iterative development, with commands for executing code, managing SAS Drive folders and triggering jobs in Viya.

Although the VS Code extension is tailored for Visual Studio Code, the SASjs CLI offers core functionality for those using alternative IDEs. Community engagement and feature requests are encouraged, with options to contribute directly or collaborate with SAS Apps developers. This approach enhances code quality, reduces manual checks and aligns SAS workflows with modern software development practices, facilitating smoother integration into automated release pipelines and continuous improvement.

SAS DevOps with SASjs CLI

The SASjs Command-Line-Interface offers a range of tools for managing development and deployment processes in SAS environments, supporting tasks such as creating structured repositories, compiling services with dependencies, building deployment packages, executing deployment scripts and generating documentation from code metadata. It includes features for testing macros and jobs, managing compute contexts, synchronising files and deploying frontend applications as streaming services, along with a variety of commands to streamline workflows from initial setup through to execution and validation.

Encrypting content with Ansible Vault

The Ansible vault documentation covers how to protect sensitive data such as passwords within Ansible by using encryption. It addresses the management of vault passwords, including how to choose between single and multiple passwords, work with vault IDs and store passwords securely. It also explains how to encrypt individual variables and files, pass passwords or vault IDs at runtime and configure default settings for working with encrypted content. Additional detail is provided on when encrypted files are decrypted during execution and the technical format used by Ansible vault for encrypted files.

Ansible Vault is a feature that encrypts variables and files to protect sensitive data such as passwords and keys, preventing them from being exposed as plaintext in playbooks or roles. It requires one or more passwords to encrypt and decrypt content, which can be managed via a command-line tool or accessed through a script if stored in a third-party secret manager. Encrypted content can be placed under source control for safer sharing and used in both ad hoc commands and playbooks by supplying the relevant passwords. It is worth noting that the protection applies only to data at rest, meaning that once content is decrypted, responsibility for preventing any disclosure of secrets falls to the play or plugin author.

How to Run Ansible Playbook Locally

There are four methods available for running an Ansible playbook on a local machine rather than on remote hosts. The first involves specifying localhost in the hosts directive of the playbook, optionally adding a connection parameter set to "local" for clarity. The second, and generally most recommended, approach uses the local_action clause within the playbook to designate specific tasks as local, which is commonly used for actions such as creating cloud instances. The third method involves explicitly defining localhost as an entry in the Ansible inventory file, where it can also be assigned a custom alias for reference within playbooks. The fourth method bypasses the hosts directive entirely by using command-line flags, specifically combining the connection, inventory and limit options to force execution on the local machine, though omitting the limit flag may cause Ansible to attribute local results to remote host names, which can be a source of confusion.

Ansible – “sudo: a password is required”

When running Ansible tasks that require privilege escalation using "become: true", users may encounter a "sudo: a password is required" error. There are four ways to resolve this. The first is to use the --ask-become-pass flag with the ansible-playbook command, which prompts for a password at runtime. The second is to pass the password non-interactively as an ansible_become_password variable, though this is discouraged for security reasons as the plain-text password may be exposed in command history and process listings. The third and recommended approach is to use Ansible Vault to store an encrypted version of the password in a separate file, which is then referenced during playbook execution alongside a vault password file with restricted permissions that is excluded from source control. The fourth option is to configure the target machine to allow the Ansible user to run sudo commands without a password at all, by adding an appropriate line to the sudoers file using the visudo command.

ansible.builtin.reboot module – Reboot a machine

The ansible.builtin.reboot module, introduced in Ansible 2.7, reboots a target machine, waits for it to go offline and come back up, and then confirms it is responsive before allowing further tasks to proceed. It offers a range of configurable parameters, including pre- and post-reboot delays, a customisable reboot command, a timeout setting for how long to wait for the machine to return and a test command to verify readiness. The module also allows users to define search paths for the shutdown command, set a message to display before rebooting and specify a command for identifying the last boot time. It supports check mode, enabling a prediction of changes without modifying the target system, and has a corresponding action plugin. For Windows machines, a separate module, ansible.windows.win_reboot, should be used instead.