Technology Tales

Notes drawn from experiences in consumer and enterprise technology

13:18, 8th January 2022

Exception handling in Julia is the process of managing unexpected conditions that arise during programme execution, preventing abrupt termination by providing an alternative path for the programme to follow. Julia achieves this primarily through try-catch blocks, where potentially problematic code is placed in the try section and any resulting exception is caught and addressed in the catch section.

Exceptions propagate up through the call stack until a suitable try-catch block is found, and they can also be stored in a variable within the catch block to allow for more precise handling of multiple exception types, a method known as the canonical approach. A finally clause can be added to ensure that certain code, such as closing an open file, runs regardless of whether an exception occurred.

Custom exceptions can be thrown deliberately using the throw() function, and the error() function generates a general ErrorException, while exceptions can also be re-thrown from within a catch block to be handled further up the call stack. Julia also provides a range of built-in exception types covering common error conditions, including ArgumentError, BoundsError, DomainError, DivideError, MethodError, OverflowError and UndefVarError, among others.

09:49, 8th January 2022

New features in DataFrames.jl 1.3: Part 1

New features in DataFrames.jl 1.3: Part 2

New features in DataFrames.jl 1.3: Part 3

New features in DataFrames.jl 1.3: Part 4

New features in DataFrames.jl 1.3: Conclusion

Version 1.3 of DataFrames.jl, a Julia data manipulation package, introduced several notable improvements across performance, usability and functionality. Row aggregation operations were made significantly faster through optimised fast-path implementations for common functions such as sum, mean, median, minimum, maximum and others when used within the package's data transformation mini-language.

The release also extended broadcasting support to allow column selectors such as Not, Between, Cols and All to be resolved within the context of a data frame during transformation operations, reducing the need for verbose syntax. Indexing improvements enabled columns to be added to data frame views for the first time, provided the view was created using a full column selector, which in turn allowed standard functions like transform! to operate directly on filtered subsets of data.

A new leftjoin! function was introduced to perform in-place left joins, offering memory efficiency for large datasets and demonstrating performance comparable to the R data.table package in benchmarks. Additional changes included greater control over group ordering in the groupby function through a sort keyword argument, a fill keyword argument for the unstack function to replace missing values with a specified default, the deprecation of the delete! function in favour of deleteat! for consistency with Julia Base conventions and a fix to incorrect sorting behaviour when an empty column selector was passed to the sort function.

18:12, 2nd January 2022

ROBOCOPY is a robust command-line file copying utility built into Windows that supports a wide range of options for copying, moving and mirroring files and directories between locations. It is compatible with Windows 10, Windows 11 and multiple versions of Windows Server, as well as Azure Local. Users can specify a source, a destination and optional file filters, then apply numerous parameters to control behaviour, including copying subdirectories, preserving file attributes and timestamps, enabling restartable or backup mode, and setting retry counts and wait intervals.

Multithreaded copying is supported to improve performance, and I/O throttling options allow bandwidth usage to be managed during transfers. File selection can be refined by including or excluding files based on attributes, age, size or last access date, and logging options allow detailed output to be written to a file for review. The tool also returns exit codes indicating the outcome of each operation, ranging from no files copied through to partial or complete failures, with any code of eight or above signifying at least one failure occurred during the process.

18:12, 2nd January 2022

The xcopy command is a Windows utility that copies files and directories, including subdirectories, and is compatible with Windows 10, Windows 11 and several versions of Windows Server. It offers a wide range of optional parameters that control how files are copied, including options to copy hidden or read-only files, verify copied files against their originals, copy only files modified after a specified date and retain file attributes such as ownership and access control settings.

It can operate across network connections in restartable mode, allowing interrupted transfers to resume once connectivity is restored, and supports network compression during transfers. Unlike the diskcopy command, xcopy does not require the source and destination storage to share the same format, making it more flexible for cross-format operations. It can also be used within batch programmes, with exit codes allowing scripts to handle errors such as insufficient memory, failed writes or user-initiated cancellations in a structured way.

18:11, 2nd January 2022

XCOPY command: syntax and examples

Copying files using XCOPY commands involves specifying parameters to control source and destination paths, filter modifications and manage exclusions. The process requires careful configuration of switches such as /D for date-based filtering, /E for recursive directory copying and /EXCLUDE to omit specific files or folders. Challenges arise from date format discrepancies, exclusion pattern conflicts and the need to suppress prompts for confirmation during automated transfers. Users must test commands thoroughly to ensure accurate file selection and avoid unintended omissions or inclusions.

15:41, 25th December 2021

DataFrames.jl mini-language explained

Mastering the transformation mini-language in DataFrames.jl involves understanding how to manipulate data through various styles of operations. This includes applying functions to grouped data, specifying source and target columns using the => syntax and handling special cases such as grouping keys.

The process requires familiarity with returning different data types like NamedTuples or AbstractMatrices, which influence how results are structured. Combining multiple transformation styles within a single operation allows for complex data manipulations, such as aggregating values, renaming columns and preserving or modifying grouping keys. The ability to integrate traditional function-based transformations with modern minilanguage syntax ensures flexibility in data processing tasks.

15:39, 25th December 2021

Tips to create beautiful, publication-quality plots in Julia

Leandro Martínez outlines practical techniques for producing publication-quality plots using the Plots library with the GR backend. There are a range of features including multi-panel layouts, sequential colour schemes, overlapping bar charts with transparency, histogram bin-width matching across datasets of differing ranges, scatter plots with non-linear exponential fits and colour-consistent annotations, and the use of LaTeXStrings to render axis labels in sans-serif fonts. A user also can format tick labels in scientific notation using LaTeX styling by combining the Formatting and LaTeXStrings packages, allowing precise control over the visual appearance of numerical axes. Saving figures in PDF format preserves vector quality, while conversions to high-resolution bitmap formats such as TIFF or PNG via image editing software can be done when needed.

14:38, 25th December 2021

One thousand and one stories in Julia

A blog post by Bogumił Kamiński, written to mark his 1,000th answer on Stack Overflow for the Julia programming language, explores the surprising and sometimes counterintuitive behaviour of Julia's sum function. Using practical examples, he demonstrates that sum does not simply use the standard addition operator but instead relies on an internal reduction operator called Base.add_sum, which promotes scalar integers to wider types but does not do so for vectors, leading to unexpected results such as integer overflow.

He further shows that sum uses reduce rather than foldl, meaning the order of operations is not guaranteed and can produce inconsistent outcomes depending on collection size and element types. Additional quirks arise when using the init keyword argument with floating-point numbers, where differing summation orders produce slightly different results, a known consequence of IEEE 754 floating-point arithmetic. His practical advice is to ensure collections contain elements of a consistent and appropriate type, and to treat floating-point results as approximations rather than exact values.

11:39, 25th December 2021

DTable – an early performance assessment of a new distributed table implementation

Addressing the need for efficient out-of-core processing of large tabular datasets, a new distributed table implementation in Julia leverages existing tools such as Dagger and Tables.jl to enable scalable computation across multiple threads, workers and machines. Early performance evaluations highlight significant improvements over direct competitors like Dask in several benchmarks, particularly for reduction and shuffle operations, though challenges remain in optimising thread safety and reducing overhead for smaller datasets.

The project is actively developed as part of the Dagger.jl package, with current focus on refining algorithms and expanding compatibility with a broader range of data processing tasks. While initial results demonstrate potential for handling complex tabular workloads, further refinement is required to enhance stability, scalability and integration with Julia's ecosystem. Users are encouraged to explore the implementation through available documentation and contribute feedback to shape its future development.

18:08, 24th December 2021

Working with dates and times in Julia involves using the Dates package to create, manipulate and format date and time objects, which includes functions for generating specific dates, extracting components like hours or minutes and converting between formats such as ISO8601 or Unix epoch time. The module allows users to define dates with varying levels of detail, apply formatting rules using specific character codes, handle time zones and utilise tools like strftime and unix2datetime for conversions, enabling precise control over temporal data in applications.

  • The content, images, and materials on this website are protected by copyright law and may not be reproduced, distributed, transmitted, displayed, or published in any form without the prior written permission of the copyright holder. All trademarks, logos, and brand names mentioned on this website are the property of their respective owners. Unauthorised use or duplication of these materials may violate copyright, trademark and other applicable laws, and could result in criminal or civil penalties.

  • All comments on this website are moderated and should contribute meaningfully to the discussion. We welcome diverse viewpoints expressed respectfully, but reserve the right to remove any comments containing hate speech, profanity, personal attacks, spam, promotional content or other inappropriate material without notice. Please note that comment moderation may take up to 24 hours, and that repeatedly violating these guidelines may result in being banned from future participation.

  • By submitting a comment, you grant us the right to publish and edit it as needed, whilst retaining your ownership of the content. Your email address will never be published or shared, though it is required for moderation purposes.