Technology Tales

LinkBlog

There are times when I just need to share some links outside the fleeting environment of Twitter, and this is where they might appear. The practice is very much in the spirit of the original weblog idea when it was about link sharing and not about writing down one’s thoughts as it subsequently became before sharing of photos, videos and other multimedia content became a mainstay of (legal) sharing for some people.

Assorted Desktop Packages

Here is some desktop software that is either commonplace in the world of Linux or needs a bit more publicity, at least in my opinion. The list is sorted alphabetically, in case you are left wondering at its first entry. As with everything in this place, it may grow or contract, but change certainly is a feature of the world of Linux anyway. That’s never a bad thing, even if it upsets some from time to time.

Calligra

This suite comes from the KDE project and includes office and graphical software. The latter includes Krita, which is described separately below, so this is an interesting collection of software.

Choqok

Linux does have a choice of Twitter/X clients and this is one of them. It’s a KDE application that also supports Pump.IO, GNU Social and Friendica instances. There are others on the support list too, though Mastodon is a surprising absence given the recent furore surrounding Twitter/X.

Darktable

The name is a play on that of Adobe’s Lightroom, and that gives you an idea of what it is about. This too allows non-destructive editing of images with the added information being kept in associated files with XMP extensions, one for each image. What the software does not have though is an image management interface like that of Lightroom or digiKam.

digiKam

This is more than an organiser, and may be the KDE project’s counterpart to Adobe’s closed source Lightroom. Its photo organising doesn’t mean automated folder creation from EXIF information like F-Spot, Shotwell or Rapid Photo Downloader. It is for that reason that I combine digiKam with the last entry on the preceding list, since I jumped ship from Shotwell. The image processing part of the application is something that I have to explore.

Eclipse

Other IDE’s have taken over me these days, but this had a use for editing PHP scripts once upon a time. It is better known for what it offers Java developers, though.

Emacs

A long-standing UNIX/Linux text editor that has been doing battle with Vi for longer than many can remember. Like the alternative, it has keyboard shortcuts that do anything but make concessions to Windows conventions, add needless steepening of any learning curve unless you find the appropriate option (CUA) that allows for some emulation of mainstream keyboard shortcuts. Nevertheless, there also is a GUI variant that makes life easier, and I have to concede that it has a history that is longer than even Microsoft itself. As if that weren’t enough compensation, it is a powerful piece of software whose functionality goes much further than text editing, whose surface I have only barely begun to scratch. The logic of the interface may differ from that to which many are accustomed, but it is consistent and well-thought-out nonetheless.

F-Spot

For a while, this was my photo organiser of choice, but it has not seen a new release since December 2010. Maybe that’s because it works well enough as it is, yet you cannot help thinking that a project with no new releases is a dead one, even if that sometime reflects how right they got things at the time.

FileZilla

Before my quest for added automation took over, this was my FTP client of choice, and its advent has made the need to buy such software extinct. That it works on both Windows and Linux is a bonus.

GIMP

The ubiquitous Photoshop challenger is maturing nicely, though its interface may not please some.

GraphicsMagick

This is very like ImageMagick (see below) with its main selling point being that it’s faster than its parent for the purpose of command line image editing; my own testing seems to support this so far. The commands that you use are similar to ImageMagick too, apart mainly from adding the gm command before the likes of convert and others. Speaking of convert, the GraphicsMagick version has yet to support the -annotate switch, so -draw needs to be used in its place.

ImageMagick

Using a command line tool for image processing may seem counter-intuitive, but there are operations where you need not have much user intervention. Included among these is image resizing and conversion between file formats, and yours truly has done both. Processing many files at a stroke comes naturally to this very useful and talented piece of software, too.

KODI

Software media centres lie largely beyond my purview, but this seems to be one of the better known of the breed. It overlays the desktop when it is running and caters for consumption of music, movies, TV, photo slideshows and games. Controversially, there even is PVR capability for recording live broadcasts as well.

Krita

For those with a more artistic bent, this is a digital drawing and illustration package that will work not only on Linux but also on Windows or OS X. The results can be striking, so it looks as if your talent may be the only limitation with this tool.

LibreOffice

Oracle’s takeover of Sun Microsystems meant that some feathers were ruffled in the open-source and free software community, and one example of a change coming from this is the forking of OpenOffice. It is that act that has brought LibreOffice into being, and it then gained so much ground that it eclipsed its parent.

LibreWolf

Mozilla may promote their wares as bing privacy-friendly, yet others are not sure, so Firefox has been forked to give LibreWolf. This removes telemetry, adds a content blocker along with other enhancements.

Mozilla Firefox

There’s no way that I could not include what once was the de facto standard web browser for Linux, though there’s competition from Chrome/Chromium now too. There is also a mobile version for phones running the Android OS.

Mozilla SeaMonkey

The original Mozilla suite still lives on, and this is what it’s called nowadays.

Mozilla Thunderbird

This has replaced Evolution on Linux systems that I use, and it comes close to eclipsing Microsoft Outlook everywhere else, too.

MythTV

The main function of this piece of software is to record broadcast TV, hence that part of the name. It also has media playback capability, and that is what makes it more of a media centre than the digital video recording functionality may suggest.

NEdit

UNIX/Linux offers plenty of text editors, so here’s another of the less well-known ones that I have encountered. Syntax highlighting is part of the offer and some menu customisation is possible too. In essence, it is a straightforward text editor that works with Windows keyboard shortcuts, but that can be no bad thing.

NetBeans

You cannot feature Eclipse in a software listing without having NetBeans too. In fact, it was NetBeans that I first encountered, and that was many moons ago. There is a PHP variant available, but that seemed very sluggish when I tried it and turned back to Eclipse, with which I have stuck ever since. That poor performance may have been caused by the variant of Java that was available to it, so I may give it another ago when I have the time.

OBS Studio

Here, OBS stands for Open Broadcaster Software, and that somewhat says what it does. In essence, we are talking about video recording and live-streaming. With the increasing pervasiveness of video like what once was the case with photography, it is easy to see the use case for this kind of software.

OpenOffice.org

Is this the office suite of choice for Linux? It certainly felt that way before Oracle bought Sun Microsystems and upset a few open-source developers. Now, the appearance of LibreOffice is going to make things look a little more interesting.

PlayOnLinux

This is a far more user-friendly way to run Windows software on Linux, using the WINE libraries in the background. The name seems to originate from game playing, though web browsers like Internet Explorer and Safari are available too, along with a selection of other software. For the adventurous, there also is the possibility of installing something you have yourself.

Privoxy

Here’s the description from the website:

Privoxy is a non-caching web proxy with advanced filtering capabilities for enhancing privacy, modifying web page data and HTTP headers, controlling access, and removing ads and other obnoxious Internet junk.

It’s available for a number of platforms, including Linux and UNIX, and offers a way of blocking ads in Google Chrome, which is how I got to hear about it. Ubuntu users can snag a copy from the usual repositories too.

Configuration is by editing text files, but the default settings have sufficient so far. Setting a browser to use it means searching through settings for the means of making it use IP address 127.0.0.1 and port 8118 for ordinary and secure HTTP connections.

Rapid Photo Downloader

When Shotwell, started to fail to download photos from ever larger memory cards, it was time to look at something else and this became the replacement. You can use it to copy images from any card reader into the directory structure of your choosing. It does nothing more than downloading, and it does it so well that it merits a mention on here.

Shotwell

This was my photo library manager of choice until its limitations when it came to handling large data volumes came to light. It is written for the GNOME desktop environment and worked well for a few years before technology overtook it. Still, it also offers limited photo editing capabilities to go with its organising skills.

UFRaw

This reader and manipulator of raw digital camera image formats acts either alone or as a plugin. It can be used via the command line or using a GUI. That makes it flexible for those times when you need things to happen without much input from yourself.

VirtualBox

All in all, this is an excellent piece of virtualisation software that makes you wonder why you’d pay for something like VMware Workstation. There is a closed source variant, but the open-source equivalent has what you’d want for personal use anyway. Windows 11 support took a while to come into place because of its TPM requirements, but that is steady these days.

VSCodium

Since the widely used VSCode is so available and appears to be open-source in nature, one does wonder why this project exists. Here is their take on that conundrum:

Microsoft’s VSCode source code is open source (MIT-licensed), but the product available for download (Visual Studio Code) is licensed under this not-FLOSS licence and contains telemetry/tracking. According to this comment from a Visual Studio Code maintainer:

When Microsoft builds Visual Studio Code, we do exactly this. They clone the VSCode repository, they lay down a customized product.json that has Microsoft-specific functionality (telemetry, gallery, logo, etc.), and then produce a build that we release under their licence.

When you clone and build from the VSCode repo, none of these endpoints are configured in the default product.json. Therefore, you generate a “clean” build, without the Microsoft customisations, which is by default licensed under the MIT licence.

The VSCodium project exists so that you don’t have to download and build from source. This project includes special build scripts that clone Microsoft’s VSCode repo, run the build commands, and upload the resulting binaries for you to GitHub releases. These binaries are licensed under the MIT licence. Telemetry is disabled.

If you want to build from source yourself, head over to Microsoft’s VSCode repo and follow their instructions. VSCodium exists to make it easier to get the latest version of MIT-licensed VS Code.

Waterfox

This is a fork of Firefox that claims to be faster and more private. From my brief test, it certainly feels faster, though I was not as able to test things on the privacy end.

Getting custom Python imports to work in Visual Studio Code

While I continue to use Spyder as my preferred Python code editor, I also tried out Visual Studio Code. Handily, this Integrated Development Environment also has facilities for working with R and Julia code as well as MarkDown text editing and adding the required extensions is enough for these applications; it helps that there is an unofficial Grammarly extension for content creation.

My Python code development makes use of the Pylance extension and it works a little differently from Spyder when it comes to including files using import statements. Spyder will look into the folder where the base script is located but the default behaviour of Pylance is that it looks in the root path of your workspace. This meant that any code that ran successfully in Spyder failed in Visual Studio Code.

The way around this was to add the required location using the python.analysis.extraPaths setting for the workspace. That meant opening Settings by navigating to File > Preferences > Settings in the menu system and entering python.analysis.extraPaths into the search box. That took me to the section that I needed and I then clicked on Add Item before entering the required path and clicking on the OK button. That was enough to fix the problem and all worked as it should after that.

A web development toolbox

Using multi-line commenting in Perl to inactivate blocks of code during testing

Recently, I needed to inactivate blocks of code in a Perl script while doing some testing. This is something that I often do in other computing languages so I sought the same in Perl. To do that, I need to use the POD methodology. This meant enclosing the code as follows.

=start

<< Code to be inactivated by inclusion in a comment >>

=cut

The =start line could use any word after the equality sign but it seems that =cut is needed to close the multi-line comment. If this was actual programming documentation, then the comment block should include some meaningful text for use with perldoc but that was not a concern here since the commenting statements would be removed afterwards anyway and it is good practice not to leave commented code in a production script or program to avoid any later confusion.

In my case, this facility allowed me to isolate the code that I needed to alter and test before putting everything back as needed. It also saved time since I did not need to individually comment out every executable line because multiple lines could be inactivated at a time.

OWASP Top 10 for Large Language Model Applications

OWASP stands for Open Web Application Security Project, and it is an online community dedicated to web application security. They are well known for their Top 10 Web Application Security Risks and late last year, they added a Top 10 for
Large Language Model (LLM) Applications.

Given that large language models made quite a splash last year, this was not before time. ChatGPT gained a lot of attention (OpenAI also has had DALL-E for generation of images for quite a while now), there are many others with Anthropic Claude and Perplexity also being mentioned more widely.

Figuring out what to do with any of these is not as easy as one might think. For someone more used to working with computer code, using natural language requests is quite a shift when you no longer have documentation that tells what can and what cannot be done. It is little wonder that prompt engineering has emerged as a way to deal with this.

Others have been plugging in LLM capability into chatbots and other applications, so security concerns have come to light, so far, I have not heard anything about a major security incident, but some are thinking already about how to deal with AI-suggested code that other already are using more and more.

Given all that, here is OWASP’s summary of their Top 10 for LLM Applications. This is a subject that is sure to draw more and more interest with the increasing presence of artificial intelligence in our everyday working and no-working lives.

LLM01: Prompt Injection

This manipulates an LLM through crafty inputs, causing unintended actions by the LLM. Direct injections overwrite system prompts, while indirect ones manipulate inputs from external sources.

LLM02: Insecure Output Handling

This vulnerability occurs when an LLM output is accepted without scrutiny, exposing backend systems. Misuse may lead to severe consequences such as Cross-Site Scripting (XSS), Cross-Site Request Forgery (CSRF), Server-Side Request Forgery (SSRF), privilege escalation, or remote code execution.

LLM03: Training Data Poisoning

This occurs when LLM training data is tampered, introducing vulnerabilities or biases that compromise security, effectiveness, or ethical behaviour. Sources include Common Crawl, WebText, OpenWebText and books.

LLM04: Model Denial of Service

Attackers cause resource-heavy operations on LLMs, leading to service degradation or high costs. The vulnerability is magnified due to the resource-intensive nature of LLMs and the unpredictability of user inputs.

LLM05: Supply Chain Vulnerabilities

LLM application lifecycle can be compromised by vulnerable components or services, leading to security attacks. Using third-party datasets, pre-trained models, and plugins can add vulnerabilities.

LLM06: Sensitive Information Disclosure

LLMs may inadvertently reveal confidential data in its responses, leading to unauthorized data access, privacy violations, and security breaches. It’s crucial to implement data sanitization and strict user policies to mitigate this.

LLM07: Insecure Plugin Design

LLM plugins can have insecure inputs and insufficient access control. This lack of application control makes them easier to exploit and can result in consequences such as remote code execution.

LLM08: Excessive Agency

LLM-based systems may undertake actions leading to unintended consequences. The issue arises from excessive functionality, permissions, or autonomy granted to the LLM-based systems.

LLM09: Overreliance

Systems or people overly depending on LLMs without oversight may face misinformation, miscommunication, legal issues, and security vulnerabilities due to incorrect or inappropriate content generated by LLMs.

LLM10: Model Theft

This involves unauthorized access, copying, or exfiltration of proprietary LLM models. The impact includes economic losses, compromised competitive advantage, and potential access to sensitive information.

SAS Data Step Hash Objects and Memory

Using hash objects in SAS data step code offers some great advantages from the speed point of view; having a set of data in memory rather than on disk makes things much faster. However, that means that you need to keep more of an eye on the amount of memory that’s being used. The first thing is to work out how much memory is available and it’s not necessarily the total amount installed on the system or, for that matter, the amount of memory per processor on a multi-processor system. What you really need is the number, in bytes, that is stored in the XMRLMEM system option and here’s a piece of code that’ll do just that:

data _null_;
mem=getoption('xmrlmem');
put mem;
run;

The XMRLMEM is itself an option that you can only declare in the system call that starts SAS up in the first place and there are advantages to keeping it under control, particularly on large multi-user servers. However, if your hash objects start to exceed what is available, here’s the sort of thing that you can expect to see:

ERROR: Hash object added 49136 items when memory failure occurred.
FATAL: Insufficient memory to execute data step program. Aborted during the EXECUTION phase.
NOTE: The SAS System stopped processing this step because of insufficient memory.
NOTE: SAS set option OBS=0 and will continue to check statements. This may cause NOTE: No observations in data set.

Those messages are a cue for you to learn to keep those hash objects and to only ever make them as large as your memory settings will allow. Another thing to note is that hash objects are best retained for rather fixed data volumes instead of ones that could outgrow their limits. There’s a certain amount of common sense in operation here but it may be that promoters of hash objects don’t mention their limitations as much as they should. If you want to find out more, SAS have a useful paper on their website and the their Knowledge Base has more on the error messages that you can get.

Selecting SAS code in the Program Editor on UNIX

Here’s a possible bugbear with programming using the SAS Display Manager in UNIX, selecting sections of code and running them. In the installations that I have encountered, the mouse selection is not retained so the code selection cannot be run. There is a fix for this and it is not the most obvious. Going to the Preferences dialogue box (Tools > Options > Preferences… from the menu bar) and selecting the Editing tab brings up the screen below:

Ensuring that "Automatically store selection" is switched off, as shown above, will allow one to select and submit sections of code from a SAS program like what is normal practice with Windows SAS. It isn’t an obvious solution but it does the trick for me.

Another way to look through WordPress code

At times in the past, I have complained about there not being sufficient documentation of the actual code used in WordPress itself. True, there is the venerable Codex but that can be incomplete and any extra help is always useful. Peter Westwood (a.k.a westi) has generated documentation from the code itself and put it up on the web for all to peruse. While we are on the subject of information on the more advanced aspects of WordPress, blogs by Mark Jaquith and Andrew Ozz are also worth a look. As the cliché goes, every little helps…

Databases & Programming

The world of UNIX appears to attract those interested in the more technical aspects of computing. Since Linux is cut from the same lineage, it is apt to include lists of computing languages. Both scripting and programming appear here despite the title, itself shortened for the sake of brevity. Since much code cutting involves working with databases, especially since data working cannot be avoided these days, these appear here too along with the stub for a list of tools.

In time, there may an opportunity to correct any imbalance between programming and scripting languages that currently exists. The original list was bare, so descriptions have been added and will be more and more needed should there be any expansion of what you find here.

Programming and Scripting Languages

Apache Groovy

My first encounter with an implementation of this language was with that belonging to a statistical computing environment (SCE) and that remains an ongoing dalliance. It is easy to think of Groovy as a way of working with a Java-based API using a scripting language, and it certainly feels like that. Saying that, it all works better if you know Java, though you do have to watch for the development of domain-specific language capability. That last comment probably applies to the aforementioned SCE in that it has its own object and method hierarchy that means that not all standard Groovy functionality is available.

Clojure

Clojure is a dynamic, functional programming language that runs on the Java Virtual Machine (JVM) and is designed for building robust and scalable software applications. It is characterised by its emphasis on immutability, persistent data structures, and seamless interoperability with Java. Clojure embraces the Lisp programming language’s principles, providing a concise syntax and powerful abstractions for managing state, concurrency, and functional programming paradigms. With its focus on simplicity, expressiveness, and the ability to leverage the vast Java ecosystem, Clojure enables developers to create efficient and maintainable code for a wide range of applications.

Erlang

Go

Computing languages often get strange names like single letters or small words like this one; that means that you need to look for “Golang” in any online search. In any case, Go was originated at Google and numbered among its inventors was one of the creators of the C programming language. The intent here is massively multithreaded system programming using stand-alone executable components, while retaining or enhancing code readability. Another facet is the ability to function efficiently in distributed computing environments like those at SoundCloud or Uber. Various tools have been written using the language, and these include the ever pervasive Docker and Kubernetes.

Julia

It remains an odd decision to give a computing language a girl’s name, but the purpose is serious. Often, there is a trade-off between speed of code writing and speed of execution, with the result being that data programming involves prototyping in one language and porting to another for production usage. The first group includes R and Python while the second includes C, C++, FORTRAN and even Java, so there is an element of translation involved that often means that different people are involved, which adds an element of error caused by misunderstandings. This gets described as the two language problem and Julia’s major raison d’être is the avoidance of that: its top-line description is that it is as quick to program as Python but runs as fast as C because of its just-in-time compilation, multiple dispatch and in-built multithreading. This also allows for extensive capabilities for scientific computing that go beyond machine learning, and an example comes in the number of differential equation solvers that are available. It also helps that meta-programming makes everything more generalisable.

Perl

It has been around since the 1980’s and still pervades, though it is not as dominant as it once was for creating dynamic websites or system administration. PHP has taken on much of the former, while Python is making inroads into the latter. Still, no list would be complete with complete without a mention of the once ubiquitous scripting language, and it once powered my online photo gallery. It may be an easier language, but there is plenty of documentation on the web with Perldoc, Perl Maven and Perlmeister being some good places to look, and Dan Massey has some interesting articles on his site too. Not only that, but it is extensible too, with plenty of extra modules to be found on CPAN.

PHP

This usurper has taken the place of Perl for powering many of the world’s websites. That the language is less verbose probably helps its case, and many if not most CMS packages make use of its versatility.

Python

It may be Google’s preferred scripting language for system administration, but it is its usefulness for Data Science where it really has shone in the eyes of many. There are numerous packages for data wrangling, data visualisation and machine learning that make the language ever present in any Data Scientist’s toolbox, and looking in the PyPi archive will allow you to find what you need. It also has its place in web scripting too, even if it is not as pervasive as PHP though CMS’s like Plone run on Python and there is the Django framework together with the Gunicorn web server.

OpenJDK

One of the acts of Jonathon Schwartz while he was head at Sun Microsystems was to make Java open source after more than a decade of its being largely proprietary, and this is the website for the project. Of course, his more notable act at Sun was to sell the company to Oracle, but that’s another story altogether…

R

This is an open-source implementation of the S language that is much appreciated by statisticians and is much used in the teaching of the subject. The base language only has so much functionality but there are many packages available that do just that and there are many to find on repositories like the CRAN and others can be found on various GitHub repositories, though these tend to be more experimental in nature. There are commonly used and well-supported mainstays that everyone uses, but there always is a need to verify that a particular package does what it claims to do. Given that, there are possibilities for data wrangling, data tabulation, data visualisation and data science. While quick to code, R is slow to execute compared with others and I have found that Python is faster, but it still has a use for smaller data sets; both keep their temporary data sets in system memory so that will help.

Rust

It came as a surprise that this Mozilla-originated language is gaining traction in scientific data analysis, possibly because it is a fast multithreaded counterpart to C and C++ with some added safety features (though these can be turned off if needed and extra care gets taken). The downsizing of Mozilla led to a sharp reduction in its team of Rust developers, and the Rust Foundation has been set up to oversee the language instead. There are online books like The Rust Programming Language and the Rust Cookbook, with the first of these also having paper and e-book counterparts from No Starch Press. For those interested in a more interactive introduction, there also is the Tour of Rust.

Databases

MariaDB

This essentially is a fork of MySQL (see below) now that Oracle owns it. The originators of MySQL are the creators of MariaDB, so their claims of it being a drop-in replacement for it may have some traction. So far, I have seen no exodus from MySQL, though.

MySQL

After being in the hands of a number of owners until it incongruously came into the custodianship of Oracle (who of course already had and still have one of their own), the database system that powers many dynamic websites almost remains a de facto standard and looks set to remain thus for now.

MongoDB

This may a document-based and not a relationship database like many of us understand them, but it still is being touted as an alternative to the more mainstream competition. Database technology isn’t just about SQL, and MongoDB champions a NoSQL approach; it sounds as if the emergence of XML might be what’s facilitating the NoSQL database technologies.

PostgreSQL

This project may have more open-source credibility than MySQL, but it seems to remain in its shadow, though that may be explained by its being a more complex piece of software to use (at least, that has been my experience, anyway). It so happens that this is what Debian installs if you specify the web server option at operating system installation time.

Tools

Git

Version control has become essential over the decades, and Linus Torvalds developed this for using on the Linux kernel project. Its usage has gone far beyond that to make it possibly the de facto standard for version control these days. The existence of GitHub and GitLab may stand testament to that, and would never have existed without Git either. There is a symbiotic relationship implicit in these.