Technology Tales

Getting rsync to resolve symbolic links

Given how Dropbox changed its handling of symbolic links in 2019 such that internal links within a Dropbox file hierarchy got fixed and links leading outside from the Dropbox area no longer worked. Thankfully, the rsync utility found in many Linux and UNIX settings does not do that, as long as you have called it correctly.

By default, symbolic links are synchronised like any other file. That is what Dropbox does now. To get rsync to resolve the links as shortcuts to either a single file or more likely a folder containing more than one file, it needs the -L switch or option in the command. When that is present, the linked file or files will get synchronised and honours the point of having these links in the first place: allowing more flexibility with folder structures and avoiding any duplication of files and folders.

Unzipping more than one file at a time in Linux and macOS

To me, it sounded like a task for shell scripting, but I wanted to extract three zip archives in one go. They had come from Google Drive and contained different splits of the files that I needed, raw images from a camera. However, I found a more succinct method than the line of code that you see below (it is intended for the BASH shell):

for z in *.zip; do; unzip "$z"; done

That loops through each file that matches a glob string. All I needed was something like this:

unzip '*.zip'

Without embarking on a search, I got close but have not quoted the search string. Without the quoting, it was not working for me. To be sure that I was extracting more than I needed, I made the wildcard string more specific for my case.

Once the extraction was complete, I moved the files into a Lightroom Classic repository for working on them later. All this happened on an iMac, but the extraction itself should work on any UNIX-based operating system, so long as the shell supports it.

A way to survey hours of daylight for locations of interest

A few years back, I needed to get sunrise and sunset information for a location in Ireland. This was to help me plan visits to a rural location with a bus service going nearby, and I did not want to be waiting on the side of the road in the dark on my return journey. It ended up being a project that I undertook using the Julia programming language.

This had other uses too: one was the planning of trips to North America. This was how I learned that evenings in San Francisco were not as long as their counterparts in Ireland. Later, it had its uses in assessing the feasibility of seeing other parts of the Pacific Northwest during the month of August. Other matters meant that such designs never came to anything.

The Sunrise Sunset API was used to get the times for the start and end of daylight. That meant looping through the days of the year to get the information, but I needed to get the latitude and longitude information from elsewhere to fuel that process. While Google Maps has its uses with this, it is a manual and rather fiddly process. Sparing use of Nomintim's API is what helped with increasing the amount of automation and user-friendliness, especially what comes from OpenStreetMap.

Accessing using Julia's HTTP package got me the data in JSON format that I then converted into atomic vectors and tabular data. The end product is an Excel spreadsheet with all the times in UTC. A next step would be to use the solar noon information to port things to the correct timezone. It can be done manually in Excel and its kind, but some more automation would make things smoother.

Pandemic camera

Back at the end of 2019, I acquired a Canon EOS 90D, possibly the swansong for mid-range Canon SLR cameras. Much effort is going into mirrorless cameras, yet I retain affection for SLR cameras because of their optical viewfinders. That may have been part of the reason for the acquisition, when I already had an ageing Pentax K5 Mark II. Buying SLR cameras is one way to keep them in production.

Little did I know what lay ahead in 2020 at that stage. Until recently, this was not to be a camera that travelled widely, such were the restrictions. Nevertheless, battery life is superb and handling is good too. The only absence is not having a level in the viewfinder like the Pentax K3 Mark III or maybe any mirrorless camera.

The newer file type of CR3 caught me out at first until I adjusted my command line tooling to deal with that. File sizes were larger as well, which has an impact on storage. Otherwise, there was little to change in my workflow. That would take other technological changes, like the increasing amount of AI being built into Adobe software.

Outdoor photography is my mainstay, and it excelled at that. The autofocus works well on its 24 to 135 mm zoom lens, except perhaps from focussing on skyscapes at times. Metering produced acceptable results, though it differed from the Pentax output to which I had become accustomed. All in all, it slipped into a role like other cameras that I had.

Throughout 2020 and 2021, it provided the required service alongside other cameras that I had. The aforementioned Pentax remained in use, like an Olympus and another Canon. Overseas travel curtailed horizons, so it was around local counties like Cheshire, Derbyshire, Staffordshire and Shropshire. In September 2020, it travelled to Llandudno in North Wales, an exception to the general trend of English hikes and cycles.

Since then, it has been superseded, though. A Pentax K3 Mark III made it into my possession to become my main camera, returning me near enough to my pre-2020 practice. Curiosity about Canon mirrorless options added a Canon EOS RP and a 24 to 240 mm zoom lens. That has shorter battery life than is ideal, and its level is not as helpful as that on the Pentax K3 Mark III or the aforementioned Olympus. If anything, it may get replaced while the EOS 90D remains. My getting a new base in Ireland means that it has gone there to save me carrying a camera there from England. That should give it a new lease of life.

Version control of large files on GitHub

When you try pushing large files to a GitHub repository, you may find that you breach its 100 MB limit. When you do, you either need to buy a data pack or exclude the file from being tracked. In my case, I decided that the monthly fee for 50 GB was not overly onerous, so I added that. Excluding such files using the .gitignore functionality makes a lot of sense, too.

If you decide to proceed as I did, you will need to install git-lfs. Since that may vary by operating system, I am leaving to you to look for those details on the website that I have linked to earlier. Activating it for your user account needs the following:

git lfs install

Following that, you need to flag the file or type of file using a command like the following:

git lfs track "[file path with name or search pattern]"

Executing the above adds the file path including the file name or the search pattern (normal operating system wildcards like * work here) to a file named .gitattributes in the root of the repository folder hierarchy. If that file no longer exists, it will get created the first time that this is done. It will also need to be added to the repository using git add like any other file. A general command like the following will also do it anyway, since it covers everything in the relevant folder:

git add .

After making a commit, the next step is to push the contents into GitHub. At this stage, the large file or files will be recognised and sent to large file storage with only a text link in the main area. Everything else will be handled as normal.

While on this subject, I need to add a few words of warning. Pushing a large file to GitHub without doing things up front will cause the operation to fail. That may make the transition over to large file storage all the more tricky, since things will be out of order. Moving everything to a temporary folder and again cloning the repository was how I got out of this impasse when it happened to me. Then, I could get the large file handling set up before getting going again. It is better to sort things like this out at the start of the process, rather than attempting to remedy things part way through the process.

Keeping a file or directory out of a Git or GitHub repository

Recently, I have begun to do more version control of files with Git and GitHub. However, GitHub is not a place to keep files with log in credentials. Thus, I wanted to keep these locally but avoid having them being tracked in either Git or GitHub.

Adding the names to a .gitignore file will avoid their inclusion prospectively, but what can you do if they get added in error before you do? The answer that I found is to execute a command like the following:

git rm -r --cached [path to file or directory with its name]

That takes it out of the staging area and allows the .gitignore functionality to do its job. The -r switch makes the command recursive, should you be working with the contents of a directory. Then, the --cached flag is what does the removal from the staging area.

While the aforementioned worked for me when I had an oversight, the following is also suggested:

git update-index --assume-unchanged [path to file or directory with its name]

That may be working without a .gitignore file, which was not how I was doing things. Nevertheless, it may have its uses for someone else, so that is why I include it above.

Remedying a warning about an ECDSA host key

During some website maintenance that I have been doing using my iMac, I encountered a message like the following at one point:

Warning: the ECDSA host key for '<server name>' differs from the key for the IP address '<server IP address>'

The cause was a rebuild of one of my web servers and changes to SSH keys that it caused. The solution in my case was to issue a command like the following:

ssh-keygen -R <server IP address>

That felt neater than editing ~/.ssh/known_hosts to remove the affected key. If the command does not remedy things for you, then editing the file should help. That, after all, is what the rest of the warning message suggested to me. My qualms about messing with files made me go with the command, and that got things sorted. There had been some use of ssh-copy-id too, which might be information that is worth having to hand.

Mouse configuration in macOS

Now having a new location from which I can work, I acquired an all-in-one desktop computer for use while there. While tempted by an HP option that runs Windows, I ended up choosing an iMac instead. That gained me extra disk space and more memory at a cost. Having UNIX-style command line capability was another attraction. After living with the Windows terminal for a while, its limitations were all too apparent to me.

While I started off desktop computing on a Macintosh Classic and having owned a MacBook Pro in the more recent past, there still was a learning curve. One of these related to the configuration of the mouse supplied with the system. Whatever about only having one button and needing to learn gestures, it was the speed at which the pointer goes that really got me reaching for my more usual Evoluent. Even so, the subsequent discovery of LinearMouse makes things much more bearable once all the requisite permissions were assigned.

Getting the Evoluent configured to my liking needed another third-party application: USB Overdrive. If Evoluent's own software fitted the bill, that would have done. However, they have done some finger pointing at Apple instead of updating it to work with the latest Mac technology. There may be truth in the accusations, but it is striking that another piece of software works when theirs does not. Nevertheless, the other option worked once it got the permissions to detect the hardware. Then, it was a matter of working out which button was which on the mouse, so I could have them assigned as I wished.

After that, I could settle into the new system and get used to its idiosyncrasies. Adding Parallels got me a Windows 11 virtual machine for business compatibility, while I got going with setting up some automation using the macOS terminal. All is becoming more settled than working out of a laptop.

Needing third-party software does have a catch, though: underlying changes to macOS could scupper things. It was the sort of thing that made me move away from GNOME Shell as my Linux desktop. The extensions on which I was depending kept getting obsoleted by every new release. It is something to watch, even if macOS evolves less dramatic than GNOME 3 in its various forms.

Avoid seeing "before you leave" messages in Firefox

After updating a page in a Textpattern instance that I have, I got a pop-up message asking me to confirm my intention to leave the editor page. While recalling that there was a way to avoid these extra dialogue boxes, I could not remember what it was. In any case, it was hardly as if I needed to add extra mouse clicks to my day, even if there may have been good intentions on the CMS developer's side.

The solution going into about:config. If doing this for the first time, you will be presented with a confirmation screen with an override tick box for later incursions. Once past the screen in question (assuming that you have not overridden it at a previous encounter), the next step is to search for dom.disable_beforeunload and set its value to true, since the default value is false.

While that did what I needed, I also noted another option called dom.require_user_interaction_beforeunload with a value of true. One has to wonder if setting that to false instead of changing dom.disable_beforeunload would have a similar effect. When I tried this, dom.require_user_interaction_beforeunload did not work as fully as dom.disable_beforeunload. Thus, I suggest sticking with the latter for now.

Turn off display of popular highlights in Kindle apps for Windows and Android

When I read books on a PC, I often make use of the Amazon Kindle web app. However, I do use its Android and iOS apps on mobile devices, and the Windows app remains available. On these, I never have taken to using annotations, though the facility does have its uses for many. Another feature that I rarely relish is the display of popular highlights, since I find this a little intrusive. Usually, I go about turning it off for that very reason.

On the Windows app, this is straightforward enough. Go to Tools > Options through the menu bar. On the dialogue box that produces, pick the Annotations screen and remove the tick mark in the Popular Highlights section. Then, click on the Save button to close the settings box and return to the main application screen.

Doing the same on Android is much less obvious. First, you need to open a book. Then, tap on the text size icon (Aa) followed by doing the same on the More menu item in the pane that appears. Scroll downward until you find Popular Highlights and toggle the setting to its off position. Lastly, swipe down the pane to close it. Though you have done this with one book open, it applies to all.

While some have commented that touchscreen devices can feel more intuitive to use, that has not been born out by what Amazon has done. It fits into the same category as how they responded when Google changed the rules for in-app purchases. Then, Amazon decided to remove this from their app. While that was a financial and business decision, their approach to user experience on their Android app does need another look.