Technology Tales

Adventures & experiences in contemporary technology

Limiting Google Drive upload & synchronisation speeds using Trickle

9th October 2021

Having had a mishap that lost me some photos in the early days of my dalliance with digital photography, I have been far more careful since then and that now applies to other files as well. Doing regular backups is a must that you find reiterated by many different authors and the current computing climate makes doing that more vital than it ever was.

So, as well as having various local backups, I also have remote ones in the form of OneDrive, Dropbox and Google Drive. These more correctly are file synchronisation services but disciplined use can make them useful as additional storage facilities in the interests of maintaining added resilience. There also are dedicated backup services that I have seen reviewed in the likes of PC Pro magazine but I have to make use of those.

Insync

Part of my process for dealing with new digital photo files is to back them up to Google Drive and I did that with a Windows client in the early days but then moved to Insync running on Linux Mint. One drawback to the approach is that this hogs the upload bandwidth of an internet connection that has yet to move to fibre from copper cabling. Having fibre connections to a local cabinet helps but a 100 KiB/s upload speed is easily overwhelmed and digital photo file sizes keep increasing. It does not help that I insist on using more flexible raw formats like DNG, CR2 or CR3 either.

Making fewer images could help to cut the load but I still come away from an excursion with many files because I get so besotted with my surroundings. This means that upload sessions take numerous hours and can extend across calendar days. Ultimately, this makes my internet connection far less usable so I want to throttle upload speed much like what is possible in the Transmission BitTorrent client or in the Dropbox client. Unfortunately, this is not available in Insync so I have tried using the trickle command instead and an example is below:

trickle -d 2000 -u 50 insync

Here, the upload speed is limited to 50 KiB/s while the download speed is limited to 2000 KiB/s. In my case, the latter of these hardly matters while the former leaves me with acceptable internet usability. Insync does not work smoothly with this, however, so occasional restarts are needed to keep file uploads progressing and CPU load also is higher. As rough as the user experience feels, uploads can continue in parallel with other work.

gdrive

One other option that I am exploring is the use of the command-line tool gdrive and this appears to work well with trickle. After downloading and installing the tool, getting going is a matter of issuing the following command and following the instructions:

gdrive about

On web servers, I even have the tool backing up things to Google Drive on a scheduled basis. Because of a Google Drive limitation that I have encountered not only with gdrive but also with Insync and Google’s own Windows Google Drive client, synchronisation only can happen with two new folders, one local and the other remote. Handily, gdrive supports the usual bash style commands for working with remote directories so something like the following will create a directory on Google Drive:

gdrive mkdir ttdc [ID for parent folder]

Here, the ID for the parent folder may be omitted but it can be obtained by going to Google Drive online and getting a link location by right-clicking on a folder and choosing the appropriate context menu item. This gets you something like the following and the required identifier is found between the last slash and the first question mark in the address string (so as not to share any real links, I made the address more general below):

https://drive.google.com/drive/folders/[remote folder ID]?usp=sharing

Then, synchronisation uses a command like the following:

gdrive sync upload [local folder or file path] [remote folder ID]

There also is the option to do a one-way upload and this is the form of the command used:

gdrive upload [local folder or file path] -p [remote folder ID]

Because every file or folder object has its own ID on Google Drive, it is possible to create two objects on there that appear to have the same name though that is sure to cause confusion even if you know what is happening. It is possible in each of the above to throttle them using trickle as well:

trickle -d 2000 -u 50 gdrive sync upload [local folder or file path] [remote folder ID]
trickle -d 2000 -u 50 gdrive upload [local folder or file path] -p [remote folder ID]

Handily, this works without the added drama seen with Insync and lends itself to scripting as well so it could be something that I will incorporate into my current workflow. One thing that needs to be watched is file upload failures but there may be ways to catch those and retry them so that would another thing that needs doing. This is built into Insync and it would be a learning opportunity if I was to stick with gdrive instead.

A collection of legal BitTorrent sites

19th October 2014

It was an article in a magazine that revealed these legal BitTorrent download sites to me, so I thought that I’d keep them on file for future reference while also sharing them with others who might need them. As far as I am aware, they are all legal in that no copyrighted material is on there. If that changes, I am happy to know and make amendments as needed.

My own interest in torrents arises from their being a convenient way to download installation disk images for Linux distributions, and at least one of the entries is devoted to just that. However, the distribution also lends itself to movies along with music and books, so that is reflected below too. With regard to downloading actual multimedia content, there is so much illegal downloading that a list like this is needed and has blackened the reputation of BitTorrent too because it only ever was conceived as a means for distributing large files in a peer-to-peer manner without the use of a single server. Of course, any use can be found for a technology, and it never has to be legal or morally acceptable either.

Archive.org

Bt.etree.org

FrostClick

Linux Tracker

Public Domain Torrents

Removing advertisements from uTorrent

12th July 2014

BitTorrent may have got some bad press due to its use for downloading copyrighted material such as music and movies but it does have its legitimate uses too. In my case, many a Linux distro has been downloaded in this way and it does take the weight off servers by distributing the load across users instead.

Speaking of Linux, my general choice of client has been Transmission and there are others. In the Windows world, there is a selection that includes BitTorrent, Inc. themselves. However, many favour uTorrent (or μTorrent) so that’s the one that I tried and there free and subscription-based options. To me, the latter feels like overkill when an eternal licence could be made available as an easy way to dispatch the advertisements on display in the free version.

As much as I appreciate the need for ads to provide revenue to a provider of otherwise free software, they do need to be tasteful and those in uTorrent often were for dating websites that had no scruples about exposing folk to images that were unsuitable for a work setting. Those for gaming websites were more tolerable in comparison. With the non-availability of an eternal licence option, I was left pondering alternatives like qBittorrent instead. That is Free Software too so it does have that added advantage.

However, I uncovered an article on LifeHacker that sorted my problem with uTorrent. The trick is to go into Options > Preferences via the menus and then go to the Advanced section in the dialogue box that appears. In there, go looking for each of the following options and set each one to false in turn:

  • offers.left_rail_offer_enabled/left_rail_offer
  • gui.show_plus_upsell
  • offers.sponsored_torrent_offer_enabled/sponsored_torrent_offer_enabled
  • bt.enable_pulse
  • gui.show_notorrents_node
  • offers.content_offer_autoexec

In practice, I found some of the above already set to false and another missing but set those that remained from true to false cleaned up the interface so I hope never to glimpse those unsuitable ads again. The maker of uTorrent need to look at the issue or revenue could get lost and prospective users could see the operation as being cheapened by what is displayed. As for me, I am happy to have gained something in the way of control.

VirtualBox OSE and 64 bit Guest Operating Systems

17th May 2008

I have gone and downloaded the next to four gigabytes of the 64 bit variant of Fedora 9 using Bittorrent and so thought that it might be a good idea to set the thing up in a VirtualBox virtual machine. However, that stratagem got scuppered by VirtualBox’s not supporting 64 bit operating systems. I do have VMware Workstation and, since that supports what I was doing, I resolved to set up Fedora there. After my plan’s getting shelved, my trying out VirtualBox is a matter that remains outstanding…

Downloading Ubuntu

14th March 2008

I was wandering around looking for a preview of Ubuntu Hardy Heron (8.04) and it took a bit of blundering before I landed on cdimage.ubuntu.com. There are ample ISO images and torrents available for anyone’s needs and I decided to keep the link somewhere handy for those who might need it.

  • All the views that you find expressed on here in postings and articles are mine alone and not those of any organisation with which I have any association, through work or otherwise. As regards editorial policy, whatever appears here is entirely of my own choice and not that of any other person or organisation.

  • Please note that everything you find here is copyrighted material. The content may be available to read without charge and without advertising but it is not to be reproduced without attribution. As it happens, a number of the images are sourced from stock libraries like iStockPhoto so they certainly are not for abstraction.

  • With regards to any comments left on the site, I expect them to be civil in tone of voice and reserve the right to reject any that are either inappropriate or irrelevant. Comment review is subject to automated processing as well as manual inspection but whatever is said is the sole responsibility of the individual contributor.