I
learn a lot about software optimizations by picking the slowest procedure to complete something. Running a software or OS update in
the background on a fast computer with a fast internet connection is
usually an easy task because it is really invisible. This is the most
common sense solution for anyone with those pre-requisites. But what
if you have a slow internet connection? I have noticed Chrome and
Firefox have what's called "differential" updates when a
browser is already installed. For minor security updates, this might
10-12MB. For a major release, such as Chrome 150, which was released
June 30th, this is a 100MB+ update. The full installation file is
somewhere around 465MB. I happened to have a lazy day today with few
obligations, so I did a little experiment. I have two Windows 10 PCs-
one a tower, and the other a laptop, with a 64 bit version of
Windows. One OS is actually Windows 10 Pro, so it's possible that the
Chrome versions for each are slightly different. However, I also had Firefox updates on each machine as well, and they were only around
12MB. These didn't take very long to install, so they completed early
this afternoon.
After
I updated Firefox, I also noticed a Chrome update on my laptop, the
one with Windows 10 Pro (both are 22H2). It did seem to take a little
longer, and upon checking, realized it was upgrading not only from a
149.0.x, but two updates to a later 149.0.x version, and then to a
150.0.x version, which required at least two restarts (of the
browser). This completed sooner, sometime in the afternoon. By 3 or 4
pm, however, I noticed my other PC was still updating Chrome.
Granted, the occasional wifi dropped, and it had to resume or
restart, but I didn't think it would need to start all over again. A
couple hours later, I started to get really puzzled. As I mentioned
in earlier blog posts, I have been using a 384kbps connection for a
month to get a feel of some early internet speeds and to also see how
slow some programs are or dependent on heavy websites. However, I am
able to download a couple hundred MBs in a few hours, if needed. I
thought, even if it's downloading the full installer (~465MB), it
should only take 3 hours max. I reasoned that it might take longer to
update a browser if it has a lot of cache or cookies and needs to
patch specific account information without deleting sensitive data.
But I wasn't willing to erase everything or at least back it up
(although I am starting to think of exporting my bookmarks and
things).
Most
of this would be a non issue on even a 1Mbps connection, but 384kbps
is fast enough to start the update but not complete it without
hiccups. There is also the added issue of needing to ensure the PC
doesn't go into sleep or suspend if leaving it unattended for 15
minutes or whatever settings the Sleep is at- I set mine at 5 hrs.
But even with all that, at nearly 10:30pm the update is still taking
a long time, on Windows 10 Home. The computer isn't slow by any
means- there is more than 13GB of storage left, SSD, and is an 8 core
processor with plenty of RAM. So I started to reason that if the
updates are incremental, it's doing it in a very haphazard fashion. I
also checked the Resource Monitor, and saw over 40 lines of Disk
Activity showing cached writing, some which may have been for Chrome,
and some for system scans (like Defender). It seemed quite odd, so I
thought maybe there is a better way of at least displaying the
progress of the install. I only saw progress a few times- sometimes
updating up to 8%, stalling near 5%, and then seeing the update later
on at 1%. I didn't see a log anywhere of whether it actually did
restart, despite the wifi drops (which happened only a couple times),
but I did get the occasional error like this:
https://support.google.com/chrome/thread/264277904/unable-to-download-chrome-due-to-code-segment?hl=en
From
Gemini:
1. Clear Update Cache and Data
Corrupted temporary files often trigger this segment-length limit.
Close all open instances of the browser.
On your keyboard, press Windows Key + R to open the Run dialogue.
Type %localappdata% and hit Enter.
Navigate to the Google folder, then delete the Chrome folder entirely to reset the update files.
Google Help +1
2. Temporarily Disable Antivirus/Firewall
Strict antivirus policies or false-positive security blocks frequently disrupt executable downloads, resulting in the 64k limit error.
Turn off your third-party antivirus software or firewall for 10 minutes.
Attempt the update or download the fresh installer again.
Remember to re-enable your protection immediately afterward.
Google Help
3. Run a Fresh Full Installer
If built-in update checks repeatedly fail, bypass them by reinstalling the app directly:
Download the fresh, standalone executable from the Google Chrome official website.
Run the installer as an administrator (Right-click > Run as administrator).
Google Help
If you can tell me which application is giving you this error and what antivirus software you use, I can give you more exact steps to fix it.
----------------
The
first option appears to delete browser files, so I was hesitant to
use that.The 2nd one seemed plausible as a minimally
invasive way to delete app data and only bypassing the AV if it was
causing the issue. The third option was my last resort, and seemed
preferrable if it could transfer my settings from the earlier
version, which usually it does.
But
then I thought, what if the third solution was the only fix, and a
number of PCs on a home network had to to the same fix?The full
download would be 465MB x 2, 3 or n. I checked if there is a remote
install option (often available for enterprise PCs, and there is,
called GPO).
https://support.google.com/chrome/a/thread/9164438/how-to-patch-chrome-on-multiple-clients-in-an-offline-environment?hl=en
https://serverfault.com/questions/364150/running-google-chrome-updates-from-a-centralized-location
However,
not everyone is an enterprise customer or sysadmin, and this option
is usually overkill (it also might not even address the direct issue
at hand, which is minimizing download traffic if it is a very limited
bandwidth network (such as India).
I
also wondered if I could share the differential/delta updates over
the network, without needing to distribute the full 465MB download
(assuming one is using the same architecture, such as x86_64, MacOS,
or AARCH64, etc). In a LAN, file sharing is well established, so that
would be a simple, but unnecessary way to update a browser needing
just a 10-100MB update.
Setting up PCs in a group policy (via Active Directory, etc) is a way to allow
sharing between devices, although there are other methods too
(sneakernet, USB drives, uploading to a cloud drive and back), but
typically this is overly permissive unless sharing is restricted to a
single folder and specified users. Even then, the capability is
typically tied to a general folder, rather than a specific type of
software. Microsoft Windows and Steam Client updates allow a more
clever and efficient way of sharing files on a local network, without
needing explicitly opt-in to sharing known directories and files between PCs:
The
concept should be simple: major software like browsers and clients
(such as Steam, Discord, Firefox, Chrome and other popular desktop software could
share core client updates on the local network without needing to
download redundant data or allowing excessive file sharing permissions.
It is possible certain updates are tailored for certain builds, like
Windows 10 Home vs Pro, but in many cases a 64 bit build is going to
be the same across an architecture. The same could be said about
Docker files, which are architecture independent files, although
these are often, if not typically entire programs , requiring a full
download for newer versions.
On
a local area network, Windows and Steam are able to share only the
game files needed to install or update a game in the gigabytes,
sometimes hundreds of GB, so this option makes sense with people with
a data cap (especially mobile plans that are not unlimiited).
Windows updates also can reach a couple gigabytes, and are often
800MB+ for monthly or quarterly updates. Thus these updates can
quickly add up in small businesses and homes that have a not
unlimited connection. Modern routers often can handle Gbps speeds, so
transferring a few hundred MB can be completed in seconds.
A
wifi hotspot on a mobile phone can function like a router, but tethering does not typically have all the LAN settings that a more featured router
has. Furthermore, it is unclear if they have a data sharing option
that Windows or Steam can take advantage of.
While
writing this, I also realized the first option might offer a data
recovery option, as deleting the Chrome cache folder doesn't
necessarily require deleting the data entirely, but moving it to
another directory while the update takes place. Nonetheless, it might
be better to delete the temp directory once the update completes,
restoring only bookmarks and passwords if necessary- probably via
another method done before that relocation step).
Tethering
is the process of sharing a data plan on a mobile network with other
devices that that cell phone or hotspot is broadcasting wifi to, somewhat like
an ad-hoc network. However, this network is not necessarily receiving
data from the devices connected to the cellular device providing the
internet, for the purposes of file sharing, but merely relaying them
to the internet.
A
couple apps for Android allow file sharing, such as LocalSend and Cx
File Explorer. These operate more as FTP servers or file transfer
protocols, rather than specific application update synchronizers.
While transferring a specific delta or differential update may
technically be packaged as a single file, initializing this update
may depend on the application, and if it could be instructed to
search for other applications on the same network, then in theory, it
could transfer itself to other devices, but this would require a few
additional steps:
It
would require the file transfer suite to index all the files on all
the devices, at least by the programs that request them (for the
folders in the other devices that have the same software installed).
It
would need some way for the file transfer software to search for
compatible applications capable of and needing to search for
updates. So for example, “CuteFTP” is running on an Android
phone which doubles as a hotspot. Android stores arch-agnostic update files in a
central directory, which can save on uploading data in the local
network (although this might be very minimal and the phone would
need to have quite a bit of extra storage, although if it is just
the latest google update for an x86_64, it might only be 10-100MB.
By
storing delta this on an Android folder or a Windows/linux folder,
the “CuteFTP” software could search for applications
periodically once a participating application gets an update, then
signal to other applications on other PCs on that WLAN hotspot that the update is downloaded or
available. This way, the updates can not only be downloaded
simultaneously to multiple clients on the WLAN, but it sidesteps the
entire process of the browser interacting with Google servers while
completing the update, saving time and preventing interruptions if
something happens during the update.
After
writing this, i searched “Steam file sharing on a mobile hotspot”
to see if Steam recognizes wifi hotspots the same way it recognizes a
LAN, and found:
https://www.reddit.com/r/gaming/comments/1bzql4k/is_it_possible_to_do_local_network_game_transfers/
https://steamcommunity.com/discussions/forum/8/601902348018668256/
Admittedly,
I have little interest in using my phone hotspot to transfer hundreds
of GB of game data (though might in the future), and see this as more
practical for the occasional hundreds of MB of a system or software
update. It would benefit the user who uses a mobile tethering hotspot
on a semiregular basis.
If
you check the Windows Optimization page again, recall that it
provides an option to share files with devices on the local network,
and files with anyone on the internet. If the home network is
trusted, it would presumably be safer than sharing with others on the
internet, but the latter also may involve a peering cost when the
device isn't being used but is on an idle (data caps may be reached
if the user is not careful but is feeling altruistic, unless
Microsoft caps the amount of upload regardless of the generosity
felt, probably for sane reasons).
If
the latter, then in theory, users on the same hotspot but less
familiar with each other- e.g. An internet cafe (if they still
exist), might share browser updates and OS updates to save the
business owner's bandwidth, but not the user's own data usage, which
they might be already paying for – another example is roommates
sharing an internet service, but not really sharing data themselves.
This could still result in an improved internet experience, since
more data is available to download while offloading identical
software distributions and updates to a local network, in the torrent
fashion.
What
types of issues can be foreseen in expanding this use of Delivery
Optimization? Checksum and digital signatures are one way to ensure data integrity. I think this is one of those overlooked cases
where bandwidth isn't usually scarce but can improve service greatly
during times of congestion. Data downloads multiplied by millions of households,
then this quicky becomes a major data & $$ saver:
"How Microsoft uses Delivery Optimization
At Microsoft, to help ensure that ongoing deployments weren't affecting our network and taking away bandwidth for other services, Microsoft IT used a couple of different bandwidth management strategies. Delivery Optimization, peer-to-peer caching enabled through Group Policy, was piloted and then deployed to all managed devices using Group Policy. Based on recommendations from the Delivery Optimization team, we used the "group" configuration to limit sharing of content to only the devices that are members of the same Active Directory domain. The content is cached for 24 hours. More than 76 percent of content came from peer devices versus the Internet."
Needless, to say, but caching content for slightly longer periods - say 3-7 days in a network where multiple users access the web on similar generation Chromebooks or laptops can probably help increase that peering percentage if they are most likely downloading the same Chrome or Firefox update. This feature would significantly improve congestion in edge cases. I have been thinking of developing a custom spin of Synaptic Package Manager, or one with Software Manager in Ubuntu that does something very similar to this Group Policy in Windows Delivery Optimization. Maybe there is an Enterprise distro that already does this, but I'd like to see it on a community edition, like Rocky Linux. And sure enough, here's what I found:
https://share.google/aimode/FiFKZ5LZAZRI4Cr6h
For Rocky Linux, "delivery optimization" typically refers to saving bandwidth and accelerating package/update distributions across your local network. The most efficient strategy is hosting an in-house DNF repository mirror and utilizing peer-to-peer (P2P) network tools to prevent redundant downloads from external mirrors. [1, 2, 3, 4, 5] 1. Host a Local Mirror
Creating a local mirror (via the official Rocky Linux Mirror tool) downloads packages once, allowing all local servers to pull updates from your LAN instead of the internet. [1, 2] - Setup: Use
rsync or the reposync utility to sync BaseOS, AppStream, and EPEL repositories directly to a local web server (e.g., Nginx). - Client Configuration: Update your client servers' repository files (located in
/etc/yum.repos.d/) to point the baseurl to your local mirror instead of $releasever. - Reference Guide: View the full setup workflow on the Setup Local Rocky Repositories Guide. [1, 2, 3, 4, 5]
2. Implement Peer-to-Peer (P2P) Synchronization
If setting up a full repository server is too heavy for your needs, you can set up a local P2P sync to share newly downloaded files and folders directly across your nodes.
- Tooling: Deploy Syncthing, a continuous file synchronization program available via the EPEL repository.
- How it works: When one server pulls a large package or ISO, Syncthing immediately shares and replicates that file locally to other nodes on your network, saturating your LAN's bandwidth and bypassing external bottlenecks. [1, 2, 3]
3. Network & Kernel Tuning
If your Rocky Linux servers suffer from packet drops or high latency during heavy network deliveries, the kernel might not be distributing interrupts optimally. [1] - IRQ Balancing: Ensure the
irqbalance service is active and properly configured to distribute network interface card (NIC) interrupts evenly across all CPU cores. - Tuning Guide: Diagnose and resolve drops using the Network IRQ Tuning Documentation. [1]
After
restarting my Chrome browser multiple times and deleting the App
data, a portable install attempt of the Chromesetup.exe failed and I
was about to post this article when, miraculously, the first 50%
update status appeared on the PC I began updating this morning with
Chrome at 10:50am. It appears that Chrome might have progressed a bit
further after a couple restarts, potentially integrating some of the
updates it picked up along the way. I was going to wait until the
next day to retry the update during a faster connection plan, but I
might have gotten lucky. Edit: the installation finally completed this morning (7/2 (the post draft began 7/1)). It required downloading the most recent ChromeSetup.exe stub (12MB), after uninstalling my Chrome and deleting %localappdata% from both the User folder and the Program Files Google Chrome folder. (Option 1). Using old Installer stubs will initialize an install but then break/or fail after 2/3rds of the download, wasting data and time.
Edit: I could have titled this "Using a Mobile Hotspot as a Router," but I thought it might be a bit vague to the specific optimization I was seeking. That said, there may be reasons to use a hotspot as a router, and reasons not to. Some Delivery Optimization features might not be transferrable between PCs even if if a router was available, hence the title of this post.
Also, three interesting videos on Rsync:
https://www.youtube.com/watch?v=QKCIi-NxJEo
https://www.youtube.com/watch?v=FLCfRs6nKW8
https://www.youtube.com/watch?v=VyF_omIjnr4
Update 2:
My phone is able to recognize the Steam client:
Edit 3: I realized that the Windows Delivery Optimization applies to apps from the Windows Store, in addition to the System updates. If I wanted to use the Windows store to download Firefox, I suppose it would also push updates to other Windows PCs on my network (that are also using Windows Store). I typically install software from the website, rather than the Windows Store. But since Ubuntu offers software in a store, I suppose they could also integrate this technique. It's now called "App Center" for Debian and Snap files: https://snapcraft.io/install/snap-store/ubuntu (and of course the trusty sudo apt-get)
But this firmware doesn't contain code to initialise the GPU, it does contain initialisation code for one rather important peripheral, which is the DRAM. The VC4 has been pretty well reverse engineered for a while (I did one of the early C compiler ports for it), but the big missing piece was figuring out how to initialise the RAM. Without that, everything had to fit in the 128kB of built-in SRAM.
This firmware's a huge step forward.