I've had this want for a while, but I haven't been able to put it in words because I have not found a good metaphor for what I am trying to accomplish. While newer fiber optic internet and 5G carriers have been able to deliver magnitudes more bandwidth, a lingering problem for speed issues concerns gaps in service, and how to maintain optimum connectivity when bandwidth is intermittent or slows to a crawl, for whatever reason. For example, when internet is throttled to just 128kbps, one can implement a data saver mode on smartphones to limit background services and notifications. But there isn't really an easy or quick way to check ALL the services, nor toggle them all without spending an hour in the settings. It took me around 2 weeks to get accustomed to some of the finer-grained controls on my Android phone, in not only enabling Data Saver mode, which I have used for years in certain instances, but the permission levels and amount of times apps will "ping home" (a server somewhere). To do that, it's best to disable unused or lightly used apps altogether. You can check how much data certain apps use, as there is a Settings page which will show you how many MB or KB were used in the past 7 days or so. That may or may not be representative of more typical months, but you may be able to get a ballpark idea of what's using much of your bandwidth, and on 128Kbps, every byte counts.
A few days ago, I downloaded Hugo. It took me several tries, because I realized I couldn't browse other websites while it was downloading. The chances of it interrupting the download was too high. But even then, I still had interruptions. Eventually I got it. Then yesterday, I wanted to download the Go language. Similar issue. I thought there might be some congestion, because I couldn't figure out why it was running at 25-50% of the speed it should be capable of. I was only getting 32-64KBps. Hugo and Go were both around 60MB, so in ideal conditions they would only take an hour to download. For people who develop websites, 128Kbps is not really going to be useful or practical, especially if there's a lof other apps one needs to download. Today I started downloading Git (even though I should have an older copy somewhere), and ran into the same issue. It's only running at 1.6-3.2KB/s. On Windows, it might say there's more data downloading, although typically it's an average. With only one device connected to the hotspot, it's possible the website is throttling, or more likely, the network is running only at 1/2 speed.
The increasing size of websites is not a new problem. A 2015 talk was given on it, and it had been observed many years before that too. So I thought, what if there was a way to dynamically control how much data is being sent AND requested whenever network conditions change? There would need to be several components working in tandem, and I can count at least 3. For example on the client side, an application will actively monitor the bandwidth, and adjust the amount of data being requested. the network card can optionally forward this information to both the host and the client, or it can be counted locally (often a network monitor does this). The host could also slow down the speed, or extend the time to renew, so that the connection doesn't get permanently terminated. Of course, with dynamic conditions, it might not always sync with what the other side is seeing, and thus the counter would need to update quite rapidly, so that there isn't late requests for slow connections when a hiccup already passed.
One metaphor that I think best help visualize it is the Sierpiński triangle:
 |
| six magnitudes of triangle sizes |
This metaphor may make more sense if you grew up with 5 magnitudes slower internet. In the 90's my first modem was a 14.4K dial up modem on a Pentium 100MHz PC. That translates into 1.8KB/s in ideal internet conditions- if the server was actually able to provide the maximum speed that you were requesting. Often times it would fall under 1024 bytes and one wouldn't see "KB/s" but "bytes per second." Yes, that slow. But I don't care about that. What I care about is that apps can be designed to be smart enough to not fail completely and become utterly useless, or worse, need to restart their download and waste even more bandwidth because they cannot save every packet that arrived. Each packet in a torrent should have instructions on how to recompile it. I think it involves a checksum but more secure files might even use some other technique to ensure it hasn't been tampered with by some malicious file sharerer. So the question I would ask a search engine, but haven't gotten around to finding the long explanation, is how do modern browsers manage a typical download whenever it briefly gets interrupted? Do they websites provide header files and offer agreements with CDNs and browsers on how they can go about that? Or is the non-technical user left completely clueless on what's actually going on behind the scenes? Packet reassembly isn't rocket science, but I won't trivialize it either. There should be an intelligent way that respects the user and developers time by showing them everything under the lid if they really want to check. "Inspect: elements is more about browser page loads than files themselves, and I would rather use a different program altogether if I don't need the browser. Tranmission is a linux client that I've used for torrenting, but I would like something that's lightweight that is browser-like, without the full stack of the browser. Kind of like a package manager - Synaptic or Software Manager in Linux, but one that actually shows you how much data was downloaded, the source (IP addresses) of where they all came from, and an optional log with granular data on when and what packets were downloaded from where throughout the entire downloaded. Yes, that might sound like a lot, except when you're wasting data, it wastes more users times.
There might only be a handful of apps that one needs to develop websites. Sometimes ordering a CD, DVD, or USB stick with all the programs preinstalled might be faster than downloading them one by one. This was far more common in the 90s and 00s than it is today, but I learned or was reminded a lot of some great practices that were lost by today's 5G and fiber optic instant gratification era.There was even an article years back that calculated the internet speed of ordering a DVD or blu-ray by mail, and even when counting the days of a typical USPS delivery (5-7 days in worst case scenario), was still on average, faster and more data transferred than something like dialup interent. Because with 128Kbps, that's only 57MB an hour, and 1.38GB a day (if you can manage a connection for 24hrs). So by that logic, ordering a blu ray from a nearby state might only be 2-3 days to deliver, and one would receive over 25GB of data. So downloading a preinstalled ISO with Python, Git, Github Desktop, GNU Utils, Go, Hugo, and all these other apps needed to create a website wouldn't be needed, even though it might save a little time in installing them. It's far better to have a DVD of typical tools like that. I know a lot of linux distros offer lite (headless), desktop OS versions with minimal apps (such as Raspberry Pi OS), and full desktop OSes with tens of applications installed (KDE once offered a 4.7GB ISO like that, and probably still do).
In the early 00's my internet speed at university was a T1- we would routinely get Megabytes of internet in the dorm rooms. At home, we got our first DSL line, which was 1.5Mbps, but some areas offered up to 6Mbps on VDSL. I never got that speed, but 160KB/s was basically the max I had for SEVERAL years, until 2013, when I needed faster internet for work. Suddenly I got 60Mbps, which was 12MB/s, although eventually I saved a bit on costs by downgrading to 3Mbps. A 1.5MBps DSL upgrade from a 14.4K modem is more than a 100x increase in speed. I've had cable internet at speeds up to 400Mbps because DOCSIS 3.0 modems can go that fast. With fiber, I've had 100Mbps but the routers are provisioned up to 10Gbps- if I want to pay that much. In a matter of 30 years, the computers I've used went from 14.4k to 1.5Mbps to 400Mbps internet on less than $60/month. If you count the current speed I've used recently, that's nearly a 10,000x speed improvement: 14.4Kbps to 100Mbps.
What hasn't improved though, is a parallel accessibility ramp that can dynamically scale between these two extremes. A given smartphone today *might* be able to dynamically hide how it limits bandwidth to make things seem fast when the connection is clearly not, but technical users want to see how that actually happens. It's unclear why Gmail on Android seems to run fast even on 128Kbps when on a desktop browser it seems to take 5 minutes even while saving cookies and website content. The browser should have all the elements it needs to recreate the site and only download the *new* emails. So why does it take so long? Hint: it's not the hotspot. It's that all these programs are not designed to seamlessly load a page as if it remembers everything it downloaded right before the browser was closed. There is still a lot of browser data that is being fetched on a desktop and mobile browser, and a lot of it is VERY heavy.
Some people might think that advances in data centers and cell phone towers in the next few decades will make this a non-issue. In developed countries, most people can find a relatively data provident internet plan at home or even on mobile, and some aren't too fussy about paying over $50 a month for that. But that also takes a lot of things for granted, and I don't think it's good to be complacent on thinking that someone will fix this in other places. No one will fix it, unless it's an actual thing people talk about. And there are few people who actually make this their job, outside of people who test software to make sure it runs in "normal" conditions.
So when you see a triangle, it represents scalability. But it doesn't have to mean a data center scalability, where virtual machines all run 256MB of RAM and require a minimum of a 3Mbps internet connection so that they don't terminate an anemic dial up connection.
No, that's not the scalability I'm referring to. It should be an infrastructure that is aware of 5 or so magnitudes of internet speeds- 14Kbps, 128Kbps, 1.5Mbps, 15Mbps, 150MBps and so on. So that the connection doesn't crap out whenever it sees a brief dip in internet speed. A delta of 3 or even 4 magnitudes shouldn't even alert the server that the speed needs more time. Everything is fine! Did a dog think otherwise? In a burning house?
So I downloade Hugo, but I also found a custom version that someone designed for blogs that allows the pages to be smaller than 10KB! Because I am not a savage, and I think Javascript should be a modern comfort. I don't want to deny it to others. What I care more about is the size of the website, rather than what components it uses. And a 10KB website loads in less than 1 second on a 128Kbps connection. Too small to matter. A 100KB webpage (a more typical Hugo site), would require 7 seconds to load on a 128KBps connection. Long enough to start wondering about the internet speed.
It only took me 90 minutes to download Go language. On average, it was around 15KB/s, or 120Kbps. A 59 megabyte file, I was able to download one of the main components to developing a website alongside Hugo and Git. Will I need more apps? Most likely. But for now, I was able to accomplish it all (or mostly all) on a very lightweight connection. I would like to compile a DVD with web development apps, because whenever I start a web design, I become surprised with so many other apps that are needed. If one can make a CD with a lightweight desktop that's less than 300MB, but then fill it with web design apps that are less than 350MB, you can create a web developer live image, or better yet, a microSD card that doesn't require much. What distros are best for this? I am not sure, but I prefer package managers that aren't obscure or have been updated in the past year (ones that are no longer maintained become hard to install if the URLS for sudo apt get aren't working, like on older distros).
If I am able to get the blog running, I will post it somewhere other that github pages, since I am trying to learn web publishing on other hosting sites (for the sake of knowing how to host)- My first ftp was in 2006- I seem to have forgotten certain skills. Website accessibility should be scalable. You should be able to do something even when the website isn't fully loaded. Useful application data should load first, and less useful data should load last.
 |
| if you see http://localhost:1313/posts/ in your browser, you have succeeded (with creating a local page- now just need to deploy)! |
No comments:
Post a Comment