The problems of scale

Swami Gulagulaananda said:
"There are always problems associated with scale... especially when my weight doesn't seem to reduce"

Scalability has been a popular buzz word for some time and with the increasing use of the internet, social media and machine learning, data collections and performing operations at scale has become very important. But scale isn't something that is necessarily confined to the realm of technology. In this post, let's have a look at how solutions that work for some problems don't work at scale.

A long time ago, I came across a fascinating website called Project Euler. Project Euler presented us with a series of mathematical problems of increasing difficulty and the objective is solve them under a minute using a computer program. The first few could be as simple as sum of the first 100 numbers or sum of first 50 even numbers which were quite easy to solve for even novice programmers. Most engineering students might remember these as practice problems as part of their curriculum. Then you may also remember a problem that asks you if a number, x, is prime or not. How do you check if a number is prime or not? You divide the number by 2 (or take the integer nearest to the square root) and ensure that the number x is not divisible by 2 or any odd number till half (or square root). If even one number can divide x, then x is not a prime number.

There was another problem that asked us to find the, say 100th, prime number. Now, the problem became somewhat interesting. We had to start with an odd number, and keep incrementing by 2 to go to the next odd number. Then, for every odd number, we have to determine if the number is prime or not. If the number is prime, then we increment a counter and proceed. This seems to be a relatively easy solution, and for smaller numbers, the answers pop out quite quickly. However as we proceed to larger numbers, the same operation of dividing larger numbers by halves (or square roots) is repeated several times resulting in a significant delay in the overall solution and you end up taking a really long time. (The solution, in case you are interested, is to use the Sieve of Sundaram or sieve of Eratosthenes)

Thus, the solution of determining if a number is prime or not cannot be used to effectively solve a related problem due to scale. Even our power computers take considerably longer than a minute as you increase it from 100th prime to say 200th prime.

The same is true for restaurants and businesses. If you have visited any restaurant where you place order and collect your food from the counters, like the self service Darshini restaurants, you will notice one of three types:

• You order along with the rest of the customers. The guy keeps placing plates on the counter. Whoever is closest picks them up and you often feel cheated if someone who came after you got his food first
• You order along with the rest of the customers. However, there is one guy who keeps collecting the tokens from you and maintaining a mental note of who came first and who came later. He ensures that food gets delivered on a first come first serve basis
• There is a numbered token system. Food is given based on the token. You show your token and collect your food

The first ones are obviously the worst. However, they are the easiest to implement because there is no additional overhead involved. This system relies on the goodness of people that people are self regulating and will pick up orders in the same order in which they gave the tokens. Let us assume that there are only 3 customers at the counter. Then, this system will work without any issues because it is easier to manage and since people are being watched by other people, they will be at their best behaviour. However, as the number of people increase, like during a rush hour, this system will simply collapse. Customers will be watching like hawks so that they don't get cheated. Some customers might even be motivated to get someone else's food first because they know how long they have to wait.

The second system is relatively better. But it has an additional overhead of a man watching. We have to rely on his memory. Also, since looking at the order isn't his full time job, he is bound to slip up occasionally and thus, it ceases to remain a scalable solution.

The third solution is the most scalable of the lot and you can see the smoothest operators using this solution. Even McDonald's that has improved their processes over the years uses this. Thus, solutions that work at lower scale don't work at higher scales.

I have made similar observations while boarding buses or metro trains. During peak hours, people wait for the doors to open. The moment the door opens, people clamour to get in to try and get a seat (or at least get into the vehicle so that they don't have to wait). However, during non-office hours, people follow the rules and breathe easy, wait till the door opens, get in and stroll to their seats. The resources are more and the competition is low.

And these observations are not limited to Bangalore or India even. I made similar observations in western nations within Europe as well as in the US. It is a myth that cars in the US or Europe don't honk as much as they do in India. It is not really a behavioural problem. They don't honk because they are not presented with the situation. When I was in New York City and in Berlin, cab drivers demonstrated behaviours very similar to those in Bangalore during peak hours where they threaded their cabs across lanes, honked and swore. 

And thus, we have to look at Bangalore and India quite differently when comparing with other countries. A lot of things that work at lower scales don't work at higher scales.