Skip to main content

Do you Scale?

Great post by Dare, Scalability: I Don't Think That Word Means What You Think It Does
following up on a post from Scott Loganbill at Google about their choice of "scalable" services.

Scalability is a funny question. Every application is designed to handle a certain type of architecture - it's usually one of those fundamental sales analysis questions "and how many orders do you typically process a day? How many people do you have working for you".

While scalability problems can be due to both technology and design, it's typically design that is the major culprit. (ok, yes, the guy that said he can run a 15,000 transaction a day system in Access - he likely chose the WRONG technology).

But everytime I think of scalability, I always think of 37 Signal's approach to scalability - in short, "Do you really need 12 servers now if you can run on two for a year?" - note read the comments from David Hansson "... Not worrying too much about scaling doesn’t mean building Basecamp on an Access database with a spaghetti soup of PHP making 250 database calls per request. It simply means having a profitable price per request, chilling the f*ck out, and dealing with problems as they arise."

Actually, their better quote is here. "If you've got a huge number of people overloading your system then huzzah! That's one swell problem to have."

Many of these posts and statements are generally made for large systems designed for mass usage. Not necessarily the same issues facing the systems that small businesses or even medium-sized businesses need. But still, just thinking about scalability when designing your databases can help in a big way.

I see silly design decisions that limit scalability all the time and they are generally made for no reason at all. A limit of 2 characters for Sequence numbers - what?!? you're never going to get over 99 items on a single order? Regardless of how unrealistic it may seem, chances are someone will hit it.

And then there are those that become requirements: A 5 character code for a customer ID - then the customer comes up and says I need to use 6. Is that really a scalability issue? Not really - until you've reached 26^5 customer codes, but the customer may perceive it as such. But the whole 26^5 argument does show there is a scalability design flaw in that definition. Will it ever show up? Hard to say.

Two Real Factors
So consider two different factors when it comes to scalability: hw uptime/access (as in can your hardware infrastructure handle a large number of users/transactions) and design (can your application handle a large number of users/transactions). Both go hand in hand yet demand different solutions.

The infrastructure factor can play off the design - if your application is designed for small groups but has lots of small groups, you can always put one group onto another server, if need be.

The Design can detrimentally affect the infrastructure. You can have as many servers as you want but if you constantly pull down lots of records when you only need one, you're asking for headaches.

And One more...
There is one more scalability factor to consider...and it's one that I think many people disregard because it's a Design issue but not so much an architecture design one - it's a user experience scalability design. Many applications were/are built for single transactions (the first web banking site for my bank only allowed one bill to be paid at a time). Just as much as uptime/downtime can play into it, if your application doesn't make it EASY to handle the increase in use, you can just forget about your other scalability problems.

Purists will say "Every software should scale " (and scale infinitely). Pragmatists will say "there will always be SOME scalability flaw in any design".

Think of this another way...even scales have scalability problems - in the design. Many only go to 300lbs. With the obesity epidemic these days, I'm not sure that's a limitation that users can overlook.


Anonymous said…
Hi Andrew,

Your post on scalability even describes my wrong use of Excel ;-). I enjoy the categorization you make when it comes down to scalability problems and I couldn't agree more that the 37signals quote is often most applicable.


Popular posts from this blog

Well, that explains CodePlex...

In a move that will be sure to anger open source (or rather anti-paid software, anti-Microsoft open source)  zealots, Microsoft is planning to buy GitHub.

A year ago, I mused about why Microsoft would shut down CodePlex and how the world needs competing source code repositories to be strong. I'm not the only one per this Slashdot article :
"...people have warned about GitHub becoming as large as it did as problematic because it concentrates too much of the power to make or break the open source world in a single entity, moreso because there were valid questions about GitHubs financial viability...." - Jacques Mattheij

I will be interested in seeing this play out - whether developers jump ship or not. Have all the efforts Microsoft has made in pushing towards open source be seen as genuine or will all the zealots jump ship or maybe even attack?

Microsoft's comment about why they shut down CodePlex referred to how spammers were using CodePlex. Well, GitHub has its own …

FoxInCloud Stats

FoxInCloud sent this link a while back about their statistics regarding visits to their site:

What's interesting here is the breakdown of people. Yes, I think it's understandable that the Fox community is getting older.

Another factor is the growth of the mobile and web environments taking over development. These environments really do push people towards the newer non-SQL or free SQL/hosted environments but more towards hosted storage options like Amazon and Google. A tool like FoxInCloud that helps MOVE existing applications to the cloud inherently competes with those environments.

But FoxInCloud also allows developers to extend their application further by giving them a starting point using Javascript and the basic CSS (such as Bootstrap). If you're not rebuilding your application from scratch, it's certainly a great step forward.

The World of Updates Today

I just received an update for Office 365. It certainly includes some cool features - including starting in one environment and picking it up in another environment. In recent years, I've certainly enjoined the use of Continuity on a Mac and in fact, I feel spoiled being able to start a message in one environment (even Google) and then finish it off on another.  This has become some pervasive when we were reviewing our most recent backlog at a client site, a similar feature was added to the current workload.

But with web applications, the trend is to reduce the amount of software on a client machine. I used to have automatic backup for all of my machines (thanks Carbonite!) but these days, many of my machines don't need anything beyond the core OS and some basic applications. Certainly that's the feeling with Chromebooks and even the lightweight aspect of many iOS apps. The functionality is mostly in the cloud.

When you upgrade your system, you expect it to a big update. So…