Made of Everything You're Not

I generally consider my professional persona to be a software guy first and a hardware guy second. My first passion is code, through and through, but I have also spent a good deal of my time performing the day to day office IT stuff and, usually, I have a good time doing it. More than that though; I’ve always found that working on the hardware is a good way to know how my software is going to interact with the hardware. Read: It makes me a better programmer.

Blow My Mind

Needless to say, I have some ideas about hardware setup and deployment; a philosophy if you will. I try to be pretty humble about it but I couldn’t help but be reminded of this a when, a few weeks ago, I was listening to Stack Overflow podcast #59.

This one was cool; they had Damien Katz on who, if you don’t know, is the creator of CouchDB and used to work on Lotus Notes (back when the Internet didn’t matter). Smart guy.

(BTW, if you don’t know who he is I highly recommend you read his blog. Start with this post called Signs You’re a Crappy Programmer.)

Anyway, like I said; good podcast. Up until the end that is when Joel and Jeff completely blew me away with the following dialog when they were discussing a question on ServerFault about disabling your page file (around 1:01:44 in the podacast):

Joel: There’s a problem that we’ve always had, and it’s more common, I hate to say this, it’s more common among Unix system administrators than Windows system administrators, which is, they get the thing out of the box, they get the operating system out of the box, they install it, and then they’re going to want to do 47 things to that system before they can use it. Mostly removing things that were put there that they don’t understand.

So they have this attitude that’s like, “What are all these services that are running; I’m going to kill all of these services and then my server will be really fast.”

And then, all of a sudden, ok, it works for a while and then you go and install FogBugz, and it doesn’t run because some basic service, that everybody else has, has been removed, severely deleted from the operating system, by some system administrator that thinks they know better but, really doesn’t.

Jeff: You sound really bitter about this.

Joel: I am bitter because it’s all over tech support calls. It comes from people who are like… There is generally a philosophy that security flaws come from things, often come from things, that you don’t even realize you have running. And that probably shouldn’t be running.

I had to rewind the podcast when I heard that part. Was Joel really suggesting that we leave the default services enabled on an operating system? Did I just hear Joel Spolsky imply it was bad to disable and remove unneeded services from a computer?

Yup, I think I did.I also don’t think it’s the best idea to keep the default configuration on a server. Why? Because an OS is released with the goal of a good out of box experience not security. For example, does your Linux web server really need CUPS running? Does your Windows server really need Windows Media Player to start every time you start the thing?

Now I’m totally willing to accept that I’m being naive; this is knowledge gained from experience not instruction. But it’d have to be a compelling argument.

But, to be clear, you disable services and programs, not to improve performance, but to improve security and reliability. (Performance improvement should be a side effect in my opinion.) The thing I think Joel might be missing is that he’s more than likely dealing with some pretty busy system administrators. They probably did something to keep FogBugz from working, and forgot what it was, so they called support.

Earlier this month my department was, finally, able to get approval for an upgraded Internet line for the office. Previously we were using a business level DSL line that just SUCKED. Whenever someone was uploading anything the download and upload would just grind to a halt and we’d have to find out who was doing what. This happened everyday and it was just painful.

New Internet Line

If you’ve never had the pleasure of using business level DSL in an Internet company count yourself lucky.

Anyway, after literally a whole year of research and proposals and meetings and compromises and revisions we finally got the new line approved.

Halle-Fucking-lujah!!

Of course there’s the increased speeds for both up and download, which really can’t be understated in it’s coolness, but there’s also a Service Level Agreement (SLA) which guarantees us an uptime of 99.99%. Having just come from the world of DSL, where the word “reliability” isn’t a part of the vernacular, this was a requirement.

Of course this begged the question of just how much downtime is allowed with a 99.99% SLA. Not really wanting to do the math myself I turned to Google and came across a post on John D. Mitchell’s Blog with the fortuitous title “What does 99.999% reliability really mean?“.

John went to all the trouble of writing a script in Java that quickly outputs the correct time based on the amount of 9s. Since I don’t work in Java I thought I’d take a shot at porting his script to php (what else?).

Here’s my take on it:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

<?php
$sp = array();
$sp['min'] = 60;
$sp['hour'] = $sp['min']*60;
$sp['day'] = $sp['hour']*24;
$sp['year'] = $sp['day']*365;
 
function determine_nines($num_nines, $seconds){
	global $sp;
 
	$seconds = (double)$seconds;
	$str =  $num_nines." 9's (";
	for ($i = 0; $i < $num_nines; $i++) {
		if (2 == $i) {
			$str .= '.';
		}
		$str .= '9';
	}
 
	$str .= '%) = up to ';
	$str .= ($seconds / $sp['hour']).'h / ';
	$str .= ($seconds / $sp['min']).'m / ';
	$str .= $seconds.' seconds of downtime per year.';
	$str .= '<br />';
	return $str;
}
 
echo "Nines of Reliability: (Hours / Minutes / Seconds)<br />";
$base = 100.0;
for($i=2;$i<=7;$i++){
	if($i != 2){
		$base = $base*10;
	}
	echo determine_nines ($i, $sp['year'] / $base);
}
?>

So using my SLA of 99.99% I get a maximum downtime of 0.876 hours or 52.56 minutes of downtime per year.

Cool.

For comparison here’s the complete output of the above script (it matches exactly what John had; so that’s nice).