IT

A New Kind of Failure Point

Posted in IT, Servers on March 10th, 2010 by Eric Lamb – Be the first to comment

It all started with a simple email from a client with the subject line “Broken Link”; looked like one of my client’s sites was down. Since this was the one that used that lovely of loveliest of programs, Dolphin CMS, I didn’t really thing much of it. Remember, Dolphin CMS sucks and, yes, it’s been known to just crumble on occasion (fucking Boonex…). So, yeah, initially, I didn’t think much of this. But then again, I have been pretty happy lately and the universe really does hate us all. I should have known this was gonna be bad…

exploding-server

A New Kind of Failure Point

Looking into the problem it was immediately obvious what was going on; my server was gone. I couldn’t access any site, much less my client’s site who alerted me to the issue, using any protocol or tool (ssh, ftp, cpanel or whm at least). It was like it didn’t exist…

Immediately, I contacted the server provider, HostGator, where I had a dedicated server requisitioned a few years ago. Thinking this was at worst a network issue I contact HostGator and submitted a support ticket; that’s the only thing to do since I couldn’t personally deal with the issue. That’s the trade off for leasing a dedicated server; on the other hand HostGator will fix anything that goes wrong so it’s worth it in my opinion.

HostGator, to their credit, got back to me within 20 minutes to inform me that the server account was closed. My first thought:

WHAT THE FUCK??!!??

Looking into just what the hell happened I found out that the client who was paying the bill (I had worked out a “deal” for him where I built and maintained his sites and he would pay the hosting bill) just decided that he was done with the whole Internet thing. Seriously. Apparently, he decided to just stop paying the bill a couple months ago forcing HostGator to cancel the account because he, in his words, “Hadn’t made shit from this crap”.

Anyway, this all highlighted a failure point in my backup strategy. Yes, I had a  backup strategy and I even had backups locally. No, my problem was that I didn’t have access to any backup newer than 2 weeks old. Here’s how my backup strategy worked:

  1. Daily backups stored to a NAS on the rack.
  2. Weekly backups FTPed to another HostGator account
  3. Bi-weekly backups were being downloaded to my local network (home) every 2 weeks.

See the problem point? Yeah, keeping too much of an interval on the network instead of within reach and not having access to my recent backups. That. Was. My. Bad.

Two weeks was just too much time to go without a physical backup. It’s an infinity on the Internet; too much stuff can happen in that time and ironically my client with the nasty, nasty, Dolphin CMS is proof of that. Unfortunately, two weeks ago her site had no traffic and no real use to anyone. Come two weeks later and she had gotten the membership up to a couple hundred users who were actually using the site. That’s the part that really sucked; I had failed my client.

This whole ordeal prompted a number of changes to my overall routine. Here’s the new setup.

  1. For starters I’m no longer leasing a server; instead I purchased my first server and have it hosted at a data center where I have personal 24/7 access to it and the rack it’s on.
  2. Daily backups are being stored to a backup drive in my server and being mirrored to an external NAS on the rack.
  3. Weekly, I will go to the data center and swap out the backup drive on the server with another keeping the drive at my house.
  4. Bi-weekly backups will be downloaded to my laptop and kept close at hand.

Overkill? Probably, but considering that the majority of this is automated I don’t see too much of an issue with it. In fact, the only thing I have to actually do is go to the data center once a week and switch out the drive so it’s really not too bad.

Hopefully, this’ll work out better than the last strategy.

Bookmark and Share

The Horrors of C99.php

Posted in Brain Dump, Code, IT on February 22nd, 2010 by Eric Lamb – Be the first to comment

If you were a sysadmin a few years ago, and you had php on your servers, you’re probably already familiar with c99. In case you haven’t had the personal pleasure, c99, or specifically c99.php (hint: check the source), is the name of a script used by hackers to gain access to a web server running php using an exploit technique called Remote File Inclusion.

The Horrors of C99.php

The Horrors of C99.php

A Little History

See, back in the day some php developers were pretty stupid. (Admit it; you were stupid once too.) What other explanation could there be for writing code that allowed the injection of arbitrary routines into a program. Trivially easy too.

To be fair, PHP was to blame a little for this as well. Given PHP’s high adoption, and design, by, and for, newbie programmers allowing such a technique by default was just ill conceived, and maybe even a little negligent. I understand the desire, and sometime need, for a technique that could be dangerous but to enable the feature by default…. damn man…

So, the risk was known, yet code was still being written (like the below example) that allowed remote file inclusion to be possible. Mostly because of the aforementioned default setting.

<?php
$color = 'blue';
if (isset( $_GET['COLOR'] ) )
{
	$color = $_GET['COLOR'];
}
require( $color . '.php' );
?>

BTW, if you currently write code that does anything like the above, frankly, you’re an idiot. You aren’t nearly as smart and clever as you think you are. I promise you this will bite you. Bad too.

About C99.php

So, using a technique like the above opens you up to learning first hand about c99.php. Finding information about the program itself is a little tricky but there are a couple examples that highlight just how devastating it can be.

When malicious intruders compromise a web server, there’s an excellent chance a famous Russian PHP script, r57shell, will follow. The r57shell PHP script gives the intruder a number of capabilities, including, but not limited to: downloading files, uploading files, creating backdoors, setting up a spam relay, forging email, bouncing a connection to decrease the risk of being caught, and even taking control of SQL databases. All these functions become readily available through an easy to use web interface, but now you can fight back.

Using the above explanation, which I agree with, c99.php acts as an interface to control your server. Once it’s on your server an attacker has easy access to view all the files and their contents, make changes to the system, upload new files, manipulate the database(s) and more.

Quite the nasty little script but pretty elegant in how it’s implemented. c99 is a completely standalone script; even the images are embedded inside using base64!

Until a month ago I would have thought the risk of encountering c99.php in the wild would have been small these days. Then, SMACK!!, a client had a site get hacked (quick CYA; that I didn’t’ work on :) ) using c99. So be warned. It’s out there and if you’re not smart, or if you’re a lazy, lazy, coder, c99 will get you.

Bookmark and Share

Parse Apache Log Files With PHP

Posted in IT, Programming on January 9th, 2010 by Eric Lamb – 1 Comment

Parsing the log files generated by Apache is one of those random tasks with a random occurrence in my world. This is a task that, until recently, hadn’t come up enough to warrant any sort of a ready solution (and it was just fun enough to be ok to write a custom solution). So every time this came up I would always fire up Google and go on a scavenger hunt for a starter script written in php.

Parse Apache Log Files With PHP

Parse Apache Log Files With PHP

This always felt like a good idea at the time the need came up. These days, for some ungodly reason, parsing Apache logs seems to come up a little too frequently to keep this up. In the spirit of making my life a hell of a lot easier for tomorrow I’ve taken a shot at writing an Apache log parser written in PHP.

One thing I decided to implement is a filtering system so you can filter out based on a provided regex. Might not be too useful to everyone but it should be trivial to remove the functionality.

Anyway, I hope someone finds this useful (even to learn from and, of course, use)

Here’s the main class:

<?php
/**
 * Apache Log Parser
 * Parses an Apache log file and runs the strings through filters to find what you're looking for.
 * @author Eric Lamb
 *
 */
class apache_log_parser
{
	/**
	 * The path to the log file
	 * @var string
	 */
	private $file = FALSE;
 
	/**
	 * What filters to apply. Should be in the format of array('KEY_TO_SEARCH' => array('regex' => 'YOUR_REGEX'))
	 * @var array
	 */
	public $filters = FALSE;
 
	/**
	 * Duh.
	 * @param string $file
	 * @return void
	 */
	public function __construct($file)
	{
		if(!is_readable($file))
		{
			return 	FALSE;
		}
 
		$this->file = $file;
	}
 
	/**
	 * Executes the supplied filter to the string
	 * @param $filer
	 * @param $status
	 * @return string
	 */
	private function applyFilters($str)
	{
		if(!$this->filters || !is_array($this->filters))
		{
			return $str;
		}
 
		foreach($this->filters AS $area => $filter)
		{
			if(preg_match($filter['regex'], $str[$area], $matches, PREG_OFFSET_CAPTURE))
			{
				return $str;
			}
		}
	}
 
	/**
	 * Returns an array of all the filtered lines 
	 * @param $limit
	 * @return array
	 */
	public function getData($limit = FALSE)
	{
		$handle = fopen($this->file, 'rb');
		if ($handle) {
			$count = 1;
			$lines = array();
		    while (!feof($handle)) {
		        $buffer = fgets($handle);
		        $data = $this->applyFilters($this->format_line($buffer));
		        if($data)
		        {
		        	$lines[] = $data;
		        }
 
		        if($limit && $count == $limit)
		        {
		        	break;
		        }
		        $count++;
		    }
		    fclose($handle);
		    return $lines;
		}		
	}
 
	/**
	 * Regex to parse the log file line
	 * @param string $line
	 * @return array
	 */
	function format_log_line($line)
	{
		preg_match("/^(\S+) (\S+) (\S+) \[([^:]+):(\d+:\d+:\d+) ([^\]]+)\] \"(\S+) (.*?) (\S+)\" (\S+) (\S+) (\".*?\") (\".*?\")$/", $line, $matches); // pattern to format the line
		return $matches;
	}
 
	/**
	 * Takes the format_log_line array and makes it usable to us stupid humans
	 * @param $line
	 * @return array
	 */
	function format_line($line)
	{
		$logs = $this->format_log_line($line); // format the line
 
		if (isset($logs[0])) // check that it formated OK
		{
			$formated_log = array(); // make an array to store the lin info in
			$formated_log['ip'] = $logs[1];
			$formated_log['identity'] = $logs[2];
			$formated_log['user'] = $logs[2];
			$formated_log['date'] = $logs[4];
			$formated_log['time'] = $logs[5];
			$formated_log['timezone'] = $logs[6];
			$formated_log['method'] = $logs[7];
			$formated_log['path'] = $logs[8];
			$formated_log['protocal'] = $logs[9];
			$formated_log['status'] = $logs[10];
			$formated_log['bytes'] = $logs[11];
			$formated_log['referer'] = $logs[12];
			$formated_log['agent'] = $logs[13];
			return $formated_log; // return the array of info
		}
		else
		{
			$this->badRows++; // if the row is not in the right format add it to the bad rows
			return false;
		}
	}
}
?>

And here’s an example of how to use it:

<?php
$data = new apache_log_parser($d->path.'/'.$entry); // Create an apache log parser
$data->filters = array(
	'path' => array('regex' => '/^.*\.(FLV|flv)$/') //pull only flv files
);
 
$data = $data->getData();
?>

A couple things to note about this script though:

1. The regex and parsing was pretty stolen from the Apache Log Parser on PHPClasses.org.
2. Without filters the script is pretty memory intensive. My needs don’t require anything client facing but heed my adivice; Don’t use this on a public web server.

Bookmark and Share

Blackberry Secret Codes

Posted in IT on December 23rd, 2009 by Eric Lamb – 1 Comment

After my little foray into unlocking my old Blackberry Curve I started to think that there were probably other special codes to play around with. I’ve done a little bit of research and it turns out, yup, there are quite a few special codes to do some random things.

Decibel meter

Normally, the signal strength meter is a series of bars like the below image. Hold “Alt” and press “N”, “M”, “L”, “L”. To undo just redo :)

Normal Signal Strength Indicator

Normal Signal Strength Indicator

Decibal Meter

Decibal Meter

View Source Code

This one every web developer should know. In the Browser, hold “Alt” and press “R”, “B”, “V”, “S”

View Source Code

View Source Code

Help Me! menu

If you ever call up for support chances are you’ve been told to enter this one. It shows a bunch of info in one place but most of it will be greek to most people.  Hold “Alt” and (left) “shift”, then press “H”.

Help Me! Menu

Help Me! Menu

Display IMEI Code

As far as I know this is really only useful for unlocking a device. Type “*”, “#”, “0″, “6″, “#” on the Home screen.

Display IMEI Code

Display IMEI Code

Event Log

The Event Log is probably only going to be useful if you’re a developer or similar. It displays all the last Java goodies that have happened since starting the device. Hold “Alt” and press “L”, “G”, “L”, “G”.

Event Log

Event Log

Show SIM Information

This code will display the details about a SIM card. You have to enter this code from the SIM Options page and press “M”, “E”, “P”, “D”

Show SIM Information

Show SIM Information

View Refld and ServiceUserId

From inside a message hold “Alt” and press “V”, “I”, “E”, “W”

View Refld and ServiceUserId

View Refld and ServiceUserId

View Data Usage & Voice Usage

This one is really only good if you have a new, unused, Blackberry. It maxes out at the below but if you buy a “new” Blackberry enter this code and verify it. From Status Options page press “B”, “U”, “Y”, “R”

View Data Usage & Voice Usage

View Data Usage & Voice Usage

Hard Reset

Doing a Hard Reset is like taking the battery out and inserting it again. Handy if you’re not in the mood to fuddle with the case. Hold ALT + Right Shift + Del

Validate Contact List

I’m honestly not really sure what these last two do but I’m listing them anyway for completeness. Hold “Alt” and press “V”, “A”, “L”, “D” in address book list

Rebuild Contact List

Hold “Alt” and press “V”, “A”, “L”, “D” in address book list.

These are all the “secret codes” I could find but that doesn’t mean there aren’t any more. Also, these codes were only verified on the Blackberry Bold 9000 so they may not work with all devices. If anyone has any new ones please leave a comment and I’ll update a list.

Bookmark and Share

How To Unlock AT&T Blackberry

Posted in Brain Dump, IT on December 21st, 2009 by Eric Lamb – Be the first to comment

My business partner recently had her phone literally break apart in her hands. Since communication between us is pretty crucial I decided to give her a backup Blackberry Curve I had as a replacement for my Blackberry Bold. Problem was that the Curve was bought through AT&T and she uses T-mobile. The phone would have to be unlocked.

How To Unlock AT&T Blackberry

How To Unlock AT&T Blackberry

Initially, I was a little nervous about the process of  unlocking the Blackberry. I’d never personally tried anything like this and my direct experience with the process was that a 3rd party company would have to be brought into the mix. I didn’t relish the idea of paying someone to handle what seemed to be a simple exercise when I’m capable of pushing buttons on a freaking phone.

Turns out it’s actually extremely easy to handle this yourself though it does require a phone call to AT&T support. Simply call them up and they’ll ask you for your IMEI code; it’s under the battery on the serial number label. They’ll send you an email, like the below, that’ll walk you through the process.

Your device unlock request was received and processed, see below for details:

IMEI: YOUR_IMEI_NUMBER

Unlock Code: YOUR_UNLOCK_CODE

Caution: If this process is unsuccessful ten times in a row, the phone will be permanently locked to the at&t network. Do not attempt to enter the code more than one (1) time total.  Instructions below will assist you in unlocking your device, if these steps are unsuccessful please contact us at 1-800-331-0500 or (916) 843-4685 from overseas.

Blackberry 8310

Follow these steps to unlock device:

1. Turn off the radio! VERY IMPORTANT

2. Go to “Options”

3. Scroll to and select “Advanced Options”

4. Click on “SIM Card”

5. Type “MEPD” (You will not see on display what is being typed. To obtain a “P” double tap “OP” key)

6. Type “MEP2” (To obtain a “P” double tap “OP” key. Press “ALT “key to obtain a “2″)

7. Enter the unlock code

8. Press enter

9. Reboot device. Device is now unlocked.

Note: To verify the IMEI, dial *#06# on device’s keypad, 15 digit # IMEI will display on the screen.  If this sequence does not work, pull the back/battery off the phone and the IMEI will be listed on the back of the phone.

All told the total time invested was only about 5 minutes. Seriously, 5 minutes. So why would I want to pay someone for this again?

Bookmark and Share

Why Not Upgrade Perl I Thought

Posted in IT on December 16th, 2009 by Eric Lamb – Be the first to comment

About a month ago I started receiving emails from my server about the installed version of PERL being too old and needing to be upgraded. Since my server uses WHM and cPanel, and thinking that upgrading PERL would have some pretty nasty consequences if something went wrong, I held off until I could familiarize myself with the ins and outs of what this was all about. Now that I’ve done the upgrade I’m not sure it was really so worth the wait and worry.

perl01

The first thing you want to do is backup all your installed CPAN modules. These are basically the library of functionality your system relies on to work. Pretty important stuff in there. It’s a pretty simple command though:

perl -MCPAN -e 'autobundle'

The above will create a “bundle” of all the CPAN modules you have installed to be used after the upgrade. Make a note of the bundle location though; you’ll need it later.

The instructions to do the actual installation were included in the email and is geared specifically towards cPanel servers:

cd /root
wget http://layer1.cpanel.net/perl588installer.tar.gz
tar -zxf perl588installer.tar.gz
cd perl588installer
./install -optimize-memory

This parts pretty simple, but time consuming. The whole process is automated and should take about an hour or three. When you come back you’re going to want to install the autobundle you created earlier. To do that you’ll need this command:

perl -MCPAN -e 'install Bundle::NAME_OF_BUNDLE_YOU_CREATED_EARLER'

This is where things started getting difficult. This process is going to require all sorts of input from you. It’ll, thankfully, give you default options but the install will sit there until you enter something. All told, this process took me about 4 hours; just sitting there watching the process.

Definitely a pain but not upgrading PERL would have been worse. If you have to do it be sure to set aside an afternoon though.

Bookmark and Share

Windows 7 Upgrade Experience

Posted in Brain Dump, IT on December 10th, 2009 by Eric Lamb – Be the first to comment

When I purchased my newest laptop back in July, from Best Buy, it came with a free upgrade to Windows 7 when it came out a few months later. Fast forward to December and I’ve since received the new operating system and have installed it on the laptop. Since a lot of people are going to be going through the same thing I put together some notes about the experience.

Windows 7

Windows 7

Because the promotion was for a version of Windows 7 that was the same flavor as the installed Windows Vista, I was given a 64bit Windows 7 Home Premium copy. Personally, I would have preferred Business or Ultimate but ok, fine Home Premium it is. At least this meant that I wouldn’t have to do a clean install so I could keep all files and programs where they were (still backing up the data of course).

The upgrade package came with 2 discs; the Windows 7 disc and a driver upgrade disc. The instructions said to insert the upgrade disc first and I’ll be prompted to enter disc 2 when required. Doing so started an upgrade program that inspected my system and warned me about deauthorizing my iTunes account which immediately made me feel good about the experience.

All told the install took around 3.5 hours and was like watching water boiling. I did it super late at night but I was still up and every time I would check on it I swear the progress rarely looked like it was making any progress. Still, it is Windows, so I was used to this; just wait and be patient, it’ll finish. And eventually it did.

Upon first booting up there were a couple issues. One was my fault. The others… not so much. Probably the worst offense was that I had no Internet connectivity. I checked both my wired NIC and the wifi and both were working  fine; I could connect to my router through both and I could find my Xbox and PS3 on the network. I just couldn’t get online. I eventually found that this was caused by a conflict between Esets firewall (which I had disabled in Vista) and the native Windows firewall. Uninstalling Eset and reinstalling it solved the issue.

I also had an issue with my local Apache webserver working. After checking the logs it turned out to be a soft link I had created under Vista to link the conf directory (makes editing the files from my working directory that much easier). Creating a new link solved the issue nicely.

The last issue is with the touchpad; and I haven’t really fixed it yet. At first, Windows thought the touchpad was a PS/2 mouse. This wouldn’t be an issue except I like the scrolling functions on the touchpad and the PS/2 drivers don’t support it. This seemed like a cut and dry driver issue except I installed all the latest drivers for my laptop that Gateway offered and it still doesn’t work all that well. Yes, it’s there but not in any real functional way. It’s jerky and hesitant when it works (around 30% of the time). Not enough to get me to downgrade but still a pain.

I’ve never really had an issue with Vista but I was still excited about Windows 7. Even though there were a couple hiccups during the install, and that my touchpad isn’t operating 100%, I’m still really happy with the experience.

Bookmark and Share

IMAP on Blackberry: The Poor Mans BES

Posted in Brain Dump, IT on December 7th, 2009 by Eric Lamb – Be the first to comment

It’s crazy to me how important email is; it’s almost 2010 and I’m still, still, relying on email for communication more than anything else. Considering just how broken email is this, along with how I know it’s broken, this isn’t a little surprising. Managing the shear amount of email I get used to be pretty easy but now… it’s a little more painful but still manageable.

IMAP on Blackberry: The Poor Mans BES

IMAP on Blackberry: The Poor Mans BES

When I had the day job I had access to a nice little network with Exchange 2003 and the latest Blackberry Enterprise Server (BES). It never really occurred to me how spoiled this would make me but, now that it’s gone, it’s pretty obvious how much I relied on it for my email management.

Why? Syncing. Loved the whole syncing aspect; open an email on your Blackberry and the email was marked as open in Exchange. Send an email from your Blackberry and it appeared in your sent folder in Outlook. Even better if you sent an email in Outlook, through Exchange, it was put in your sent folder on your Blackberry. Simple. This was great for things like filing and responding to emails. Put simply, using the Exchange/BES combo allowed you to keep all your communication in sync.

Then I get unplugged from the BES and have to setup a new solution. First, I went to POP3 (as I was most familiar with the protocol) but this was a cluster fuck of a bad idea. Yes, email was put on the Blackberry but it’s a one way method and nothing was kept in sync. I’d do something on the device and when I would go to Gmail nothing would be there; this couldn’t do.

I also tried the native Gmail Blackberry app but this too wasn’t ideal; I hated how it wasn’t truly a “native” app in that other apps would still open the default Blackberry message program to do anything email related. For example, if I clicked on an email on a web page while on the Blackberry it wouldn’t open the Gmail app but the native messaging one instead. I was surprised how much this happened and how much it bothered me but it did.

Then I learned that Gmail has IMAP support and have been using it ever since. So far, IMAP is the closest approximation of the BES experience I’ve found. It’s not without it’s issues or anything, and technically it’s not supported by Google, but it’s the best solution I’ve found.

Probably the biggest issue I have with the IMAP integration is that mail manipulated on the Gmail site doesn’t get synced to my Blackberry. This isn’t too bad since I usually only use the Gmail site for sending but if I want to keep things in sync I have to treat the Blackberry as the master.

To keep the calendar and contacts in sync requires the use of another Blackberry app called Google Sync. Google Sync it’s keeps your contacts and calendar synced up between Google Calendar and Gmail contacts and your Blackberry.

While it’s not a perfect solution I’ve been mostly happy with IMAP and Gmail on the Blackberry. It’s way, way, better than paying money for an outsourced BES / Exchange account.

this couldn’t do.;
Bookmark and Share

Hey!! You There, Pussy! Don’t Be A Pussy.

Posted in Brain Dump, Business, IT, Programming, Rant on September 28th, 2009 by Eric Lamb – Be the first to comment

Working in IT requires balls; you have to make some really tough choices with very real consequences. It’s not really a problem for programmers; very few of us work on projects that has the potential to destroy lives or break companies apart. On the other hand, in IT, you’re dealing with the backbone of an organization. Make a mistake here and: You. Are. In. Trouble.

Don’t Be A Pussy

Don’t Be A Pussy

Not to worry though; try as hard as you want to not fuck up and it’s just going to happen that much sooner.

I can say with absolute certainty that there’s going to come a time in your career when you fuck up. Big. Like really BIG. The type of mistake that has the potential to sink the company or client you’re working for/with. When it happens it’s going to be bad. So bad that you’ll have the fear of Dad in you. You remember that right? When Dad was coming home and you knew he knew what you did and you knew your life was over. If you didn’t have a Dad; think shear panic mixed with absolute paranoia and terror. Yeah, that’s the stuff.

What you did/will do isn’t important. What is important is how you deal with it. You’re going to have options when it  comes to dealing with the issue(s) and how you act is going to determine how your colleagues and peers look at you for the next few months. Make the wrong call and you’re in for some real uncomfortable silences and some really awkward sidelong glances.

If this has already happened to you; congratulations. Just know it probably won’t be the last. On the other hand if it hasn’t happened yet get ready; it will. You’re going to make some stupid mistakes in your career; mistakes so idiotic and so demoralizing your confidence will shatter and you’ll have a hard time getting back on the horse.

Like I said above, I have absolutely no idea what you do or what you can do to fuck it up so, as anecdotal examples only, I’m going to rely on my personal experience. I can honestly say, with absolute pride, that I have done the following:

  • Deleted a database and couldn’t restore the data
  • Deleted all the rows in a table and didn’t have a backup
  • Deleted a user account and all the email and files associated with it.
  • Changed every users password to “password” in a database
  • Sent an internal cost analysis report for a client project to the client

And that’s only what was off the top of my head; I’m sure I’ve blocked out some of the worse things. The one constant between the above list (aside from the stupidity involved) was that I owned the mistake. You have to immediately handle the situation whatever that means (it’ll depend on the situation).

After that though a funny thing will happen; it’s very likely your confidence will be shot. This is important because you need confidence (read: balls) to work in IT. There are too many things, that you just don’t know how to do, that you’re going to have to do, and that requires the confidence to know you can do these things. It’s why we make the big bucks.

In my experience the only thing you can do in these situations is get back on the horse ASAP. The sooner you do something, anything, that has consequences the better. You can’t wallow in the past and getting hung up isn’t the answer.

BTW: After reviewing the above I have to say:

Thank fucking God I don’t work in IT anymore.

Bookmark and Share

When Did Performance Stop Being Important?

Posted in IT, Programming on September 21st, 2009 by Eric Lamb – 9 Comments

Now that I’m finally starting to “get” the Zend Framework I’m starting to have some serious doubts on whether I made the right choice; not in choosing Zend over another framework but in choosing any framework at all. The memory usage is just abysmal across the board and after working with the Zend Framework for about a month or so it’s not entirely clear if it’s going to scale as I need it to.

When Did Performance Stop Mattering?

When Did Performance Stop Mattering?

Which lead to the question of why.  At the moment it seems like a question of speed of development versus performance (which is ironic because Zend Framework is not easy or speedy to develop with).

<disclaimer>
To be fair, it’s not just frameworks that have an uncomfortable overhead. Just take a look at Joomla and Drupal; 2 popular content management systems with an absurd overhead. It’s just easier to focus on my current interest rather than the CMS’es.
</disclaimer>

One thing I’m having a hard time getting comfortable with is how much memory is required when using a php web framework. Out of the box both Zend and Symfony (for example) use around 5MB per request. Understand, this is without any custom code. Just setting up the MVC and Autoloader for the default views and models. Nothing impressive or useful and 5 fucking MB to run that?

After having been on the wrong end of this issue on my own code I’m pretty sensitive to how my code performs; I’ve written some nasty algorithms and watching them crumble in real time has a tendency to turn you around ;)

Researching the issue doesn’t really help. There’s a lot of advice on how to improve the performance but it seems to always center around common sense improvements you should be using anyway.

The most touted improvement I’ve heard is that you have to use a PHP accelerator and opcode cache. I just find that response flawed but not because it’s bad advice but because it’s common sense. Yes, it’s true, but not using a framework in combination with a PHP accelerator and opcode cache is still better in my experience. All relying on those tools does is move the baseline for performance, which you’re supposed to do already, and a framework still consumes a good amount of resources on it’s own.

In my experience you get about a 50% reduction in memory usage when using something like x-cache but using the Zend Framework  still leaves a total of 2.5MB of memory usage to accomplish the bare minimum setup.

One saving grace is that hardware is cheap. Scaling with hardware is usually the go-to escape when the bottleneck is the code but it’s not without it’s own set of issues. For one thing while it’s true that hardware is cheap the labor to maintain that hardware is not. Especially if you want to maintain the server in a proper and responsible manner.

Another option, that’s really only available when using the Zend Framework, is decoupling the project from a direct dependence and not use the MVC components. In anticipation of doing this I’ve been writing a lot my recent code and projects in a style that’ll allow easy(ish) separation when the time comes.

At this point I haven’t used a framework in a production environment so all of this consternation might be for nothing. I just have a hard time accepting the performance hit of half a MB for using something trivial like a content management system (drupal) or, for example, a component like Zend_Navigation compared to the benefit. What are they actually doing to make the cost worth while?

Still another option is to just walk away from this whole OOP thing and head back to the familiar touch of procedural php and using functions and classes as more of decorators to apply than core components.  From my personal experience, and only my experience, using OOP is way more expensive than procedural. At the end of the day I need my programs to work fast, be easy to operate for my users and have a low impact on the server. How does using OOP help that?

At the moment I’m not sure how this is going to work out. I am confident it’ll be an adventure though. Hopefully, I find out how Zend will scale before a project of mine goes viral or gets popular. Hopefully.

Bookmark and Share