Made of Everything You're Not

One question I get asked from time to time by, well, pretty much anyone I've ever worked for or with, is why their hard drive was marketed at one size yet the available size is far smaller. For example, they may buy a hard drive that's marketed at 320 Gigabytes but when they get it installed there are only 286 Gigabytes of space available. On the surface this can be pretty frustrating in the "Bait and Switch" kind of way; you buy one thing and get another, lesser, thing but like most things about computers, the reality is actually pretty rational if not simple.

Traditionally, when confronted with the question I had always just gone with the tried and true line that I cribbed from Tom's Hardware:

Hard Disk Drive (HDD) makers define "GB" or "Gigabyte" as 1,000,000,000 bytes. Microsoft (in Windows) defines it as 1,024 x 1,024 x 1,024 bytes, or 1,073,741,824 bytes.

Thanks to Stack Overflow I now know that the real, thorough, answer is:

There are 3 reasons why the amount of space you can actually use is different from that listed for the drive, all of which work against you:

1. Hard drive manufactures treat 1GB as one billion bytes, while the operating system calls it 1,073,741,824 bytes (1000 * 1000 * 1000 vs 1024 * 1024 * 1024).
2. You lose some space for file tables when formatting.
3. Disk space is divided into chunks larger than 1 byte (typically 4K). Using typical Windows defaults, a 1 byte file takes up 4K of space on disk.

Of these, the first two can influence the amount of space reported by the drive (though IIRC the 2nd one was more of an issue with FAT32 than NTFS). The last one only influences the amount of free space remaining, but will still prevent you from using the full capacity of your 80GB drive.

It's almost reminiscent of the Mars rover debacle about metric vs standard measurements; HDD makers use a decimal metric while the operating system uses binary. Anywho, this means that if you have a 320 Gigabyte HDD Windows will recognize it as 286 Gigabyte HDD. Pretty straightforward really and aside from having to dodge the occasional rant about how illogical that is and how come I just don't know more about the ins and outs of HDD business practices it's not really been that big of a concern for me professionally. Well, now as my client's needs for storage are increasing well, it's, in a sorta kinda way, becoming a bit of an annoyance.

A recent project I had was to design a system for obscene storage space; the client is a civil engineering firm so they deal in those HUGE CAD files for city planning and architectural plans and they want to do version control on them. So, lots and lots of space was needed and while investigating a possible solution that involved a dedicated RAID array it dawned on me that the specs were a little more complicated than they aught to be. Math was going to be required above the normal RAID calculations you have to do for setting up a RAID array.  I mean, God forbid I say they'll have one amount of space when the reality is they'll get a lesser amount; I don't need that kind of attention.

On the surface setting up a RAID 5 device with 8 2 terabyte HDDs would yield a total of 16 terabytes before the RAID and, according to the RAID calculator, around 13 terabytes of usable space after the RAID setup. The reality though is that each 2 terabyte HDD is only going to yield 1.8 terabytes each and once the RAID is configured the total size is going to be 11.7 terabytes. A little over a full terabyte is missing; a non trivial amount to be sure.

So it's a knowable problem and it just requires a little math but the more I think about it the more I'm starting to empathize with the reactions of previous clients. Put simply; wtf man? Why don't the HDD makers put on the box the amount of space most people will experience; at least as a sub text on the box? For example, putting both the decimal and the binary amounts side by side or maybe listing the amount of space available for the big 3 operating systems (Windows, Mac and *nix) would be way more helpful and truthful than just listing the binary amount and leaving customers feeling ripped off.

canThere are 3 reasons why the amount of space you can actually use is different from that listed for the drive, <em>all</em> of which work against you:</p>

<ol>
<li>Hard drive manufactures treat 1GB as one billion bytes, while the operating system calls it 1,073,741,824 bytes (1000 * 1000 * 1000 vs 1024 * 1024 * 1024).</li>
<li>You lose some space for file tables when formatting.</li>
<li>Disk space is divided into chunks larger than 1 byte (typically 4K).  Using typical Windows defaults, a 1 byte file takes up 4K of space on disk.</li>
</ol>

<p>Of these, the first two can influence the amount of space reported by the drive (though IIRC the 2nd one was more of an issue with FAT32 than NTFS).  The last one only influences the amount of <em>free space</em> remaining, but will still prevent you from using the full capacity of your 80GB drive.

It's a strange day when an opinion piece condemning a device for not a few inadequacies actually sells someone else on the supposedly flawed device; but that's exactly what happened while reading Omar Shahine's recent rant against the iPhone. In a nutshell, he's been using the iPhone for the last couple years and, while he initially loved the iPhone, he's grown tired of all the problems he has (which are mostly due to his work situation because of Windows tech).

However, I have grown tired of you. As an enterprise device, you have completely failed. Each update of iOS brings more problems with Exchange support. You fundamentally don't understand what it means to talk to an Exchange server since your programmers don't use Exchange for their jobs. I put up with months of email hangs, years of poorly formated emails, calendar appointments went missing from my calendar, and you still don't understand that it's important when you look someone up in the corporate directory that you show their office location. After 2 years of this, I'm convinced you'll never figure this out. I put up with it though because you were the best lifestyle phone around.

All good points and the above is almost picture perfect for why I've been sticking with the Blackberry for the last 4 years. I always looked at the iPhone (and I hate to say it but pretty much anything Apple) as just a toy used by amateurs. Yeah, I'm a snob about computers (Surprise!). But it was true; once upon a time Blackberry had the better reputation for email and, with email being as important to me as it is, Blackberry wins. It was really that simple.

Blackberry really took care of me and most of my needs for a LONG time. I've used the Blackberry Enterprise Server (BES), IMAP and POP with the Blackberry for email and it's always been pretty cool (POP less so but what do you expect from POP?). There's a pretty nice App ecosystem too, not as good as the iPhone (obviously), but I never really felt like I was missing anything. I had Google Maps, a great RSS reader with Viigo, a couple cool games, an SSH client and a pretty nice phone to boot. Along with the email, which, seriously, I thought couldn't be beat, I was a pretty happy man.

And then, about a year and a half ago things started to get a little... crappy with the Blackberry. I've been using a Blackberry Bold, once a really good device, but there was a constant issue with the trackball not working. For days at a time I couldn't scroll down. That probably doesn't sound so bad at first glance but when you consider that the Blackberry doesn't have keyboard shortcuts for navigation and moving the trackball to the extreme of any direction doesn't loop to the opposite side the device in the interface this made my Blackberry pretty much useless for anything except email (painful, but usable anyway) because of a keyboard shortcut for opening that application.

I couldn't use Google Maps and I couldn't read any articles. I couldn't use the web browser (unless I clicked a link in an email). Hell, a couple times I couldn't even turn the ringer back on once I'd turned it off. Oddly though, the trackball would miraculously start working after about a week though I could never figure out what happened.

And that's where I found myself last week as I was reading Omar's post; I'd been ham fisting my way through using my Blackberry for basic, email only, stuff and I was fucking sick of it. Hearing all of Omar's reasons for dropping the iPhone were reasons that don't apply to me anymore. I don't work in a corporate job so no more Exchange or IT policies to comply with that aren't of my making. This was a revelation; I don't need a Blackberry just a good smart phone that does email alright. Enter the iPhone.

So I took the plunge. I dropped a couple hundred, signed a 2 year contract with AT&T (sigh...) and after playing with the iPhone for the last 24 hours I really think I made the right choice. The whole tapping thing is pretty nice though typing is challenging right now (makes me feel like I have fat fingers...) but I'm told it gets easier.

The weird thing is just how much I love the email management. It. Is. Slick!! The iPhone really makes the Blackberry look like shit; it's like comparing Windows 3 to Windows 7. It's that good. Animations, smooth interface, decent search and really thorough Google Apps integration; wow was I impressed. It's that good.

Still, the juries out on whether or not this is a good move; it's entirely possible I'm going to regret moving to the iPhone but for now at least everything's nice. I still have my old Blackberry Bold, ready to use at a moments notice (sans trackball moving down of course), so I should be cool but as of right now I'm happy.

It all started with a simple email from a client with the subject line "Broken Link"; looked like one of my client's sites was down. Since this was the one that used that lovely of loveliest of programs, Dolphin CMS, I didn't really thing much of it. Remember, Dolphin CMS sucks and, yes, it's been known to just crumble on occasion (fucking Boonex...). So, yeah, initially, I didn't think much of this. But then again, I have been pretty happy lately and the universe really does hate us all. I should have known this was gonna be bad...

Looking into the problem it was immediately obvious what was going on; my server was gone. I couldn't access any site, much less my client's site who alerted me to the issue, using any protocol or tool (ssh, ftp, cpanel or whm at least). It was like it didn't exist...

Immediately, I contacted the server provider, HostGator, where I had a dedicated server requisitioned a few years ago. Thinking this was at worst a network issue I contact HostGator and submitted a support ticket; that's the only thing to do since I couldn't personally deal with the issue. That's the trade off for leasing a dedicated server; on the other hand HostGator will fix anything that goes wrong so it's worth it in my opinion.

HostGator, to their credit, got back to me within 20 minutes to inform me that the server account was closed. My first thought:

WHAT THE FUCK??!!??

Looking into just what the hell happened I found out that the client who was paying the bill (I had worked out a "deal" for him where I built and maintained his sites and he would pay the hosting bill) just decided that he was done with the whole Internet thing. Seriously. Apparently, he decided to just stop paying the bill a couple months ago forcing HostGator to cancel the account because he, in his words, "Hadn't made shit from this crap".

Anyway, this all highlighted a failure point in my backup strategy. Yes, I had a  backup strategy and I even had backups locally. No, my problem was that I didn't have access to any backup newer than 2 weeks old. Here's how my backup strategy worked:

1. Daily backups stored to a NAS on the rack.
2. Weekly backups FTPed to another HostGator account
3. Bi-weekly backups were being downloaded to my local network (home) every 2 weeks.

See the problem point? Yeah, keeping too much of an interval on the network instead of within reach and not having access to my recent backups. That. Was. My. Bad.

Two weeks was just too much time to go without a physical backup. It's an infinity on the Internet; too much stuff can happen in that time and ironically my client with the nasty, nasty, Dolphin CMS is proof of that. Unfortunately, two weeks ago her site had no traffic and no real use to anyone. Come two weeks later and she had gotten the membership up to a couple hundred users who were actually using the site. That's the part that really sucked; I had failed my client.

This whole ordeal prompted a number of changes to my overall routine. Here's the new setup.

1. For starters I'm no longer leasing a server; instead I purchased my first server and have it hosted at a data center where I have personal 24/7 access to it and the rack it's on.
2. Daily backups are being stored to a backup drive in my server and being mirrored to an external NAS on the rack.
3. Weekly, I will go to the data center and swap out the backup drive on the server with another keeping the drive at my house.
4. Bi-weekly backups will be downloaded to my laptop and kept close at hand.

Overkill? Probably, but considering that the majority of this is automated I don't see too much of an issue with it. In fact, the only thing I have to actually do is go to the data center once a week and switch out the drive so it's really not too bad.

Hopefully, this'll work out better than the last strategy.

If you were a sysadmin a few years ago, and you had php on your servers, you’re probably already familiar with c99. In case you haven’t had the personal pleasure, c99, or specifically c99.php (hint: check the source), is the name of a script used by hackers to gain access to a web server running php using an exploit technique called Remote File Inclusion.

A Little History

See, back in the day some php developers were pretty stupid. (Admit it; you were stupid once too.) What other explanation could there be for writing code that allowed the injection of arbitrary routines into a program. Trivially easy too.

To be fair, PHP was to blame a little for this as well. Given PHP’s high adoption, and design, by, and for, newbie programmers allowing such a technique by default was just ill conceived, and maybe even a little negligent. I understand the desire, and sometime need, for a technique that could be dangerous but to enable the feature by default…. damn man…

So, the risk was known, yet code was still being written (like the below example) that allowed remote file inclusion to be possible. Mostly because of the aforementioned default setting.

<?php
$color = 'blue';
if (isset( $_GET ) )
{
	$color = $_GET;
}
require( $color . '.php' );
?>

BTW, if you currently write code that does anything like the above, frankly, you’re an idiot. You aren’t nearly as smart and clever as you think you are. I promise you this will bite you. Bad too.

About C99.php

So, using a technique like the above opens you up to learning first hand about c99.php. Finding information about the program itself is a little tricky but there are a couple examples that highlight just how devastating it can be.

When malicious intruders compromise a web server, there’s an excellent chance a famous Russian PHP script, r57shell, will follow. The r57shell PHP script gives the intruder a number of capabilities, including, but not limited to: downloading files, uploading files, creating backdoors, setting up a spam relay, forging email, bouncing a connection to decrease the risk of being caught, and even taking control of SQL databases. All these functions become readily available through an easy to use web interface, but now you can fight back.

Using the above explanation, which I agree with, c99.php acts as an interface to control your server. Once it’s on your server an attacker has easy access to view all the files and their contents, make changes to the system, upload new files, manipulate the database(s) and more.

Quite the nasty little script but pretty elegant in how it’s implemented. c99 is a completely standalone script; even the images are embedded inside using base64!

Until a month ago I would have thought the risk of encountering c99.php in the wild would have been small these days. Then, SMACK!!, a client had a site get hacked (quick CYA; that I didn’t’ work on ) using c99. So be warned. It’s out there and if you’re not smart, or if you’re a lazy, lazy, coder, c99 will get you.

Parsing the log files generated by Apache is one of those random tasks with a random occurrence in my world. This is a task that, until recently, hadn't come up enough to warrant any sort of a ready solution (and it was just fun enough to be ok to write a custom solution). So every time this came up I would always fire up Google and go on a scavenger hunt for a starter script written in php.

This always felt like a good idea at the time the need came up. These days, for some ungodly reason, parsing Apache logs seems to come up a little too frequently to keep this up. In the spirit of making my life a hell of a lot easier for tomorrow I've taken a shot at writing an Apache log parser written in PHP.

One thing I decided to implement is a filtering system so you can filter out based on a provided regex. Might not be too useful to everyone but it should be trivial to remove the functionality.

Anyway, I hope someone finds this useful (even to learn from and, of course, use)

Here's the main class:

<?php
/**
 * Apache Log Parser
 * Parses an Apache log file and runs the strings through filters to find what you're looking for.
 * @author Eric Lamb
 *
 */
class apache_log_parser
{
	/**
	 * The path to the log file
	 * @var string
	 */
	private $file = FALSE;
 
	/**
	 * What filters to apply. Should be in the format of array('KEY_TO_SEARCH' => array('regex' => 'YOUR_REGEX'))
	 * @var array
	 */
	public $filters = FALSE;
 
	/**
	 * Duh.
	 * @param string $file
	 * @return void
	 */
	public function __construct($file)
	{
		if(!is_readable($file))
		{
			return 	FALSE;
		}
 
		$this->file = $file;
	}
 
	/**
	 * Executes the supplied filter to the string
	 * @param $filer
	 * @param $status
	 * @return string
	 */
	private function applyFilters($str)
	{
		if(!$this->filters || !is_array($this->filters))
		{
			return $str;
		}
 
		foreach($this->filters AS $area => $filter)
		{
			if(preg_match($filter, $str, $matches, PREG_OFFSET_CAPTURE))
			{
				return $str;
			}
		}
	}
 
	/**
	 * Returns an array of all the filtered lines
	 * @param $limit
	 * @return array
	 */
	public function getData($limit = FALSE)
	{
		$handle = fopen($this->file, 'rb');
		if ($handle) {
			$count = 1;
			$lines = array();
		    while (!feof($handle)) {
		        $buffer = fgets($handle);
		        $data = $this->applyFilters($this->format_line($buffer));
		        if($data)
		        {
		        	$lines+):(\d+:\d+:\d+) (]+)\] \"(\S+) (.*?) (\S+)\" (\S+) (\S+) (\".*?\") (\".*?\")$/", $line, $matches); // pattern to format the line
		return $matches;
	}
 
	/**
	 * Takes the format_log_line array and makes it usable to us stupid humans
	 * @param $line
	 * @return array
	 */
	function format_line($line)
	{
		$logs = $this->format_log_line($line); // format the line
 
		if (isset($logs)) // check that it formated OK
		{
			$formated_log = array(); // make an array to store the lin info in
			$formated_log = $logs;
			$formated_log = $logs;
			$formated_log = $logs;
			$formated_log = $logs;
			$formated_log = $logs;
			$formated_log = $logs;
			$formated_log = $logs;
			$formated_log = $logs;
			$formated_log = $logs;
			$formated_log = $logs;
			$formated_log = $logs;
			$formated_log = $logs;
			$formated_log = $logs;
			return $formated_log; // return the array of info
		}
		else
		{
			$this->badRows++; // if the row is not in the right format add it to the bad rows
			return false;
		}
	}
}
?>

And here's an example of how to use it:

<?php
$data = new apache_log_parser($d->path.'/'.$entry); // Create an apache log parser
$data->filters = array(
	'path' => array('regex' => '/^.*\.(FLV|flv)$/') //pull only flv files
);
 
$data = $data->getData();
?>

A couple things to note about this script though:

1. The regex and parsing was pretty stolen from the Apache Log Parser on PHPClasses.org.
2. Without filters the script is pretty memory intensive. My needs don't require anything client facing but heed my adivice; Don't use this on a public web server.

After my little foray into unlocking my old Blackberry Curve I started to think that there were probably other special codes to play around with. I've done a little bit of research and it turns out, yup, there are quite a few special codes to do some random things.

Decibel meter

Normally, the signal strength meter is a series of bars like the below image. Hold "Alt" and press "N", "M", "L", "L". To undo just redo

View Source Code

This one every web developer should know. In the Browser, hold "Alt" and press "R", "B", "V", "S"

Help Me! menu

If you ever call up for support chances are you've been told to enter this one. It shows a bunch of info in one place but most of it will be greek to most people.  Hold "Alt" and (left) "shift", then press "H".

Display IMEI Code

As far as I know this is really only useful for unlocking a device. Type "*", "#", "0", "6", "#" on the Home screen.

Event Log

The Event Log is probably only going to be useful if you're a developer or similar. It displays all the last Java goodies that have happened since starting the device. Hold "Alt" and press "L", "G", "L", "G".

Show SIM Information

This code will display the details about a SIM card. You have to enter this code from the SIM Options page and press "M", "E", "P", "D"

View Refld and ServiceUserId

From inside a message hold "Alt" and press "V", "I", "E", "W"

View Data Usage & Voice Usage

This one is really only good if you have a new, unused, Blackberry. It maxes out at the below but if you buy a "new" Blackberry enter this code and verify it. From Status Options page press "B", "U", "Y", "R"

Hard Reset

Doing a Hard Reset is like taking the battery out and inserting it again. Handy if you're not in the mood to fuddle with the case. Hold ALT + Right Shift + Del

Validate Contact List

I'm honestly not really sure what these last two do but I'm listing them anyway for completeness. Hold "Alt" and press "V", "A", "L", "D" in address book list

Rebuild Contact List

Hold "Alt" and press "V", "A", "L", "D" in address book list.

These are all the "secret codes" I could find but that doesn't mean there aren't any more. Also, these codes were only verified on the Blackberry Bold 9000 so they may not work with all devices. If anyone has any new ones please leave a comment and I'll update a list.

My business partner recently had her phone literally break apart in her hands. Since communication between us is pretty crucial I decided to give her a backup Blackberry Curve I had as a replacement for my Blackberry Bold. Problem was that the Curve was bought through AT&T and she uses T-mobile. The phone would have to be unlocked.

Initially, I was a little nervous about the process of  unlocking the Blackberry. I'd never personally tried anything like this and my direct experience with the process was that a 3rd party company would have to be brought into the mix. I didn't relish the idea of paying someone to handle what seemed to be a simple exercise when I'm capable of pushing buttons on a freaking phone.

Turns out it's actually extremely easy to handle this yourself though it does require a phone call to AT&T support. Simply call them up and they'll ask you for your IMEI code; it's under the battery on the serial number label. They'll send you an email, like the below, that'll walk you through the process.

Your device unlock request was received and processed, see below for details:

IMEI: YOUR_IMEI_NUMBER

Unlock Code: YOUR_UNLOCK_CODE

Caution: If this process is unsuccessful ten times in a row, the phone will be permanently locked to the at&t network. Do not attempt to enter the code more than one (1) time total.  Instructions below will assist you in unlocking your device, if these steps are unsuccessful please contact us at 1-800-331-0500 or (916) 843-4685 from overseas.

Blackberry 8310

Follow these steps to unlock device:

1. Turn off the radio! VERY IMPORTANT

2. Go to "Options"

3. Scroll to and select "Advanced Options"

4. Click on "SIM Card"

5. Type “MEPD” (You will not see on display what is being typed. To obtain a "P" double tap "OP" key)

6. Type “MEP2” (To obtain a "P" double tap "OP" key. Press “ALT "key to obtain a "2")

7. Enter the unlock code

8. Press enter

9. Reboot device. Device is now unlocked.

Note: To verify the IMEI, dial *#06# on device’s keypad, 15 digit # IMEI will display on the screen.  If this sequence does not work, pull the back/battery off the phone and the IMEI will be listed on the back of the phone.

All told the total time invested was only about 5 minutes. Seriously, 5 minutes. So why would I want to pay someone for this again?

About a month ago I started receiving emails from my server about the installed version of PERL being too old and needing to be upgraded. Since my server uses WHM and cPanel, and thinking that upgrading PERL would have some pretty nasty consequences if something went wrong, I held off until I could familiarize myself with the ins and outs of what this was all about. Now that I've done the upgrade I'm not sure it was really so worth the wait and worry.

The first thing you want to do is backup all your installed CPAN modules. These are basically the library of functionality your system relies on to work. Pretty important stuff in there. It's a pretty simple command though:

perl -MCPAN -e 'autobundle'

The above will create a "bundle" of all the CPAN modules you have installed to be used after the upgrade. Make a note of the bundle location though; you'll need it later.

The instructions to do the actual installation were included in the email and is geared specifically towards cPanel servers:

cd /root
wget http://layer1.cpanel.net/perl588installer.tar.gz
tar -zxf perl588installer.tar.gz
cd perl588installer
./install -optimize-memory

This parts pretty simple, but time consuming. The whole process is automated and should take about an hour or three. When you come back you're going to want to install the autobundle you created earlier. To do that you'll need this command:

perl -MCPAN -e 'install Bundle::NAME_OF_BUNDLE_YOU_CREATED_EARLER'

This is where things started getting difficult. This process is going to require all sorts of input from you. It'll, thankfully, give you default options but the install will sit there until you enter something. All told, this process took me about 4 hours; just sitting there watching the process.

Definitely a pain but not upgrading PERL would have been worse. If you have to do it be sure to set aside an afternoon though.

When I purchased my newest laptop back in July, from Best Buy, it came with a free upgrade to Windows 7 when it came out a few months later. Fast forward to December and I've since received the new operating system and have installed it on the laptop. Since a lot of people are going to be going through the same thing I put together some notes about the experience.

Because the promotion was for a version of Windows 7 that was the same flavor as the installed Windows Vista, I was given a 64bit Windows 7 Home Premium copy. Personally, I would have preferred Business or Ultimate but ok, fine Home Premium it is. At least this meant that I wouldn't have to do a clean install so I could keep all files and programs where they were (still backing up the data of course).

The upgrade package came with 2 discs; the Windows 7 disc and a driver upgrade disc. The instructions said to insert the upgrade disc first and I'll be prompted to enter disc 2 when required. Doing so started an upgrade program that inspected my system and warned me about deauthorizing my iTunes account which immediately made me feel good about the experience.

All told the install took around 3.5 hours and was like watching water boiling. I did it super late at night but I was still up and every time I would check on it I swear the progress rarely looked like it was making any progress. Still, it is Windows, so I was used to this; just wait and be patient, it'll finish. And eventually it did.

Upon first booting up there were a couple issues. One was my fault. The others... not so much. Probably the worst offense was that I had no Internet connectivity. I checked both my wired NIC and the wifi and both were working  fine; I could connect to my router through both and I could find my Xbox and PS3 on the network. I just couldn't get online. I eventually found that this was caused by a conflict between Esets firewall (which I had disabled in Vista) and the native Windows firewall. Uninstalling Eset and reinstalling it solved the issue.

I also had an issue with my local Apache webserver working. After checking the logs it turned out to be a soft link I had created under Vista to link the conf directory (makes editing the files from my working directory that much easier). Creating a new link solved the issue nicely.

The last issue is with the touchpad; and I haven't really fixed it yet. At first, Windows thought the touchpad was a PS/2 mouse. This wouldn't be an issue except I like the scrolling functions on the touchpad and the PS/2 drivers don't support it. This seemed like a cut and dry driver issue except I installed all the latest drivers for my laptop that Gateway offered and it still doesn't work all that well. Yes, it's there but not in any real functional way. It's jerky and hesitant when it works (around 30% of the time). Not enough to get me to downgrade but still a pain.

I've never really had an issue with Vista but I was still excited about Windows 7. Even though there were a couple hiccups during the install, and that my touchpad isn't operating 100%, I'm still really happy with the experience.

It's crazy to me how important email is; it's almost 2010 and I'm still, still, relying on email for communication more than anything else. Considering just how broken email is this, along with how I know it's broken, this isn't a little surprising. Managing the shear amount of email I get used to be pretty easy but now... it's a little more painful but still manageable.

When I had the day job I had access to a nice little network with Exchange 2003 and the latest Blackberry Enterprise Server (BES). It never really occurred to me how spoiled this would make me but, now that it's gone, it's pretty obvious how much I relied on it for my email management.

Why? Syncing. Loved the whole syncing aspect; open an email on your Blackberry and the email was marked as open in Exchange. Send an email from your Blackberry and it appeared in your sent folder in Outlook. Even better if you sent an email in Outlook, through Exchange, it was put in your sent folder on your Blackberry. Simple. This was great for things like filing and responding to emails. Put simply, using the Exchange/BES combo allowed you to keep all your communication in sync.

Then I get unplugged from the BES and have to setup a new solution. First, I went to POP3 (as I was most familiar with the protocol) but this was a cluster fuck of a bad idea. Yes, email was put on the Blackberry but it's a one way method and nothing was kept in sync. I'd do something on the device and when I would go to Gmail nothing would be there; this couldn't do.

I also tried the native Gmail Blackberry app but this too wasn't ideal; I hated how it wasn't truly a "native" app in that other apps would still open the default Blackberry message program to do anything email related. For example, if I clicked on an email on a web page while on the Blackberry it wouldn't open the Gmail app but the native messaging one instead. I was surprised how much this happened and how much it bothered me but it did.

Then I learned that Gmail has IMAP support and have been using it ever since. So far, IMAP is the closest approximation of the BES experience I've found. It's not without it's issues or anything, and technically it's not supported by Google, but it's the best solution I've found.

Probably the biggest issue I have with the IMAP integration is that mail manipulated on the Gmail site doesn't get synced to my Blackberry. This isn't too bad since I usually only use the Gmail site for sending but if I want to keep things in sync I have to treat the Blackberry as the master.

To keep the calendar and contacts in sync requires the use of another Blackberry app called Google Sync. Google Sync it's keeps your contacts and calendar synced up between Google Calendar and Gmail contacts and your Blackberry.

While it's not a perfect solution I've been mostly happy with IMAP and Gmail on the Blackberry. It's way, way, better than paying money for an outsourced BES / Exchange account.