Made of Everything You're Not

Here’s a quick fix for a weird issue I ran into while working with MySQL on Windows.

MySQL

The first time I moved code from my Windows box to one of my Linux servers I ran into an error about a MySQL table not existing in one of my queries. I checked the table and the code to verify everything was as it should be and noticed that all my tables were lowercase instead of CamelHumped. Odd….

Here’s the SQL:

$sql = "SELECT * FROM TableName WHERE id = '".$DB->es($id)."'";

It turns out that MySQL on Windows will convert all tables created to a lowercase name. To remain consistent with Linux you should add the below to your my.cnf file:

set-variable=lower_case_table_names=0

Adding the above line will preserve the naming conventions between both platforms.

Like most web developers, I use Linux on a daily basis. Because I always have to revisit how to do some of the basics from time to time I thought I’d put together a list of some of the commands I use the most along with some of their examples. NOTE: the paths are all from RHEL 4 and CENTOS 4 & 5.

Linux

Enjoy!

MySQL

Example of mysqldump::

mysqldump -u USER -pPASSWORD DATABASE > filename.sql

Example of restore DB::

mysql -u USER -p DBNAME < dump.sql

Start the MySQL service::

/etc/init.d/mysqld start

Stop MySQL service::

/etc/init.d/mysqld stop

Add User::

GRANT ALL PRIVILEGES ON *.* TO 'USER'@'localhost' IDENTIFIED BY 'PASSWORD' WITH GRANT OPTION;

Export Single Table::

mysqldump --opt -u USER-pPASSWORD DataBaseName TableName > FileName

rsync

rsync is used primarily synchronize data between two directories or servers.

Sync 2 servers

rsync -p -r -a -z -v /path/to/dir 2.2.2.2:/path/to/dir

Sync 2 directories

rsync -p -r -a -z -v /path/to/dir /path/to/new/dir

Lighttpd

Start the lighttpd service::

/etc/init.d/lighttpd start

Stop the lighttpd service::

/etc/init.d/lighttpd stop

Path to conf::

/etc/lighttpd/lighttpd.conf

SSH

Restart SSH Server

/etc/rc.d/init.d/sshd restart

Start SSH Server

/etc/rc.d/init.d/sshd start

Stop SSH Server

/etc/rc.d/init.d/sshd stop

Tar/GZ

Create tarball

tar -cvf directory.tar directory/

Create gzipped tar file

tar -czvf directory.tgz directory/

Extracting tarball:

tar -xvf directory.tar

Extracing gzipped tar:

tar -xzvf directory.tgz

Misc.

Delete the mail queue

rm /var/spool/mqueue/*

Count files in directory recursively

ls -R -1 proc | wc -l

I’ve been using lighttpd as my main production web server since the summer of 2007. I first heard about lighttpd from a server administrator I was working with on a project a few years earlier but since my clients weren’t getting traffic that Apache couldn’t handle I didn’t really pay it too much attention. As far as I was concerned it was a cool idea, building a better webserver, but since lighttpd was so new and support and documentation being a little sparce, compared to, say, Apache’s documentation, all using lighttpd meant was that I had to stay away from Apache specific functionality and I’d be ok.

lighttpd

Fast forward a couple years and I’m working on projects that Apache is really struggling with. The point of removing Apache from my toolbox started because one of our projects at StreetWise, the official community for one of our biggest client’s programs, was having serious load issues. The site was very popular, getting over 2,000,000 hits a day, and Apache was having a hard time dealing. Here we were with 8 web servers, on a load balancer, 3 DB servers, and all of the web servers were just getting hammered; it was cascading, where one by one the servers would go into swap and all traffic would be shifted to a different server, only to have the same thing happen again until all servers died. It was a very, very, stressful time for my team and I.

We tried configuring the hell out of Apache, upgrading the server’s RAM and adding additional CPUs but finally came to realize that continuing with Apache meant we’d just have to get more servers. Apache can only handle so many users at one time, like all webservers really, so we needed a web server that could handle more users at one time. It’s a pretty basic While adding additional servers to a project is cheaper than throwing people at it, if it’s a software issue, there’s additional costs with hardware in my team especially because we manage the servers ourselves.

This wasn’t good; it meant explaining to the client that we would need more money outside of our projections, which is embarrassing and, worse, my team would have to manage more servers (we were already managing 18 in total at the time).

It was at this time that lighttpd came back into play. Since Apache wasn’t working out so hot in the load we were under it had to be replaced. I remembered how lighttpd was started as a proof of concept for the C10K problem and had some pretty impressive benchmarks so I started researching more about it.

While lighttpd’s penetration wasn’t even a blip compared to Apache or IIS we installed it on a one of our dev servers and started testing it.

lighttpd Overall Usage

Installing lighttpd was pretty straightforward though setting up php to run in fast-cgi mode was a little more challenging than usual. Right away, one of the biggest challenges was the configuration though. After using Apache for, well, forever really, moving to how lighttpd was configured was pretty difficult.

lighttpd.conf example

$HTTP["host"] =~ "(www.)?your-domain.com" {
 
  server.document-root = "/www/your-domain.com/html"
 
#### auth module
## read authentication.txt for more info
auth.backend               = "htpasswd"
auth.backend.htpasswd.userfile = "/www/your-domain.com/.htpasswd"
#auth.backend.plain.groupfile = "lighttpd.group"
auth.require               = ( "/" =>
                               (
                                 "method"  => "basic",
                                 "realm"   => "Server Info",
                                 "require" => "valid-user"
                               )
                             )
 
}

Compare the above to the below.
httpd.conf example

<VirtualHost *>
    ServerName your-domain.com
    DocumentRoot /www/your-domain.com/html
    ServerAdmin contact@your-domain.com
    <IfModule mod_suphp.c>
        suPHP_UserGroup nobody nobody
    </IfModule>
</VirtualHost>

Right away, you can see how Apache and Lighttpd have different philosophies on layout and structure. Apache takes an XML structure for their configuration files while lighttpd formats its configurations like an array.

Anyway, after familiarizing ourselves in how lighttpd was configured we started the process of moving all the production servers away from Apache over to lighttpd. We went one server at a time, installing lighttpd, configuring, benchmarking and then changing the server’s init so lighttpd was started and Apache wasn’t. All told the process took about a week.

Since then, I haven’t had a single issue related to load on one of lighttpd boxes. If you’re serious about performance and care about how your projects function check out lighttpd ASAP.

I was going through some legacy code for a client and ran across a whole bunch of calls to sprintf(). The calls were all similiar to the below:

$query = sprintf('SELECT * FROM table WHERE field1 = "%s" AND field2 = "%s" LIMIT 1',$var1, $var2);

It seemed an odd use to call the above instead of the straight forward, and low impact, way of declaring the variable.

$query = "SELECT * FROM table WHERE field1 = '$var1' AND field2 = '$var2' LIMIT 1";

I’ve never really had the need to use sprintf() so I’m not familiar with it enough to know much so I asked a colleague of mine, Devin, “What process would require the use of sprintf?”. According to Devin, sprintf is used for complicated and dynamic formatting. Here’s an example from the php manual on the subject:

<?php
$n =  43951789;
$u = -43951789;
$c = 65; // ASCII 65 is 'A'
 
// notice the double %%, this prints a literal '%' character
printf("%%b = '%b'\n", $n); // binary representation
printf("%%c = '%c'\n", $c); // print the ascii character, same as chr() function
printf("%%d = '%d'\n", $n); // standard integer representation
printf("%%e = '%e'\n", $n); // scientific notation
printf("%%u = '%u'\n", $n); // unsigned integer representation of a positive integer
printf("%%u = '%u'\n", $u); // unsigned integer representation of a negative integer
printf("%%f = '%f'\n", $n); // floating point representation
printf("%%o = '%o'\n", $n); // octal representation
printf("%%s = '%s'\n", $n); // string representation
printf("%%x = '%x'\n", $n); // hexadecimal representation (lower-case)
printf("%%X = '%X'\n", $n); // hexadecimal representation (upper-case)
 
printf("%%+d = '%+d'\n", $n); // sign specifier on a positive integer
printf("%%+d = '%+d'\n", $u); // sign specifier on a negative integer
?>

The above outputs:

%b = '10100111101010011010101101'
%c = 'A'
%d = '43951789'
%e = '4.39518e+7'
%u = '43951789'
%u = '4251015507'
%f = '43951789.000000'
%o = '247523255'
%s = '43951789'
%x = '29ea6ad'
%X = '29EA6AD'
%+d = '+43951789'
%+d = '-43951789'

Pretty cool, but I’m not still not sure what the purpose of using sprintf() like the first example demonstrated would accomplish. Surely, there must be some sort of performance penalty, right? According to a programmer Devin used to work, there’s a HUGE penalty for using sprintf. To find out I wrote a simple test script, that’s the farthest thing from scientific, to just get a generic baseline on memory usage and sprintf().

<?php
 
for($i=0;$i<100000;$i++){
	//Call sprintf
}
 
echo filesize_format(memory_get_usage());
echo '<br>';
echo filesize_format(memory_get_peak_usage());
 
function filesize_format($bytes, $format = '%01.5lf %s', $force = '')
{
	$force = strtoupper($force);
	$defaultFormat = '%01d %s';
	if (strlen($format) == 0)
			$format = $defaultFormat;
 
	$bytes = max(0, (int) $bytes);
 
	$units = array('B', 'KB', 'MB', 'GB', 'TB', 'PB');
 
	$power = array_search($force, $units);
 
	if ($power === false) {
		$power = $bytes > 0 ? floor(log($bytes, 1024)) : 0;
	}
 
	return sprintf($format, $bytes / pow(1024, $power), $units[$power]);
}
?>

First the baseline. This is without running the loop.

/*
Baseline of 0 loops just php start stop:
Total: 71.39062 KB 
Peak: 76.84375 KB
*/

Next, I did a run of sprintf() calls like the below:

/*
Total: 73.76562 KB
Peak: 77.22656 KB
*/
for($i=0;$i<100000;$i++){
	$t = 'SELECT col1,col2,col3 FROM table WHERE col1="1" AND col2="2" LIMIT 1';
}

Followed by a straight variable declaration:

/*
Total: 72.82812 KB
Peak: 77.17969 KB
*/
for($i=0;$i<100000;$i++){
	$t = 'SELECT col1,col2,col3 FROM table WHERE col1="1" AND col2="2"  LIMIT 1';
}

Huh… the numbers don’t really add up to all that much. I really thought it’d be more of a difference than that. Well, just to be sure I ran two more tests like the above, except with 4 variables being created instead of 1.

/*
Peak: 76.69531 KB
Peak: 78.28125 KB
*/
for($i=0;$i<100000;$i++){
	$t = sprintf('SELECT col1,col2,col3 FROM table WHERE col1="%s" AND col2="%s" LIMIT 1','1', '2');
	$r = sprintf('SELECT col1,col2,col3 FROM table WHERE col1="%s" AND col2="%s" LIMIT 1','1', '2');
	$k = sprintf('SELECT col1,col2,col3 FROM table WHERE col1="%s" AND col2="%s" LIMIT 1','1', '2');
	$o = sprintf('SELECT col1,col2,col3 FROM table WHERE col1="%s" AND col2="%s" LIMIT 1','1', '2');
 
}

And

/*
Peak: 73.95312 KB
Peak: 77.48438 KB
*/
for($i=0;$i<100000;$i++){
	$t = 'SELECT col1,col2,col3 FROM table WHERE col1="1" AND col2="2" LIMIT 1';
	$r = 'SELECT col1,col2,col3 FROM table WHERE col1="1" AND col2="2" LIMIT 1';
	$k = 'SELECT col1,col2,col3 FROM table WHERE col1="1" AND col2="2" LIMIT 1';
	$o = 'SELECT col1,col2,col3 FROM table WHERE col1="1" AND col2="2" LIMIT 1';
}

Look at that; nothing. There doesn’t appear to be any significant increase in memory in relation to usage. For a loop that runs 100,000 times compared to a memory increase of only about 1 KB this really isn’t bad. Use sprintf if you want to.

Oh yeah, Devin you were lied to

I wrote a script a while ago, to help me in the search for freelance work, to parse craigslist every few minutes and send an email if a match was found to a list of terms (similar to the script created in A journey into php-cli and scraping) to . I’ve since stopped looking and have settled back into my day job but I’ve left the script running just in case something cool comes up.

Stupid Person

Any experienced freelancer will tell you how the quality of the listings on Craigslist can be really bad sometimes. I have personally seen ads from people looking for a developer to build a whole social networking site (think myspace clone) with a budget for the project at $250. Just ludicrous ideas coming from people who don’t know any better; when you talk to them you find out how their brother in law told them $250 was plenty and how $250 was all the money they had. They just don’t know any better; like talking to your grandmother about her website.

I’ve gotten into the habit of ignoring ads like those because, well, it’s just not worth it; the client’s expectations aren’t realistic and educating them is just too painful to bother. Just leave it alone, that’s what I do. Not so for other people though.

Over the weekend (1/17 & 18) there was a bunch of back and forth where a man had posted an ad stating:

NEED EXPERIENCED AND LOCAL IN LOS ANGELES WEB DESIGNER FOR MUSIC ARTIST WEBSITE…..MUST WORK ON SHOESTRING BUDGET………..PREFER STUDENT OR SOMEONE WHO NEEDS TO BUILD THEIR PORTFOLIO…..ABSOLUTELY NO OVERSEAS OR OUT OF STATE REPLIES……..AGAIN LOW LOW PAY SO DON’T RESPOND I
CT

He clearly has a small budget and is being up front about this. The sentence “MUST WORK ON SHOESTRING BUDGET” stands out and told me not to even bother. I deleted the email from my blackberry and went about my day (which was awesome BTW).

Apparently, others didn’t feel the same as me though. I got numerous emails from people replying to the ad, hammering on the guy, acting like children, with name calling and insults for posting the ad. They tried to wrap the sentiment into a warning but you get the impression they got more glee out of pointing out the obvious than satisfaction from “warning” others about ad.

This guy Crosby Tyler will only pay 100 dollars for what I would call an extensive website. He shamelessly responded that he’s looking for a “kid in school” to build it for him. It has a store, music samples, upcoming dates. And lots more.

Normally I don’t agree with posting replies to ads just because you dont approve of the posters budget. But I want to encourage everyone here to email this individual and try and explain that this is unheard of and if cannot afford to pay more than 100 then having a website is not for him.

And most importantly, do not accept this kind of compensation or get taken advantage of.

And

Yeah, I spoke with him too… he’s an idiot. He wants to pay designers less than what kids at In-N-Out make, and has the nerve to talk down to ppl… definite idiot… Designers beware.

This just seems like overkill to me. I mean, the guy said, in the post, that there was a shoestring budget and people are amazed and insulted when they contact him only to find that, yup, there’s a shoestring budget.

In the end Crosby actually posted an apology:

Dear Paul S and any others who felt my ad was insulting to you as web developers, I deeply apologize and have decided to remove my post off Craigslist….Other than commenting on your work samples from my point of view and setting a budget which I could afford I’m sorry again if this has angered you….. Any other resolutions that can amend your feelings I’d be happy to oblige to….. Again, accept my apologies and much success in your ventures……..
Crosby

This was unfortunate and completely unnecessary.

Version 0.2 of wp-click-tracker has been out for almost 4 days and it looks like I left a couple bugs in there. I’ve updated the repository with the latest version and you can also download it from the main page:

wp-click-track download

This release is just a bug fix for the below:

• Parsing of attributes with single quotes
• added parsing / tracking of Archive links
• added parsing / tracking of Bookmark links
• added parsing / tracking of Comment Author URL links
• fixed option updates

Well it’s taken a couple weeks but I’m finally done with 0.2. This release includes an extensive reporting area, the addition of tracking on links created in the Links area of the admin, incompatibility checking for Google Analytics plugin and an upgrade notification nag.

Top Referrers

Top Referrers

Individual Clicks

Individual Clicks

Clicks by Hour

Clicks By Hour

Clicks by Day

Clicks By Day

Upgrade Checking

Upgrade Click Tracker

Plugin Incompatibility Checking

Compatibility Warning

I usually format my computer about once a year like clockwork; whether it needs it or not. I find that it just helps. Windows collects a lot of gunk over time that just needs to be cleaned out (in fact, if I have to spend more than a couple hours troubleshooting an issue I format the computer and start over). Doing this may take about a day, from start to finish, but when It’s complete the computer I’m left with is always a lot stronger and faster than the one I started with.

Since I manage the network for my day job, as just one of my responsibilities, I’ve carried this idea forward into taking care of client computers with a couple of additions for the corporate workplace. If a virus or any piece of spyware infects one of my machines we format it and start over (it’s too much work trying to disinfect it and I’m never 100% satisfied the virus or spyware won’t return).

The process for completing a complete reset goes something like this:

1. Backup Files
2. Install Windows (w/ format)
3. Install Drivers
4. Activate OS (this SUCKS)
5. Update OS
6. Add to Domain
7. Install Needed Programs
8. Restore Files
9. Hand Off to User

The above can usually take anywhere from 5 to 7 hours depending on activation issues and the programs needed. When doing this on my personal computer it sucks, but not nearly as bad as when it has to be done to 8; I’ve just spent the last 3 days creating Windows XP boxes for my domain. Of course, during this time I didn’t do any “real” work (coding) or make money for the company in any substantive way.

Now, once upon a time I had a staff. I had them always make sure there was a few spare computers around in case something happened; to be used to replace a users computer and get the user back up and running ASAP. If we needed a computer it worked like such:

1. Backup Files
2. Install Needed Programs
3. Restore Files
4. Hand Off to User

Like a pit crew, we usually had the turn around time down to an unpit crew like time of about 20 minutes. This was usually followed by restoring the computer back to factory install and adding it to the queue of stand by computers.

Now that I only have a staff of one it becomes a little tougher to manage the network in a preventive way without taking time away from my real love; programming. Since we’re about to head into a couple months of project work it behooved us to prepare the network for the reality of a Windows life; we needed to do this.

We started Monday morning with 5 computers to do with an order of 3 to provide to users which then needed to be reset as well. We set up computers at 3 different, unused, workstations and began the process. Starting at 10AM, we finished the first one at 3PM and the third about 30 minutes later (of course all the computers are different models which required different drivers and updates).

Repeat above paragraph two more times.

Sigh…

There has to be a better way. I wish I could lay this pain squarely on the shoulders of Microsoft but that’s not fair. Sure, the update process for Windows takes it’s fair share of time to complete, and the shear volume of updates to complete is amazing but, no, it’s not all painful because of Windows.

Here are the minimum programs I need to install on a workstation with the time it takes to finish:

1. MS Office 2003 (10 minutes)
2. Adobe CS3 (60 minutes)
3. Adobe Acrobat (30 minutes)
4. Eset Nod32 (3 minutes)
5. Edit Plus 3 (1 minutes)
6. MS Project (15 minutes)
7. MS Visio (15 minutes)

As you can see, Adobe installers take a while too. And that’s not even including the updates, of which there are many.

How come there isn’t some way to automate this; I know it’s possible but a solution’s never come into my world. Considering how frequent setting up a computer happens in the corporate world I’m surprised.

Of course I could be wrong…

So a couple days ago I finalized the beta of a new WordPress plugin: wp-click-track. But, well, no one knows about it yet. Which is a shame because it’s pretty cool (IMHO anyway). What to do… what to do?

Turns out there are all sorts of options available to promote your plugin.

A warning though: you will need to create multiple online identities with a few websites and none of them offer OpenID so be sure to crack open your identity manager before starting. (BTW, WTF is up with that? I mean, even WordPress makes you create different identity to host a plugin. This is in addition to your current WordPress account. Get it together people!)

Anyway, here’s a list of places to start:

Plugins/Plugin Compatibility Wiki*
Select the version(s) of WordPress your plugin is compatible with and add your plugin to the compatibility list.
*You have to sign up for an account first!

Add Your Plugin to WordPress.com
By signing up with WordPress you get access to the subversion repository for managing your plugin. You already have to know subversion (but of course you already do; I’m sure). Just fill out the form and you’ll receive an email with instructions on how to add your files to the repo.

One more note on the WordPress Plugin Repository; pay attention to the format of your readme.txt. The format of this effects your plugin page on the site; if the format is off your page won’t be accurate and there’s about a 5 minute delay between when you update the repository and when the changes show up online. Be sure to check out the readme.txt validator to make sure the format’s ok.

Fill out the form and submit. NOTE: You’ll need a different set of credentials.

wp-plugins.net
Pretty basic here: Create account, fill out form. I’m not really sure how trafficked this site is but it’s recommended by WordPress.com for plugin so there you go.

Weblog Tools Collection
They seem to have an ongoing list of new and updated plugins.

A while ago I used to own the domain ericlamb.com and all was good in the world. The sun came out every morning, birds would sing and women loved me. Then, in a stupendous act of laziness on my part, the domain expired and someone else swooped in and bought it.

Panicking, I hurriedly purchased the domain ericlamb.net, where you’re at, and reset all my files, I have a LOT of dev sites, and email addresses and moved on with my life. Sure, it was a sore spot for me, losing my domain like that, but what the hell; life goes on.

Recently, I found myself at GoDaddy.com attempting to access my domain so I could change the DNS to a new server. Having forgotten my account number I went to the “forgot information” area and attempted to retrieve everything. You enter in the domain you want the information for and they send an email with that information to the registered domain owner. Not giving it a second thought I put in ericlamb.net, clicked submit, and .

Woops! Unfortunately, the account was created with my old email address eric@ericlamb.com; which I don’t have anymore. Sigh…

This is ok though because you can request the merger of separate accounts into a single account.

Go to:
https://www.godaddy.com/gdshop/change/ChangeRequest.asp?ci=6149&prog_id=GoDaddy

fill out and fax to:
(480) 624-2547

Or scan and email it to: change@secureserver.net

They will get back to you in about 3 days with either questions or confirmation the request is completed.

Easy; like your sister.