Made of Everything You're Not

In my pursuit for financial independence I’ve been taking on random freelancing gigs from some really smart and interesting clients. One of the more respected clients I work with has been using Expression Engine for their main platform for years, and while I was initially skeptical, I’m beginning to believe there is potential for Expression Engine to be a useful tool too. There’s just one little thing; it’s possible to create a debugging nightmare pretty easily.

Expression Engine Escaping Madness

Expression Engine is built by the same company who put together pMachine, one of my favorite blogging software from back in the day, so I had some pretty high hopes for it. Then I started reading some off the cuff comments about Expression Engine, especially in comparison to my mortal enemy Dolphin CMS, and I started getting a little nervous. Then, when I started seeing how the flow worked, my head almost exploded.

See, all the style and creative stuff is stored in the database. Because Expression Engine has it’s own meta templating language (similar to Smarty in syntax and style; to me anyway) all the templates are available and ready for anyone to make modifications to. On top of that, Expression Engine allows for the inclusion of custom php inside of the stored template files which gets executed with the dreaded and evil “eval()” tag.

Confused? Me too. To help clear things up here’s a snippet of Expression Engine templating code:

{assign_variable:my_weblog="default_site"}
{assign_variable:my_template_group="site"}
{embed="global/header"}
Page Content Here.

It should be pretty obvious what the code above is doing, but because I get a lot of shit for not being verbose (I’m looking at you Reddit), here’s what’s happening:

1. A variable called “my_weblog” is being created with the variable “default_site”.
2. A variable called “my_template_group” is being created with the variable “site”.
3. The header template file is being included.

Not so bad right? I didn’t think so either but there’s also the inclusion of raw php. The below is perfectly valid to do in Expression Engine (assuming the “Allow PHP in Tempaltes” setting is enabled):

<?php
$my_weblog = 'default_site';
$my_template_group = 'site';
include 'global/header.php';
?>

The above is a translation of the Expression Engine code by the way (if you hadn’t picked up on that). This, too, isn’t bad per se, but it does break a few very important rules which I’ll get into in a moment. Annoying and sort of dangerous? Absolutely. But I can see where the appeal lies in allowing this sort of functionality (and, yes, even if you have to use eval() to do so).

That being said, my head almost exploded when I saw how the logic was laid out when mixing both the Expression Engine template tags with php functionality. Keep in mind that Expression Engine has a setting that allows you to set when in the processing flow you want the php to be executed. If that sounds confusing just know that in the below example the Expression Engine stuff is executed before the php code.

Here’s what I mean:

{exp:query sql="SELECT name FROM exp_freeform_entries WHERE entry_id = '1'"}
<?php 
$name = '{name}';
?>

The above simply grabs the name from the table and then sets it up for use by php. Once again, perfectly valid usage it would seem, though the more astute people will immediately see the issue.

Since Expression Engine executes the template tags first this is kind of a snap. The thing is though there’s no escaping going on there. The above will work great when the value of name is something like Eric or John but what if the value is “Eric O’Reily”?

Yeah; it’s gonna break with a parse error. But worst of all when it does break the error message you’re going to get is going to reference the call to eval() and not the actual template file. This is going to make debugging a bit of a bitch. On top of that, there’s no native method to escape anything within Expression Engine itself. So adding the usual call to addslashes() isn’t possible.

So, while Expression Engine is pretty snazzy and nice it isn’t without it’s pitfalls. Mind you, the escaping issue isn’t impossible to avoid; it’s more a question of design than anything. It is something that needs to be watched out for because, yeah, doesn’t seem there’s going to be a change anytime soon.

I’ve been working with Zend Framework a lot lately, which is one reason for the drought in posting, and have kept running into a recurring uncertainty using the URL view helper. The URL view helper included with the Zend Framework is a little confusing at random times for me so in the hopes of making sense out of everything long term the below is a brief outline of how the Zend Framework URL view helper works and what not.

Zend Framework URL View Helper

The Zend Framework URL view helper is used to render a URL that follows the rules setup using the Zend Route module. This is nice because you can change the routes defined for your application and not have to worry about how your URLs are structured.

The basic syntax is below:

<?php 
$this->url(array $urlOptions = array(), $name = null, $reset = false, $encode = true);
?>

Using the above for a template below is an example of the usage and output:

<?php
//outputs /a/b/c/
echo $this->url(array('module' => 'a','controller'=>'b','action'=>'c'), null, FALSE); 
?>

As you can see the URL view helper outputs a simple URL to the module “a”, controller “b” and the action “c”.

Things get a little trickier when you want to pass along some variables though. By default the above example will append any existing variables, that are outside of the MVC paradigm, onto any new URLs created. For example, if the page url is the below:

/*
/a/b/c/foo/4/bar/yes
*/

And you call the below call to the URL view helper:

<?php
echo $this->url(array('module' => 'a2','controller'=>'b2','action'=>'c2'), null, FALSE); 
?>

You’ll get the below:

/*
/a2/b2/c2/foo/4/bar/yes
*/

When I first ran into this issue I was flummoxed. It was kind of a problem (to put it mildly). To get around this you have to set the “reset” value to TRUE. Doing so will keep any existing query variables out of your URL.

To add fresh variables to the URL view helper you use the below syntax:

<?php
//outputs /a2/b2/c2/bar/yes
echo $this->url(array('module' => 'a2','controller'=>'b2','action'=>'c2','bar'=>'yes'), null, TRUE); 
?>

That will, hopefully, keep you from making the same mistake and the subsequent head bashing that would be sure to ensue.

After the server lockout and subsequent move last week I had to setup all my old sites on the new server. Aside from the loss of a few posts my blog (which I could restore from Google cache) and a few comments (which I couldn’t restore unfortunately) everything was pretty smooth. That was until I had to move over one of my clients sites that was using Dolphin CMS.

Remove Boonex Footer From Dolphin CMS

The issue was that the client’s site was now being tagged with a Powered By Boonex footer; not cool because the client had purchased a license and shouldn’t have the callout to Boonex. It looked like the site couldn’t reach the licensing server so it was acting like the site wasn’t a valid and licensed version.

I had set up the new server in a pretty locked down way, using a pretty paranoid strategy with firewalls and port changes and all that fun stuff. Unfortunately for my client, this included mod_security which Dolphin requires special configuration for:

If some security module is installed on the server (such as mod_security for Apache), it should be able to be disabled or set up for specific folders.

Not wanting to allow such a blatant security hole into my server following the above just wasn’t acceptable. Instead I decided to just remove the call to the licensing server in the code; it’s just php so I didn’t think it would be too difficult. It wasn’t but it was a little confusing though so here’s the code and process in case anyone else has the need.

BECAUSE I DON’T WANT TO GET SUED: only use this if you’ve already purchased a license. Blah, blah, blah. Oh, and this has only been tested in Dolphin 6.1.

1. First, open up “/inc/design.inc.php”
2. look for a HUGE base64 encoded line (one really long and one underneath is short). You’re going to need to remove both lines.
   It should be around line 500 and indented a few pages in. If you can’t find it search for “base64_decode” and it’ll come up.
3. Replace the both lines with the below:

$s813518='Y3JlYXRlX2Z1bmN0aW9u';$s534634='base64_decode';$s434741='YmFzZTY0X2RlY29kZQ==';$s865127='ZWNobw==';$s734874='Z2xvYmFsICRfcGFnZTsNCg0KJHM0MzUyMzYgPSBiYXNlNjRfZGVjb2RlKCAnWW1GelpUWTBYMlJsWTI5a1pRPT0nICk7DQokczU4OTM1NSA9ICdYMTlpYjI5dVpYaGZabTl2ZEdWeWMxOWYnOw0KJHM3NDM3NjUgPSAnWjJ4dlltRnNJQ1J6YVhSbE93MEtaMnh2WW1Gc0lDUjBiWEJzT3cwS0RRb2tjMFp2YjNSbGNuTWdQU0FuSnpzTkNtbG1JQ2huWlhSUVlYSmhiU2duWlc1aFlteGxYMlJ2YkhCb2FXNWZabTl2ZEdWeUp5a3BJSHNOQ2lBZ0lDQU5DaUFnSUNBa2MwRm1aa2xFSUQwZ2RISnBiU2huWlhSUVlYSmhiU2duWW05dmJtVjRRV1ptU1VRbktTazdEUW9nSUNBZ2FXWW9JSE4wY214bGJpZ2dKSE5CWm1aSlJDQXBJQ2tnSkhOQlptWkpSQ0F1UFNBbkxtaDBiV3duT3cwS0RRb2dJQ0FnYjJKZmMzUmhjblFvS1RzTkNnMEtJQ0FnSUNSelJtOXZkR1Z5Y3lBOUlDY25PdzBLZlEwS0RRcHlaWFIxY200Z0pITkdiMjkwWlhKek93PT0nOw0KJHM1ODYyODQgPSAnVkcxd2JFdGxlWE5TWlhCc1lXTmwnOw0KJHM5ODU0OTUgPSAnTDE5ZktGdGhMWHBCTFZvd0xUbGZMVjByS1Y5Zkx3PT0nOw0KJHM3ODI0ODYgPSAnYzNSeWNHOXonOw0KJHM5NTAzMDQgPSAnYzNSeVgzSmxjR3hoWTJVPSc7DQokczk0Mzk4NSA9ICdjSEpsWjE5eVpYQnNZV05sWDJOaGJHeGlZV05yJzsNCiRzNjc3NDM0ID0gJ1dXOTFJR2hoZG1VZ2JXRnVkV0ZzYkhrZ2NtVnRiM1psWkNBOFlTQm9jbVZtUFNKb2RIUndPaTh2ZDNkM0xtSnZiMjVsZUM1amIyMHZJajVDYjI5dVJYZzhMMkUrSUdadmIzUmxjbk1nZDJsMGFHOTFkQ0J3WVhscGJtY2dabTl5SUhSb1pTQnlhV2RvZENCMGJ5NGdVR3hsWVhObExDQm5ieUIwYnlBOFlTQm9jbVZtUFNKb2RIUndjem92TDNkM2R5NWliMjl1WlhndVkyOXRMM0JoZVcxbGJuUXVjR2h3UDNCeWIyUjFZM1E5Ukc5c2NHaHBiaUkrUW05dmJrVjRMbU52YlR3dllUNGdZVzVrSUc5eVpHVnlJSFJvWlNCaFpDQm1jbVZsSUd4cFkyVnVjMlZ6SUhSdklHSmxJR0ZpYkdVZ2RHOGdkWE5sSUhsdmRYSWdjMmwwWlNCM2FYUm9iM1YwSUR4aElHaHlaV1k5SW1oMGRIQTZMeTkzZDNjdVltOXZibVY0TG1OdmJTOGlQa0p2YjI1RmVEd3ZZVDRnWm05dmRHVnljeTRnVkdobGVTQjNhV3hzSUdKbElHRjFkRzl0WVhScFkyRnNiSGtnY21WdGIzWmxaQ0JoY3lCemIyOXVJR0Z6SUhsdmRTQnlaV2RwYzNSbGNpQjViM1Z5SUdGa0lHWnlaV1VnYkdsalpXNXpaWE11SUZCc1pXRnpaU3dnY0hWMElIUm9aU0E4WWo1ZlgySnZiMjVsZUY5bWIyOTBaWEp6WDE4OEwySStJR3RsZVNCaVlXTnJJR2x1ZEc4Z1JHOXNjR2hwYmlCMFpXMXdiR0YwWlM0PSc7DQokczU0NjY5MyA9ICdibUZ0WlY5cGJtUmxlQT09JzsNCg0KJHM1NDU2MjQgPSAkczQzNTIzNiggJHM1ODYyODQgKTsNCiRzNDM0NjQzID0gJHM0MzUyMzYoICRzOTg1NDk1ICk7DQokczkzNzU4NCA9ICRzNDM1MjM2KCAkczc4MjQ4NiApOw0KJHMwMjM5NTAgPSAkczQzNTIzNiggJHM5NTAzMDQgKTsNCiRzOTM3NTA0ID0gJHM0MzUyMzYoICRzOTQzOTg1ICk7DQokczM4NTk0MyA9ICRzNDM1MjM2KCAkczU0NjY5MyApOw0KDQokczk4NzU2MCA9ICRfcGFnZTsNCiRzOTQ2NTkwID0gZmFsc2U7DQokczg1OTM0OCA9IGFycmF5KCAyOSwgNDMsIDQ0LCA1OSwgNzksIDgwLCAxNTAgKTsNCg0KaWYoIGluX2FycmF5KCAkczk4NzU2MFskczM4NTk0M10sICRzODU5MzQ4ICkgb3IgJHM5Mzc1ODQoICRzNjUzOTg3LCAkczQzNTIzNiggJHM1ODkzNTUgKSApICE9PSAkczk0NjU5MCApIHsNCiAgICAkczY1Mzk4NyA9ICRzMDIzOTUwKCAkczQzNTIzNiggJHM1ODkzNTUgKSwgZXZhbCggJHM0MzUyMzYoJHM3NDM3NjUpICksICRzNjUzOTg3ICk7DQogICAgJHM2NTM5ODcgPSAkczkzNzUwNCggJHM0MzQ2NDMsICRzNTQ1NjI0LCAkczY1Mzk4NyApOw0KICAgIGVjaG8gJHM2NTM5ODc7DQp9IGVsc2UNCiAgICBlY2hvICRzOTg3NTYwWyRzMzg1OTQzXSAuICcgJyAuICRzNDM1MjM2KCAkczY3NzQzNCApOw==';
 
$s545674=$s534634( $s813518 );$s548866=$s534634( $s434741 );$s947586=$s534634( $s865127 );$$s947586=$s545674( '$s653987', $s548866( $s734874 )

Boonex uses base64 to encode and obfuscate the licensing code so it can’t be modified without a bare minimum of trouble. Not that they had much of an option; php is notoriously hard to encode with any elegance or reliability. Anyway, they chose base64.

All that was needed was to base64_decode the code, and then base64_decode that code (yup, they did it twice). After that I made the changes to remove the HTML that displays the Boonex footer, base64_encoded that, then did it again to create the above.

So, once again, only use the above code if you’ve already purchased a license. Yes, it should work if you didn’t but I don’t want to get sued so it has to be said.

It all started with a simple email from a client with the subject line “Broken Link”; looked like one of my client’s sites was down. Since this was the one that used that lovely of loveliest of programs, Dolphin CMS, I didn’t really thing much of it. Remember, Dolphin CMS sucks and, yes, it’s been known to just crumble on occasion (fucking Boonex…). So, yeah, initially, I didn’t think much of this. But then again, I have been pretty happy lately and the universe really does hate us all. I should have known this was gonna be bad…

A New Kind of Failure Point

Looking into the problem it was immediately obvious what was going on; my server was gone. I couldn’t access any site, much less my client’s site who alerted me to the issue, using any protocol or tool (ssh, ftp, cpanel or whm at least). It was like it didn’t exist…

Immediately, I contacted the server provider, HostGator, where I had a dedicated server requisitioned a few years ago. Thinking this was at worst a network issue I contact HostGator and submitted a support ticket; that’s the only thing to do since I couldn’t personally deal with the issue. That’s the trade off for leasing a dedicated server; on the other hand HostGator will fix anything that goes wrong so it’s worth it in my opinion.

HostGator, to their credit, got back to me within 20 minutes to inform me that the server account was closed. My first thought:

WHAT THE FUCK??!!??

Looking into just what the hell happened I found out that the client who was paying the bill (I had worked out a “deal” for him where I built and maintained his sites and he would pay the hosting bill) just decided that he was done with the whole Internet thing. Seriously. Apparently, he decided to just stop paying the bill a couple months ago forcing HostGator to cancel the account because he, in his words, “Hadn’t made shit from this crap”.

Anyway, this all highlighted a failure point in my backup strategy. Yes, I had a  backup strategy and I even had backups locally. No, my problem was that I didn’t have access to any backup newer than 2 weeks old. Here’s how my backup strategy worked:

1. Daily backups stored to a NAS on the rack.
2. Weekly backups FTPed to another HostGator account
3. Bi-weekly backups were being downloaded to my local network (home) every 2 weeks.

See the problem point? Yeah, keeping too much of an interval on the network instead of within reach and not having access to my recent backups. That. Was. My. Bad.

Two weeks was just too much time to go without a physical backup. It’s an infinity on the Internet; too much stuff can happen in that time and ironically my client with the nasty, nasty, Dolphin CMS is proof of that. Unfortunately, two weeks ago her site had no traffic and no real use to anyone. Come two weeks later and she had gotten the membership up to a couple hundred users who were actually using the site. That’s the part that really sucked; I had failed my client.

This whole ordeal prompted a number of changes to my overall routine. Here’s the new setup.

1. For starters I’m no longer leasing a server; instead I purchased my first server and have it hosted at a data center where I have personal 24/7 access to it and the rack it’s on.
2. Daily backups are being stored to a backup drive in my server and being mirrored to an external NAS on the rack.
3. Weekly, I will go to the data center and swap out the backup drive on the server with another keeping the drive at my house.
4. Bi-weekly backups will be downloaded to my laptop and kept close at hand.

Overkill? Probably, but considering that the majority of this is automated I don’t see too much of an issue with it. In fact, the only thing I have to actually do is go to the data center once a week and switch out the drive so it’s really not too bad.

Hopefully, this’ll work out better than the last strategy.

If you were a sysadmin a few years ago, and you had php on your servers, you’re probably already familiar with c99. In case you haven’t had the personal pleasure, c99, or specifically c99.php (hint: check the source), is the name of a script used by hackers to gain access to a web server running php using an exploit technique called Remote File Inclusion.

The Horrors of C99.php

A Little History

See, back in the day some php developers were pretty stupid. (Admit it; you were stupid once too.) What other explanation could there be for writing code that allowed the injection of arbitrary routines into a program. Trivially easy too.

To be fair, PHP was to blame a little for this as well. Given PHP’s high adoption, and design, by, and for, newbie programmers allowing such a technique by default was just ill conceived, and maybe even a little negligent. I understand the desire, and sometime need, for a technique that could be dangerous but to enable the feature by default…. damn man…

So, the risk was known, yet code was still being written (like the below example) that allowed remote file inclusion to be possible. Mostly because of the aforementioned default setting.

<?php
$color = 'blue';
if (isset( $_GET['COLOR'] ) )
{
	$color = $_GET['COLOR'];
}
require( $color . '.php' );
?>

BTW, if you currently write code that does anything like the above, frankly, you’re an idiot. You aren’t nearly as smart and clever as you think you are. I promise you this will bite you. Bad too.

About C99.php

So, using a technique like the above opens you up to learning first hand about c99.php. Finding information about the program itself is a little tricky but there are a couple examples that highlight just how devastating it can be.

When malicious intruders compromise a web server, there’s an excellent chance a famous Russian PHP script, r57shell, will follow. The r57shell PHP script gives the intruder a number of capabilities, including, but not limited to: downloading files, uploading files, creating backdoors, setting up a spam relay, forging email, bouncing a connection to decrease the risk of being caught, and even taking control of SQL databases. All these functions become readily available through an easy to use web interface, but now you can fight back.

Using the above explanation, which I agree with, c99.php acts as an interface to control your server. Once it’s on your server an attacker has easy access to view all the files and their contents, make changes to the system, upload new files, manipulate the database(s) and more.

Quite the nasty little script but pretty elegant in how it’s implemented. c99 is a completely standalone script; even the images are embedded inside using base64!

Until a month ago I would have thought the risk of encountering c99.php in the wild would have been small these days. Then, SMACK!!, a client had a site get hacked (quick CYA; that I didn’t’ work on ) using c99. So be warned. It’s out there and if you’re not smart, or if you’re a lazy, lazy, coder, c99 will get you.

Among the few hassles to being a citizen of the Internets, along with multiple login credentials, comment trolls and pedophiles (obviously), is the hassle of multiple profile pics. Yes, it’s an extremely small problem but it is a problem solved by Gravatar.

Easily Add Gravatar to Your PHP Apps

Gravatar, which stands for Globally Recognized Avatar (seriously), is a service provided by Automattic (yes, the same Automattic that provides WordPress) that allows users to upload a single image which developers can then use on their websites. The outcome is that users have a single icon (or avatar) representing them, that they don’t have to repeatedly upload. Another, less publicized benefit, is that us developers don’t have to write all the photo uploading and resizing functionality. Again.

The Basics

From an implementation standpoint Gravatars couldn’t be simpler. At the base level it’s just a call to a URL that includes an md5() hash of a users email. For example my gravatar URL is the below:

http://www.gravatar.com/avatar/8a55f4e419d6d5314e420ba26bf7455e?s=75&d=wavatar&r=X

Looking at the URL (which hopefully, you’re comfortable looking at) it’s broken up like so:

Base URL: http://www.gravatar.com/avatar/

My Identifier (eric@ericlamb.net in md5()): 8a55f4e419d6d5314e420ba26bf7455e

Query String Parameters: s=75&d=wavatar&r=X

As far as the query string goes, none of the below are required but using them allows to customize the output. You have the following options:

Image Size: s (between 1 and 512)

Default: d (wavatar, identicon, monsterids or custom URL)

Rating: r (think movie ratings with G – X)

Considering all of the above, and how simple it is, I’m kind of amazed that there’s still an issue with wide adoption (though this could be due to Gravatar scaling the service for the obscene amount of traffic this would create) .

The Codes

Obviously, writing a little snippet to create the Gravatar URLs in PHP would be trivial to most programmers:

<?php
$url = 'http://www.gravatar.com/avatar/';
$email = 'eric@ericlamb.net';
$default = 'monsterid';
$size = 75;
 
$grav_url = $url.'?gravatar_id='.md5( strtolower($email) ).
'&default='.urlencode($default).'&size='.$size; 
?>
<img src="<?php echo $grav_url;?>" />

Simple but the code above could be a lot more useful. As expected though, a few php classes have been written that’ll handle all the particulars for you. One, available on phpclasses.org, I looked at, and even used within a project, but it didn’t have native support for “defaults” so I decided against using it as an example here (still a cool script if the next one doesn’t float your boat).

The other script is available through TalkPHP, so it’s not really clear who the actual author is (sorry for not giving credit whoever you are). The code is clean, clear and dead easy to implement though.

<?php
include_once('./TalkPHP_Gravatar.php');
$pAvatar = new TalkPHP_Gravatar();
$pAvatar->setEmail('adam@talkphp.com')->setSize(80)->setRatingAsPG()->setDefaultImageAsIdentIcon();
?>
<img src="<?php echo $pAvatar->getAvatar(); ?>" />

Very, very simple. I have to say I’m a fan of Gravatar. I wish more sites implemented Gravatars, instead of having me constantly uploading a different image, but now you can do your part. Get cracking.

With the release of Google Buzz last week a lot of people have been screaming bloody murder over some privacy concerns they have and Google’s perceived lack of forethought on the matter.

Google Didn't Fuck You; You Did

First, Google Buzz appears to be a FriendFeed clone that Google just launched about a week (or 2) ago. Initially, it was enabled inside of all gmail accounts by default without any authorization to the contrary. I haven’t had the opportunity to try it though. Not because I don’t use gmail (I do; sorta) but because I use Google Apps gmail which wasn’t a part of the rollout.

From what I can glean; Google Buzz works by parsing your contact list and then making connections between everyone in it and displaying their social network activity info publicly for all to see (seriously, just like FriendFeed). Make sense? No? Here’s the Crunchgear explanation of Google Buzz:

Google Buzz is a social network and sharing product built by Google. Based within Google Profiles, Buzz offers a stream of status updates, pictures, links, and videos from your friends. You can “like” these items and you can comment on them. Updates from Flickr, Picasa, Google Reader, or Twitter can also be automatically imported into a Buzz stream. Buzz will recommend items you might like based on your friends’ activity.

So, apparently, one of the “features” of Google Buzz is that when it was initially released it displayed your contact list publicly which raised all sorts of hell from people who can’t afford for this to happen (think lawyers, journalists, etc).

This smacks of a high level of naivete on most of the users. Under what delusion are people living in to think that they have any expectation of privacy from a publicly traded company. Yes, I know they claim to care about your privacy, and I’m sure on a personal level the people working for Google do, in fact, care about your privacy. But the organization itself? Not a fucking chance.

Let’s get serious here; as stated above, Google is a publicly traded company which means their priorities start and end with cash ($$$). Frankly, it’s naive to think otherwise. Ask any corporate officer and they’ll tell you they have a responsibility to their shareholders. This is a notorious lose for consumers but it’s the reality nonetheless. Cry all you want but Google fucking their users in this way did ensure they launched a new social network with millions of users. From a fiscal standpoint, this was a HUGE win even with all the bitching and moaning. Even taking into account any users who would leave Google (along with any ill will this may have created) this was still a winning strategy for launch.

If privacy is an issue then, it seems to me, that you really should have taken greater measures to protect yourself. Relying on Google to protect something like this screams of escapism and finger pointing. Guess what? It’s your fault. Deal with that instead of crying that a publicly traded company that provides a service you use for free does something in a way that you don’t like.

Do I think that Google was right in any way for doing what they did? Not for a second. That said, people need to take responsibility for their own needs instead of blindly trusting a for profit company to do it for them. Yes, even when that company claims to “do no evil”.

I was working on a project recently that was using WordPress as the base platform and had a need for a scheduled task function. Ordinarily, this would be the type of functionality that I would just setup a cron job for; but since WordPress has a pseudo cron system in place I decided to investigate that as an option. Long story short; fuck that man.

Working With WP Cron

As mentioned, WordPress has a pseudo cron mechanism available to WordPress developers for scheduling tasks called WP Cron. WP Cron works using series of functions available for plugin developers to create their own cron style tasks. In theory WordPress makes this easy(ish) using the built in cron API but the reality is that it’s not only confusing and awkward to implement but once you’re complete you have to allow for a bit of latitude on the scheduling.

Take a look at scheduling a task in WordPress. Basically, it works by adding a function to your activation hook, within a plugin, that like the below:

<?php 
register_activation_hook(__FILE__, 'my_activation');
add_action('my_hourly_event', 'do_this_hourly');
 
function my_activation() {
	wp_schedule_event(time(), 'hourly', 'my_hourly_event');
}
 
function do_this_hourly() {
	// do something every hour
}
?>

Pretty simple implementation really though not very “cron” like in syntax. The important part is the call to “wp_schedule_event()” which takes 4 parameters (start date, recurrence, hook and arguments). According to the API manual the function:

Schedules a hook which will be executed by the WordPress actions core on a specific interval, specified by you. The action will trigger when someone visits your WordPress site, if the scheduled time has passed. See the Plugin API for a list of hooks.

The problem I had was that this function only allows for 3 options on recurrence; hourly, twicedaily and daily. I needed my function to run at a specific time. (It was EXTREMELY important that it ran during a specific interval.)

This isn’t to say that it’s not possible to schedule a task for a specific time; it’s just a pain in the ass. You have to write all the logic to determine when to add a new task and removing expired tasks. Ughh. To me the question becomes “Should I write a couple hundred lines of code or just add a single line to my crontab?”.

Crontab baby.

Recently I had a problem; the plugin I was using to display my resume on my site mysteriously started causing WordPress to throw a weird error. Not to get too technical about it but the issue was a little more complicated than I was willing to deal with personally and I was prepared to resign myself to not having an online resume for a while. Sigh…

Fast forward about a month and the hoped for “fix” for the plugin never came. This left me with no alternative than to write my own. Yay!!

Looking at the plugin I was using, LinkedIn hResume, and noticing some of the flaws (like the plugin ONLY working with LinkedIn) I realized I could one-up the plugin and make something a lot more useful to more people by writing a new, custom resume plugin.

And that’s what we have here; WP-hResume. WP-hResume is a wordpress plugin that takes any hresume encoded webpage and allows you to place the content on your site. It’s been tested using both LinkedIn and Stack Overflow Careers as well as quite a few stand alone hresume pages. It works wonderfully.

Please take a look and let me know if you like

Parsing the log files generated by Apache is one of those random tasks with a random occurrence in my world. This is a task that, until recently, hadn’t come up enough to warrant any sort of a ready solution (and it was just fun enough to be ok to write a custom solution). So every time this came up I would always fire up Google and go on a scavenger hunt for a starter script written in php.

Parse Apache Log Files With PHP

This always felt like a good idea at the time the need came up. These days, for some ungodly reason, parsing Apache logs seems to come up a little too frequently to keep this up. In the spirit of making my life a hell of a lot easier for tomorrow I’ve taken a shot at writing an Apache log parser written in PHP.

One thing I decided to implement is a filtering system so you can filter out based on a provided regex. Might not be too useful to everyone but it should be trivial to remove the functionality.

Anyway, I hope someone finds this useful (even to learn from and, of course, use)

Here’s the main class:

<?php
/**
 * Apache Log Parser
 * Parses an Apache log file and runs the strings through filters to find what you're looking for.
 * @author Eric Lamb
 *
 */
class apache_log_parser
{
	/**
	 * The path to the log file
	 * @var string
	 */
	private $file = FALSE;
 
	/**
	 * What filters to apply. Should be in the format of array('KEY_TO_SEARCH' => array('regex' => 'YOUR_REGEX'))
	 * @var array
	 */
	public $filters = FALSE;
 
	/**
	 * Duh.
	 * @param string $file
	 * @return void
	 */
	public function __construct($file)
	{
		if(!is_readable($file))
		{
			return 	FALSE;
		}
 
		$this->file = $file;
	}
 
	/**
	 * Executes the supplied filter to the string
	 * @param $filer
	 * @param $status
	 * @return string
	 */
	private function applyFilters($str)
	{
		if(!$this->filters || !is_array($this->filters))
		{
			return $str;
		}
 
		foreach($this->filters AS $area => $filter)
		{
			if(preg_match($filter['regex'], $str[$area], $matches, PREG_OFFSET_CAPTURE))
			{
				return $str;
			}
		}
	}
 
	/**
	 * Returns an array of all the filtered lines 
	 * @param $limit
	 * @return array
	 */
	public function getData($limit = FALSE)
	{
		$handle = fopen($this->file, 'rb');
		if ($handle) {
			$count = 1;
			$lines = array();
		    while (!feof($handle)) {
		        $buffer = fgets($handle);
		        $data = $this->applyFilters($this->format_line($buffer));
		        if($data)
		        {
		        	$lines[] = $data;
		        }
 
		        if($limit && $count == $limit)
		        {
		        	break;
		        }
		        $count++;
		    }
		    fclose($handle);
		    return $lines;
		}		
	}
 
	/**
	 * Regex to parse the log file line
	 * @param string $line
	 * @return array
	 */
	function format_log_line($line)
	{
		preg_match("/^(\S+) (\S+) (\S+) \[([^:]+):(\d+:\d+:\d+) ([^\]]+)\] \"(\S+) (.*?) (\S+)\" (\S+) (\S+) (\".*?\") (\".*?\")$/", $line, $matches); // pattern to format the line
		return $matches;
	}
 
	/**
	 * Takes the format_log_line array and makes it usable to us stupid humans
	 * @param $line
	 * @return array
	 */
	function format_line($line)
	{
		$logs = $this->format_log_line($line); // format the line
 
		if (isset($logs[0])) // check that it formated OK
		{
			$formated_log = array(); // make an array to store the lin info in
			$formated_log['ip'] = $logs[1];
			$formated_log['identity'] = $logs[2];
			$formated_log['user'] = $logs[2];
			$formated_log['date'] = $logs[4];
			$formated_log['time'] = $logs[5];
			$formated_log['timezone'] = $logs[6];
			$formated_log['method'] = $logs[7];
			$formated_log['path'] = $logs[8];
			$formated_log['protocal'] = $logs[9];
			$formated_log['status'] = $logs[10];
			$formated_log['bytes'] = $logs[11];
			$formated_log['referer'] = $logs[12];
			$formated_log['agent'] = $logs[13];
			return $formated_log; // return the array of info
		}
		else
		{
			$this->badRows++; // if the row is not in the right format add it to the bad rows
			return false;
		}
	}
}
?>

And here’s an example of how to use it:

<?php
$data = new apache_log_parser($d->path.'/'.$entry); // Create an apache log parser
$data->filters = array(
	'path' => array('regex' => '/^.*\.(FLV|flv)$/') //pull only flv files
);
 
$data = $data->getData();
?>

A couple things to note about this script though:

1. The regex and parsing was pretty stolen from the Apache Log Parser on PHPClasses.org.
2. Without filters the script is pretty memory intensive. My needs don’t require anything client facing but heed my adivice; Don’t use this on a public web server.