Made of Everything You're Not

Last June I decided to do something unprecedented for me; I dipped my toe into the ExpressionEngine add-on market to try and sell some custom add-ons I’d written. Being the open source loving hippie that I am this felt pretty out of character for me at the time and I got a bit of grief for it from friends and my own id. But I’ve been talking about moving out of client services for far too long and figured a little experiment by writing some add-ons and selling them through Devot:ee (the de facto marketplace for ExpressionEnigne add-ons) would be a good introduction into full time commercial development. Looking at things 4 months later I realize that deciding do that was one of most important things I could have done for my career.

mithra62

Those of you who read this blog regularly (which, I admit hasn’t been that regular of late) probably noticed that I’ve been writing more and more about ExpressionEngine over the last year and a half. Obviously, I’ve been doing a lot of ExpressionEngine development. Mostly for client work but, like most things a good programmer does, I eventually built up a go to library of custom stuff for when new projects started. Just your normal stuff really; a couple extensions and modules to make my client’s lives easier and give that little “extra” to make a project sparkle.

Then, my CartThrob project ended. This is a project that will forever have a place in my heart. Not the site specifically mind you, which is pretty awesome BTW, but the clients tight budget combined with needing more from the site than they had initially outlined created a dilemma that presented an opportunity. I could either hand off the site as is, which was to the client’s spec but not to their expectations, or I could continue working without getting paid to make the client happy. Then, a third option came to mind.

I proposed to the client that I build an add-on to handle their needs on the condition that I retain ownership of the add-on. The plan was simple; I’d write this add-on and then sell it on Devot:ee in the hopes that eventually I’d recoup my loss in time investment. Hence, the birth of my first commercial add-on now called CT Admin. That… I then sat on for a couple months while I worked up the courage to put it up for sale.

The more I thought about releasing CT Admin the more nervous I became over the prospect of putting out an add-on that’s only useful for users of another third party add-on. Starting out with a customer base that’s a small slice of another company’s customer base isn’t exactly a good way to measure an industry. So, I decided to release a second add-on around the same time as CT Admin.

This second add-on, now named Backup Pro, was at the time a specialized backup add-on I’d written for my hosted clients which due to certain configuration conflicts couldn’t use any of the existing ExpressionEngine backup add-ons on the market. Releasing Backup Pro was also a gamble though of a different sort; as mentioned there were existing add-ons within that niche and and one of those competitors has fiercely loyalty customers (Hi Tom!). But still, two add-ons are better than one. Right?

So on June 10th I had a stiff drink and posted my add-ons to Devot:ee. And a whole 24 hours later I sold my first add-on. Immediately, I realized I was in. I was hooked. The joy I felt when I got the email notification from Devot:ee about the sale was absolute and total. Someone had paid me money for proprietary code I had written that I was going to be able to sell again. Fuck. That was the sweetest $24 I ever made.

But $24 also came with a weird sense of responsibility I never really felt writing OSS code. With OSS I’d always kept things at my pace and, unless there was anything critical that affected me personally, individual issues other users experienced weren’t that big of a priority. But, with the ExpressionEngine add-ons, that someone had paid real cash money for, the paradigm shifted from random and faceless users to clients. In essence my customers became my clients and that’s a dynamic I’ve always been comfortable with in terms of knowing how to respond. Put simple; complete and total support until they’re happy.

And that’s really the key to everything right there. What initially started out as a small experiment I personally likened to an OSS project morphed into me sharing some responsibility with other developer’s client work. Shit got real all of a sudden. People were paying money on my code with the belief that the code would help them solve their client’s problems. For those users who had issues (and there were more than a few) my responsibility could not be denied.

Initially, I was naive enough to think I could just post my add-ons to Devot:ee and go about my life. I had no concept of how much care would have to go into support and customer satisfaction or the amount of hours I’d spend in the Devot:ee forums and answering questions on Twitter. To give an idea of just how many updates there have been; at the time of this writing Backup Pro is at version 1.8.1. That’s a lot of bug fixes and feature requests in just under 4 months. Almost to the point of ridiculousness.

But it’s addictive. Extremely so. The more feedback I got, and the more features and bugs that were fixed or added, the better the products became. The process creates a really nice positive feedback loop wherein we’re all winners. The customer gets a better product, I get to create a better product which, in turn, makes my products more appealing to others. It’s really something to be a part of.

So after 4 months of progressively improving my products, networking within the community, building a dedicated site for marketing and having an absurd advertising appetite and a library of 7 add-ons now, I look around and realize that I’m a part of something I had never anticipated; the ExpressionEngine community. And it feels like home.

I had a small task come to me recently wherein a site needed to allow for verification of ExpressionEngine credentials but couldn’t use the normal controllers for access. The challenge was in how ExpressionEngine encrypts the passwords and replicating that behavior. Taking a look at the Login controllers made things very clear though; as usual ExpressionEngine was very well written.

Here’s an example of how to do it (note that this will only work within the CP):

<?php
$user = 'test';
$pass = 'test';
$this->EE->db->select('members.password, members.unique_id, members.member_id, members.group_id, member_groups.can_access_cp');
$this->EE->db->where('username', $user);
$this->EE->db->where('member_groups.site_id', $this->EE->config->item('site_id'));
$this->EE->db->where('members.group_id = '.$this->EE->db->dbprefix('member_groups.group_id'));
$query = $this->EE->db->get(array('members', 'member_groups'));	
if ($query->num_rows() != 0)
{
	$password = do_hash($pass);
	if ($query->row('password') == $password)
	{
		//good user credentials :)
	}
	else
	{
		//bad password/good username
	}			
}
else
{
	//bad username
}
 
?>

According to the site admin who passed this my way the above won’t work outside the CP. He was kind enough to send along an example that worked fine for their situation:

<?php
$this->EE->load->library('auth');
$this->EE->lang->loadfile('login');
$authorized = $this->EE->auth->authenticate_username($this->EE->input->post('username'), $this->EE->input->post('password'));
if ( ! $authorized)
{
	set_status_header(500);
	exit(lang('unauthorized_request'));
}
?>

Fuck. We spent all this time convincing one of the oldest clients with the agency into upgrading their old and kludgy, PHP 3 era, nightmare website into a snazzy ExpressionEngine 2.2 gem of modern technology. But, like the assholes we are, we didn’t quite think through the fact that they have a complicated data structure (having been built when PHP really sucked) that we’d have to import into ExpressionEngine. So, you know, fuck…

But then, after the pants come on and a moment to reflect is had and the obviousness of the overreaction becomes clear. This is ExpressionEngine after all; there’s bound to be some utility available to handle this with the least amount of pain possible. And there is: the Member Import Utility and, even better, it’s included into the core of ExpressionEngine so it’s already there.

And in traditional ExpressionEngine fashion its built in much the same and logical way most would design an import system. There are two different modes to work with: one to import an ExpressionEngine Member XML file and another to create said file from a CSV formatted file.

The CSV format conversion wasn’t really all that for the purposes of this project (mostly due to conditional rewriting of specific column entries) but it does have it’s uses. For instance, with smaller, self contained, exports it should be a snap to easily hand over the export straight from phpMyAdmin and convert it into an ExpressionEngine Member XML file. One big drawback though is that the CSV import doesn’t appear to create new custom member fields on import so manual mapping is required per import.

The flip side of the CSV format convertor is the straight ExpressionEngine Member XML import utility. The big drag with this, and only if you don’t use the CSV convertor, is that a script has to be written to handle the conversion. Not too big a deal but, considering most installations will vary, this is going to require manual intervention every time a new system is imported. But, unlike the CSV convertor, the Member XML importer does actually create missing custom fields so updating the system shouldn’t be too big  a drag.

For my purposes I wrote a dirty little script to create the XML file for me so it was simply a matter of importing. I’m lazy, in the long term, so keeping as much of this as simple as possible is always a priority for me. But, since it’s a throw away thing cleanliness and best practices is less important than getting me more time to do real work. A word of caution though; the XML parser within ExpressionEngine isn’t very helpful when it comes to error messages and telling you what went wrong. You will curse.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

<?php
$sql = "SELECT * FROM users";
$result = mysql_query($sql);
if(!$result)
{
	echo mysql_error();
	exit;
}
$ee = "<members>\n";
while($row = mysql_fetch_assoc($result))
{
	$ee .= "\t<member>\n";
	$ee .= "\t\t<username><![CDATA[".$row['first_name']." ".$row['last_name']."]]></username>\n";
	$ee .= "\t\t<screen_name><![CDATA[".$row['first_name']." ".$row['last_name']."]]></screen_name>\n";
	$ee .= "\t\t<legacy_subscriber_id><![CDATA[".$row['subscriber_id']."]]></legacy_subscriber_id>\n";
	foreach($row AS $key => $value)
	{
		if($value == '')
		{
			continue;
		}
		$ee .= "\t\t<".$key."><![CDATA[".$value."]]></".$key.">\n";
	}
	$ee .= "\t</member>\n";
}
 
$ee .= '</members>';
 
echo $ee;
exit;
?>

The above code is fine if your existing database is cool but, and this is weird, ExpressionEngine goes full retard when it encounters anything it doesn’t like in the XML and won’t let you proceed. It’s really quite stunning. The XML importer knows that duplicate email addresses are bad in the database but instead of, say, allowing you to skip or force things the importer will just die and scream about a duplicate leaving you the only option of then manually pruning your data. Why they couldn’t be bothered with a checkbox or something allowing you to skip bad entries is beyond me but it makes the experience really, really, painful if your data sucks.

Still though, if your data is good to begin with the importer is a nice touch.

An unfortunate issue with ExpressionEngine is how internal messaging is handled. It’s really unfortunate that such a nice platform like ExpressionEngine shows such a wart when it comes to managing system pages and templates. Anyone who’s ever had to modify the message pages can tell you; it’s a dirty and counter intuitive deed requiring a lack of best practices that will leave you feeling pretty gross.

Customizing ExpressionEngine 2.x Message Pages

The current method of modifying the message pages works as you would expect; this being an ExpressionEngine site you just go to the Design section and you’ll find them under “Message Pages” . There are 3 different types of message pages you can edit (Email Notifications, User Messages and Offline Template) and while it would seem logical on how what to do to get going you’d probably be wrong (if you hadn’t ever tried before). There are a couple problems here…

First, you can’t use EE template tags within those templates. It’s the damnedest thing too because some of those templates (User Messages and Email Notifications) do allow specific tags; they just don’t parse any others. So, if you’re a good little programmer and you build your site using global embeds for building your templates (head & footer for example) you’re hosed. Your only option is to hard code your HTML which, if you’ve built even a simple site, you’ll instantly be filled with remorse and guilt over creating such a maintenance nightmare.

Second, and after the above it’s a minor issue though if you’re good enough for the above to be an issue this’ll probably piss you off too; you can’t save those templates as files. This means version control is going to be a bit of a problem. After nearly 10 years of having version control beat into me this hurt pretty bad. To be honest, because of tight deadline and even tighter budgets I could have lived with the guilt and shame of hard coding just to finish the project but taking away my version control was like adding insult to injury.

It’s quite the head scratcher when you get right down to it; a smart and clever company redesigns their entire product from the ground up and completely drops the ball when it comes to the messaging system. It’s a shame…

There is another option though it does require money; a really nice extension called Custom System Messages. The extension was written by Brian Litzinger and is available for $12 through the ExpressionEngine marketplace Devot:ee.

Custom System Messages

Custom System Messages works exactly as you would expect ExpressionEngine to work; through a nice interface you choose custom templates for specific messages. You can set a different template for both message scenarios (General Error and Form Submission Error) as well as change the behavior of the {link} template tag so it doesn’t use the lame ass JavaScript “back link”.

The best part though is that Custom System Messages allows you to set special templates based off of all the actions your site has (even those from other modules and extensions). So, for example, if you have a CartThrob site you can set a special template for when adding an item to your cart fails and separate template for when updating an item within a cart fails. It’s really quite slick and for only $12 it’s very reasonable. Priced just enough to dissuade any developer from rolling their own solution.

I do have to say how disappointing it is to see ExpressionEngine lacking this built in. To me, it’s reminiscent of buying a car and being charged extra for the gas cap. Sure, I can just go buy a gas cap for a couple bucks but it really should be a part of the car to begin with.

About a year ago I reviewed various shopping cart platforms for what was to be an “upcoming” project. After numerous delays and client hand holding (with the requisite back and forth) one year later that project finally began. And, as has lately become a trend in my professional life, the client’s spec combined with a drastically shortened and accelerated deadline forced us to go with something completely unknown and with only cursory inspection; the 3rd party e-commerce ExpressionEngine module CartThrob 2.0 (beta at the time of this review BTW).

CartThrob

Of course this presented the problem of actually using an unknown package to build a site with. Thankfully though, while CartThrob has a steep learning curve and is most certainly missing key functionality, it’s an overall nice package that allowed us to do what we needed. Sure, there’s pain involved (lots of configuration, testing and set up) but with a little elbow grease it all worked out.

As mentioned, CartThrob is an ExpressionEngine module for managing an online store. It’s available for both ExpressionEngine branches (1.x and 2.x) and was voted Module of the Year by Devot:ee in 2010 (for the 1.x version only). This being an ExpressionEngine project CartThrob would, in theory, be a nice fit.

One of the really nice things about using CartThrob is that it uses the ExpressionEngine admin panel for managing your store. This meant there wouldn’t be a second administration site that the client would have to learn. (Personally, I always found dual admins to be janky at best and sloppy/lazy at worst so this was a definite bonus in favor CartThrob.)

But this is also double edged because, well, pretty much everything about your store is saved into the entries system within ExpressionEngine. This, of course, causes your Entries panel to become an eye sore whenever you have to do anything there. Instead of having a dedicated section to manage products you use the Entries section. Instead of a dedicated section to manage your orders you use the Entries section. Instead of a dedicated section to manage… well, you get the idea; everything is managed through the entries. This makes sense for the product management but when you have to dig through entries to find orders and coupons and discounts (etc) things get a little jarring for some people (even internally with my team) so it’s a concern.

Still, single freaking admin so it was kind of a lesser evil compromise.

The unusual thing though is that, for how thoroughly the management of items was punted (though, to be fair, it looks to be a feature versus bug debate more than “punting”), CartThrob has an extremely deep configuration section where you can customize CartThrob to function just right. CartThrob has all the basic customization functionality one would expect from any modern shopping cart platform (tax, shipping, payments with lots of merchant options, etc) plus a bunch of settings just to get CartThrob working (CartThrob is NOT an out of the box solution). It’s all presented in an easy to use interface that’s designed to mesh with the ExpressionEngine administration panel design which makes for an easier experience.

As far as merchant accounts go (for accepting payments through your site) there’s the usual suspects of Authorize.net, Paypal and SagePay built right in but also some offline solutions for pay by check and credit accounts. For this project we used Authorize.net which went smooth for the most part but I can’t speak for the other options though if they’re anything like the Authorize.net one they’re probably cool. CartThrob also includes some extensions and hooks so writing your own payment gateway is possible if you need.

CartThrob can also hook into the ExpressionEngine member module so your customers can be registered internally. Like pretty much everything else with CartThrob there’s a bit of configuration involved to get it up and working though. Unfortunately though, there’s not really any out of the box functionality for your members to take advantage of. For example, there isn’t any way for your customers to view their past orders when logged in and when using the control panel and viewing a member’s profile there’s no relationship with CartThrob. You’re gonna have to write all that stuff yourself but, and I don’t know why, but CartThrob doesn’t store any relationships between ExpressionEngine members and it’s customers so expect a challenge.

In terms of the user side, the actual storefront, everything is done using ExpressionEngine templates and template tags. There are lots of module methods available and even though it’s far from a complete package of the expected functionality. In fact, on our team, we got into a habit of thinking of CartThrob as a Lego set for creating what we want instead of having what we want already.

For example, to view the details about a users cart you’d use the template code below:

{exp:cartthrob:cart_info}
	{total_items} <br />
	{total_unique_items} <br />
	{cart_entry_ids} <br />
	{cart_shipping} <br />
	{cart_tax_rate} <br />
	{cart_tax} <br />
	{cart_subtotal} <br />
	{cart_total} <br />
{/exp:cartthrob:cart_info}

On the surface this appears to be nice because it keeps things within the core ExpressionEngine development process. And it would be except, well, CartThrob isn’t abstract enough. Too many times I would try to do something that seemed obvious, like checking for shippable items within a users cart, and would come up short; that functionality doesn’t exist.

Still though, don’t get me wrong. Yes, some of the above sucks. Absolutely. But CartThrob is still under development. Plus, since it’s a commercial script ($99) there’s financial incentive to improve it. And, truth be told, for as painful as certain points were it’s still way more fun to build a site with CartThrob than most of the other options I’ve used or reviewed like OpenCart or Avactis.

Plus, again, single freaking admin!

Recently, I came across a weird issue while developing an ExpressionEngine 2.0 module. The issue was that, for some reason, my module would output a random string instead of the intended content. For example, instead of displaying the HTML, that the module created, the module would display the string M00o93H7pQ09L8X1t49cHY01Z5j4TT91fGfr instead. Tracking down this issue was symptomatic of the proverbial needle in a haystack but I learned a lot about how ExpressionEngine’s internals work and was once again reminded to be pragmatic about my development strategies.

random, the lion

My first and automatic reaction when coming up against some random and meaningless output string is to just Google it and see what happens. Unfortunately though, this particular search, at the time at least, returned around 35,000 sites that were outputting the string and only 1 that had a reference to the issue with a possible explanation. All of a a sudden this became an actual thing.

Putting aside the fact that there are around 35,000 sites with the string within Google for the moment (and that’s really a problem to be discussed) it took a bunch of digging but I finally found a few things:

First, this is an issue that has been within ExpressionEngine for years. Doing a search on the ExpressionEngine site lists the first entry of this string at 2007 (sorry for the lack of a link; ExpressionEngine search is cached so old search URLs don’t work sometimes). My initial reaction was that this was kind of sloppy; EllisLab updated ExpressionEngine to 2.0 recently, from the ground up to use CodeIgniter I believe, yet this issue got ported over?

After doing further research though, I came to believe the above is more to do with the design of ExpressionEngine than anything else like copying code. Turns out, there’s some logic to that random and oblique string.

Basically, that string (M00o93H7pQ09L8X1t49cHY01Z5j4TT91fGfr) is a template holder for the content you want to output. It’s a little complicated in how it works but thanks to Derek Jones over at the ExpressionEngine forums there’s a succinct, though still a little complicated but less complicated than I could explain it, explanation of what’s going on.

…

The template parser is an extremely complex gizmo, so let me try to explain it stripped of what techno mumbo jumbo I can. This will still be long, so bear with me.

The template parser intelligently saves on resources by only processing a given tag once, no matter how many times it occurs on the template. This means that if your tag is exactly the same, including its tagdata, EE only execute’s that modules code once, and replaces all copies of that tag with the output.

So first, when the template parser goes through the template finding tags, it replaces them all with temporary markers (the gibberish), and adds each tag to a list of tags to be parsed, in the order they are encountered on the template, top down. Each marker starts with the letter “M” followed by the number (starting at 0) that tag is sitting in on The List.

Second, the parser goes down its list of tags. It parses tag #0, and then replaces all of #0’s marker with that tag’s output.

——————-

Ok that’s the setup. Here’s why it affected you, and why this side effect is not a problem (and also extremely, extremely rare). Remember, top down, tags will be assigned 0, 1, 2, etc. So I’m abstracting it out to show the nesting and the positioning only. Just focus on the numbers.

{exp:tag0}foo{/exp:tag0}
{exp:tag1}
    bar
    {exp:tag0}foo{/exp:tag0}
{/exp:tag1}

Step 1, the tags are grabbed and replaced with markers. The first tag encountered is replaced first.

M0
{exp:tag1}
    bar
    M0
{/exp:tag1}

And then the next tag:

M0
M1

All tags have been collected and put on The List, so step 2, the parser goes down The List, and parses tag #0:

foo
M1

See the problem? The second M0 is sitting protected inside the marker for tag #1, so it doesn’t get replaced. Then tag #1 gets parsed.

foo
    bar
    M0

And the marker is left in the output. EE doesn’t go back and parse tag #0 again, it’s already done that once, and will not do it again.

So the circumstances required for this problem to exist each are uncommon, and combined, very very rare:

1) Multiple exact copies of a given tag
2) One or more copies nested inside another tag (tag nesting itself is also rare, and usually not recommended)
3) The tag of which there are copies must sit at a position in the template above any occurrences of it being nested in another tag.

From the above it was easy to conclude that the template parsing stuff was breaking down in my module. Why, I had no idea, but at least I had a known starting point. While it was easy to conclude that my issue lay with the template parsing though, it was 100% off point for my particular issue which had to do with, of all things, the case of the module name. But, I digress.

The point is that for years now, and through multiple versions and rewrites, ExpressionEngine has consistently carried the same “issue”. Understanding the intent of the design for the template parsing helps put some perspective on things: it’s definitely a side effect of the particular approach. It bugged, and surprised, me that ExpressionEngine would have such oblique and useless output on such a widespread basis (35,000 site’s with that string within Google is a bit much in my opinion) but the more I thought about it the more I came to understand the complexity of what was involved in being helpful when the issue cropped up. Aside from abandoning the approach what could really be done?

It’s a nice reminder that development is often a balancing act between getting what you want and dealing with a counter issue. The ExpressionEngine team developed a strategy to enable them to effectively and efficiently meet their requirements but there’s sometimes an issue; a random string gets output instead of the template data. Plus, since the cause could be for anything from bad module installation data (as I had) to an issue within a template there’s no real way to provide helpful error messages when the template parsing does fail. But, on the other hand, the template parser does exactly what it was designed to do (and it does it well).

Which would you choose?

I’ve been hard at work on a dynamic navigation system for an ExpressionEngine project this last week. The project has been a lot of fun in all the right ways; complicated, technical and way outside my comfort zone. Basically, the project was to build a suckerfish style navigation bar using jQuery, superfish (a jQuery implementation of suckerfist) and ExpressionEngine. And therein lies the issue…

I had anticipated making the nav all efficient-like with a single query to pull out all the items and build the nav using unordered lists. I started doubting this as a good idea though when I came to learn how complicated the data model was for this particular project. Now, don’t get me wrong, ExpressionEngine by itself is a blank slate when it comes to the data model for a project and, technically, they have a good and normalized structure for everything. Which was the problem.

An ExpressionEngine site is traditionally (at least as far as the sites I’ve seen and learned from) broken up like the below:

Weblogs
    Categories
        Entries
            Sub-Entries...

So, weblogs have categories which have entries attached to them which, in turn, can have sub-entries. Pretty straightforward I think but when you consider that this all represents around 6 tables that would require some pretty complex joins it’s one of those cases where the “right” way isn’t the ideal way. On the other hand, breaking the SQL up into multiple queries done in a loop is usually a pretty bad idea especially when you don’t know how many times the loop is going to occur.

So, I was fucked with 2 crappy choices and needed to pick the least evil; I settled on looping queries with the caveat of caching. This was ideal because the cache could be set to a pretty large interval so the nasty SQL wouldn’t execute but once a week or so. Having never worked with the native caching mechanism within ExpressionEngine learning about it became the new problem.

To be honest, I’m a little embarrassed that I didn’t delve into learning about this sooner; it turns out ExpressionEngine has a few different caching options available built right in. I’m pretty new to ExpressionEngine, having only been developing with it for a few months, but it was pretty lame that I didn’t immediately look into this option. There are so many problems the built in caching would have solved :/

Query Caching in ExpressionEngine stores the output of certain database queries to a flat file (in fact, all the caching mechanisms store the cache as a flat file). It has to be manually turned on in the administration panel but once it is turned on you’re good; all the queries that can be cached will be cached in (what else?) the cache directory. It should be noted that EllisLab doesn’t recommend using Query Caching because it’s much more effective to just turn on the native caching in MySQL itself.

Tag Caching is useful for storing the output of ExpressionEngine tags on a tag by tag basis. Tag Caching allows for very granular caching so you can have certain portions of a page cached while leaving others completely dynamic. It should noted that Tag Caching only works on ExpressionEngine tags and, unless it’s coded for caching, won’t have any effect on plugins or extensions.

To enable Tag Caching all you have to do is add “caching=’true’ refresh=’60′” to any ExpressionEngine tag like the below:

{exp:channel:entries cache="yes" refresh="30"}

The first parameter just tells ExpressionEngine to turn caching on for the tag; the second is the amount of time, in minutes, to keep the cache active before it is refreshed. Pretty simple really though depending on how many tags you’re trying to cache it could be a bitch to implement I think.

Template Caching is what I think of as the most effective caching mechanism ExpressionEngine has to offer. Essentially, ExpressionEngine saves the entire output of a template to a flat file. The really cool thing is that ExpressionEngine caches not only most of the output for most EE Tags used (like the above) but also any php you may have added as well.

It’s a little misleading to say that it’s perfect though; there are a quirk to how it works. The entire, processed, template doesn’t get cached just the heavy database or PHP stuff does; any EE Tag for conditional logic and URLs are still processed after the fact. Not a thing that’s initially worrisome though that may depend on the circumstances of the project.

I recently had a project come up that could have gone one of two ways from a  strategic design stand point; either as a straight up script custom built for the occasion or, time warranting, I could build an extension for ExpressionEngine. As I’d already looked into writing an ExpressionEngine plugin the idea of writing an extension was pretty enticing regardless of the added time costs I would have to swallow (can’t bill the client for my fun now can I?).

Developing ExpressionEngine 1.6x Extensions

The only problem is that the project in question was built on ExpressionEngine 1.6.7 and ExpressionEngine, having recently released 2.0 with all new bells and whistles and ways of doing things, updated their documentation as well. I couldn’t find any documentation for writing extensions for 1.6x though there’s a really nice help center for 2.0. A little advice third party developers; don’t just remove documentation and replace it with the new stuff because, as shocking as this may sound, people still use the old stuff.

To ExpressionEngine an extension is a script built to modify how ExpressionEngine works as opposed to adding additional functionality like a plugin would do. For example, a plugin would be good for adding a drop in template tag while an extension would be good for changing how ExpressionEngine parses all template tags. If you’re used to the WordPress development paradigm think of extensions as hook filters in WordPress plugins.

Those aren’t the only distinction though; extensions can also have a settings section, language files and database tables too so you’re really going  to need at least an extension for any deep functionality or customizations (there are also ExpressionEngine modules and expansions but that’s for another day).

Codewise, ExpressionEngine extensions are pretty similar to plugins insofar as structure and elements. At the core an extension is a PHP class that follows the same naming structure for both the class and the file as a plugin except the prefix changes and where you store the file. ExpressionEngine extensions should be named with a prefix of “ext.” and the class name, include a couple class variables for information about the extension, a couple methods to activate, deactivate and remove the extension and, ideally, a method to be called when the “hook” is executed.

As an example let’s put together an extension stub for a demonstration of how things should be setup

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125

<?php
class Pissoff_client
{
	/**
	 * Name of the extension
	 * @var string
	 */
	public $name = 'Enrage The Client';
 
	/**
	 * Version of the extension
	 * @var string
	 */
	public $version = '1.0';
 
	/**
	 * The description of the extension
	 * @var string
	 */
	public $description = 'Replaces every instance of one word with another from all entries';
 
	/**
	 * Flag for whether this extension has settings 
	 * @var string
	 */
	public $settings_exist = 'n';
 
	/**
	 * URL to the documentation for the extension
	 * @var stirng
	 */
	public $docs_url = 'http://url_to_documentation.com';
 
	/**
	 * The available settings for the extension
	 * @var array
	 */
	public $settings = array();
 
	/**
	 * Constructor
	 *
	 * @param 	mixed	Settings array or empty string if none exist.
	 */
	function Pissoff_client($settings = '')
	{
		$this->settings = $settings;
	}
 
	/**
	 * Activate Extension
	 *
	 * This function enters the extension into the exp_extensions table
	 *
	 * @see http://codeigniter.com/user_guide/database/index.html for
	 * more information on the db class.
	 *
	 * @return void
	 */
	function activate_extension()
	{
		global $DB;
 
		// default settings
		$settings =	array();
		$hook = array(
				'extension_id'	=> '',
				'class'			=> __CLASS__,
				'method'		=> 'run',
				'hook'			=> 'hook_to_process_extension_on',
				'settings'		=> serialize($settings),
				'priority'		=> 1,
				'version'		=> $this->version,
				'enabled'		=> 'y'
		);
 
		$DB->query($DB->insert_string('exp_extensions',	$hook));
	}
 
	/**
	 * Update Extension
	 *
	 * This function performs any necessary db updates when the extension
	 * page is visited
	 *
	 * @return 	mixed	void on update / false if none
	 */
	function update_extension($current = '')
	{
		global $DB, $EXT;
 
		if ($current < $this->version)
		{
			$query = $DB->query("SELECT settings FROM exp_extensions WHERE class = '".$DB->escape_str(__CLASS__)."'");
 
			$this->settings = unserialize($query->row['settings']);
			$DB->query($DB->update_string('exp_extensions', array('settings' => serialize($this->settings), 'version' => $this->version), array('class' => __CLASS__)));
		}
 
		return TRUE;
	}
 
	/**
	 * Disable Extension
	 *
	 * This method removes information from the exp_extensions table
	 *
	 * @return void
	 */
	function disable_extension()
	{
		global $DB;
		$DB->query("DELETE FROM exp_extensions WHERE class = '".__CLASS__."'");
	}
 
	/**
	 * Method to execute when hook is called
	 * @param mixed $data
	 */
	public function run($data)
	{
 
		//process
	}
}

Most of the above should be self explanatory but just in case what’s going on is that most of the class variables are used for extension details and there are 3 methods for activating, deactivating and removing the extension. The methods for activation management are important, especially the activation method “activate_extension” which contains the details about the extension including the hook to execute on along with the method name to execute when the hook is called. In the above example, the extension will execute the method “run” when the hook “hook_to_process_extension_on” is called. Thankfully, ExpressionEngine hasn’t done away with the list of hooks available in the 1.x branch.

Obviously, the above extension is pretty useless, what with not doing anything, but that’s all you’re gonna need to have the extension listed in the Extension Manager. To further personalize and customize (huh… I’m a poet) the details of your extension you can add settings and, if you want people to have the ability to change those settings, you can add a settings page by.

To add a settings page to your extension things get a little complicated at first but it’s not so bad. All you have to do is set the class variable $settings_exist to “y”, create a new method called “settings()” that EE uses and modify your extension language file (should be created inside the “lang” directory using the convention of “lang.extension_name”). The settings() function should contain an associative array with the key being the name of the setting and any default value for that setting like the below:

<?php
function settings()
{
	$settings = array();
	$settings['pissoff_find']		= '';
	$settings['pissoff_replace']	= 'this is the default value';
	return $settings;
}

Using the above will simply display a form with 2 text boxes for input. If no value is provided for the second item then there’ll be a default used. Simple. But, if you want to use, say, radio buttons or select boxes you have to make a couple modifications like the below:

<?php
function settings()
{
	$settings = array();
 
	$settings['pissoff_find'] = array('t', 'Default!!');
	$settings['pissoff_replace'] = array('r', array('nothing' => "Nothing", 'something' => "Something"), 'something');
	return $settings;
}

The above will create a new setting form with a textarea for the first option and a radio fieldgroup for the second. Make sure you add the radio option text to your language file though; it won’t do that automatically. Also, you’re not locked into just radio and textareas; you can also have select boxes (s) and multi-select boxes (ms) by updating the relevant array key.

The more I work with ExpressionEngine the more I keep running into the same issues. ExpressionEngine hides most error messages, especially those related to configuration, probably for security, but this doesn’t make debugging any easier. To be fair, I don’t know if this is how ExpressionEngine works out of the box or if this is a configuration setup done by the projects original developers but it does make fixing the issue that much harder.

ExpressionEngine White Screen

Admin White Screen

I’ve only come across the administration area throwing a white screen when using ExpressionEngine 1.67 on a PHP 5.3 server and only if extensions are enabled. I’m not sure if the newer versions of the 1.x branch have this fixed so this might not work for you.

The issue has to do with how the variables are being passed and called; PHP 5.3 changed how references were handled so the method ExpressionEngine 1.67 uses no longer works. To fix you have to modify “/system/core/core.extensions.php” with the below changes that are on the ExpressionEngine forums:

<?php
//system/core/core.extensions.php around line 115 modify:
if (sizeof($args) == 1)
{
    $args = array($which, '');
}
 
if (version_compare(PHP_VERSION, '5.3') >= 0)
{
    foreach ($args as $k => $v)
    {
        $args[$k] =& $args[$k];
    }            
}  
?>
 
<?php
//and likewise around line 174 modify:
{
    $php4_object = FALSE;
    $args = array_slice(func_get_args(), 1);
}
 
if (version_compare(PHP_VERSION, '5.3') >= 0)
{
    foreach ($args as $k => $v)
    {
        $args[$k] =& $args[$k];
    }            
}  
?>

Front End White Screen

Then there’s the front end white screen; so far I’ve encountered this type of white screen in both the 1.6 and 2.0 branches of ExpressionEngine. Luckily, whenever I’ve ran into a white screen on the front site it’s always due to various path configurations which is easily fixed by over riding the configuration file.

Expression Engine is one of those programs that stores as much as possible in the database including file and path directory paths. To get around this permanently I’ve gotten in the habit of using a default config.php file for all any Expression Engine site I work on; it’s the first thing I do before anything else.

This new configuration file uses the $_SERVER super global to dynamically determine the paths and makes allowances for development, staging and production environments.

<?php
if ( ! defined('EXT')){
	exit('Invalid file request');
}
 
$conf['app_version'] = "167";
$conf['license_number'] = "";
$conf['debug'] = "0";
$conf['install_lock'] = "1";
$conf['db_hostname'] = "";
$conf['db_username'] = "";
$conf['db_password'] = "";
$conf['db_name'] = "";
 
if('dev.site.com' == $_SERVER['HTTP_HOST'])
{
	$conf['db_hostname'] = "";
	$conf['db_username'] = "";
	$conf['db_password'] = "";
	$conf['db_name'] = "";
}
elseif('stage.site.com' == $_SERVER['HTTP_HOST'])
{
	$conf['db_hostname'] = "";
	$conf['db_username'] = "";
	$conf['db_password'] = "";
	$conf['db_name'] = "";	
}
 
$conf['avatar_url'] = "http://".$_SERVER['HTTP_HOST']."/images/avatars/";
$conf['avatar_path'] = $_SERVER['DOCUMENT_ROOT']."/images/avatars/";
$conf['photo_url'] = "http://".$_SERVER['HTTP_HOST']."/images/member_photos/";
$conf['photo_path'] = $_SERVER['DOCUMENT_ROOT']."/images/member_photos/";
$conf['sig_img_url'] = "http://".$_SERVER['HTTP_HOST']."/images/signature_attachments/";
$conf['sig_img_path'] = $_SERVER['DOCUMENT_ROOT']."/images/signature_attachments/";
$conf['prv_msg_upload_path'] = $_SERVER['DOCUMENT_ROOT']."/images/pm_attachments/";
$conf['theme_folder_url'] = "http://".$_SERVER['HTTP_HOST']."/themes/";
$conf['site_url'] = "http://".$_SERVER['HTTP_HOST'];
$conf['captcha_url'] = "http://".$_SERVER['HTTP_HOST']."/images/captchas/";
$conf['captcha_path'] = $_SERVER['DOCUMENT_ROOT']."/images/captchas/";
$conf['emoticon_path'] = "http://".$_SERVER['HTTP_HOST']."/images/smileys/";
$conf['theme_folder_path'] = $_SERVER['DOCUMENT_ROOT']."/themes/";	
$conf['db_type'] = "mysql";
$conf['db_prefix'] = "exp";
$conf['db_conntype'] = "0";
$conf['system_folder'] = "system";
$conf['cp_url'] = "http://".$_SERVER['HTTP_HOST']."/".$conf['system_folder']."/index.php";
$conf['doc_url'] = "http://expressionengine.com/docs/";
$conf['cookie_prefix'] = "";
$conf['is_system_on'] = "y";
$conf['allow_extensions'] = "y";
$conf['multiple_sites_enabled'] = "n";
?>

The above config is for the 1.x branch though most values should work for the 2.x branch with the addition of $config['tmpl_file_basepath'].

Hopefully, this should take care of those white screens.

In Expression Engine Escaping Madness I laid out an issue I was experiencing with a client site. The issue was that there didn’t appear to be a method available to escape Expression Engine markup when it is mixed with php so there is a definite risk of parse errors using that technique (it’s not a security issue or anything; I want to be clear on that). This made me nervous enough that I couldn’t let it go and kept thinking about how to get around the issue. The answer: write an Expression Engine plugin. Of course this meant I had to actually learn how to write an Expression Engine plugin first. Here are my notes

Create Expression Engine Plugin

First though credit where it’s due: for this example I used the number_format plugin as a base so a lot of credit goes to Robert Wallis for the nicely written plugin I’m basically leaching from:)

An Expression Engine plugin is essentially a php class with at least one method that translates into an Expression Engine tag and used inside of Expression Engine templates (I don’t think they can be used inside of entries directly though I haven’t confirmed this). They are best left for small tasks though because they can’t have an administrative backend or integration with the Expression Engine l10n stuff or form processing or any manageable settings. No fun stuff for plugins…

Like most platforms there’s a few conventions that have to be followed but, also like most platforms, they aren’t too troublesome to work with. It should be noted that the syntax for the class uses the php4 syntax so dumb yourself down accordingly.

For example our plugin class will look like:

<?php
class Add_Slashes
{
	function Add_Slashes()
	{
 
        }
}

And the plugin will be executed with the below Expression Engine template tag:

{exp:add_slashes}O'Reilly{/exp:add_slashes}

Now, it’s possible to create template tags that aren’t done in pairs but for this example I’m going to stick with pairs. In case it wasn’t obvious, if you wanted to have a plugin that has more than a single method you would call that method like:

{exp:add_slashes:somethingelse}O'Reilly{/exp:add_slashes:somethingelse}

So that’s the basic syntax. The next thing that needs to be done is that a file has to be created inside the “/system/plugin/” folder. Note that I’m using the default name for that folder (system) so if you renamed it during the installation process use that instead. The file name must be lower case the same as the class name and it must have pi. as the prefix, and begin with the second segment of the tag. So, with our example plugin, the plugin name would be: pi.add_slashes.php. Simple enough. Once the file is saved to the location the plugin is installed. That’s how Expression Engine rolls.

Now we have a working plugin in theory but if you go into the plugin manager you’ll see an error about a missing variable as well as a distorted view of the page. We’re missing something; the $plugin_info array.

Every Expression Engine plugin should have a variable outside of the class called $plugin_info. For our example it should look like the below:

<?php
$plugin_info = array(
	'pi_name'        => 'Add Slashes',
	'pi_version'      => '1.0',
	'pi_author'       => 'Eric Lamb',
	'pi_author_url'  => 'http://blog.ericlamb.net/',
	'pi_description' => 'Exposes PHP\'s <a href="http://php.net/manual/en/function.addslashes.php">addslashes()</a> function via EE tags.',
	'pi_usage'        => Add_Slashes::usage()
);
?>

Obviously, the above lists all the details about the plugin for display in the plugin administration module:

Expression Engine Plugin Manager

You may have noticed the pi_usage key in the array; this method is recommended by the Expression Engine Developer Center for describing the usage of a plugin. Inside the plugin create a method called usage() and just return the instructions. Those instructions will be used on the plugin description page:

Expression Engine Plugin Information

Using all of the above the completed plugin is below:

<?php
$plugin_info = array(
	'pi_name'        => 'Add Slashes',
	'pi_version'      => '1.0',
	'pi_author'       => 'Eric Lamb',
	'pi_author_url'  => 'http://blog.ericlamb.net/',
	'pi_description' => 'Exposes PHP\'s <a href="http://php.net/manual/en/function.addslashes.php">addslashes()</a> function via EE tags.',
	'pi_usage'        => Add_Slashes::usage()
);
 
class Add_Slashes
{
	function Add_Slashes()
	{
		global $TMPL;
		$this->return_data = addslashes($TMPL->tagdata);
        } 
 
	function usage()
	{
		return "This is really just a wrapper for PHP's add_slashes function:
http://php.net/manual/en/function.addslashes.php
 
{exp:add_slashes}
O'Reilly
{/exp:add_slashes}
returns: O\'Reilly
 
";
	} 
 
}

Now, this example was used using Expression Engine 1.6.9 so the process might not work for the upcoming 2.0 (I haven’t looked into that version just yet). It also doesn’t look like EllisLabs is accepting any new submissions to their plugin library so if you’re hoping to distribute your plugin prepare to do it solo and without any help from them.

Still, an easy enough process that any custom functionality you may need for an Expression Engine site should be trivial to achieve.

UPDATE February 21, 2011
I completely forgot to include any info on customization through passing parameters. Note: this is totally cribbed from the ExpressionEngine forums.

{exp:add_slashes return="FALSE"}
O'Reilly
{/exp:add_slashes}

Doing the above will create a variable within the plugin called “return” which can be accessed like:

<?php
global $TMPL;
$return = $TMPL->fetch_param('return');  
?>