Made of Everything You're Not

Being the Director of Technology for a marketing agency is full of surprises. The best surprise was learning that everyone is an expert. Yup; regardless of experience, or knowledge, you're going to run into all sorts of people who know everything about subjects that are incredibly complex and deep.

This was made apparent the other day when I received a link to an article called Hot Ecommerce Trend: Embedded Video in Email.

The article starts out with the general, and true, premise that linking to videos is a good way to improve click through rates:

Anna Yeaman reports one retailer boasting a 20-27% click through rate without linking to video, and 51-65% with links to video. And Forrester Research reports video in email can increase click through by 2-3X.

Of course there's no mention of what the campaigns were about, or who the sample targets were in relation to previous email campaigns. But what the hell, it's the Internet, so we grain of salt the numbers and don't give them any real weight. Interestingly, though the article never comes right out and says it, from the tone of the article it looks like the idea is that if linking to a video is good embedding the video directly is better.

The article goes into some detail about the obvious challenges of actually embedding a video in an email, instead of just linking to the video, (like spam flagging and file size mostly) but just doesn't offer a satisfactory response on how to accomplish it. Basically, it says embedding a video is a good thing to do but doesn't provide any solutions to do it.

This is probably because, oh, I don't know, IT'S NOT A GOOD THING TO DO (hmm, bold, italics, all caps? serious text). Just don't do it if you want your email to, you know, be seen by people.

According to the article:

What’s so tough about embedding actual video in email?

Deliverability is the issue. Large video attachments are often a red flag for spam filters, and ISPs (Internet service providers) block “complex data” including Javascript for security reasons.

ISPs have banned Javascript Sending video in an e-mail has been a challenge for deliverability, since large video attachments often alert spam filters. The way that Goodmail gets around this issue is that their e-mail class, called CertifiedEmail, is a paid service that does not go through typical e-mail filters.

Forgiving for a second the nonsensical flow of the second paragraph, it seems there's a service called Goodmail that, somehow, allows email to bypass the spam filters. Unfortunately, Goodmail is currently only in use by AOL at the moment so it's not realistic unless you have a list of only AOL users. Does anyone, besides AOL, send to just AOL? Exactly.

Next the article talks about sending to just gmail; provided the recipient signed up for YouTube video embedding in their gmail account settings they shouldn't have a problem. This, too, suffers from the Goodmail problem of only working with one email provider. Not much of a solution really.

But, all of the above are minor issues that don't really deal with the larger problem; email wasn't made for embedded video. Sure, it can do it, in a kinda-sorta, if you squint and tilt your head kind of way, but it's similar to how HTML isn't a design language but has been re-purposed for that use.

Email was built for text; attachments weren't even a possibility until 1996 with the introduction of RFC 2045. (The initial specification only allowed for text communication using 7bit US-ASCII as the encoding and had a limit on characters around 1,000 total.) Email with embedded video wasn't even a thought at the time.

How much was video not considered an option? Just take a look at the complete lack of support for certain HTML tags required to embed a video in the popular email clients.

To be honest, there weren’t a lot of surprises here. The OBJECT and EMBED tags remain as poorly supported now as they were 3 years ago. This instantly wipes out Flash, Quicktime, and Windows Media formats. As predicted, Java support was also a no show.

So, what are your options then? That really depends on whether quality matters.

If quality actually means something to you then you definitely should avoid the much touted animated gif "trick". Years ago this might have been a good idea but now, in the 21st century, it just looks dated. Once, videos and animated gifs were almost comparable but now the quality of online video just shines way too bright compared to an animated gif. It may be easy to do but definitely looks like amateur hour.

On the other hand, you could always link to the video. Why, for the love of god, this has to be stated plainly escapes me but c'mon people; just use an image and link to the video on your site. This has the added benefit of creating visitor to your property at the same time, and you never know, maybe they'll think about sticking around.

To be honest, I don't really see video in email EVER being a viable option. There are just too many problems inherit with the idea to make it possible, much less practical.

First, you have bandwidth, which is still one of the biggest barriers to doing anything cool online. Videos are big and delivering them via email is a sure way to drive people crazy. Have you ever tried to download an email with a big attachment? Same thing.

Second, as mentioned above the email clients don't even support the HTML tags required to render a video even if it's downloaded. This one is pretty big because those tags aren't allowed for a reason: malware. If an EMBED or OBJECT tag is used the bad people have a bigger sandbox to play with. The rule of not opening attachments to help protect against malware goes right out the window. Changing the email client programs is just not going to happen.  Sorry.

So, let's all get together and recognize that this isn't going to happen. Ever.

I'm back to working on a couple projects so I haven't been able to work on Wp-Click-Track for a couple months. Hell, the last update was in March so, to be honest, it's been about 3 months since the release of 0.4; though there were a couple, small, feature and bug fixes put out.

This release includes a bunch of features that a lot of people have been asking for but also a fix for some stupidity on my part.

I made a HUGE mistake in how the data was being recorded and the flagging of unique versus individual clicks. The patch can't be automated so you'll be presented with a link to click once the upgrade is complete. I highly recommend you do this ASAP after upgrading.

Features and Modifications

Added settings to disable user clicks
Added ignore IP address for click tracking
Added link statistics reset
Added link deletion
Changed graphs to Open Flash Chart
Added additional line chart vectors to display unique clicks
Added link parsing of next and prev template tags
Added link parsing of categories in posts/pages as well as sidebar widget
Added link parsing of tags links template
Improved title extraction to reduce No Name Given auto-label
Added global history and report page

Bug Fixes

Click graph date descrepancy issue
Added bypass for external links being double tracked when entered in page
Fixed backwards tracking flags
Changed admin widget ordering to list most clicked to least click
Fixed php short tag usage

Be sure to update your plugin ASAP and please let me know of any issues in the comments.

I admit, I have a bit of a maniacal attitude towards code structure and formatting. I definitely drive my team (of one smart programmer) crazy with my coding standards. Sorry for that dude...

(Quick aside: the above image is a screenshot of the winning entry for the 1998 International Obfuscated C Code Contest. It's a flight simulator that actually compiles )

To be clear, I'm not talking about good code; that's just way, WAY, too subjective of a discussion. No, I'm talking about structure, layout and, most of all, consistency.

I tend to take this approach from a maintainability standpoint. The way I look at it, I've spent too much time in my career wading through some seriously bad code; there's just no reason not to do better for those who come after me.

That's only a part of it though. I have to go through my code; no fucking way I'm going to mess with myself or my team.

On the other side though, we also have to write code on behalf of our clients. We give them code that they paid us to do so it's pretty much a  foregone conclusion  that the client is going to go through the code. If we hand them crap they're, obviously, going to think we suck. That's no good for anyone.

It seems like such an obvious thing to write good, clean and well formatted code, that it's a little surprising how much bad code is out there.

Apparently, I'm not alone. Going through Reddit the other day I came across an interesting discussion of the importance of read-optimizing your code; it was nice to see the rallying and discussion. All of this stemmed from a really good article, on Brendel Consulting going into the importance of good source code.

When you design a software system, database or data structures, you take into consideration its most common use case. For example, you organize your data differently when it is read frequently, but only rarely written to. You read optimize your data.

I am convinced that the same applies to source code. In almost all cases, source code will be read much more often than it is written. Who writes the source code? You. A function is developed over some limited time, and once it is done it changes rarely, unless being refactored or modified to accommodate some changed requirements.

And, of course, there's a great post on Coding Horror discussing the similarities between, of all things, programmers and chefs.

For software developers, working clean means constant refactoring. If you don't stop occasionally-- frequently, actually-- to revisit and clean up the code you've already written, you're bound to end up with a big, sloppy ball of code. If you forget to regularly clean up behind yourself, things get smelly. Working clean means following your nose and addressing those nagging issues before they become catastrophic.

In addition to working clean, cooks also spend a lot of time thinking about mise en place, how their cooking stations are arranged for optimal work.

It's just too important we make sure to write good code and there's just no reason not to do it. If nothing else, remember that as a programmer you're judged by your code. Are you a professional?

Fuck, I hate Twitter; yet another tool to allow people to collect, well... people really. As an example, I signed up for a Twitter account so I could test Twitter functionality for some projects I'm working on. I don't use Twitter for anything but testing yet I get notices daily about users now following me (and none of them are readers of this blog).

Still, Twitter is all the rage and I do have to admit it is fun working with all the little toys people are making with the Twitter API (playing with the API is really FUN).

One such tool is TweetMeme.

Accoring to the TweetMeme site:

Tweetmeme is a service which aggregates all the popular links on twitter to determine which links are popular. Tweetmeme is able to categorize these links into categories and subcategories, making it easy to filter out the noise to find what your interested in.

We make it easy for you to subscribe to each category and the most popular through aur RSS feeds and Twitter accounts, you can find out more about theses through our help.

TweetMeme has a few ways to integrate their data into your site. The easiest way works by giving site owners a little sliver of JavaScript to embed in their site. The JavaScript adds a little button to the page for one click tweeting on Twitter.

<script type="text/javascript"><!--mce:0--></script>
<script src="http://tweetmeme.com/i/scripts/button.js" type="text/javascript"><!--mce:1--></script>

The above creates a little icon:

In theory, the above is all you'll need to do; just drop that slip into a page and you're good to go. The reality is that there are a couple rules you need to follow.

First, make sure you're using a custom title meta for your pages. If you don't all your Tweets will have the same title. Not good.

Second, make sure the page in question is open to the Internet. Don't try and lock the page behind a log in barrier or TweetMeme won't be able to parse the page to get the URL and title information.

Last, TweetMeme caches the data so don't expect the update to be instant; you will be disappointed. For example, on the Dark Void site we are constantly being hit up by the client about the delay. You probably won't see the number in the widget increment for a bit; there's nothing you can do about it.

TweetMeme also offers a widget, which I admit I haven't used and but it looks like it's basically the same thing as Tweetizen.

It should be noted that I wasn't able to get the TweetMeme Widget to work. Ever. It just shows a blank page. Good job, TweetMeme.

And of course, there's the TweetMeme API. You just have to have an API to be taken seriously. It does seem a little redundant to have an API that's populated with data from the Twitter API but, whatever. I haven't looked at the API too much either but it doesn't look too outside the norm for most APIs.

Anyway, there you go; TweetMeme. Marketers love it. I'm ambivalent.

I wish I could say I have moral reasons for not using pirated/cracked software but I'm pretty practical about the issue. Cracked software isn't something new to me; I've used it a lot. No, like I said, I'm more practial about it.

BTW, because I've had too much to drink while I write this, I'm using pirated and cracked interchangeably though I know they both mean different things. Try to keep up...

I was once a big fan of not paying for software. While I was a kid and all through college I never paid for any computer programs; not one, not ever. I used to have collections of stuff that my friends and I would share with each other and our families (computers were HUGE in my family). Basic disk swapping really.

And then the Internet came into my world. I became very good at finding cracked programs (warez). First warez sites, then Morpheus, then Limelight, then Kazaa (sometimes all 4).

That said, a lot of people find it surprising that I hate to use cracked software now.

Anyway, I've found that pirated programs are a little unreliable. This could of course be attributed to the main software product, not just the cracked version, but I find that hard to believe (unless, of course, it's version 1 of a new Microsoft product). I've used both cracked and commercial versions of many programs and more often than not the cracked or pirated version is just more unstable.

Now, I admit, I've installed cracked programs that didn't have any viruses and worked as stable as you could want. But I've also ran into the other side where it just becomes a pretty big time investment.

There's also the whole virus thing which is definately a crap shoot; sometimes you're good, other times you get hit. But it's still an issue that, if you do get infected, now requires time to clean up. Kind of a bummer when it happens.

Another, smaller reason, is the whole update thing. A lot of companies (I'm looking at you Adobe) have started finding what versions are cracked and when updates are applied they disable the program. This means if you want to use a cracked program you had better not update it. Ever. No new security patches and no bug fixes. Sorry.

All the above is pretty lightweight though. I hate to say it, but if that's all there was to worry about I'd probably be all about it.

No, the biggest reason is because you just never know. You never know if the issue you're having with the program is because your computer sucks or because you're using a cracked version that's become unstable. How can you know?

If you rely on software for professional reasons (not a kid or student) it's just not worth the headache.

Really, I should call this post "I whine about how Bethesda told me to Fuck Off and hand over the money" but it's a little too wordy. Instead, just a quick little rant on why exclusive downloadable content (DLC) is a really bad call for consumers.

I never really care too much about "deals" made between companies. I understand that nearly everything is a business, especially in the US, so deals are pretty common and I just accept it. I've never really been burned by them until recently though.

This particular case was with the Bethesda video game Fallout 3. I definately wouldn't classify myself as a gamer but I do play them; probably more than I should. I just don't follow any games. To be even more clearer: I DO NOT CARE ABOUT VIDEO GAMES. I just like to play.

One day I saw a commercial for Fallout 3, and having played the first 2 Fallout games, as well as Oblivion, I immediately went out and bought the game for the Playstation 3 (PS3).

Excellent game. Just a really, really, deap and immersing experience. I enjoyed it a lot.

Once I was done, and the game makes sure you are done once the game is beat  (unlike Oblivion), I put the game up and went back to my life.

Then I heard Bethesda was coming out with downloadable content for Fallout 3 and I started looking into it. Turns out the DLC was only for the Xbox and not the PS3; Microsoft, the fuckers, paid a shitload of money to lock this in.

Sigh...

Here's the problem; I had paid the same amount for the PS3 version as the 360, $60, and while I had a 360 I decided on the PS3 version because, well, the 360 was having non-RROD issues and I didn't want to go through the trouble of getting the damn game to work.

And I got hosed for it.

The lesson, apparently, is that Microsoft has no issue forcing my decisions. If I want to get full value out of the games I buy, and they are available for both the PS3 and 360, I had better buy for the 360 because they will ensure I regret that decision. I resent the crap out of that.

Like I said, I understand the business decision to do this (obligation to turn a profit and all that). But I also think it's a slimy, short thinking, decision that runs an unacceptable risk of alienating customers.

It's a good thing both Bethesda and Microsoft make good video game products.

There's a pretty positive review of wp-click-track on Daily Blog Tips, written by Daniel Scocco, today and it's official; wp-click-track doesn't suck. Not that it's good, it just doesn't suck  

It's always nice to see that someone has an opinion about something I'm involved in (good and bad). This is especially nice because it's on Daily Blog Tips, a site I actually read and enjoy.  

Thanks for the review Daniel.

Everyone likes the pretty. The pretty makes people feel nice and adds a sense of peace and comfort. The pretty is good and brings joy to small children.

We all want our data to be pretty. Tables of numbers just aren't fun for anyone. Except programmers; we don't give a shit.

Oddly though, PHP hasn't had many options for pretty charts and graphs. Sure, we have JpGraph but it's a little cumbersome and... well ugly's a good word for it. Recently I heard about pChart, which I'll be writing about another time, but aside from the cosmetic upgrade it doesn't seem too different than JpGraph. 

I've already gone into some detail about the Google Visualization API, which is a pretty good JavaScript implementation for creating charts and graphs, but that too is a little challenging to implement.

I've recently become enthralled with Open Flash Charts (OFC) though. OFC is an open source flash charting system that uses external data-files to keep the program language agnostic. You can use pretty much any web programming language, so long as it can output strings, and you're in business. In fact, there are libraries for a slew of languages including .NET, perl, python and ruby packaged right along with the download.

OFC was started because, I shit you not, the initial developer was burned by a vendor when looking for support on a PAID product. So he did what any programmer would do; wrote an open source version to help put that crappy company in hot water. Kudos

It's a pretty slick setup though. OFC works by embedding a swf in a webpage and then setting a flashvar to point to a datafile on the server. Simple. Well... sort of...

There are 2 different versions to use; each given the very original titles v1 and v2 respectively. Near as I can tell the latest version uses JSON as the data format while version 1 uses something a little more... generic. The problem is that version 2 has only a couple examples and version 1 has a ton so while I like to use the latest and greatest I have to stick with version 1 for now. That's what this article is going to focus on.

First, to get started, be sure to download the files from SourceForge. The download for version 1 was a little hard to find so, in the interest of helping others, I'm providing a direct link to download Open Flash Charts version 1.9.7. The download includes the swf, which you'll obviously need to embed in your page, and the wrapper code which is provided in quite a few different lanuages (if you don't do php).

Using the wrapper code makes creating the really complicated strings for the swf to absorb. Sure, you could do it by hand, but why the fuck would you?

The official site acutally has some really good examples of how to work with the code, so I won't go into that too much here. Suffice it to say it's pretty easy to create charts and graphs. The hardest part is getting used to the process of instantiating the code in the template (HTML) and then creating a stand alone script that gets hit by the swf for the data.

The HTML portion of the code is the simplest; you include the library (if it hasn't been done already) and call a function like so:

1
2
3
4

<?php
include_once 'ofc-library/open_flash_chart_object.php';
open_flash_chart_object( $width, $height, $url, $use_swfobject=true, $base='' );
?>

The only two that should be confusing is $base and $url.

$base is useful if you don't store the swf inside the site's root directory; just set $base to the path, from your sites document root and you're good.
For example:

2
3
4
5
6
7
8

<?php
//BAD
//$base = $_SERVER.'/path/to/open-flash-chart.swf';
 
//GOOD
$base = '/path/to/open-flash-chart.swf';
?>

$url is the full URL to the sister script to provide the options and data to the swf. It's important to keep in mind that all the options for the swf are provided by this script. Seriously, all the options. What type of graph to generate, the data for the graph, the style for the graph, everything is generated on the server. The swf just interprets that data and draws accordingly.

The code for the sister script is a little more complicated but pretty straight forward once you understand it. NOTE, this was taken line for line from the official site:

<?php

// generate some random data:

srand((double)microtime()*1000000);

$max = 50;

$data = array();

for( $i=0; $i<12; $i++ )

{

  $data

OFC is definitely worth a look for the pretty.
	
	

I don't really follow video games, except for the ones with web sites I'm working on, but I play them all the time with my brother. I have both the PS3 and 360 and really like 1st person shooters and RPGs (of course Right now, we're hugely addicted to Call of Duty 5's Nazi Zombies multiplayer. It's fucking SICK.

I say all of that so I can say I've been working my ASS off on two new projects so, as a bit of a thank you from a client, I was given a ticket to this year's E3 conference. Since I'm a geek I was really excited to go.

I don't really follow video game news or anything like that; I hear about a game, and if it sounds cool, I'll buy it. Still, I've been playing video games since the Atari 2600 so I've known about E3 for years.

I was able to play quite a few upcoming games like Red Faction 3, Dark Void and inFamous, which was pretty cool, and get a little drunk (in case you're wondering, yes, they do serve alcohol at the event).

Here are some of the better pics I took.

" src="/images/uploads/2009/06/img00040-20090604-1356-300x225.jpg" alt="Nurse " width="300" height="225" />

It was really interesting and pretty fun.

As programmers, we can get obsessed with the small things. I personally have no problem spending hours trying to optimize a section of code, not for performance or to improve user experience or anything like that, but because there's just something funny about the algorithm.

The professional in us tries, sometimes futily, to keep this behavior in check. Sometimes though you just have to do "just" one more thing to really make it perfect.  This, more often than not, balloons up into a weeklong chore to, in the end, dig yourself out of the hole you made with zero progress on the initial problem.

This is the way of the code monkey.

Which is why it's a little surprising to see how HTTP cookies are implemented. According to the official RFC (HTTP State Management Mechanism):

The user agent makes a series of requests on the origin server, after each of which it receives a new cookie. All the cookies have the same Path attribute and (default) domain. Because the request URLs all have /acme as a prefix, and that matches the Path attribute, each request contains all the cookies received so far.

Think about that for a second. On every request the cookies are sent to the server. ON EVERY REQUEST.

Try this; install a FireFox plugin called Live HTTP Headers (if you don't already have it installed). Start the plugin and refresh this page.

You should notice that every request for ANYTHING (images, js, css) sends the cookies up to the server.

I don't know about you, but I only have one or maybe two, points in an application that can evaluate cookies so to send on every request is a little... wasteful.

I don't know for sure if there's more work on the server side to deal with the cookies (though I would imagine the HTTP server has to do something to make the cookies available to a scripting language) but what I focus on is the bandwidth.

Now, I know it's the 21st century and bandwidth is now fast and cheap. Yay us.

But consider the state of the Internet and bandwidth in 1997 when the specification was first drafted. Most people were using 28.8 and 33.6 baud modems to browse the Internet and 56k was still a year away.

According to the spec:

Practical user agent implementations have limits on the number and size of cookies that they can store. In general, user agents' cookie support should have no fixed limits. They should strive to store as many frequently-used cookies as possible. Furthermore, general-use user agents should provide each of the following minimum capabilities individually, although not necessarily simultaneously:

* at least 300 cookies

* at least 4096 bytes per cookie (as measured by the size of the characters that comprise the cookie non-terminal in the syntax description of the Set-Cookie header)

* at least 20 cookies per unique host or domain name

User agents created for specific purposes or for limited-capacity
devices should provide at least 20 cookies of 4096 bytes, to ensure
that the user can interact with a session-based origin server.

So, unless I'm crazy, the above makes it acceptable for a site to use 20 cookies, each with a maximum size of 4096 bytes (4Kb). This equals out to a possible 81920 (82Kb) bytes of cookie data being sent on EVERY REQUEST.

This basically means that an image that weighed in at a cool 2Kb comes out to need 82Kb of bandwidth to transfer. Doing the math on a full site with say 20 images, 1 HTML file, 1 CSS file and 3 JS files and it really starts to add up. So at a time when bandwidth was scarce cookie usage was a good way to screw up the user experience if you didn't pay attention.

Granted, you'd have to be an idiot to write a program that used 20 cookies with each containing 4kb of data, but we're programmers. Most of us are stupid; some are really stupid.

It's easy to point at cookies and laugh.

The thing we need to keep in mind, though, is that this isn't a that big a problem anymore. Sure, once, it may have been an possible issue. Technology almost solved it though. Faster bandwidth, with faster computers made will eventually make any issue just disappear.

Which is kind of the point; something I try to keep in mind.