Made of Everything You're Not

Dreamhost is slandering me! They think I suck and have no problem telling people as much. Seriously; I write bad code by them. How do I know? Because Chris Duke over at AppModo told me so. Problem is, Dreamhost was 100% right. Yup; I fucked up.

Wp-Click-Track 0.7.1

On Monday I got an email from Chris asking for some help with one of the WordPress plugins I wrote; wp-click-track. Chris was cool enough to let me post the message he received from Dreamhost:

Hello,

I’m writing you about your database “appmodo”, please write me back as soon as possible.  There seems to be a plugin you are running that is poorly coded (missing table joins) and this 1300+ second query that it is running is causing high server load spikes. If this plugin is not fixed, or disabled, and it causes server problems, we may need to disable the wp_tracking_clicks and wp_tracking_links tables.  You can write me at justin@dreamhost.com.

The problem queries:

# Query_time: 2181  Lock_time: 1115  Rows_sent: 1  Rows_examined: 38703 use appmodo;

SELECT date_format(click_date,"%Y-%m-%d") AS first_click,
date_format(click_date,"%j") AS day_of_year FROM wp_tracking_clicks tc,
wp_tracking_links tl WHERE tc.click_id != '0'  GROUP BY first_click ORDER
BY first_click DESC LIMIT 1;

# Query_time: 1995  Lock_time: 924  Rows_sent: 1  Rows_examined: 38703 use appmodo;

SELECT date_format(click_date,"%Y-%m-%d") AS first_click,
date_format(click_date,"%j") AS day_of_year FROM wp_tracking_clicks tc,
wp_tracking_links tl WHERE tc.click_id != '0'  GROUP BY first_click ORDER
BY first_click DESC LIMIT 1;

# Query_time: 1387  Lock_time: 309  Rows_sent: 1  Rows_examined: 38703 use appmodo;

SELECT date_format(click_date,"%Y-%m-%d") AS first_click,
date_format(click_date,"%j") AS day_of_year FROM wp_tracking_clicks tc,
wp_tracking_links tl WHERE tc.click_id != '0'  GROUP BY first_click ORDER
BY first_click DESC LIMIT 1;

# Query_time: 2016  Lock_time: 930  Rows_sent: 1  Rows_examined: 38703 use appmodo;

SELECT date_format(click_date,"%Y-%m-%d") AS first_click,
date_format(click_date,"%j") AS day_of_year FROM wp_tracking_clicks tc,
wp_tracking_links tl WHERE tc.click_id != '0'  GROUP BY first_click ORDER
BY first_click DESC LIMIT 1;

Thanks!
Justin K

Justin did a nice job highlighting what the issue was; I didn’t join the tables together so it would take a really long time to complete the scans if there were a lot of rows in the tables. Another reason to always develop with large data sets; something I obviously didn’t do with wp-click-track.

Your application is useful and popular. Your users love it. Your users love you. But over the next week, something curious happens. As people use the application, it gets progressively slower and slower. Soon, the complaints start filtering in. Within a few weeks, the app is well-neigh unusable due to all the insufferable delays it subjects users to– and your users turn on you.

Anyway, there was no excuse for that; this was shoddy on my part. The only explanation I have is poor quality on my part.

This particular bug affected the line chart on the admin dashboard. Ever wonder why that particular graph took forever to load? Yup, that’s why.

What’s funny, to me at least, was that I had already fixed this particular issue; I just hadn’t released it yet. This particular bug was planned to be released with 0.8 but 0.8 is taking longer to complete than I initially thought it would. So the bug fix languished and sat there. Unreleased.

So yeah, wp-click-track 0.7.1. It’s a good release to get.

I hadn’t really had much time to announce this (what with the holidays and stuff) but last week WP-Click-Track 0.7 was released. This release includes a few bug fixes and a couple new features. I’d planned to release this a while ago but testing took a little longer than anticipated but it’s finally done.

WP-Click-Track 0.7 Released (Umm Last Week)

There were a couple minor, edge, bugs that I don’t want to go into detail about but there were 2 in particular that deserve a mention.

First, there was a bug fix for SSL enabled admins. This one I hadn’t really anticipated; I’d never thought someone would use ssl for the admin panel. The problem was that if you went to the admin through ssl the graphs wouldn’t render properly. My bad.

Then there was an issue in the graphs. If the charted number was over 1,000 the chart would basically throw up. Rookie move on my part but some escaping fixed that right up. If you ran into this bug I’m really sorry.

0.7 also includes an improved cookie model. I admit, I’ve been completely lazy about cookie management for years now. Yes, sessions are nice but when you’re building for a multi-server environment sessions can get a little tricky. Problem is that there are some pretty specific rules for cookies and just setting a cookie willy nilly, without any thought to the implications, can screw things up. Now it’s much, much, better.

Among the additions there’s a new configuration value to disable tracking of internal links. This should help with all the clicks in peoples galleries I’ve been hearing so much about

Speaking of configuration the entire configuration section has been rewritten to be more user friendly. Instead of the HUGE and confusing single page everything is organized into tabs and should, hopefully, make for a better experience when you do have to configure something.

Anyway, you can install it within the WordPress plugin manager or if you want to take a  look at the code here you go.

When I first started wp-click-track it was because I had a specific need; I wanted to find out how helpful my posts would and what better indicator than how many people clicked on the links in my posts. It was only to drive traffic and get my name out on the Intertubes that I released the plugin to other WordPress users and sites.

Roadmap

I never imagined people would actually use it much less have so much great feedback and improvement suggestions. As great as the insight is though it has pretty much made my plans for features and upgrades secondary to the users requests. As it should; because of the users wp-click-track is on a great track to become something truly useful.

There are only 3 releases left before the final, 1.0, version and to reach that I have the following outline:

0.7

The features for this release are going to focus on Data Export, Localization and a reworking of the configuration page.

Data export is a little complicated because two offerings have to be created to allow for older systems; normal csv/tsv and complete XLS export for users with newer systems. The XLS export will use the Excel 2007 XLS file format built using phpExcel. Since phpExcel requires php 5.2.5, at least, the exporter needs to be smart enough to degrade elegantly when the system can’t perform.

The Localization phase has already started with the release of 0.6. Initially, wp-click-track will be localized in french and spanish using Google Translate (until a native speaker can contribute). If anyone knows of a better method for transcription I’m open to suggestions.

There’s also going to be a refactoring of the cookie functionality. I admit, I was a little lazy about this, something I’ll go into more detail about at a later time. Now it’s time to clean this up.

0.8

It’s important that wp-click-track act like a native WordPress module; that’s what 0.8 is going to focus on. This one is obviously going to include more AJAXy goodness  to make link management easier and a little more inline with the rest of WordPress.

One thing 0.8 will definitely have is a removal of the Edit Link page. I’ve never really been happy with how that functionality worked; it’s very… standard. Instead, it will be modified into a simple “click to edit” style similar to how the permalinks are edited on the post edit page.

0.9

0.9 will include some advanced functionality like backup and import capabilities and the creation of WordPress hooks so other developers can extend the plugin. I have some ideas for a Bookmarklet for link creation but I don’t know how relevant that might be to other users.

By this point, I should have a fix for the little bugs that have been with the plugin since the beginning. I can’t say they won’t be done before 0.9 but they will be done by then.

1.0

Unless there’s any objection this release is also going to include functionality to send me usage and error information. I guarantee there won’t be any sensitive data being sent; EVER. It’s just important to know how the plugin is being used and what errors happen. This will be opt-in too so it won’t be a requirement.

The other thing the plugin will be ready for is donations. I never really wanted  to go this route but I had a day job then. This too will be opt-in only and there won’t be any nag screens or anything annoying. This will be as unobtrusive as possible I promise.

It should be noted that there’s no real guarantee that the functionality mentioned above will be done in that order. Need drives the direction more than anything else and the users certainly aren’t shy about steering the ship so to speak.

After 1.0 I don’t really anticipate any more updates; except for bug fixes or specific requests. Obviously, when WordPress changes their core functionality again the plugin will be rewritten and tested and a new version cycle will probably begin.

After about a month of working on the plugin around 20 hours a week I’m happy to announce the release of wp-click-track 0.6. This release includes a couple small bugs as well as one really bitch of a bug that’s been with the system since the release of 0.1. To make it worthwhile to upgrade there’s also a few features that have been requested and a couple I’ve personally been wanting for a while.

WP-Click-Track 0.6: Going Native

I have to admit, I didn’t think this update would take as long to deploy as it did. I don’t have a day job so, my time being free, I honestly thought I’d be done in a couple days. Lesson: estimates are hard even for your own projects.

One of the reasons was caused by the addition of a dedicated menu instead of the single entry point. Because the functionality has grown to the point of a full CRUD program the old menu style just wasn’t that useful anymore. Making the change required paying down a lot of built up technical debt in the system.

Another new feature is a search system for the links. I’ve been planning on this one for a while but because there was changes to the database, and I wasn’t really confident in deploying those changes until now, I’ve been sitting on the addition. The system will attempt to convert  the link table to the MyISAM format and, if it can’t, will use a basic LIKE search syntax as a fallback. This should really help those with bunches of links.

And, much to the delight of many I’m sure, this release includes a couple sidebar widgets for making the top links and “todays” click count data public. These work as you would expect a widget to work; they’ll have to be activated from the Appearance->Widget administration interface as any other widget should.

Finally, as far as the features and additions go, the interface has been changed to match the native WordPress interface. This should keep the experience of looking at the reports a little more fluid and less jarring.

The best thing about this release isn’t a feature but a single bug fix. For a while now (too long really) there’s been an issue that could keep certain links from being tracked accurately if the link didn’t open in a new window on FireFox 2&3. This was a bitch of a bug that was difficult to isolate and fix; but it’s finally, finally, dead.

In anticipation of 0.7, which focus primarily on l10n and i18n, 0.6 is ready to be localized. (BTW, if anyone is willing to localize the plugin into a different language please drop me a line.)

As always please report any bugs or issues with wp-click-track; I’ll get them fixed ASAP.

I’m back to working on a couple projects so I haven’t been able to work on Wp-Click-Track for a couple months. Hell, the last update was in March so, to be honest, it’s been about 3 months since the release of 0.4; though there were a couple, small, feature and bug fixes put out.

WP-Click-Track 0.5: The BIG Update

This release includes a bunch of features that a lot of people have been asking for but also a fix for some stupidity on my part.

I made a HUGE mistake in how the data was being recorded and the flagging of unique versus individual clicks. The patch can’t be automated so you’ll be presented with a link to click once the upgrade is complete. I highly recommend you do this ASAP after upgrading.

Features and Modifications

Added settings to disable user clicks
Added ignore IP address for click tracking
Added link statistics reset
Added link deletion
Changed graphs to Open Flash Chart
Added additional line chart vectors to display unique clicks
Added link parsing of next and prev template tags
Added link parsing of categories in posts/pages as well as sidebar widget
Added link parsing of tags links template
Improved title extraction to reduce No Name Given auto-label
Added global history and report page

Bug Fixes

Click graph date descrepancy issue
Added bypass for external links being double tracked when entered in page
Fixed backwards tracking flags
Changed admin widget ordering to list most clicked to least click
Fixed php short tag usage

Be sure to update your plugin ASAP and please let me know of any issues in the comments.

There’s a pretty positive review of wp-click-track on Daily Blog Tips, written by Daniel Scocco, today and it’s official; wp-click-track doesn’t suck. Not that it’s good, it just doesn’t suck  

Daily Blog Tips

It’s always nice to see that someone has an opinion about something I’m involved in (good and bad). This is especially nice because it’s on Daily Blog Tips, a site I actually read and enjoy.  

Thanks for the review Daniel.

It’s taken way too long to do but I’ve finally fixed a bunch of issues people have been experiencing. In fact, this was supposed to just be a bug release but I got a little carried away with the design and added in a couple overdue features.

First, The Bugs

Big Bug!! (by Denis Collette)

To be blunt, there were a couple nasty bugs in 0.3 that pretty much ruined the experience for a lot of people. I’m sorry for that. I swear, it worked on my blog

Anyway, the bugs that were fixed include 2 that prevented the system from tracking (Gary) and a bug that pretty much destroyed a users blog (Joerg) plus a one that prevented the comment body from being parsed as well as a couple spelling issues.

The Gary bug prevented the tracker from working because of a simple pathing issue. I didn’t take into account that people might have their blog in a sub-directory of the domain so my calls were going to the ether instead of the WordPress installation. So, you know, my bad…

The Joreg bug was a little more insidious and harder to track down though. It basically consisted of a couple issues, that when combined, caused a memory failure on the users blog. This is pretty much the worst case scenario when developing for other people (to destroy the system) and I, unfortunately, didn’t experience this myself so it only became apparent on other peoples blogs. This one bugged the shit out of me.

To fix I had to take content from Joreg’s blog and add it to my system. After I did that my computer instantly burst into flames. I didn’t know that was even possible but there you go…

The actual cause of the bug came from 3 different causes:

1. Email (mailto:) links
2. Anchor (#top) links
3. Missing (empty hrefs) links

Turns out that my regex skills were lacking because I didn’t account for any of the above. To fix I just added checking for those types of links to the parser and am ignoring them for now. This is a no brainer for the missing and anchor links (why would you want to track those?) but I can see a reasonable argument for wanting to track mailto links.  I’ll be working on getting that working with the next release.

The comment parsing bug was easily fixed by changing the parse priority in the WordPress hook to a higher level than 1. This was an issue because WordPress uses it’s own plugin system to create links placed into comments to href links automagically so when I was parsing the href wasn’t created yet. I went with a priority of 9 and it worked great.

Now, The Features

New Gold Dream... (by law_keven)

So, while I was trying to figure out the Joerg bug I got a little motivated and started work on a couple new features no really asked for but I felt the system could use. The biggest is a mod to add links for external distribution. If I’m going to be honest, this really started as a bit of  a copout on my part. I originally wanted the entire experience to be seemless to the user so they wouldn’t have to go through the pain of manually adding links to the system but with all the problems people were having with the parser and me not being able to figure it as fast as I wanted I thought having a fallback mechanism to add links made sense. It should be noted that this decision was made at 3AM after a night on the town.

Like most coding decisions made while, ahem…”comfortable“, this mod required more work than I initially thought. I don’t want to bore with the technical crap but suffice it to say  a HUGE refactoring was required so I wouldn’t gouge my eyes out while working on the plugin. Technical debt, you’ve got to pay the price at some point…

I also built in a little configuration panel so blog owners can explicitly state what portions of the code they want to parse and which they don’t. This was, again, meant as a stop gap for the Joerg bug with the idea being that blog owners could lower the parsing coverage if performance issues came up.

And that’s 0.4. I hope this one goes better than 0.3 but I have no idea at this point. I did include the activation monitor so I can keep an eye on how many times it’s activated and/or deactivated so I’ll find out in about a week if people are actually using the plugin.

In the meantime, if anyone experiences any bugs or issues, no matter how small, please don’t hesitate to post a comment on the official WP-Click-Track 0.4 page.

I’ve updated the repository with the latest version tonight.

This release includes a couple bug fixes and a few improvements.

Improvements

• Added Indexes on tables for better performance
• Updated Install system
• Added pie charts for clicks by day and hour

Bug Fixes

• fixed hour click report formatting
• fixed blank link haunting
• fixed option saving

Screenshots

Clicks By Hour Pie Chart

Day Clicks Pie Chart

Version 0.2 of wp-click-tracker has been out for almost 4 days and it looks like I left a couple bugs in there. I’ve updated the repository with the latest version and you can also download it from the main page:

wp-click-track download

This release is just a bug fix for the below:

• Parsing of attributes with single quotes
• added parsing / tracking of Archive links
• added parsing / tracking of Bookmark links
• added parsing / tracking of Comment Author URL links
• fixed option updates

Well it’s taken a couple weeks but I’m finally done with 0.2. This release includes an extensive reporting area, the addition of tracking on links created in the Links area of the admin, incompatibility checking for Google Analytics plugin and an upgrade notification nag.

Top Referrers

Top Referrers

Individual Clicks

Individual Clicks

Clicks by Hour

Clicks By Hour

Clicks by Day

Clicks By Day

Upgrade Checking

Upgrade Click Tracker

Plugin Incompatibility Checking

Compatibility Warning