Made of Everything You're Not

Personal blog of PHP programmer Eric Lamb.
  • Blog
  • Portfolio

Archive for May, 2009

Setting Up A Linux Web Server

Posted in IT, Servers on May 25th, 2009 by Eric Lamb – 0 Comments

I've been setting up my own servers for years; it's something I'm pretty passionate about as a programmer. I've learned soooo much about programming from seeing how the computer operates; setting up a server is HUGELY enlightening in that respect.

Setting Up A Linux Web Server

On the other hand though, I have a lot of things to do with my day and setting up a server, something that's pretty rote, doesn't require my direct attention. Just someone who can really follow instructions.

With that in mind, I started to compile a list so I could delegate this activity to my team. It's pretty useful from a historical stand point too so in case anyone else is interested in just how to setup a linux web server; here you go.

I've broken the setup into multiple sections, each outlining a specific type of setup, so it should be easier to digest. I'm just over the whole "series" thing, so instead of breaking this post into several smaller posts I opted for one REALLY long post. Plus, there's the whole disconnect thing with a series and setting a Linux box up shouldn't be done piecemeal.

It should be noted that the below is just a vanilla setup; it should be considered the bare minimum that needs to be done.

Basics

These are just a couple things that have to be done at no particular time. Ideally, the Firewall (below) would be configured first and then everything else would follow, but the rest could be done in any order. To make life easier it would be a good idea to take care of the basic stuff ASAP.

Setup locate

There's a sweet little command that is extremely helpful in locating files on the server; locate.

From the manual page:

locate reads one or more databases prepared by updatedb(8) and writes
file names matching at least one of the PATTERNs to standard output,
one per line.

It's way faster and efficient than 'find' and it's just what you need to find all sorts of things quickly. A basic use works like so:

locate php.ini

Which outputs:

/scripts/php.ini
/usr/lib/php.ini
/usr/local/lib/php.ini

To use it though, the first thing you need to do is make a call to 'updatedb' which will create the 'locate' database. The first time it's ran it'll take around 5 minutes to complete so you want to get this done ASAP so it doesn't break up your flow when working on something else.

updatedb

Setup Host Name

This part will register the server with the rest of the Internet. It's a different way on quite a few of the Linux servers I've worked with so I won't go into detail on how to do configure the server; I'm just going to go over the steps. You can always ask your host to set it up if needed.

Probably the most important part is choosing the name. There sees to be a couple different camps on this subject. Some like naming it as the role (so if it's a web server it'd be web1 or web5 (whatever) or if it's a db it'd be db1 or db5. Other's feel this is a security risk because it broadcasts what the server does.

Still, others, choose names that are just arbitrary; wizardtower, methlab, etc...

Personally, I find the convenience of recognition (knowing web1 is a web server and db3 is a database server) far outweighs any security concerns so I go this route.

Whatever you choose make sure to register the name with your registrar as an A DNS entry. Once that's done then you can update the server with the new hostname.

Email Forwarding

For every user on the system that receives email, like root for example, you'll want to create a .forward file in that users home directory. This good so email can be sent to you personally outside of logging into the server.

echo you@domain.com > /root/.forward

Security

Just to reiterate the above, this is by no means a complete list of everything that can be done to secure a server. Consider this a list of the minimum that needs to be done; nothing more. You should still research secure based off of your particular situation.

Seriously, your needs may require additional levels of security. You've been warned.

Server Security

Firewall

Most modern installations of Linux come with a firewall installed called iptables which is really reliable and stable. I use iptables in conjunction with either ConfigServer's csf/lfd or, usually if I'm not on a cPanel or Webmin server, apf.

Personally, I prefer csf/lfd for managing my firewall. Not only does it take care of iptable configuration but it also:

  • Sends an email on ssh login or su usage
  • Blocks connections with excessive connections
  • Login failure notifications for a lot of common services (cpanel, ftp, ssh, etc)
  • Port scan tracking and blocking
  • Temporary and permanent IP blocking
  • System Integrity checking and alerts
  • Suspicious process alerts
  • Suspicious file alerts

apf requires adding a few other programs on to attain the same amount of coverage; I prefer simple.

To install csf/lfd ConfigServer made the process extremely easy; they even put together step by step instructions that have yet to fail.

Setup Users

Add non privileged user and add to wheel group. This is important because we're going to seriously limit access to the shell. I like to have just one user who can ssh into a server but who can't do anything but use 'su' to up their privileges. No access to anything but 'su', not even 'wget'.

Mount /tmp securely

It's important to mount your tmp directory securely so nothing contained inside can be executed. It really helps when your site allows file uploads or if the server has been exploited (the tmp directory is a prime target).

You want to mount the partition as noexec,nosuid.

Install RKHunter

RKHunter is a tool that scans your system looking for any rootkits. It's a good tool but it does report some false positives; nothing too annoying but it does happen.

It's pretty simple to install and I've yet to see it fail on any flavor of Linux I've used (and I've used a bunch).

wget http://superb-west.dl.sourceforge.net/sourceforge/rkhunter/rkhunter-1.3.4.tar.gz
tar -zxvf rkhunter-1.3.4.tar.gz
cd rkhunter-1.3.4
./installer.sh --layout /usr/local --install

Once RKHunter is installed you'll want to set a daily cron job so your system is checked regularly. To do that just create a shell script and place it in /etc/cron.daily/ as outlined in this tutorial for installing RKHunter

#!/bin/bash
(rkhunter -c --cronjob 2>&1 | mail -s "Daily Rkhunter Scan Report" root)

SSH

SSH is incredibly vulnerable for no bigger reason than visibility; it's the de facto entrance point for most linux servers. I like to do a couple things to secure my SSH installation.

To make any of the below changes you'll need to edit your sshd_config file. On most systems it's going to be in:

/etc/ssh/sshd_config

First, I always disable root login. Most brute force (BF) attacks on your server will be for the user 'root' so simply disabling this allows most BF attacks will be futile for the attacker. If you've added a new user to the system, and that user is the only user who can ssh in, you're in a pretty good spot. The attacker has to know the username in order to even try passwords.

Second, I also change the port ssh listens on. The default, 22, is what most attackers will try for getting into ssh. Change that to something different and you've added another level of complexity onto the system.

It's important to let your host know of the change so they can access ssh when they need to. This shouldn't be a problem for most hosts but you may have a fight on your hands for some.

There are quite a few options you can use to configure ssh for; it's definitely recommended that you research as much about ssh as possible to configure it specifically for your needs.

Disable General Commands

This next one isn't exactly critical but I find it useful and it definately adds peace of mind so there's that.

I first heard about this from a forum for securing a cPanel server.

Many php exploit scritps use common *nix tools to download rootkits or backdoors. By simply chmod'ing the files so that no none-wheel or root user can use them we can eliminate many possible problems. The downside to doing this is that shell users will be inconvenienced by not being able to use the the commands below. Mod_security really removes the need to chmod this, but it is an added layer of protection.

#chmod 750 /usr/bin/rcp
#chmod 750 /usr/bin/wget
#chmod 750 /usr/bin/lynx
#chmod 750 /usr/bin/links
#chmod 750 /usr/bin/scp

As mentioned above, this is probably overkill but it does prevent anyone who does gain access from being able to do much of anything. If you really want to have that warm, fuzzy, feeling of safety you could also just chmod everything under /usr/bin like so

chmod 750 /usr/bin/*

That should really make you feel safe.

Disable Unneeded Services

Chances are that your server is going to be running quite a few services you're just not going to need. For example there's 'cups' the Linux print service. Is your webserver going to be connected to a printer? Probably not.

Leaving these enabled is bad because it's an avoidable entry point into your server by the "bad" people. From my experience I've learned to disable a bunch so I put together a little shell script to just handle it for me. Copy the below and put it into a file on your server called 'disable_services.sh' and chmod it to 0755

#!/bin/bash
service cups stop
chkconfig cups off
 
service xfs stop
chkconfig xfs off
 
service atd stop
chkconfig atd off
 
service nfslock stop
chkconfig nfslock off
 
service canna stop
chkconfig canna off
 
service FreeWnn stop
chkconfig FreeWnn off
 
service cups-config-daemon stop
chkconfig cups-config-daemon off
 
service iiim stop
chkconfig iiim off
 
service mDNSResponder stop
chkconfig mDNSResponder off
 
service nifd stop
chkconfig nifd off
 
service rpcidmapd stop
chkconfig rpcidmapd off
 
service bluetooth stop
chkconfig bluetooth off
 
service anacron stop
chkconfig anacron off
 
service gpm stop
chkconfig gpm off
 
service saslauthd stop
chkconfig saslauthd off
 
service avahi-daemon stop
chkconfig avahi-daemon off
 
service avahi-dnsconfd stop
chkconfig avahi-dnsconfd off
 
service hidd stop
chkconfig hidd off
 
service pcscd stop
chkconfig pcscd off
 
service sbadm stop
chkconfig sbadm off

HTTP Server

I've gotten a renewed appreciation for Apache lately so I'm not going to focus on one more than the other. With the exception of mod_security, everything below should be possible on pretty much all your popular webservers like Apache or Lighttpd.

Web Server
If you're going to stay with the default web server that's installed on the server you should, at the very least, rebuild it to make sure you're using the most up to date version.

Once you're dealing with a new(ish) installation of a web server the next thing you need to do is create the default site. This is the site people will see when they put either the IP address or the hostname of the server into a browser. I always set it up to use a blank page instead of the standard or default page. This way it doesn't look janky when users stumble upon the server.

Next, you want to disable Indexes. This setting is useful to prevent people from hitting a directory and seeing all the contents. If a user does try to read a directory, "images" for example, they will get a 403 (Forbidden) page.

Another thing I like to do is change the cPanel and http server skeleton files to blank pages. This is nice so when another site is created the site gets setup with blank pages instead of the "advert" pages for the system.

ModSecurity is an open source intrusion detection and prevention engine for web applications. It operates embedded into the web server, acting as a powerful umbrella - shielding applications from attacks. It's really cool.

ModSecurity works with Apache but there's always people out there experimenting so, hopefully, other http servers should get coverage some day. If you're using Apache you should definately, 100%, no excuses not to, install ModSecurity

PHP

For all the jokes about PHP being a sub-par programming language, accoring to the TIOBE Programming Community Index for May 2009 it's the most popular web development language available. So suck it.

php

It is true that php's the only language with a configuration file. I admit; that's just fucked up man...

Improve PHP

PHP is an interpreted language so right away your scripts are going to have a performance penalty (compared to a compiled language like C# for example). To help alleviate this you should always, always, install some sort of OP code cache. My personal favorite is xCache but there's also eAccelerator to name just one.

Installing xCache is pretty straightforward and setting it up is just as easy. Once it's done you should notice a considerable improvement in performance.

You'll also need to upgrade PEAR to make sure you're using the latest versions of packages and such. It's pretty easy; from a command prompt:

pear upgrade pear

After that you'll want to make sure the below packages are installed and up to date. These are just what I personally use and what the majority of open source php projects I've seen use.

pear install HTML_QuickForm
pear install Table
pear install Cache
pear install Cache_Lite
pear install Mail
pear install Mail_Mime

Secure PHP

Pretty much all of the security stuff is done by configuring PHP in php.ini. If you don't know where it is you can either create a phpinfo() call and look for the path to php.ini or, better, just use the 'locate' command from the command line:

locate php.ini

Unless you're using an old version of php (and why the fuck would you?) it's going to come out of the box with the WORST setting php ever introduced; 'register_globals'. If you require this setting to be active, for new projects, you're an idiot. I do unfortunately know about legacy apps requiring register_globals to be on but you can just set it with 'ini_set' on a per project basis so turn it off FOR EVERYTHING NEW YOU DO.

You're going to want to disable quite a few functions too, if possible. There's definate use in a lot of the below functions and sometimes the project you're hosting is going to require some of them. For example, on php 5.2.9 popen is required for some PEAR packages (and itself I think). This should be done on a case by case basis. But if you don't need to keep these open DON'T.

Look for the string 'disable_functions' in your php.ini and add any of the below to that string.

disable_functions = show_source, system, shell_exec, passthru, exec, phpinfo, popen, proc_open, allow_url_fopen

Configure MySQL

One thing really; change the god damn root password!! Set it to something, anything is better than NOTHING.
MySQL

I've had some people recommend disabling LOAD DATA LOCAL but while it sounds like a good idea it doesn't really gel with the way I work. I need that enabled to import databases on the server when the file is too big to upload into phpMyAdmin. I'm sure I could just enable it, do my thing, then disable it again but that sounds... troublesome.

Run Benchmarks

Running the benchmarks on the server is probably the most important part of setting up a server. It's important because the results of the benchmarks are going to tell you what you have to do next.

Benchmarking

There are a few different tools for benchmarking a server but the most popular is ApacheBench. Accoring to Wikipedia:

ApacheBench is a command line computer program for measuring the performance of HTTP web servers, in particular the Apache HTTP Server. It was designed to give an idea of the performance that a given Apache installation can provide. In particular, it shows how many requests per second the server is capable of serving.

ApacheBench comes standard with the Apache distribution so on most systems it's already going to be there. There's already a really good tutorial on NixCraft on how to use ApacheBench so I won't go into it. Just check out the tutorial above and you'll have a good understanding to start with.

I will say that this portion should take a good a while to do properly because you'll be doing it a lot. You're going to want to tweak your HTTP server configuration to get the optimal performance and every time you make a change you're going to want to confirm the improvement.

It will get old.

Now, as I mentioned above this is by no means a definitive guide or anything. It's just the bare minimum that should be done when you're setting up a Linux web server.

Just How Good Are You?

Posted in Brain Dump, Programming, Rant on May 22nd, 2009 by Eric Lamb – 5 Comments

What do doctors, lawyers, programmers and strippers have in common? The need for continuing education (ok, strippers probably don't but, you know, strippers. Think about it).

I realized early on that I was never, ever, EVER, going to be done learning about programming. Until I retire, and probably not even then, I'm going to spend a good portion of my life with my nose in a book, reading blogs, and/or diving into subjects WAY above my head.

stripper

I don't think this is as bad as it might sound though. I really love this shit. You have to if you want to be taken seriously. I accept that I'm not a Wozniak or Cutler; I have to actually work to understand this profession.

Oddly, I'm in the minority here. In my, limited, exposure to other programmers I can say definitively that the majority just plain suck; mostly because they refuse to grow and learn.

I've heard all the arguments before, "My weekends are mine", "I work hard enough; I don't have the energy", and the best ever, "My employer should pay for this like Google does. Whah!!". (I know Google doesn't, in fact, do this but people still say it.) All just pure crap excuses for maintaining a level of competence just high enough to not get fired.

Bottom line: working 8 hours a day is just not enough to matter. If you think you're a programmer and you don't spend time improving your skills you'll quickly, really quickly, become obsolete. It just doesn't matter if .Net is going to be around forever and your employer won't ever upgrade from 1.1; you're a hack (and not in a good way).

The crappy developers rarely, if ever, read blogs, books or articles on anything related to development. And when they do, it's usually just to reinforce a preconceived notion they already had. You've seen it; "See! Look, I've been saying all along Java sucks and here's an article on Reddit."

On the other hand, good developers spend time thinking about their projects. They read books about programming concepts like Code Complete and The Mythical Man Month. They're interested in the past; learning about how Windows NT was built or how AOL was founded is a good read to them. When asked for weekend plans they actually weigh programming against it.

I'm a little sick of the discrepancy here; there are just way, way, too many crappy programmers out there. Just lazy, untalented, bastards.

If you're not living programming to your bones chances are you're just not going to succeed. Sure, you can make a living but you'll never be anything more than what you are right now. If the idea of never excelling doesn't scare you do us all a favor and just stop writing code. Just stop right now.

How Do You Diff?

Posted in Brain Dump, Programming on May 20th, 2009 by Eric Lamb – 0 Comments

The other day I was working on a page that made heavy use of JavaScript. I'd been cool about it actually; I was using Prototype and Scriptaculous as the framework. I was even testing with virtual machines and everything!

DiffMerge

Then, mysteriously, the page stopped working in IE. Really frustrating because, well, fuck man, it WORKED. Why it happened doesn't matter; I did something pretty stupid.

What is important is that it was fixed. For that, I went through my source (SVN) repository and found the file that worked. Easy really.

This still left the issue of why it happened though.

Quick aside: I don't understand people who just accept that something works (ie; it broke, I fixed it, I'm done). How can you not need to figure out what happened? How will you prevent this from happening again?

To solve that little mystery I busted out my trusy diff tool; SourceGear's DiffMerge.

I've tried a bunch of diff tools, on both Windows and Linux, but at this point I'm really digging DiffMerge. According to the site:

  • Diff. Graphically shows the changes between two files. Includes intra-line highlighting and full support for editing.
  • Merge. Graphically shows the changes between 3 files. Allows automatic merging (when safe to do so) and full control over editing the resulting file.
  • Folder Diff. Performs a side-by-side comparison of 2 folders, showing which files are only present in one file or the other, as well as file pairs which are identical or different.
  • Configurable. Rulesets and options provide for customized appearance and behavior.

It's a really great diff tool totally worth checking out.

Advanced Bad Behavior

Posted in Code, Programming on May 18th, 2009 by Eric Lamb – 1 Comments

I really wanted to move away from Bad Behavior; there's only so much I'm interested in this topic. But the first two posts didn't cover everything I wanted to talk about so I wasn't left with that warm fuzzy feeling of completeness.  If, like me, you're over the whole Bad Behavior series I'm really sorry.

More Bad Behavior; Again

This time I'm going to go over the last little tid-bits so you can protect your sites and make sure the "bad" people stay away. For the most part anyway.

White Listing

You can white list IP addresses and user-agents. IP addresses can be white listed using ranges (in the CIDR format) or single IP by editing the file 'whitelist.inc.php'. Open it up and edit the  below:

14
15
16
17
18
19
20
21
22
// Includes four examples of whitelisting by IP address and netblock.
$bb2_whitelist_ip_ranges = array(
	"64.191.203.34",	// Digg whitelisted as of 2.0.12
	"208.67.217.130",	// Digg whitelisted as of 2.0.12
	"10.0.0.0/8",
	"172.16.0.0/12",
	"192.168.0.0/16",
//	"127.0.0.1",
);

User-agents can be white listed in the same file but require an exact match to what you add to the array.

37
38
39
$bb2_whitelist_user_agents = array(
//	"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) It's me, let me in",
);

It's important to use white listing conservatively. Extremely so. You don't want to use white listing unless you positively, absolutely, have no other option.

Black Listing

There are two different ways to use black listing in Bad Behavior; manually adding entries to your black lists and using the http:BL feature.

To use Bad Behavior's http:BL features you must have an http:BL Access Key. It's a fairly simple process that requires registration with Project Honeypot. They'll give you a BL Access Key and you place it in the settings array of 'bad-behavior-generic.php'.

47
48
49
50
51
52
53
54
55
56
$bb2_settings_defaults = array(
	'log_table' => 'bb_logs',
	'display_stats' => true,
	'strict' => true,
	'verbose' => true,
	'logging' => true,
	'httpbl_key' => 'PLACE_YOUR_KEY_HERE',
	'httpbl_threat' => '25',
	'httpbl_maxage' => '30',
);

Once that's done your install of Bad Behavior will use your local black lists as well as the http:BL lists.

On the other hand, you may encounter some rare cases where your site is being spammed by a new agent. In this case you might want to manually add entries to your local black lists.

The black lists are placed within 'blacklist.inc.php'. It only accepts user-agents, probably because IP address blocking is essentially useless. You'll need to edit 3 different areas of the script:

The first is for strings that occur at the beginning of the user-agent.

7
8
9
10
11
12
$bb2_spambots_0 = array(
	"<sc",			// XSS exploit attempts
	"8484 Boston Project",	// video poker/porn spam
	"adwords",		// referrer spam
	"autoemailspider",	// spam harvester
	//etc...

The next is for strings that occur anywhere within the user-agent string.

57
58
59
60
61
$bb2_spambots = array(
	"\r",			// A really dumb bot
	"; Widows ",		// misc comment/email spam
	"a href=",		// referrer spam
	//etc...

And, best of all, there's also a regular expression (regex) array for the really difficult user-agents.

87
88
89
90
91
92
93
94
// These are regular expression matches.
$bb2_spambots_regex = array(
	"/^{10}$/",	// misc email spam
	"/^Mozilla...$/i",	// fake user agent/email spam
	"/{8,}/",
//		"/(;\){1,2}$/",		// misc spammers/harvesters
//		"/MSIE.*Windows XP/",	// misc comment spam
);

It's also possible to add your own blacklists into your Bad Behavior install. This is pretty helpful if you have multiple installs and are sane enough to recognize the absurdity in maintaining multiple lists. Just build a blacklist service and add the info to 'blackhole.inc.php'.

Yeah, just build a blacklist server. Ummm... it's easy?

Customizing the Template

Every time a request gets blocked the system doesn't really know, 100%, that the request is bad; it just can't. So, instead of just dying, Bad Behavior displays a page with instructions on how to "unblock" yourself usually by just clicking on a link.

Unfortunately, the page looks like ass.

Bad Behavior Blocked Screenshot

The idea, I'm sure, is that the page should have as small a footprint on the server as possible. One of the selling points is to lower bandwidth by blocking spam requests. No images, CSS or pretty allowed at all.

The flip side of this argument is that legitimate users who get this page will have an experience that's lacking in, ahem, quality.

You can change the look of the page by editing 'banned.inc.php'. It should be pretty self explanatory once you open the file what needs to be done.

If you want, you can also change the response messages. Those are stored in 'responses.inc.php'.

Well, that about does it; there's more to Bad Behavior but this series pretty well covered all the good parts.

This will be my last post on Bad Behavior; I swear.

More Bad Behavior

Posted in Code, Programming on May 15th, 2009 by Eric Lamb – 0 Comments

In the first post in this "series" (wtf? when did I start doing "series"? Oh, right...) I went over the basics of what Bad Behavior is and how to get it installed. Bad Behavior's advanced setup required some investigation and forethought in order to work out it so it was best to break the post up; so, you know, here you go.

More Bad Behavior

By default, if all you do is follow the instructions laid out in the first post you'll have a working setup. For some of the cooler logging functionality you're going to have to edit the included file 'bad-behavior-generic.php'.

I've been feeling kind of down on Bad Behavior about this process for a few days now. It seemed kind of lame that they came up with such a great idea, wrote a really cool script but then killed the implementation. After working with the script for a couple days it's starting to make some sense but not really enough to convince me it shouldn't have been done.

To help others who've had this delima, I've compiled a list of steps to get Bad Behavior logging up and running.

Install Bad Behavior

The first thing you're going to want to do is create a database and add the connection code to your version of 'bad-behavior-generic.php'.

Here's the SQL:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
CREATE TABLE IF NOT EXISTS `bb_logs` (
  `id` int(11) NOT NULL auto_increment,
  `ip` text NOT NULL,
  `date` datetime NOT NULL default '0000-00-00 00:00:00',
  `request_method` text NOT NULL,
  `request_uri` text NOT NULL,
  `server_protocol` text NOT NULL,
  `http_headers` text NOT NULL,
  `user_agent` text NOT NULL,
  `request_entity` text NOT NULL,
  `key` text NOT NULL,
  PRIMARY KEY  (`id`),
  KEY `ip` (`ip`(15)),
  KEY `user_agent` (`user_agent`(10))
)

For this simple demo I'm going to use the native php functions but it's more than likely you'll have a database class. I put the below directly past the comments above any function declaration:

33
34
35
36
37
38
39
40
41
42
$link = mysql_connect('localhost', 'user_name', 'password');
if (!$link) {
   die('Not connected : ' . mysql_error());
}
 
// make foo the current db
$db_selected = mysql_select_db('bad_behavior', $link);
if (!$db_selected) {
   die ('Can\'t use foo : ' . mysql_error());
}

Next, you need to populate the skeletal functions with their appropriate code:

60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
// Return current time in the format preferred by your database.
function bb2_db_date() {
	return gmdate('Y-m-d H:i:s');	// Example is MySQL format
}
 
// Return affected rows from most recent query.
function bb2_db_affected_rows() {
	return mysql_affected_rows();
}
 
// Escape a string for database usage
function bb2_db_escape($string) {
	return mysql_real_escape_string($string);
}
 
// Return the number of rows in a particular query.
function bb2_db_num_rows($result) {
	if ($result !== FALSE)
		return mysql_num_rows($result);
	return 0;
}
 
// Run a query and return the results, if any.
// Should return FALSE if an error occurred.
// Bad Behavior will use the return value here in other callbacks.
function bb2_db_query($query) {
	return mysql_query($query);
}
 
// Return all rows in a particular query.
// Should contain an array of all rows generated by calling mysql_fetch_assoc()
// or equivalent and appending the result of each call to an array.
function bb2_db_rows($result) {
	return mysql_fetch_assoc();
}
 
// Return emergency contact email address.
function bb2_email() {
	// return "example@example.com";	// You need to change this.
	return "badbots@ioerror.us";	// You need to change this.
}

Then you'll also need to place a call to close the database connection at the bottom of the script. Place the below on the very last line:

160
mysql_close();

Doing that will make sure you don't have any rogue connections eating up your queue.

Once the above is complete you should have a fully setup and working install of Bad Behavior.

In case anyone has any issues with the above I've prepared a stand alone version of the script anyone can download bad-behavior-generic.

It should also be noted that incorporating a database into your Bad Bahavior installation ups the load on every request (which may be why it's not in there by default). You just have to choose whether the need for logging out weighs the increased load. For me, it did.

Simple Twitter Search with PHP

Posted in Programming on May 14th, 2009 by Eric Lamb – 0 Comments

So, I've already gone into detail on the Arc90 Twitter API Service and it's pretty fun to work with. One small problem though; it's pretty big, complicated and, well, bulky. There's a lot you have to do to get something up and running.

XKCD

Enter PHPTwitterSearch; a pretty easy to use, lightweight, Twitter API Search class. It's based on the class originally developed by David Billingham, which I admit, I didn't check out so I don't know how much it's based on.

The thing I didn't really like about the Arc90 script was how complicated the search process was. In case you didn't notice, I didn't go into any detail about the search process; for precisely because it was just to complicated. I stopped having fun when working with the search functionality.

Simplicity was what immediately drew me to PHPTwitterSearch. Here's a basic example of a search:

1
2
3
4
5
6
<?php
$query = 'findme';
$search = new SWTwitterSearch();
$search = new TwitterSearch($query);
$results = $search->results();
?>

Very, very simple. The above searches the API for $query.

Here's the best part though; for a little more complication all you have to do is chain the calls together.

1
2
3
4
5
<?php
$search = new SWTwitterSearch();
$search = new TwitterSearch();
$results = $search->from('username')->with('hashtag')->to('username2')->results();
?>

The above searches the API for tweets from "username" with the hash string "#hashtag" that was sent to the user "username2".

The class returns an array with objects that's really basic and simple.

Why I Hate Facebook Connect

Posted in Programming on May 13th, 2009 by Eric Lamb – 0 Comments

In the Facebook Connect Primer I barely scratched the surface about the Facebook Connect service; I fully expected to research more and write about it. I had it all worked out in my head; in this post I wanted to outline how I was able to use the service and outline deeper features and functionality.

I Hate Facebook

Unfortunately, this just wasn't possible because, well, all the documentation I could find on Facebook Connect was terribly outdated and inaccurate. So, instead, I'm going to talk about how painful it was trying to work with the system. Enjoy smile

I needed to find out as much as possible about the Facebook Connect service so the first thing I thought to do was the Facebook demo site, "The Run Around".

I first downloaded the file and extract and setup a development site on my development web server. I work on my Vista machine so it was pretty straightforward.

This was a mistake though. It turns out that development of Facebook Application projects requires an open connection to the Internet. It can not be behind a password screen.

FUCK! You have to work live on the Internet when developing; amateurs...

Anyway, after setting up the project on my computer the first thing I got was a screen full of undefined variable notices from php; the developers didn't take the time to kill notices in the code or throw a call to:

error_reporting(0);

sigh.... have to fix...

Either, open up init.php and add the above code below the first <?php, or do like I did and fix all the errors yourself.

The php notices aside, the first time it's opened in a browser an error screen shows up saying to follow the instructions in lib/config.php.sample

Initial Error

You have to register for a developer account if you don't already have one, like I did, and then create an application. This means, to me anyway, that Facebook Connect sites have to abide by the same Terms of Use as Facebook Applications.

Create a Facebook Application and go to the Connect tab. From here you can setup the callback URLs, upload a new image for the Connect confirmation screen and enter a base domain (this helps for cross domain access).

Of particular interest are these lines from the Terms of Service:

You can only cache user information for up to 24 hours to assist with performance. The only exceptions are those listed in the Facebook Platform Documentation.

However, you must get signoff from us before releasing any formal press releases.

After reading the above, combined with the above frustrations, I decided to walk away from Facebook Connect entirely. This was a HORRIBLE research experience.

That was until recently when I found out about an upcoming project that requires integration with the service. Oh, well. At least I already have some research out of the way...

Good Reading:
Facebook Platform Application Guidelines
Platform Policy
Facebook Privacy Policy
Facebook Site Terms of Use
Facebook Platform Developer Copyright Information (if your application permits file-sharing)
Facebook Platform Documentation

How to Build a Facebook Application
Facebook Application Tools
Getting Started With Facebook Connect
FBConnect - 5 reasons why we dumped it
API Functions

Virtualization Actually Works!

Posted in IT on May 11th, 2009 by Eric Lamb – 3 Comments

I had wanted to have a full time Linux desktop for about a year now. I think I've gotten to a point with Linux that I'm comfortable enough to use it for development. The problem is that I manage a Windows network and I work solely from a laptop. (Surprise! I'm pretty stubborn about my computer setup; I likes what I likes.)

I'd used Cygwin before but it always seemed... clunky at certain things like, well, everything. It really does works though and it's pretty nice for doing things like integrating SpamAssassin into Exchange, which is also a pretty cool exercise, just not for a full desktop experience.

Virtualization

Because of that, I was a little skeptical about the whole virtual machine thing. It sounds like a good idea, having another operating system installed on top of another, but until I recently tried it I just didn't really understand how cool it really is.

First, some details. My personal computer is a Gateway MT6705 with 2GB of RAM running Vista Business. I use the classic theme, fuck you Aero, to keep my computer snappy too. After boot up my computer has 1.5GB of RAM available.

With those specs, my biggest concern was performance; it didn't really make sense that my computer would still be usable when a virtual desktop was running.

Then I remembered that "usable" is a relative state. Right now, I'm using the fastest computer I've ever used, so I'm already accustomed to crappy performance. Plus, I'm already running a stripped down version of the latest, stable, Windows so the only place to go was a lower end operating system.  The biggest question then becomes "What virtual machines do I want to run?".

The first place I went was VMware; I still wasn't sure all the details and as usual Wikipedia left me with questions. VMware offers a workstation version that costs $189 at the moment with a free trial to check it out. They also have a free virtual machine client, VMware Player. Unfortunately, when I was researching the two products the VMware site wasn't allowing downloads so I couldn't check it out.

Instead, I moved onto VirtualBox; and I'm really happy I did. VirtualBox is a free virtual machine that is also open source. According to their site:

VirtualBox is a family of powerful x86 virtualization products for enterprise as well as home use. Not only is VirtualBox an extremely feature rich, high performance product for enterprise customers, it is also the only professional solution that is freely available as Open Source Software under the terms of the GNU General Public License (GPL). See "About VirtualBox" for an introduction.

Presently, VirtualBox runs on Windows, Linux, Macintosh and OpenSolaris hosts and supports a large number of guest operating systems including but not limited to Windows (NT 4.0, 2000, XP, Server 2003, Vista), DOS/Windows 3.x, Linux (2.4 and 2.6), Solaris and OpenSolaris, and OpenBSD.

VirtualBox is being actively developed with frequent releases and has an ever growing list of features, supported guest operating systems and platforms it runs on. VirtualBox is a community effort backed by a dedicated company: everyone is encouraged to contribute while Sun ensures the product always meets professional quality criteria.

On this site, you can find sources, binaries, documentation and other resources for VirtualBox. If you are interested in VirtualBox (both as a user, or possibly as a contributor), this website is for you.

Installation was easy; just like any other Windows program really. It's definitely worth while to read through manual before diving into the install though; they have some conventions that are really useful to know before diving in.

To install a virtual machine you're essentially installing the operating system onto a your hard drive so the process can take a little time. I was able to get Fedora 10, Windows XP and Windows 7 (yup, Windows 7!)  running in a long night.

VirtualBox

The way it works is that you allot a set amount of your spare system resources to each of your virtual machines. So, for example, I set Windows XP to have 1 GB of RAM and 10GB of hard drive space and VirtualBox only allows that virtual machine to those specs.

VirtualBox handles the resource detail pretty elegantly; in that it doesn't use the resource until it needs it. This means that instead of instantly having 10GB of your hard drive used up VirtualBox will only use the amount already taken. You can tweak the settings for a VM whenever you want so you can get just the right mix.

Fedora VirtualBox

That does make me think my current setup isn't really the best. Specs aside, which are a pretty big issue for me at the moment, running Windows Vista as my base Operating System is kind of wasteful. I'm thinking it might be a good idea to upgrade my computer, I've had this laptop for over 2 years and 64 bit is pretty appealing, or more likely, downgrade to Windows XP or a base Linux install and run virtual machines full time.

The Bad Behavior Spam Blocker Part 1

Posted in Code, Programming on May 08th, 2009 by Eric Lamb – 6 Comments

Anyone with a blog has seen comment spam. This is the stuff that shows up talking about Viagra, written in Russia and are usually stuffed with links.

There are a couple tactics for combating this sort of thing; some sites require registration to comment, some people manually delete the stuff and some sites use technology to help.

Bad Behavior

What to do? What to do...?

Well, I don't think it's a good idea to add barriers in front of users participating in a discussion, so registration is out. I'm pretty lazy and don't want to manually delete comment spam so moderation isn't going to work. I am a programmer though so I have an innate confidence in technology to deal with this (mostly anyway). To that end I like to use 2 different services to deal with comment spam; Akismet, which I'm not going to talk about now, and Bad Behavior.

According to the official site:

Bad Behavior complements other link spam solutions by acting as a gatekeeper, preventing spammers from ever delivering their junk, and in many cases, from ever reading your site in the first place. This keeps your site’s load down, makes your site logs cleaner, and can help prevent denial of service conditions caused by spammers.

Thankfully, there are already Wordpress plugins for both Akismet and Bad Behavior, so my blog is pretty well protected, but I also work on custom programs and need to protect them too. This got me thinking about how to to get Bad Behavior up and running on your systems; which is why you're here I'm sure.

Like most things php, installing Bad Behavior is pretty easy. To install just download the files, unzip and place the files somewhere in your applications include path. Then just include the below preferably in a file included in all your pages after you upload the files. Using the below will only protect your site

1
2
3
4
<?php
$path_to_bb = '/path/to/';
require_once("$path_to_bb/bad-behavior-generic.php");
?>

The above is nice and all; your site's pretty well protected from there but it would be nice to know what was happening behind the scenes. Just how many spam attempts are being blocked?

Bad Behavior does include a logging system but, oddly, at the time of this writing using 2.0.26, they don't include any sort of install script. The instructions state:

If you just can’t live without logging, you will need to provide a database connection. Bad Behavior uses callbacks whenever it needs to run a database query; in order to provide this functionality, you will need to provide the appropriate hooks into your PHP-based software’s database and add them into the bad-behavior-generic.php file. The code has stub functions which show what is needed, and you can use the bad-behavior-wordpress.php file as an example to work from, though your implementation will necessarily be different.

I'll go into detail in the next post; I'm still sick so I can't write anymore.

Arc90 Twitter API Service Part 2

Posted in Code, Programming on May 06th, 2009 by Eric Lamb – 3 Comments

As promised, here's the follow up to Arc90 Twitter API Service Part 1. If you haven't read that piece yet you probably should...

Twitter

Here are the rest of the methods for interacting with Twitter via the Arc90 Twitter API Service.

It should be noted that nearly everything in the below is taken from the generously documented source code.

Get Messages

Returns a list of the 20 most recent direct messages sent to the authenticating user. The XML and JSON versions include detailed information about the sending and recipient users. This method can return XML< JSON, ATOM and RSS.

1
2
3
4
5
6
7
8
9
10
11
12
13
<?php
$params = array();
/*
$params = '0';
$params = mktime(date("H")-4, date("i"), date("s")); //unix timestamp;
$params = '0';
*/
$response = $twitter->getMessages('json', $params);
 
$return = $response->getData();
 
print_r( json_decode($return) );
?>

Get Sent Messages

Returns a list of the 20 most recent direct messages sent by the authenticating user.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
<?php
$params = array();
/*
$params = '0';
$params = mktime(date("H")-4, date("i"), date("s")); //unix timestamp;
$params = '0';
*/
$response = $twitter->getSentMessages($format = 'json', $params);
 
// Print the XML response
$return = $response->getData();
 
print_r( json_decode($return) );
?>

Send Message

Sends a new direct message to the specified user from the authenticating user. This shows up under their "Direct Messages" section.

1
2
3
4
5
6
7
8
<?php
$text = 'Hello from my script!';
$user = 'USER_TO_SEND_TO'; //can be username or user_id
$response = $twitter->sendMessage($user, $text, 'json');
 
// Print the XML response
$return = $response->getData();
?>

Destroy Message

Destroys the direct message specified in the required ID parameter. The authenticating user must be the recipient of the specified direct message.

1
2
3
4
5
6
7
8
9
<?php
$id = 'ID_OF_MESSAGE_TO_DELETE';
$response = $twitter->destroyMessage($id,'json');
 
// Print the XML response
$return = $response->getData();
 
print_r( json_decode($return) );
?>

Create Friendship

Befriends the user specified in the ID parameter as the authenticating user. Returns the befriended user in the requested format when successful.

1
2
3
4
5
6
7
8
9
<?php
$id = 'ID_OF_USER_TO_BEFRIEND';
$response = $twitter->createFriendship($id,'json');
 
// Print the XML response
$return = $response->getData();
 
print_r( json_decode($return) );
?>

Destroy Friendship

Discontinues friendship with the user specified in the ID parameter as the authenticating user.

1
2
3
4
5
6
7
8
9
<?php
$id = 'ID_OF_USER_TO_DESTROY_FRIEND';
$response = $twitter->destroyFriendship($id, $format ='json');
 
// Print the XML response
$return = $response->getData();
 
print_r( json_decode($return) );
?>

Check if Friendship Exists

Tests if a friendship exists between two users.

1
2
3
4
5
6
7
<?php
$user_a = 'USER_1';
$user_b = 'USER_2';
$response = $twitter->friendshipExists($user_a, $user_b, $format ='json');
$return = $response->getData();
print_r( json_decode($return) );
?>

Update Location

Updates the location attribute of the authenticating user, as displayed on the side of their profile and returned in various API methods. Please note this is not normalized, geocoded, or translated to latitude/longitude at this time.

1
2
3
4
5
6
<?php
$location = 'Home';
$response = $twitter->updateLocation($location,'json');
$return = $response->getData();
print_r( json_decode($return) );
?>

Update Delivery Device

Sets which device Twitter delivers updates to for the authenticating user. $device Must be one of: sms, im, none.

Sending 'none' as the device parameter will disable IM or SMS updates.

1
2
3
4
5
<?php
$device = 'sms';
$response = $twitter->updateDeliveryDevice($device, 'json');
$return = $response->getData();
?>

Update Profile Colors

Sets one or more hex values that control the color scheme of the authenticating user's profile page on twitter.com. These values are also returned in the /users/show API method.

At least one of the $params is required.

1
2
3
4
5
6
7
8
9
10
<?php
$params = '#FFFFFF';
$params = '#000000';
$params = '#CCCCCC';
$params = '#999999';
$params = '#FFFFFF';
 
$response = $twitter->updateProfileColors($params, $format ='json');
$return = $response->getData();
?>

Update Profile Image

Updates the authenticating user's profile image. Image must be a valid GIF, JPG, or PNG image of less than 700 kilobytes in size. Images with width larger than 500 pixels will be scaled down.

1
2
3
4
5
6
7
<?php
$image = '/path/to/image/'; //Local file path of the image to be uploaded
$response = $twitter->updateProfileImage($image, $format ='json');
$return = $response->getData();
print_r( json_decode($return) );
 
?>

Update Profile Background Image

Updates the authenticating user's profile background image. Image must be a valid GIF, JPG, or PNG image of less than 800 kilobytes in size. Images with width larger than 2048 pixels will be scaled down.

1
2
3
4
5
6
<?php
$image = '/path/to/image/'; //Local file path of the image to be uploaded
$response = $twitter->updateProfileBackgroundImage($image, $format ='json');
$return = $response->getData();
print_r( json_decode($return) );
?>

Update Profile

Sets values that users are able to set under the "Account" tab of their settings page. Only the parameters specified will be updated; to only update the "name" attribute, for example, only include that parameter in your request.

At least one of the $params is required.

1
2
3
4
5
6
7
8
9
10
11
<?php
$params = 'Name_TO_Use';
$params = 'Email_To_Use';
$params = 'URL_TO_Use';
$params = 'Location_TO_Use';
$params = 'Description_TO_Use';
 
$response = $twitter->updateProfile($params, $format ='json');
$return = $response->getData();
print_r( json_decode($return) );
?>

Get Favorites

Returns the 20 most recent favorite statuses for the authenticating user or user specified by the ID parameter.

1
2
3
4
5
6
7
8
9
<?php
$params = 'ID_TO_Use';//The ID or screen name of the user for whom to request a list of favorite statuses.
$params = '0';
 
$response = $twitter->getFavorites($format ='json', $params );
$return = $response->getData();
print_r( json_decode($return) );
 
?>

Create Favorite

Favorites the status specified in the ID parameter as the authenticating user. Returns the favorite status when successful.

1
2
3
4
5
6
7
<?php
$id = 'ID_TO_Use'; //The ID of the status to favorite
 
$response = $twitter->createFavorite($id, $format ='json');
$return = $response->getData();
print_r( json_decode($return) );
?>

Destroy Favorite

Un-favorites the status specified in the ID parameter as the authenticating user. Returns the un-favorited status in the requested format when successful.

1
2
3
4
5
6
7
<?php
$id = 'ID_TO_Use'; //The ID of the status to favorite
 
$response = $twitter->destroyFavorite($id, $format ='json');
$return = $response->getData();
print_r( json_decode($return) );
?>

Follow

Enables notifications for updates from the specified user to the authenticating user. Returns the specified user when successful.

NOTE: The Notification Methods require the authenticated user to already be friends with the specified user. Otherwise the error "there was a problem following the specified user" will be returned.

1
2
3
4
5
6
7
<?php
$id = 'ID_TO_Use'; //The ID or screen name of the user to follow
 
$response = $twitter->follow($id, $format ='json');
$return = $response->getData();
print_r( json_decode($return) );
?>

Leave

Disables notifications for updates from the specified user to the authenticating user. Returns the specified user when successful.

NOTE: The Notification Methods require the authenticated user to already be friends with the specified user. Otherwise the error "there was a problem following the specified user" will be returned.

1
2
3
4
5
6
7
<?php
$user = 'ID_TO_Use'; //The ID or screen name of the user to leave
 
$response = $twitter->leave($user, $format ='json');
$return = $response->getData();
print_r( json_decode($return) );
?>

Block

Blocks the user specified in the ID parameter as the authenticating user. Returns the blocked user in the requested format when successful.

1
2
3
4
5
6
7
<?php
$id = 'ID_TO_Use'; //The ID or screen name of the user to block
 
$response = $twitter->block($id, $format ='json');
$return = $response->getData();
print_r( json_decode($return) );
?>

Un-Block

Un-blocks the user specified in the ID parameter as the authenticating user. Returns the un-blocked user in the requested format when successful.

1
2
3
4
5
6
7
<?php
$id = 'ID_TO_Use'; //The ID or screen name of the user to block
 
$response = $twitter->unblock($user, $format ='json');
$return = $response->getData();
print_r( json_decode($return) );
?>

Graph Friends

Returns an array of numeric IDs for every user the specified user is following.

1
2
3
4
5
6
7
<?php
$user = 'ID_TO_Use'; //ID or screen_name of the user to retrieve the friends ID list for
 
$response = $twitter->graphFriends($format ='json', $user );
$return = $response->getData();
print_r( json_decode($return) );
?>

Graph Followers

Returns an array of numeric IDs for every user the specified user is followed by.

1
2
3
4
5
6
7
<?php
$user = 'ID_TO_Use'; //ID or screen_name of the user to retrieve the followers ID list for
 
$response = $twitter->graphFollowers($format ='json', $user );
$return = $response->getData();
print_r( json_decode($return) );
?>
« Older Entries
  • Subscribe: Entries | Comments
  • About Me

    Email Email
    Twitter Twitter
    310.739.3322
  • Categories

    • Brain Dump
    • Business
    • Code
    • IT
    • Programming
    • Rant
    • Servers
  • Archives

    • February 2012
    • October 2011
    • August 2011
    • July 2011
    • June 2011
    • May 2011
    • April 2011
    • March 2011
    • February 2011
    • January 2011
    • December 2010
    • November 2010
    • October 2010
    • September 2010
    • August 2010
    • July 2010
    • June 2010
    • May 2010
    • April 2010
    • March 2010
    • February 2010
    • January 2010
    • December 2009
    • November 2009
    • October 2009
    • September 2009
    • August 2009
    • July 2009
    • June 2009
    • May 2009
    • April 2009
    • March 2009
    • February 2009
    • January 2009
    • December 2008
    • November 2008
    • October 2008
  • Advertisement

Copyright © 2008 - 2013 Eric Lamb - All rights reserved