Ruby Enterprise Edition and the MySQL gem

| |

There are a lot of moving parts in a Rails development environment these days. RVM helps us develop with exactly the version of Ruby on the production server, and allows us to keep up with the proper gemsets for each project as well. However, getting all the versions to play properly together can be a trial.

This morning we ran into some trouble installing the MySQL gem version 2.7 for a project using Ruby Enterprise Edition (REE) version 1.8.7-2011.03. The error looked like this:

ree-1.8.7@global $ env ARCHFLAGS="-arch x86_64" gem install mysql -v 2.7 -- --with-mysql-config=/usr/local/mysql/bin/mysql_config
Building native extensions.  This could take a while...
ERROR:  While executing gem ... (TypeError)
    can't dup NilClass

Searching the error didn’t provide much help, because the relevant keywords turn up all over the place, but eventually I found a page suggesting that versions of rubygems might be the issue. (More specifically, that some gems don’t play well with some versions of the rubygems binary.)

At this point I went to the #rvm IRC channel, where “telemachus” put me on track for how to change versions of rubygems within each RVM environment. So I tried downgrading from 1.6.2 to 1.5.2:

ree-1.8.7@global $ gem --version
1.6.2
ree-1.8.7@global $ rvm rubygems 1.5.2
Removing old Rubygems files...
Installing rubygems dedicated to ree-1.8.7-2011.03...
Installing rubygems for /Users/parker/.rvm/rubies/ree-1.8.7-2011.03/bin/ruby
Installation of rubygems completed successfully.
ree-1.8.7@global $ gem --version
1.5.2

And that let me install the MySQL gem successfully:

ree-1.8.7@global $ env ARCHFLAGS="-arch x86_64" gem install mysql -v 2.7 -- --with-mysql-config=/usr/local/mysql/bin/mysql_config
Building native extensions.  This could take a while...
Successfully installed mysql-2.7
1 gem installed
Installing ri documentation for mysql-2.7...
Installing RDoc documentation for mysql-2.7...

Under-Promise and Over-Deliver

| |

While most potential clients are wary of the lowest cost solution, why aren’t more wary of the vendor that says “Yes” to everything? I can’t tell you the number of times we’ve come in to clean up a broken relationship when the specs were not clear enough and the developer over-promised and under-delivered.

Our rates are professional (but reasonable, especially in relation to New York, Boston and L.A. firms), and we make a point of writing good specifications. We make sure the client knows what they’re paying for and what is “Phase 2,” we hit deadlines and we come in on budget. We’re hoping our proposal recipients see through the lowball bids and the bidders who promise the world on an unreasonable schedule – not just because we’d love to work with them, but to save them money and grey hair in the long run.

Always search the error message (Thin and Eventmachine)

| |

I kept crashing the SproutCore development server with this error:

FATAL ~ undefined method `associate_callback_target' for #<Thin::Connection:0x0000010141d438>

As usual the solution turned out to be pretty simple; there’s a known bug (albeit one without a known solution yet) with Thin talking to eventmachine 0.12.10. The solution (described at the link above) is to uninstall eventmachine 0.12.10 and reinstall it; others have reported that downgrading eventmachine to 0.12.8 helped as well.

But the big lesson is that it’s always worth doing a web search for your error message. (Even if that web search is not Google.) Pretty basic stuff for most of us, but it bears reinforcement.

Project: Happily Ever After now in print

| |

We mentioned in 2009 that we’d done some WordPress work for my former colleague Alisa Bowman on her blog, Project: Happily Ever After (or PHEA, as we called it).

The book Alisa wrote based on the experiences which spawned the blog is now in print and on sale. Congratulations to Alisa for making the jump from blog to book!

Mac OS X git client: GitX and branches

| |

I’m perfectly happy doing most of my git interaction right in the shell, but there are people who like to use a nice glossy desktop application. In the case of git, GitX offers significant advantages when it comes to surveying changes since the last commit and staging the next commit; for one example, it facilitates staging changes by chunk rather than by file, so it’s possible to make commits which ignore some changes to a file but commit others. GitX also provides a much better history-browsing experience than the command line.

When GitX hadn’t updated in a few months, however, and a Colorado developer named Nathan Kinsinger took it on himself to fork the project and add features like push/pull interaction with remote repositories and branch navigation. He claims the fork is experimental, but I’ve been using it for a week and I’m almost completely out of the command line. If you’re using GitX, you should take a look at the fork.

(Of course it’s on GitHub.)

Optimizing Apache, MySQL and PHP for content management

| |

We don’t focus on LAMP packages like WordPress or Drupal, but sometimes we pick them up in order to work with particular design partners. While the packages are generally optimized to allow deployment on shared hosting, factors from client preference to site traffic often drive our installations to VPS hosting, where the software infrastructure is not necessarily tuned to handle them. We’ve assembled a suite of tools to help us optimize the LAMP stack on a VPS.

These scripts and steps were developed tuning an Ubuntu 10.4 LTS (“lucid”) system installed at SliceHost. Generally there are two concerns: memory use (because these tools will use as much memory as they’re given, and MySQL will often throw a tantrum if it doesn’t get what it wants) and site speed.

Apache

The default Apache install slurps quite a lot of memory. The main culprit is the MPM configuration, which is where Apache decides how many instances to keep active and how much work they should each do before they shut down: basically controlling Apache’s process spawn rate. Default installations often have too many workers open by default, and spawn more too easily, thus using more memory than is really needed.

The block in /etc/apache2/apache2.conf where this is configured looks like this:

<IfModule mpm_prefork_module>
    StartServers          1
    MinSpareServers       1
    MaxSpareServers       2
    MaxClients           20
    MaxRequestsPerChild   4000
</IfModule>

The default numbers are generally much higher; this block has been tuned. However, don’t swallow these numbers either; instead, figure your own. MinSpareServers is how many processes are sitting idle; a higher number may improve site response time, but too high a number will let Apache use more memory than it needs. MaxSpareServers caps this number.

MaxClients tells Apache when to give up because it’s about to run out of memory. To figure a good setting for this, we use a script from the great folks at RimuHosting:


#!/bin/bash
echo "This is intended as a guideline only!"
if [ -e /etc/debian_version ]; then
    APACHE="apache2"
elif [ -e /etc/redhat-release ]; then
    APACHE="httpd"
fi
RSS=`ps -aylC $APACHE |grep "$APACHE" |awk '{print $8'} |sort -n |tail -n 1`
RSS=`expr $RSS / 1024`
echo "Stopping $APACHE to calculate free memory"
/etc/init.d/$APACHE stop &> /dev/null
MEM=`free -m |head -n 2 |tail -n 1 |awk '{free=($4); print free}'`
echo "Starting $APACHE again"
/etc/init.d/$APACHE start
echo "MaxClients should be around" `expr $MEM / $RSS`

MySQL

The best tool for tuning MySQL is the MySQL Performance Primer script. Download that to your home directory on the server being tuned and chmod 755 tuning-primer.sh to make it executable. You should also run which bc to see if the bc calculation language is installed; if it isn’t, try sudo apt-get install bc to get it.

This script prefers to have at least 48 hours of continuously-running server to work with. If you let it, it will produce a my.cnf configuration file with a number of suggested directive changes, generally pointed at optimizing your memory usage and MySQL performance. It’s pretty impressive.

If you’re serious about tuning your application to get fast results from MySQL, enable the Slow Query Log while you’re at it, but don’t expect this to be too helpful with packages like WordPress or Drupal, which don’t really let you get at the query builders directly.

PHP

The best thing to speed up PHP is the APC or Alternative PHP Cache. This caches PHP’s “opcodes”, the compiled versions of the PHP scripts, so they won’t be recompiled with each request.

This is a good primer for the installation on Ubuntu. One catch is that the initial APC install (the sudo pecl install apc part) often throws an error. If it fails, try sudo pecl install apc-beta and see if that works better.

Definitely install the cache status page somewhere when you’re done. It provides some interesting data about how the cache is speeding things up.

Baseline Solutions Demo

| |

We’ve mentioned Baseline Solutions before – it is a project we’re very proud of.

Baseline® is a suite of tools which allows a contract reviewer to integrate best practices and shared knowledge directly into a document. The tool is for use by anyone who reviews contracts — controllers, CFO’s, CEO’s, paralegals, contract administrators, contract managers and lawyers.

Baseline works by reading the uploaded contract and then generating a snapshot summary of the contract. With a further click of the mouse, Baseline generates a markup of the document incorporating best practices into the document in Microsoft Word track changes mode. The best practices take the form of inserted clauses, deleted clauses and hyper links to a Knowledgebase with a further explanation of the clauses.

Scott Soloway, founder and President of Baseline Solutions Corporation, has recorded a video demo if you’re interested in seeing exactly what the app does:

Simple Perl scripts for OS X users using Platypus

| |

Maybe everyone who’s ever done anything with Perl has written an email extraction script, but a recent client request asked us to take it one step further: allow the script to function as a Mac OS X “droplet.” In other words, the script should be an icon in the Finder, and when files are dragged and dropped on the icon, it should run the script.

This was easy in the days of Mac OS 9 and MacPerl; because there was no Unix core to the Mac OS, MacPerl was the only Perl environment, and the MacPerl environment took care of things like drag-and-drop file access (after all, that was the only way to provide a filename as an argument to a Perl script on a Mac.) Nowadays, with a standard Unix-y Perl shipping with the Mac OS, MacPerl is no longer needed, but the handy drag-and-drop functions aren’t there.

Filling the gap is Platypus, a handy little utility which makes application bundles out of Unix scripts. Notice I didn’t say “Perl” there. Platypus plays nicely with shell scripts, Python, PHP, Ruby, and Tcl just as easily as Perl. As the developer describes it, “this is done by wrapping the script in an application bundle directory structure along with an executable binary that runs the script.”

Advanced options allow you to configure how script output is shown, whether to allow dropped files as input (you can also filter which file types are accepted) and what to do at script conclusion.

In the case of our email extraction utility, which originally spat out the addresses it found on STDOUT, we found that displaying the script output in a text window for some reason only read one input file. We adjusted the script to write output to a file, then used the text window to show which files were read and the path to the output file (as well as any errors), and that adjustment allowed for multiple input files.

If you’ve ever wanted to double-click your scripts rather than running them from the command line, give Platypus a look.

Our graduate work still going strong

| |

While we’re tooting our own horns, an interesting link arrived at our browser this afternoon via a roundabout route. According to EduDemic, the college search site collegesurfing.com ran a feature during the Winter Olympics of “The Web 2.0 College Olympics,” running down “50 Social Media Innovators in Higher Education.”

Noah and Parker’s graduate alma mater, Tufts University, topped the “gold medal winners” list, and while the description leaned heavily on Twitter and Facebook for all contestants, both EduDemic and collegesurfing.com specifically cited Tufts’ “Spark” project. Parker was part of the original Spark team as part of his graduate work, hacking blog software and working on the XSLT bridges Spark used to tie together the project components.

Quick HTML/CSS demo

| |

People often ask us what we do when a designer sends over a PSD, so I made some screen captures from the early stages of a recent build. Next time we’ll do a longer one, right through getting all the fonts and images perfect.

Design by our friend Justin Zucco, and I’ll post a link to the live site (Balmat Law) when it goes live in a week or two.

http://www.youtube.com/watch?v=mSC84CUx_Uk