Ryan Schenk

Several weeks ago I was approached to contribute an art installation to an upcoming show at the Woods Hole Historical Museum commemorating the Woods Hole drawbridge. Our drawbridge, originally installed in the late 1930’s, was replaced with a new one this winter. The show is centered around two time lapse movies that document the entire demolition of the old bridge and construction of the new one. There is also a documentary film about the old bridge, historic photographs, and my project.

I was given the old control box to the bridge, and turned it into an interactive “virtual bridge” installation. The control box is a 40lb metal contraption, with a set of industrial buttons on the face, and a huge nest of wires on the inside. The buttons lights up, each one containing its own 120vac-to-6v step down transformer to power the bulb. Also interesting is the mechanical aspect of the button is a separate module from the electrical contacts. Normally-open or normally-closed electrical contact modules are screwed onto the back of the button module, I assume so one can replace the contacts when they wear out.

Buttons inside the Control Box

Attached to the buttons was a huge mess of wires, which ran out of the box through a conduit. When the bridge was operational, this conduit connected to some sort of logic unit that was lost during demolition. Having talked to bridge operators, and given the vintage of this control system, I suspect the logic was performed with a large bank of relays.

To create a computer input controller for the virtual bridge, my friend Dan ripped out the original wiring, and re-wired just the 120vac circuit to make all the buttons light up. Next, I got the controller to interface with the computer. To do this, I used the USB keyboard encoder hack. This is a very easy technique, accomplished by opening up a keyboard, plugging it in, and shorting pins on the keyboard encoder circuit. With a little trial-and-error, you can determine a set of combinations that reproducibly “type” specific key codes into the computer. For instance, shorting a certain two pins on the encoder might always type the letter “w” onscreen. I came up with one pin combination for each of the buttons on the control box, soldered on some wire leads, and connected them to the buttons on the control box.

USB keyboard circuit wired to the control buttons

Once I had the control box talking to the computer, the rest was done in software. The idea for the installation was simply to “create a virtual drawbridge,” so I had quite a bit of creative freedom. Initially, I was given video footage of the bridge in operation, and I experimented with using the control box to manipulate video playback. I really wanted the control box to perform realtime datamoshing on the bridge video, but ultimately this proved unfeasible and way too weird for the (elderly) population of Woods Hole.

Instead, I scrapped the video footage entirely and used Flash to create a bridge tending video game. Museum patrons – with the aid of onscreen prompts – use the real control box to operate an animated drawbridge, allowing cars to drive over and boats to sail under. The game required quite a bit of illustration, which is not really my forte, but came out looking great. While I had originally designed the game for kids, during the opening reception it was a bigger hit with old people than children! I suspect these folks have been wanting to play with that bridge controller for years, and now they finally can.

Screenshot of the Drawbridge Video Game

I have been working on version 2 of the LigerCat tool since October. Today, we flipped the switch and turned it on, at http://ligercat.ubio.org.

LigerCat is the brainchild of Dr. Neil Sarkar, and was created by me.

It’s hard to explain its use to someone outside the scientific or medicine field, but I’ll try my best to break it down.

The basic idea is that “tagging” is nothing new. The Web 2.0 folks got the idea from librarians, who have been tagging literature for many years. Librarians “tag” things using a “controlled vocabulary,” which is just a set of tags that are curated and maintained by some authoratative body. For instance, scientific articles published in life sciences field are tagged with a controlled vocabulary called Medical Subject Headings (MeSH), which has over 20,000 tags in the set.

The problem is, the journals that publish these articles are not annotated with MeSH descriptors; journals are annotated with another controlled vocabulary called Journal Subject Terms…but there’s only 120 of them! So, an article is published in a journal. This article could be tagged with any number of 20,000 MeSH descriptors. However, the journal that publishes the article is only tagged with one or more of the 120 Subject Terms.

That seems silly, doesn’t it?

That’s where LigerCat comes in. It can find every article published in a journal, and is capable of generating a tag cloud of the MeSH terms applied to every article published in a journal. The National Library of Medicine has a database of life sciences articles called PubMed, so we downloaded all the records, and built indices from that.

From a journal’s MeSH tag cloud, you can click one or more tags. LigerCat will do a live search back to PubMed to see the articles tagged with your selections.

That’s pretty cool to begin with, but there’s a lot more that we can do with the technology.

The articles tab will allow you to do a plain old search into the PubMed article database. Ligercat will download all the results, and build a MeSH tag cloud from the articles returned by your search. You can search for a topic, a person, or an organism, and LigerCat will build you a MeSH cloud based on the results.

Or you can go nuts, and click over to the Genes tab. You can paste a FASTA-formatted sequence of ATC’s and G’s into the text box. (This is a good one to try if you’re curious.) LigerCat uses an algorithm called BLAST to find all the known genes that are similar to the one you pasted. LigerCat will use its own magic to find all the articles written about those genes, then display a MeSH cloud built from all those articles. You could start with an unknown gene sequence, and in the span of a few minutes, be reading articles about that sequence. Pretty neat stuff.

For the geeks reading this, you may notice that the searches to PubMed are performed live outside the HTTP request/response cycle. I used BackgrounDRB to pull this off, and it works quite well. If and when LigerCat experiences a lot of load, BackgrounDRB will allow us to pull the background search workers off onto their own server(s), allowing us to isolate the computationally intensive part of the application away from the app server.

Pretty easy

yum install mysql-devel
gem install mysql -- --with-mysql-config=/usr/bin/mysql_config --with-mysql-lib=/usr/lib/mysql

I have just gone through hell attempting to install ImageMagick, RMagick2 on our CentOS5 server, and thought others might find this information useful.

It should have been as easy as

yum install ImageMagick
gem install rmagick

But unfortunately it wasn’t. The reason is because yum will install ImageMagick 6.2.8.0, while RMagick2 requires ImageMagick 6.3.0 or higher. Fantastic. My sysadmin friend says, “It’s what is in the distribution, to keep the package base consistent.” I say, “But I need ImageMagick 6.3.0+ and this is a royal pain in the neck.”

These are the steps I took to successfully install ImageMagick 6.4.3 and RMagic2 on CentOS5. It took me an entire morning to figure this out (I am not a Linux expert), but with this guide it should take you 15 minutes or less.

0. Tell Yum to Get a Grip

I wasn’t able to install any of the following RPMs, because Yum had a gripper about them being “unsigned.” You need to tell Yum otherwise. Open up /etc/yum.conf

vi /etc/yum.conf

At the very end of the file, add the following line

gpgcheck=0

That will tell Yum to install packages, regardless if they are “unsigned.”

WARNING: There’s probably a reason why gpgcheck exists. Do this at your own risk, and be sure to flip gpgcheck back to 1 when you are done.

1. Install a couple dependencies for ImageMagick

ImageMagick 6.4.3 requires two dependencies that were not in yum, libjasper.so.1 and libdjvulibre.so.15. Using http://rpm.pbone.net/, I found the RPMs that contained these files:
jasper-libs-1.900.1-7.el5.kb.i386.rpm and djvulibre-3.5.19-4.el5.kb.i386.rpm

wget ftp://ftp.pbone.net/mirror/centos.karan.org/el5/extras/testing/i386/RPMS/jasper-libs-1.900.1-7.el5.kb.i386.rpm
wget http://centos.karan.org/el5/extras/testing/i386/RPMS/djvulibre-3.5.19-4.el5.kb.i386.rpm
 
yum install jasper-libs-1.900.1-7.el5.kb.i386.rpm
yum install djvulibre-3.5.19-4.el5.kb.i386.rpm

2. Install ImageMagick and Various Development Packages

Next, we need to install ImageMagick. To simply install ImageMagick, it’s a single package. However, to make RMagick happy, we also need to install several development packages.

wget ftp://ftp.imagemagick.org/pub/ImageMagick/linux/fedora/i386/ImageMagick-6.4.3-6.i386.rpm
wget ftp://ftp.imagemagick.org/pub/ImageMagick/linux/fedora/i386/ImageMagick-devel-6.4.3-6.i386.rpm
wget ftp://ftp.imagemagick.org/pub/ImageMagick/linux/fedora/i386/ImageMagick-c++-6.4.3-6.i386.rpm
wget ftp://ftp.imagemagick.org/pub/ImageMagick/linux/fedora/i386/ImageMagick-c++-devel-6.4.3-6.i386.rpm
 
yum install ImageMagick-6.4.3-6.i386.rpm
yum install ImageMagick-devel-6.4.3-6.i386.rpm
yum install ImageMagick-c++-6.4.3-6.i386.rpm
yum install ImageMagick-c++-devel-6.4.3-6.i386.rpm

When you install the first ImageMagick package, yum should do its magic and install a bunch of dependencies, such as ghostscript, lcms, and others.

3. Install RMagick, finally

With all those RPM packages installed, you can install RMagick. If all the ImageMagick stuff is there, this should go smoothly.

gem install rmagick

4. Undo the Change to yum.conf

It’s probably wise to remove or comment-out the modification you made to yum.conf

# gpgcheck=0

When displaying certain elements as inline, the whitespace between tags does matter in your layout.

When trying to display some LI tags as inline, I really needed to spit out the tags with no whitespace in between. Unfortunately HAMLs handy-dandy tag indenting was wrecking me. It was so bad, that I resorted to calling a helper method that spat out the HTML as a string, with no whitespace between the tags.

But today, I discovered that if you append the HAML ‘tag’ with a “>“, it will not add any whitespace to the tag.

Example:

%ul.filter_navigation.tabnav
  %li.go.first>= link_to_function 'Gene Ontology', 'Filter.select("go")'
  %li.age>=      link_to_function 'Longevity',     'Filter.select("age")'
  %li.gaz.last>= link_to_function 'Gaz Ontology',  'Filter.select("gaz")'

Results in this HTML:

<ul class='filter_navigation tabnav'><li class="go first"><a onclick='Filter.select("go"); return false;' href="#">Gene Ontology</a></li><li class="age"><a onclick='Filter.select("age"); return false;' href="#">Longevity</a></li><li class="gaz last"><a onclick='Filter.select("gaz"); return false;' href="#">Gaz Ontology</a></li></ul>

No Whitespace! You got a date Wednesday baby!

If you have an AR Model whose table name does not follow the convention, and you have needed to set it explicitly using set_table_name.

class ChiliDog < ActiveRecord::Base
  set_table_name 'dogs_made_of_chili'
end

Then your test fixtures file must be named the same name as the table, not the model…

dogs_made_of_chili.yml

In your rspec, you must use the table name to call your fixtures, then specify the AR class that models it…

fixtures :dogs_made_of_chili
set_fixture_class :dogs_made_of_chili => 'ChiliDog'

Note that little guy set_fixture_class!! He does not appear in the online Rails API docs (at least not Rails Brain), so I had originally assumed that you would pass in a Constant, not a String to it. I was wrong!!

The class name given to set_fixture_class must be a String and not a Constant like you might assume/think/wish-for/hope/desire. If you put a constant in there, you will get all sorts of weird, will bang your head against your desk repeatedly, pull your hair, tear your clothes, pour ashes on your head, and various other Old-Testament-style lamentations.

So, word of warning, use a String, not a Constant when using set_fixture_class!

I have an ActiveRecord class called JournalQuery that performs a large webservice call out to the National Library of Medicine, then caches the result in our local database. Oftentimes, this call can take well over a minute to run, which is not good for the HTTP request/response cycle. In the Rails applicatin got around this limitation of HTTP by using an after_create callback to fork this web-services call into a background process using the library BackgrounDRB. My Rails app creates a new JournalQuery, and after it is saved to the database, the after_create callback launches the BackgrounDRB worker transparently.

That’s fine and dandy for the Request-Response cycle, but right now I need to write a Rake task to pre-compute and pre-cache a bunch of canned JournalQueries. I do not want to deal with the overhead and complications of forking into a background process; it’s totally overkill and unnecessary, since this Rake task is going to be fired by a cronjob, and cron obviously has no request-response cycle!

So, what about that callback? How can I avoid tripping it in my Rake task, without changing the code in my Model class?

The Easy but Bad Solution:

I could simply Monkey-Patch my Model class, redefining my callback function to return true, instead of forking off the background process. Monkey patching is the process of writing code that, at runtime, opens up a class, and redefines methods inside that class. Yes, Ruby lets you do that, and yes, it is often as terrible an idea as it sounds (although sometimes extremely handy).

I didn’t want to do this, because this egregious a monkey patch would probably lead to unforeseen and extremely weird behavior at some point down the line. Then I got to thinking…what if I saved the code inside the original callback, applied said egregious monkeypatch, then monkeypatched my monkeypatch with the original callback, thus restoring harmony to the universe?

The Elegant Solution That Took Two Hours Instead of Two Minutes:

…And what if I did it all inside a class method, that accepted a block? Yeah baby! So I wrote this method, which is sitting at the bottom of this post. The method is called allow_skipped_callbacks_on, and accepts a Ruby Constant (assumed to be the name of a Class that you want to violate). This method opens up the victim class with a class_eval, and defines a class method called without_callback. Without_callback accepts a symbol, which is the name of your callback function, and a code block; without_callback saves the original callback method into a local variable, overwrites it with a stub, then executes the code in your block, and finally undoes all the changes it made to your poor little class.

Example:

Let’s say you have an ActiveRecord class called MyTable, and MyTable has a before_create callback method named my_callback. Now let’s say I want to populate the my_table database table with some data, and do not want my_callback to fire. Here’s how to do it

allow_skipped_callbacks_on MyTable
 
MyTable.without_callback(:my_callback) do
 100.times{ |n| MyTable.create(:name => "my_table_#{n}") }
end

Pretty cool huh? The code for allow_skipped_callbacks_on and without_callback is below:

Code

def allow_skipped_callbacks_on(model)
  model.class_eval do 
    def self.without_callback(method_name, &block)
      raise "#{self.name}##{method_name} doesn't exist, Hosehead" unless method_defined?(method_name)
      original_callback = instance_method(method_name)
      remove_method(method_name)
      define_method(method_name){ true }
      begin
        yield
      ensure
        remove_method(method_name)
        define_method(method_name, original_callback)
      end
    end
  end
end

Ya ever notice how when you do page or action caching in Rails, it ignores the GET query parameters?

Sup with that?

The REST bible says you should include inputs to algorithmic resources as GET query params. Unfortunately, this means that Rails will ignore the inputs to your algorithmic resources, and you can’t cache them. Not so good if you’ve got a big honkin’ algorithm, for instance Gruff sitting on a URL like this:
/species-discoveries/94/graph.png?years=1750-2000

So what do you do? Well, in the words of Rowdy Roddy Piper , “Either you put on these glasses, or start eatin’ that trash can!”¹.

While I couldn’t figure out how to modify page caching, and perhaps there isn’t a way I did figure out how to modify action caching to include GET params. It was, in fact, quite trivial. All you need to do is monkey patch ActionController::Caching::Actions::ActionCachePath.initialize. You can simply stick the code at the bottom of this post in your lib, and then require ‘action_cache‘ in your environment.

I have NO idea if there’s a good reason why Rails doesn’t cache get parameters. Perhaps there is, but for now, this is the best I’ve got. I’m sure this could be tweaked further. For instance, you might be able to subclass ActionCachePath with one specific for your algorithmic resources — in the example above, I could write a custom ActionCachePath that only allowed the years parameter through, because that’s the only one we care about.

module ActionController
  module Caching
    module Actions
      class ActionCachePath      
        def initialize(controller, options = {})
          @extension = extract_extension(controller.request.path)
          # path = controller.url_for(options).split('://').last
          host = controller.url_for(options).split('://').last.split('/').first
          path = host + controller.request.path + controller.request.query_string # Booya!
          normalize!(path)
          add_extension!(path, @extension)
          @path = URI.unescape(path)
        end
      end
    end
  end
end

1. http://www.youtube.com/watch?v=wqKFadyJxwg [↩]