Mike's Page - Tech

Ruby's shallow copies of hashes

nospam@example.com (Mike Boone) — Mon, 06 Jun 2011 12:31:01 -0700

When you try to copy a Ruby hash using .dup or .clone, you get what is called a "shallow" copy. The data in the hash below the first level just seems to be referenced, so if you have a hash within the hash, and try to change a value in the deeper hash, the value is changed even for the original hash you ran the .dup on. To get a full ("deep") copy of a hash, you have to run an inelegant hack using Marshal to copy it: copy_hash = (Marshal.load(Marshal.dump(source_hash))). This apparently applies to arrays too. See the code below for an example, and I hope this saves you hours of debugging.

Rails validations with accepts_nested_attributes_for and _destroy

nospam@example.com (Mike Boone) — Sat, 29 Jan 2011 19:51:04 -0800

I was recently working on a Rails app that has a form with a parent item and child items on the same form. Ryan Bates' complex form examples is a good place to start with this. That code will give you a simple form setup with some Javascript for adding and removing rows of children.

It works pretty well, except that in my case, I needed to ensure that each parent had a minimum of one child. I had a validation which checked for this, but it only worked for creating new records. If I removed all the child rows during an edit, the form would still save successfully. It turns out that the key is in the Rails documentation: "Note that the model will not be destroyed until the parent is saved." So my validation was still happily finding a child row, even though it was set to be deleted. It took me some time before I found the right way to check for this, but the marked_for_destruction? method seems to do the trick. Here's my code:

class Project < ActiveRecord::Base

  has_many :tasks
  accepts_nested_attributes_for :tasks, :reject_if => :all_blank, :allow_destroy => true

  def validate
    # require a minimum of one task
    undestroyed_task_count = 0

    tasks.each { |t| undestroyed_task_count += 1 unless t.marked_for_destruction? }

    errors.add_to_base 'There must be at least one task' if undestroyed_task_count < 1
  end

end

Hope this helps someone!

My first attempt at a mobile web app

nospam@example.com (Mike Boone) — Thu, 25 Nov 2010 12:04:27 -0800

Is this thing on? So it's been a forever since I've blogged, but we'll see if I still can. Happy Thanksgiving at least!

Anyway, I've been wanting to build stuff for my iPhone. While I've been experimenting with Objective C for a native app, I wanted to actually build and publish something. So I have abandoned my Objective C code snippets for now and decided to go with things I know and build a Ruby on Rails site. There are various libraries to let you style a site for mobile browsers. I went with jQTouch, which is covered pretty well by these Railscasts and Peepcode screencasts.

I like using the German-English dictionary site Beolingus, but I wanted to use it more easily on my iPhone. It turns out the data is downloadable, so I decided to build a web application with mobile front-end. Here it is:

Denglisch (that's a term for English and German mixed together)

I really wanted to build the site in Rails 3, but most of my stuff is hosted on Dreamhost, and I've read about various problem getting those apps running right now. So it's done in Rails 2.3.10 instead. The Rails side is pretty simple, just one model for the data, a rake script to import the textfile into the database, one controller and two actions. Then there's a little bit of jQuery Javascript code to handle the Ajax form submission.

I knew how to do a lot of these bits, but I learned things along the way too. For one, I didn't know that the iPhone could make a native-looking app out of a website. The website can include icons and a splash screen, and when you add the bookmark to your home screen, the icon will show up there. The splash screen shows up when the website is loading, and then the web app uses the full screen without the Safari toolbars. Cool. I found jQTouch to be fairly easy to use, but the download from their home page is a good bit older than the code they have on Github. I ran into an issue with the latest version that I had to look for a solution from the bug list. They don't have much documentation either. The Peepcode screencast was very helpful, but it isn't free, and jQTouch has changed a bit since it was published (e.g. the new "jqt" div).

Anyway, I've tested this on my iPhone...I'm curious to hear if it works for the iPad and other mobile devices. It was a good hobby project anyway. Have fun!

One of those Non-descriptive Errors

nospam@example.com (Mike Boone) — Wed, 17 Jun 2009 12:53:13 -0700

This one stumped me for a while today. I had an application to deploy which worked fine in development. Deploying to production led to this error:

/usr/local/lib/ruby/gems/1.8/gems/activesupport-2.0.5/lib/active_support/dependencies.rb:249:in `load_missing_constant': Expected /myapp/releases/20090617160914/app/models/widget.rb to define Widget (LoadError)

Googling that error turned up lots of pages for Rails 1.2.x and issues with underscores in names. None of it applied. Widget was definitely being defined in widget.rb. Following the stack trace it appeared that the error was thrown in the Ultrasphinx plugin. After trying a lot of other things, I decided to comment out the Ultrasphinx code in the model and retry. The error changed to this, which was helpful:

Errno::EACCES: Permission denied - /myapp/releases/20090617160914/public/widget/myfile

This model was also using the file_column plugin, and the underlying error was just a permissions problem in the file attachment target directory. I fixed the permissions and put the Ultrasphinx code back in the model, and all was happy. If only that permissions error had shown up first!

MAMP and the Ruby MySQL Gem

nospam@example.com (Mike Boone) — Thu, 29 Jan 2009 08:31:01 -0800

Update 2011-09-18: See here for the most up-to-date instructions (MAMP 1.9). I got it working on 2.0, but experienced some of the same problems other commenters noted). See also these instructions for MAMP 1.8.

After much frustration, I found a way to get the Ruby mysql Gem installed and talking to my MySQL server on MAMP. Credit goes largely to "Hootbah" who followed a similar path to get MAMP talking to Perl's MySQL library. Here's the steps I followed for MAMP 1.7.2:

Install the regular MAMP 1.7.2 dmg package.
Download the MAMP 1.7.2 source code.
Unzip the source code file.
Open the terminal and go into the source code directory.
Untar the mysql file and go into the untarred directory:
$ tar -xzvf mysql-5.0.41.tar.gz
$ cd mysql-5.0.41
This is the Hootbah magic...we're basically compiling libraries here for the Gem to link against.
$ ./configure --with-unix-socket-path=/Applications/MAMP/tmp/mysql/mysql.sock --without-server --prefix=/Applications/MAMP/Library
$ make -j2
Copy the compiled libraries into MAMP:
$ cp libmysql/.libs/*.dylib /Applications/MAMP/Library/lib/mysql
Copy the MySQL headers into MAMP...Hootbah didn't do this but I had been doing it earlier trying to get around some missing mysql.h problems, and figured I'd keep doing it:
$ mkdir /Applications/MAMP/Library/include
$ cp -R include /Applications/MAMP/Library/include/mysql
Install the Ruby MySQL Gem:
$ sudo env ARCHFLAGS="-arch i386" gem install mysql -- --with-mysql-config=/Applications/MAMP/Library/bin/mysql_config

After being used to seeing errors spit out by this statement, finally a pleasant surprise:

Building native extensions. This could take a while...
Successfully installed mysql-2.7
1 gem installed

Success!

Grepping from Ruby

nospam@example.com (Mike Boone) — Sun, 07 Dec 2008 08:30:06 -0800

I recently needed to run through one data file, pull a number, and look that number up in several other data files. So in Ruby, I first opened the initial file, pulled the number, and then opened the other files, iterating through every line until I found a match, then call break to move on. This worked, but it was sort of slow.

It occurred to me that I could use grep to see if the number was in the other files. So I had to figure out how to call it from Ruby. Here's what I came up with:

The speed improvement depends on how many searches you're doing and the size of the files you're searching, but in a quick test the grep calls were 5x faster.

'Shoulda' Tested My App

nospam@example.com (Mike Boone) — Fri, 10 Oct 2008 19:42:23 -0700

Shoulda is a handy plugin for Rails (now a Gem), which makes writing tests easier. It's worth checking out. Shoulda has been very helpful with a big Rails project I've been working on all summer, and I submitted a couple small contributions and bug fixes. I was surprised to see my name mentioned in the newest release, but I appreciate it, and it was fun helping out!

Rake and Constants

nospam@example.com (Mike Boone) — Tue, 16 Sep 2008 15:12:36 -0700

Here's a gotcha that I came across today: don't define a constant as something in one Rails rake task file, and then define the same constant as something different in another file, because the last constant will supercede them all:

a.rake:

DATA = 'Hi there!'

desc "Test A"

task :test_a do

  puts DATA

end

b.rake:
 
DATA = 'Hallo!'

desc "Test B"

task :test_b do

  puts DATA

end

$ rake test_a

Hallo!

$ rake test_b

Hallo!

I guess this happens because Rake is loading all of the task files before running the requested task. There's probably a way to namespace the constant, but I need to do more research on that. In the meantime I'll use different constant names!

Git is Fun

nospam@example.com (Mike Boone) — Thu, 22 May 2008 18:27:27 -0700

I was initially disappointed when it was announced that Ruby on Rails was moving its version control system from Subversion to Git. There's enough to keep up with in Rails that I didn't relish having to keep up with this as well. It wasn't so much that Rails was moving, but related code began to move too, so we're currently in this limbo where many plugins have both git and soon-to-be-deprecated SVN repositories, and it's just another detail to keep track of. Hopefully there will be tools to sort all this out soon.

I like my SVN GUI tools, but it turns out that git works pretty well on Windows under Cygwin, if you can live with the command line. It's certainly sufficient for cloning plugin repositories.

I could take or leave git by itself, but I love GitHub. They make it so easy to fork an existing project, make your own changes to it, and share it back with the world. Your changes can easily be pulled back into the original projects, too. I've been able to make a couple of small contributions to shoulda and acts_as_xapian.

So git is fun after all, and I'm looking forward to using it more; especially when the GUI tools show up.

Printer held Hostage by Ink Cartridge

nospam@example.com (Mike Boone) — Mon, 07 Apr 2008 17:57:03 -0700

I've had a Canon Pixma MX700 multifunction printer since the beginning of the year. I've been pretty happy with it. It's worked well for scanning, copying, faxing and printing. Until today. Today I needed to scan something, and was greeted with the following message when I turned it on:

U150 The following ink tank cannot be recognized.

The yellow cartridge was indicated. I though that it was an odd problem, but I didn't care for the moment, because I didn't want to print, just scan. But no button on the printer would bypass this error and let me scan.

I ended up calling Canon, and happily received a domestic phone rep. He had me try a couple things, but eventually decided that the yellow ink would need to be replaced. They would send me one free. I asked if there was any way to bypass the error so I could just scan, and he said there was not. If I wanted to use it today I would have to go out and buy a yellow ink cartridge. So after driving all the way across town and spending $16, I have my printer back...for now.

It seems like questionable design to cripple all the functions of the printer just because one color cartridge is faulty. A better failure mode would be to only prevent color printing. Black and white printing, and the other printer functions should still be usable. Hey Canon, how about updated firmware for this? Please?

Determining Image File Types in Ruby

nospam@example.com (Mike Boone) — Mon, 10 Mar 2008 19:22:03 -0700

Today I came across a PNG file that had been uploaded from a browser with a .JPG extension and image/jpeg MIME type. It's too bad that MIME types are apparently unreliable when it comes to file uploads. I went looking for a way to determine the file type by actually reading the file. This is probably a solved problem, but I was unsuccessful Googling for the answer. I came up with the following Ruby method which decides the image file type using up to the first 10 bytes:

def image_type(file)

  case IO.read(file, 10)

    when /^GIF8/: 'gif'

    when /^\x89PNG/: 'png'

    when /^\xff\xd8\xff\xe0\x00\x10JFIF/: 'jpg'

    when /^\xff\xd8\xff\xe1(.*){2}Exif/: 'jpg'

  else 'unknown'

  end

end

This works well on a small set of test files (400+ from a browser temp files directory). Let me know if there's a case where this code doesn't work, or if there's a better solution in general.

Update: This idea was expanded and found its way into a gem: ruby-imagespec. Credit for that goes to Brandon Anderson, Michael Sheakoski, and Dimitrij Denissenko.

I'm in Rails

nospam@example.com (Mike Boone) — Sat, 15 Dec 2007 04:55:47 -0800

I've made a code contribution to Ruby on Rails. It was minor, but I'm glad I could help. See ticket 10435 and changeset 8386.

Supposed to get some rain today...keeping my fingers crossed.

Update 2007-12-17: Looks like my patch made it into Rails 2.0.2. And we did get that rain, but more would be good.

Update 2008-03-17: I managed to get another small patch committed. Go open source!

Formatting a Javascript Date for MySQL

nospam@example.com (Mike Boone) — Sat, 08 Dec 2007 05:59:50 -0800

I don't like to code in JavaScript, but that's the Ajax-y wave of the future. So I write it when I have to. This week I needed to format a Javascript date as a MySQL-style date like 2007-12-08.

JavaScript apparently doesn't have a nice date formatting function or even a generalized sprintf. Another Javascript quirk is that the Date.getMonth() function returns 0-11, while the getDate() function returns 1-31. I'm not sure what the designer was thinking on that one.

I came up with this function...feel free to use it in your code with or without credit:

function formatDate(date1) {

  return date1.getFullYear() + '-' +

    (date1.getMonth() < 9 ? '0' : '') + (date1.getMonth()+1) + '-' +

    (date1.getDate() < 10 ? '0' : '') + date1.getDate();

}

Improving upon Rails' validates_inclusion_of

nospam@example.com (Mike Boone) — Tue, 28 Aug 2007 13:36:34 -0700

I'm fairly new to the Ruby language and the Ruby on Rails framework, but it worked nicely for quickly building my website BirdSite.

I was using validates_inclusion_of fairly regularly to make sure the data in my models was valid. But then, watching the development log, I noticed extra SQL queries being made when I accessed my main model. It seems that validates_inclusion_of was forcing the model to make queries to the related models even when I didn't need data from them.

Some Googling led to a blog post that did the validation a different way. It worked well, until I added the code to a new model today and it broke. The code was translating the model field name into a string from which to call the model itself. But the code assumed that the model would be a single word. I added a change to strip the spaces out, and it worked again.

I contacted the company that hosts the blog and they confirmed that it was a problem, and they didn't know of a nicer Ruby way to correct it short of dropping the spaces with search/replace. They updated their code.

I'm glad I could help out. It really works better than the standard validates_inclusion_of.

Rails acts_as_ferret without DRb

nospam@example.com (Mike Boone) — Tue, 14 Aug 2007 14:17:38 -0700

I wanted to add a search feature to my Ruby on Rails application BirdSite. There is a plugin called "acts_as_ferret" which allows Rails applications to use the Ferret search engine (based on Lucene).

Using this tutorial, I was able to get search up an running. It worked great on my development system. But there was a warning that you needed to run the indexer using a DRb server instead of directly from the Rails app. This is because the index cannot handle multiple processes writing to it simultaneously. (I also ran into this problem using PHP with another engine called Xapian).

I was hoping my new site was low traffic enough to avoid problems. Since I'm hosting it on a shared server, I can't run a DRb server there anyway. But I got my first indexing exceptions less than 24 hours after I loaded my stuff onto the live server.

So I decided to try periodic indexing by a cron job. This would allow me to update the index once an hour, from a single process. The downside is that the search index is only updated hourly, but I decided I could live with that.

The first step is telling your Rails app to not index the content when updating. My model already had a before_save method, so I added this code:

# in the model

def before_save

  # other stuff goes here

  # disable automatic ferret indexing...move it to a cron job

  self.disable_ferret(:always)

end

Then I had to create a rake task which would build the index:

# ferret_index.rake

desc "Updates the ferret index for the application."

task :ferret_index => [ :environment ] do | t |

  MyModel.rebuild_index

  # here I could add other model index rebuilds

  puts "Completed Ferret Index Rebuild"

end

This task is simplified: I'm telling it to rebuild the entire index each hour. I'm guessing when my dataset gets big enough, this will be really slow. In that case I'll need to track all the model instances that got updated in the past hour and just index those.

Finally, I needed a cron job to run the rake task, making sure to set the environment to "production":

cd /rails_app && rake ferret_index RAILS_ENV=production

So far this is working well, and I haven't received any indexing exceptions since.