geekQ tech blog

Add column safely in MySQL and other idempotent db migrations

2010-08-11T00:00:00+02:00

Add column safely in MySQL and other idempotent db migrations

With all the NoSQL hype nowadays this post regarding MySQL probably appears a bit archaic. But this is what kept me busy last two days.

In our current Rails project we agreed with admins to use DDL and SQL scripts to implement the db structure- (and sometimes db content-) adjustments.

Context

as little downtime as possible and sometimes even less is required
if off time is approved, then such upgrade is done in the night
developers have no access to production system

Advantages of SQL

Using hand crafted DDL and SQL scripts gives following advantages:

our admins have better feeling while being able to run single statements from the migration script and recover in trivial cases
when data changes, e.g. normalization/denormalization needed, the SQL statements are much faster than Rails code involving creation of huge amounts of Ruby objects

MySQL solution

I also wanted to make the scripts idempotent so in case there were some problems and the structure/data conversion run only partially it would be possible to do some changes and rerun the whole script. It should be easy, I thought, just prepend every change statement with an if. With Oracle or Microsoft SQL Server it is just a matter of seconds.

In fact, there is an IF NOT EXISTS clause for CREATE TABLE in MySQL. But unfortunately there is no one for ALTER TABLE ADD COLUMN.

Searching the internet does not help in case of MySQL. The official documentation is insufficient and the signal to noise ratio in MySQL relatet forums is lower, than for every other technology I’ve ever seen. Even stackoverflow contains unprecise or unhelpful.

It took me some time to explore all the ways and byways of MySQL programming. The problems I had to deal with:

IF statements only work in stored procedures, not when run directly, e.g. in mysql client
more elegant and concise SHOW COLUMNS does not work in stored procedure
the syntax for delimiting statements is strange in MySQL, so you have to redefine the delimiter to be able to create stored procedures. Do not forget to switch the delimiter back!
INFORMATIONAL_SCHEMA is global for all databases, do not forget to filter on TABLE_SCHEMA=DATABASE(). DATABASE() returns the name of the currently selected database.

Here is the working solution (tried out on MySQL 5.0 for Solaris):

DELIMITER $$

DROP PROCEDURE IF EXISTS upgrade_database_1_0_to_2_0 $$
CREATE PROCEDURE upgrade_database_1_0_to_2_0()
BEGIN

-- rename a table safely
IF NOT EXISTS( (SELECT * FROM information_schema.COLUMNS WHERE TABLE_SCHEMA=DATABASE()
        AND TABLE_NAME='my_old_table_name') ) THEN
    RENAME TABLE 
        my_old_table_name TO my_new_table_name,
END IF;
 
-- add a column safely
IF NOT EXISTS( (SELECT * FROM information_schema.COLUMNS WHERE TABLE_SCHEMA=DATABASE()
        AND COLUMN_NAME='my_additional_column' AND TABLE_NAME='my_table_name') ) THEN
    ALTER TABLE my_table_name ADD my_additional_column varchar(2048) NOT NULL DEFAULT '';
END IF;

END $$

CALL upgrade_database_1_0_to_2_0() $$

DELIMITER ;

workflow 0.4 gem released

2010-06-27T00:00:00+02:00

workflow 0.4 gem released

Changes in this release:

completely rewritten the documentation to match my branch
switch to jeweler for building gems
use gemcutter for gem distribution
every described feature is backed up by an automated test

The documentation now lives on my personal site. Your own copy will be generated locally on gem install workflow.

Vim as External Editor for Thunderbird 3 under Ubuntu

2010-05-13T00:00:00+02:00

Vim as External Editor for Thunderbird 3 under Ubuntu

Recently I upgraded to Ubuntu 10.04 Lucid Lynx which includes the newest Thunderbird 3.0 email application. This version is really nice: offers tabbed interface for reading emails (unfortunately composing a new email does not work in a tab) and a very powerful search.

But as always with the bleeding edge version, all add-ons stopped to work. Because I had to reconfigure everything anyway and find new ways for doing usual things like signatures wanted, at this occasion I decided to finally set up the vim as external editor for new messages.

There a lot of howtos out there, none of them worked for me though and it took me some time to figure out, how to get the stuff working. So here is my documentation…

Assumptions:

Ubuntu 10.04 Lucid Lynx
hence Thunderbird 3.0
and gnome-terminal as the terminal of choice
vim 7.0, not gvim

Solution:

install the External Editor 0.8.0 Add-on from authors site
in Thunderbird menu Tools > Add-Ons > Preferences for “External Editor” set Text Editor to gnome-terminal --geometry=110x40 --disable-factory -x vim
create a new message with Ctrl+N
press Ctrl+E to edit the message in vim
:wq to save the message, then the Thunderbird GUI is automatically updated with the new content

Parameter explanation:

I use --geometry to make the editing window a bit bigger than default
--disable-factory ensures, that the call does not return immediately but waits until you have edited and saved the message
-x vim and not -e vim is also important because the Add-on seems to simply append the parameters to the end of the line and to get the temporary file name properly propagated to vim it is crucial that all the remaining command line is interpreted as a single shell command

Signatures

Using a powerful and comfortable editor solves lot of other problems - for signatures, for example, I can now use vim abbreviations. Just put following in the .vimrc file

iab brr
  \<CR>Best Regards,
  \<CR>
  \<CR>Vladimir

Now I can type brr<Enter> to get the complete signature. You can also easily create a macro which extracts the first name of the recipient and puts a greeting together.

Fix keyboard shortcuts

One of annoyances of Thunderbird 3.0 is that the keyboard shortcuts changed under linux. “J” and “K” keys were traditionally used to go to the next or previous message in the email message overview (like in vim). And now “J” means “Mark as junk and delete”!? WTF!? It took me three deleted messages till I got, what happened. You can reconfigure shortcuts with the keyconfig addon though.

Just write an email

Sometimes while in the middle of doing something I remember that I have to tell X to person Y. So just switch to the email application to write new message. Oh, there are new emails in the inbox! So just skim over the subjects! Oh, this one looks interesting/important, lets check what is inside… and the context is lost.

Better solution is to bind thunderbird -compose to a global keyboard shortcut. This opens a window just for composing an email without the distracting main email overview window.

Fix pragprog ebook fonts for source code listings

2010-02-10T00:00:00+01:00

Fix pragprog ebook fonts for source code listings

Pragmatic programmers are pioneers of the ebook technology and have offered their books in both PDF and DRM free EPUB format for a long time.

Now I can carry all my ebooks with me on my Sony PRS-505 ebook reader and read whatever I like while on commute. The eInk display is easy on eyes. Using an iPhone or iPad with their high glare displays for reading books is a joke compared to eInk!

There is one shortcoming regarding books containing lots of source code though. Pragprogs use the default monospaced font. And the default monospaced font on Sony is unfortunately the owful Courier New - very wide and very ugly. So people at Sony are as clueless regarding typography as their counterparts at Apple. On PRS-505 some source code is cut off on the right side and what remains is very hard to read - so it is impossible to learn from this source code listings.

It is easy to fix though.

An EPUB file is simply a zip archive with

the book text in html format
some additional meta data like table of contents in XML files
CSS

all standardized by International Digital Publishing Forum.

So we are going to:

unpack the EPUB archive
adjust the style sheet
pack the archive again
and transfer the newly created file to the reader

In addition we are going to install nicer additional fonts on the reader. Inconsolata and DejaVuSans are free fonts and perfect for displaying source code listings. The condensed version of DejaVuSans allows for even more characters per line.

Just grab my Rakefile and stylesheet on github, adjust as needed and enjoy!

Here is the result - very nice source code listing, isn’t it?

HTML partials in Sinatra

2009-12-22T00:00:00+01:00

HTML partials in Sinatra

I was looking for a way to generate an HTML partial (e.g. for web site navigation) within a Sinatra application. Sinatra itself does not have any opinion regarding partials. There are some add-ons out there e.g. sinatra_more aiming for Rails-like helpers content_tag, input_tag etc.

But it seems more natural to me to directly use an HTML/XML library. If I am referencing Nokogiri anyway (Nokogiri is a popular ruby wrapper around the libxml XML library), why not just use that?

builder ||= Nokogiri::HTML::Builder.new do |doc|
  doc.ul {
    doc.li 'hello'
    doc.li 'admin', :class => 'current' if current_user.is_admin?
  }
end
puts builder.doc.inner_html

produces

<ul>
<li>hello</li>
<li class="current">admin</li>
</ul>

We do not need any plug-ins and add-ons, it looks easy, does not it? Here is a more elaborated example:

I’ve also done some performance checking:

Results:

                  total for 1000 loops, in milliseconds
Nokogiri, singleton builder     0.090000 
Nokogiri, recreate builder      0.270000 
HAML, singleton engine          0.170000 
HAML, recreate engine           1.220000

So HAML is significantly slower than xmllib, who could imagine that? And caching (evil) can make things faster, and certainly more messy. So I have to think about a helper for helpers that would help you organize your helpers and hide the caching and Nokogiri builder instantiation from you, without hiding too much and introducing leaky abstractions. Stay tuned!

Minimalist testing for Ruby applications

2009-11-25T00:00:00+01:00

Minimalist testing for Ruby applications

Last four months I have been developing web and backend applications for fun and profit without Rails - using the light weight Sinatra framework. Besides technical solutions there is also an important philosophical attitude I have learned in that Sinatra community.

In opposite to Rails they completely decline the cargo cult (also mentioned by Koz in a slightly different context) where one writes a line or two, he has seen in the README of the plugin and expects something magical to happen. And often it does not happen or happens in an unexpected way.

By contrast, in Sinatra community, if one develops a chunk of functionality, he thinks can be helpful for others, instead of creating a plugin or gem one publishes a gist at github so others can copy, paste and use it as inspiration for implementing the 5 percent of functionality they really need.

I found out that for me the RSpec approach does not work. The tests should give me more confidence, that everything works, especially for refactoring. Through all the magic and extensive metaprogramming RSpec creates more uncertainty for me instead.

Test::Unit is on the other side a bit “underkill” but it is easy to extended it with features I am missing:

use full sentences to name the tests
declare tests as pending or non-critical
enable the same level of abstraction in the test implementation and provide an overview of a test

If folded in the editor, a test looks like following, providing an overview of what is going on in the test:

-class SignupTest < FunctionalTestCase 

-  test 'registration workflow' do
+    prepare 'a customer with wrong bank details' do
+    expect 'errors only after the offline check' do
+    expect 'update should clean errors and bank details check identifier' do
   end
 end

This single test steps can be also printed out during the test execution if you set the environment variable VERBOSE_TEST=true.

For comparision here is the full source code of the integration test:

class SignupTest < FunctionalTestCase 

  test 'registration workflow' do
    prepare 'a customer with wrong bank details' do
      put '/customers/inva', create_customer_xml(
        'account' => '123456',
        'bank-number' => '36010043'
      ).to_xml, {'HTTP_X_ON_BEHALF_OF' => 'functional test'}
      assert_equal 202, last_response.status
    end

    expect 'errors only after the offline check' do
      assert_equal 0, Customer.find_by_customer_id('inva').customer_errors.size
      
      Customer.check_and_promote_all

      a = Customer.find_by_customer_id('inva')
      assert_equal 1, a.customer_errors.size
      assert_match /invalid/, a.customer_errors.first.message
    end
    
    expect 'update should clean errors and bank details check identifier' do
      get '/customer/inva', {}, {'HTTP_X_ON_BEHALF_OF' => 'functional test'}

      new_doc = response_doc()
      new_doc.select_single_node('//bank-name').set_text('Other Bank')
      put '/customer/inva', 
        new_doc.to_xml, 
        {'HTTP_X_ON_BEHALF_OF' => 'functional test'}
      a = customer.find_by_customer_id('inva')
      assert_equal 202, last_response.status
      assert_equal 0, a.customer_errors.size
      assert a.schufa_request_identifier.blank?, 'request identifier is not cleaned.'
    end

  end
end

So my testing infrastructure mainly consists of

the excellent Rack::Test facility, is also standard for new Rails versions
my Test::Unit extension, grab it on github
my project-specific test_helper, which can programatically create entities filled with test data and contains project-specific assertion helpers

Deploy JRuby web application with jetty-rackup

2009-09-18T00:00:00+02:00

Deploy JRuby web application with jetty-rackup

I already reported about our problems with different Ruby libraries. Bad library support is the main cause, why in our current project we decided to switch from MRI to JRuby for all the new application development.

The tradition in the JRuby world is to do it the same way it is done in Java:

create a lengthy configuration in WEB-INF/web.xml
package everything to jars and wars
throw it in a servlet container - tomcat, jetty or glassfish

and all the howtos and tutorials assume that way.

The Ruby way, in my opinion, is to use Rack and a rackup script. So I wanted to “invert the control” and embed a Java web server inside my Ruby script. That way is also supported by the Java’s light weight web server jetty.

So I ported rackup and adapted it for the usage with jetty. You can install it from github. Now you can take your existing, say Sinatra, application and reuse your existing config.ru

Enjoy!

Switching to JRuby

2009-09-17T00:00:00+02:00

Switching to JRuby

Currently I am working on an “enterprise-y” integration solution, where we offer a RESTful interface making heavy usage of HTTP and XML on the implementation side. While suitable for HelloWorld and blogging applications, all the standard Ruby libraries simply do not work in the corner cases:

XML builder generates illegal XML
REXML - occasionally the stack level is too deep :O
Net::HTTP can not proper handle HTTP 1.1 and “100 Continue”
hpricot - tag soup parsing and creation

While being thankful to all the great hackers, who developed the mentioned libraries, partly in their personal spare time, it is simply too much pain to use them in the real world application, that has to 100% work.

And at this point of time we would also like to get rid of ImageMagick.

The JRuby is pretty mature nowadays and promises to eliminate debugging in mentioned areas thanks to java libraries, that evidently work. For years.

That is why we decided to switch to JRuby for all the new application development. For automation tasks (organizing my life with Rake) the MRI still remains the platform of choice, especially due to its incredibly short start up time - 12ms on my machine.

P.S. Just for completeness: you could use libxml and libcurl (known 100% working solutions written in C) with their respective Ruby wrappers, but

I found the quality of libcurl wrappers not convincing
searching for cross-platform compiling issues (production servers are Sun Solaris based) does not belong to our hobbies
we wanted to solve all our problems at once. ;-) You know, the silver bullet.

workflow 0.3 gem released

2009-08-13T00:00:00+02:00

workflow 0.3 gem released

Better error messages

Helpful error messages is one of the hardest thing to achieve. Thanks to the workflow user Kyle Burton for the proposal regarding transitions to an illegal state! Now if you have not defined e.g. solved event but try to transition to it, you will get more meaningful message like Event[solve]'s transitions_to[solved] is not a declared state. and immediately know, what to fix.

Separation of concerns

Intermixing of transition graph definition (states, transitions) on the one side and implementation of the actions on the other side for a bigger state machine can introduce clutter.

To reduce this clutter it is now possible to use state entry- and exit- hooks defined through a naming convention. For example, if there is a state :pending, then instead of using a block:

state :pending do
  on_entry do
    # your implementation here
  end
end

you can hook in by defining method

def on_pending_exit(new_state, event, *args)
  # your implementation here
end

anywhere in your class. You can also use a simpler function signature like def on_pending_exit(*args) if your are not interested in arguments. Please note: def on_pending_exit() with an empty list would not work.

If both a function with a name according to naming convention and the on_entry/on_exit block are given, then only on_entry/on_exit block is used.

Have fun!

You can easily install the newest version with

sudo gem install workflow

Editing a LISP program on Ubuntu with vim

2009-07-13T00:00:00+02:00

Editing a LISP program on Ubuntu with vim

Trying to fill the gap in my computer science education I started to learn LISP again.

I am using the SICP “Structure and Interpretation of Computer Programs, 2nd Edition” book from MIT, that is used to be the book for the first semester students (!) to learn programming.

So I wanted to use the mit-scheme - a LISP interpreter written by the authors of the book. Unfortunately, there is no precompiled version for the 64 bit version of Ubuntu 8.10 Intrepid. But so far the scheme48 seems to be compatible to the examples from the book.

sudo aptitude install scheme48

The command line of the scheme48 is not very convenient so I ended just copying and pasting snippets from the editor into the interpreter window.

All the trolls say, the only way to edit and run LISP programs is to use emacs. There is a working automation solution for vim though - VIlisp.vim. It took me some time to get it work - here is my howto for Ubuntu 8.10 Intrepid:

1. Download and extract the VIlisp.vim

mkdir ~/.vim/lisp
tar -C ~/.vim/lisp --strip-components 1 -xf vilisp.2.3.tgz

2. Install the required Perl modules

sudo aptitude install libterm-readline-perl-perl \
  libterm-readline-gnu-perl libenv-ps1-perl

In addition I had to install the Pty package through the Perl package-maangement CPAN

sudo cpan
cpan> install IO::Pty
cpan> exit

3. Now we can run VIlisp which connects two windows

In the scheme interpreter window:

export PERL_RL=gnu
perl funnel.pl $HOME/.lisp-pipe scheme48

I’ve created an alias for that

alias lisp='PERL_RL=gnu perl $HOME/.vim/lisp/funnel.pl $HOME/.lisp-pipe scheme48'

so I can just type lisp instead.

In the editor window:

vim exercise1_3.scm

In vim:

:so ~/.vim/lisp/VIlisp.vim

or put something like following to your .vimrc file:

autocmd BufRead,BufNewFile *.lsp,*.lisp,*.scm so ~/.vim/lisp/VIlisp.vim

so the functionality is loaded automatically for every scheme/lisp source file.

4. Now you can use shortcuts described in the README of VIlisp

\ec to evaluate current s-expression
\eb sends current visual block to the interpreter

RailsWayCon Berlin

2009-05-27T00:00:00+02:00

RailsWayCon Berlin

The RailsWayCon in Berlin is over. All the talks I’ve seen were extremely interesting. Here are my personal highlights:

It seems that Ola Bini can create a new programming language effortlessly. On the examples of JRuby and Ioke he offered a very interesting look behind the scenes and showed the way from the AST (abstract syntax tree) via the runtime environment (frames, bindings) to the interpretation and compilation. He told that Ruby is especially hard to implement because of its very flexible syntax (little parentheses and semicolons, lot of whitespace) and other flexibility - eval, closures, the fact, that many functions accept an explicit binding object. The harder it is to implement Ruby, the better it is for developers, that use Ruby: they get a very powerful tool and can write very terse and at the same time very expressive source code.

Neal Ford showed, that naming the things in a particular way immediately makes them sound more respectable. By introducing of terms “cut points”, “advice” instead of “monkey patching” everything sounds like advanced design patterns and not anymore as a hackery.

I found the discussions with members of the Rails core team between the sessions and in the evening very helpful. In that environment even previously rejected patch can take a shortcut into the main repository.

P.S. I gave a talk on metaprogramming:

Metaprogramming considered harmful

…not all kinds of it, of course. People just discovered the power of this technique extensively use it, both in cases where it’s a good match, but unfortunately also where it creates a lot of unexpected problems. With a selection of practical ‘Metaprogramming gone wild’ examples I would like to show how less and more appropriate metaprogramming leads to simpler and better maintainable code.

rubygems 1.3.3 released

2009-05-25T00:00:00+02:00

rubygems 1.3.3 released

Some time ago I modified the gem server to provide a convinient search for the documentation of locally installed gems.

Now it is part of the official rubygems 1.3.3 release. Just update your rubygems with

sudo gem update --system

After you restart the gem server you can see the new search box in the upper right corner of the page. You can type in a part of the gem name and after pressing the enter key you will be immediately redirected to the rdoc of the desired gem or presented with a list of matching gems.

You can also define a custom search word in your browser (at least Firefox and Opera support this). I’ve connected shortcut:rdoc to http://localhost:8808/rdoc?q=%s. Now I can type ‘rdoc hp’ to jump directly to the documentation of the hpricot gem.

And if you use the sdoc for generating your local Ruby documentation you can immediately continue with typing the name of then method. That way I typically need only about two seconds to get to the documentation of the desired method:

Ctrl+T in Firefox for a new tab
‘rdoc core’, press ENTER
‘insteval’, press ENTER and I can read the documentation for the Object#instance_eval method.

Düsseldorf on Rails with Gregg Pollack

2009-04-23T00:00:00+02:00

Düsseldorf on Rails with Gregg Pollack

Yesterday we had Gregg Pollack from Rails Envy as a guest speaker at our small local user group in Düsseldorf.

His Rails Envy webcasts are very popular. But to see and to hear his presentations live is even better!

He talked about recent and upcoming changes in Rails and about other Rails 3 related stuff:

modularization of Rails and how it should improve the competition between different O/R mappers, support for different JavaScript libraries etc.
introducing Rack and Metal to Rails and scenarios, where performance can be significantly improved by introducing additional middleware like Rack::Cache
reverse HTTP proxies and how it works (or does not work) in diffrent scenarios

He also talked about implementation details:

how more consious definition of the Rails public API promises to decrease the pain caused by a typical switch to a newer minor version of Rails
how less and less invasive meta programming and revival of object oriented techniques improves both the performance and the readability of Rails internal implementation (also my favorite topic)

All in once it was a great fun!

Rails internationalization - removing validation message prefix

2009-04-09T00:00:00+02:00

Rails internationalization - removing validation message prefix

Current ActiveRecord implementation always prepends an error message with the attribute name. This is because of design weakness in the handling of human messages in Rails.

The data should be always interpolated into the message, string concatenation should be never used.

Good:

MY_MESSAGE = '%{attribute} is invalid'
return MY_MESSAGE % {:attribute => 'email'}

Bad:

MY_MESSAGE = 'is invalid'
return MY_MESSAGE + ' ' + 'email'

… because the correct order of the words in a sentence depends on language.

Generally it is not possible to create a message in human language by string concatenation. Only interpolation universally works!

Still bad (from validations.rb):

attr_name + I18n.t('activerecord.errors.format.separator', 
  :default => ' ') + message

I18n.t would not help, if you use +.

Our custom message ‘Gadget should have at least one screenshot.’ is corrupted by a prefix to ‘Screenshots Gadget should have at least one screenshot.’

I hope one day it can be fixed in Rails by substantially refactoring the ActiveRecord validations and internationalization subsystem. For now I’m fixing it with a dash of metaprogramming and monkey patching.

module ActiveRecord
  class Errors

    # allow a proc as a user defined message
    def add(attribute, message = nil, options = {})
      if options[:default].is_a?(Proc) # this is new
        message = options[:default].call(attribute)
        def message.full? do
          true
        end
      end
      message ||= :invalid
      message = generate_message(attribute, message, options) if 
        message.is_a?(Symbol)
      @errors[attribute.to_s] ||= []
      @errors[attribute.to_s] << message
    end

    def full_messages(options = {})
      full_messages = []

      @errors.each_key do |attr|
        @errors[attr].each do |message|
          next unless message

          if attr == "base"
            full_messages << message
          else
            attr_name = @base.class.human_attribute_name(attr)
            if message.respond_to? :full? and message.full? # new switch
              full_messages << message
            else
              # messages for humans based on string concatenation are 
              # inherently broken - should not be used!
              full_messages << attr_name + 
                I18n.t('activerecord.errors.format.separator', 
                  :default => ' ') + message
            end
          end
        end
      end
      full_messages
    end

  end
end

Explanation:

copy a paste the code into a initializer (underneath config/initializers folder)
all translated messages created by a proc get the special full? tag which is later used in full_messages to bypass the attr_name addition
you can use attribute name in your proc to interpolate into the localized error message

This dirty hack is only needed because of design weekness of rails. Some conceptial thoughts:

Short and full error messages

Distinction between short and full error message does not make sence

one can not be universarly infered from another for every human language
specifing both manually hardly makes sense So there should be just error message with optional placeholder for the attribute name.

String interpolation everywhere

The data should be always interpolated into the error messages, never concatenated, because the order of the words in a sentence and gramatical rules differs from language to language.

Non-programmer capable translation files

YAML files and ruby symbol driven translation keys are unsuitable for non-programmers and people without deep knowledge of the application to be translated.

See GNU gettext for a deeper explanation of internationalization concepts that work!

ActiveRecord validations and internationalization

2009-04-08T00:00:00+02:00

ActiveRecord validations and internationalization

We recently switched to Rails 2.3 in our project. This version offers a new internationalisation system, which is a clear step forward, but its yaml based translation file storage and symbol driven message selection are unfortunately unsuitable for the real enterprise environment - handling translation files and separation of roles of developers and translators. The latter are driven by each country’s very own marketing department.

So we are going to continue using Gettext for localization. The old Masao’s library we have been using before is not compatible with current Rails. So after some evaluation we desided to use fast-gettext library as a foundation for our development, which was created by Michael Grosser.

The Rails 2.3 release feels rough and the transition to the new internationalization system unfinished. You can see different generations of code in validations.rb that do not really fit together.

There are still problems with custom validation messages. The build-in backend with symbol-based message keys is not an option for us or any other big-scale development environment (role separation etc., s. above). And ActiveRecord provides no other API for custom translated messages.

validates_length_of :archives, :minimum => 1,
  :message => _('Gadget requires at least one archive ')

obviously does not work, as accurately described by Michael.

The underscore method is evaluated during the class loading. So instead of string we need a proc at this point, so it can account for locale, currently selected by user. We need a possibility to define a validation like this:

validates_length_of :archives, :minimum => 1,
  :message => proc {_('Gadget requires at least one archive ')}

I’ve created a monkey patch for that, which you can put into a Rails initilizer, e.g. config/initializers/gettext_hacks.rb

module ActiveRecord
  class Errors

     # allow a proc as a user defined message
     def add(attribute, message = nil, options = {})
       message = options[:default].call if options[:default].is_a?(Proc)
       message ||= :invalid
       message = generate_message(attribute, message, options) if message.is_a?(Symbol)
       @errors[attribute.to_s] ||= []
       @errors[attribute.to_s] << message
     end

  end
end

Now I’m working on a more advanced version, which will solve the problem of attribute name making troubles as validation message prefix. More on that - tomorrow.

New blog - blogging like a geek

2009-04-07T00:00:00+02:00

New blog - blogging like a geek

Now I am blogging like a geek - with the article content residing in text files managed by git revision control software and HTML generation powered by a ruby software. My new blog is movabletype free.

jekyll was originally written by Tom Preston-Werner, a founder of github.com. I did some adjustments to the markdown handling so it better suites my DRY attitude. I think, I’ll further expand the software and probably merge it with git-wiki so it eventually leeds to a bliki.

Web design of the new site is currently more a kind of scientific style - without any bells and whistles, but I am going to change this soon.

The new blog is available at http://blog.geekq.net/

The old one is still available at http://www.innoq.com/blog/vd/

Things to do:

copy the posts from the old blog
enable comments

impress!ve - presentation for geeks

2009-03-29T00:00:00+01:00

impress!ve - presentation for geeks

Recently I’ve discovered a great presentation tool chain for real geeks:

beamer latex class to describe the slides in latex
vim for editing
impressive (former keyJnote) by keyJ for displaying

There are two points for improvement in this chain though.

Markdown plus latex

A pure latex file demands too much markup for my taste, exactly as an html file requires too much angle brackets. I am going to use something like markdown with latex fragments the same way you can use html in markdown as a fallback for the case you need something special.

Motivation for the impressive fork

Macs have the perfect presentation software: while an external beamer shows the slide you are talking about, your notebook shows

the current slide
the next slide
time elapsed for the current slide
total elapsed time

This is exactly what I need and what impressive is unfortunately missing - the dual monitor (dual head) feature.

impressive is implemented in python and the source code seems to be a perfect foundation:

the code looks clean
it is a bit C style, not object oriented - a lot of global, not (deeper) structured variables, but it is OK

So keep an eye on http://github.com/geekq/impressive

Translator hints with gettext

2009-02-27T00:00:00+01:00

Translator hints with gettext

In our current project we use gettext not only for translating, but also for localizing date and time formats.

One day I’ve found out, that the user registration date looks strange if I switch the language to italian. The cause was, that the translator translated %Y-%m-%d to %A-%m-%g. It is logical, because “year” is “anno” in italian, so he translated %Y to %A. The result was

"Friday-02-09"

neither italian nor a correct full date.

My first idea was to embed the translation hints into the msgid itself. But the problem is, if there is no translation for particular language, the msgid is used as date format, so we can not do that.

I’ve found out that the GNU gettext provides a special feature for such comments given by the programmer and directed at the translator; these comments are called extracted comments and inserted by xgettext (for C programs) into the po file and appear as follows:

#. extracted-comments
msgid untranslated-string
msgstr translated-string

So the prefix is a hash sign followed by a dot. There is a bigger example on the GNU web site.

Unfortunatly this feature is not implemented in ruby-gettext yet. I’ll probably implement it some day. Then the following code will generate appropriate comments in the po file.

def format_date(user, t)
  # TRANSLATORS: please adjust the order of the date parts and the separator
  # %Y - Year with century
  # %m - Month of the year (01..12)
  # %d - Day of the month (01..31)
  # Example: %d.%m.%Y generates 23.02.1976
  user.timezone.utc_to_local(t.utc).strftime(_('%Y-%m-%d'))
end

`acts_as_state_machine` successor `workflow`

2009-02-25T00:00:00+01:00

`acts_as_state_machine` successor `workflow`

While acts_as_state_machine was not maintained for some years (the revived version can be now found on github), a couple of successor libraries sprang up. My favourite was workflow by Ryan Allen.

Unfortunately, after we had adopted it in our current project, we’ve found some undeterministic behaviour in the ActiveRecord integration area. So I’ve streamlined the integration code by:

strictly separating the implementation between the state machine/workflow specification (states, transitions) and the model entity state
putting all the model enhancements into modules and initiating the admixing the state machine functionality in the self.included method. The modules are::
- WorkflowClassMethods
- WorkflowInstanceMethods
- ActiveRecordInstanceMethods

But you do not need to know all that implementation details to be able to use the workflow library. Just grab my version from github.

Rails internationalization - locale-selector released

2009-01-02T00:00:00+01:00

Rails internationalization - locale-selector released

Just released the first public version of the locale_selector. locale_selector is an internationalization library and provides a wrapper around the ruby-gettext gem.

You can install it simply by

gem install locale_selector

or see the complete installation instructions

Features:

Offers a convinient way to specify the list of locales supported by your application.
Provides a html UI control for locale selection.
Maintains the user preference in a cookie and provides callback for saving the preference e.g. in a database.
Offers rake tasks for maintaining the translations. The suggested translation maintenance workflow is really enterprise proven. E.g. updating single po-files according to the real world responsibility distribution.

Provides some fixes and improvements for the ruby-gettext library:

better parsing for ActiveRecord models in some edge cases:
- namespaced classes
- multiple model classes per file
- and most important - fix for models, that are loaded in environment.rb due to e.g. referencing by a observer
html escaping built into the underscore _() method
better customization for ActiveRecord validation methods - you can now avoid including the field name if you wish

The fixes will be eventually merged into and released with the next releases of ruby-gettext. But for the impatient - just install the locale_selector with the monkey patches.

You can also browse the source code.

geekQ tech blog

Add column safely in MySQL and other idempotent db migrations

Add column safely in MySQL and other idempotent db migrations

Context

Advantages of SQL

MySQL solution

workflow 0.4 gem released

workflow 0.4 gem released

Vim as External Editor for Thunderbird 3 under Ubuntu

Vim as External Editor for Thunderbird 3 under Ubuntu

Signatures

Fix keyboard shortcuts

Just write an email

Fix pragprog ebook fonts for source code listings

Fix pragprog ebook fonts for source code listings

HTML partials in Sinatra

HTML partials in Sinatra

Minimalist testing for Ruby applications

Minimalist testing for Ruby applications

Deploy JRuby web application with jetty-rackup

Deploy JRuby web application with jetty-rackup

Switching to JRuby

Switching to JRuby

workflow 0.3 gem released

workflow 0.3 gem released

Better error messages

Separation of concerns

Have fun!

Editing a LISP program on Ubuntu with vim

Editing a LISP program on Ubuntu with vim

1. Download and extract the VIlisp.vim

2. Install the required Perl modules

3. Now we can run VIlisp which connects two windows

In the scheme interpreter window:

In the editor window:

4. Now you can use shortcuts described in the README of VIlisp

RailsWayCon Berlin

RailsWayCon Berlin

rubygems 1.3.3 released

rubygems 1.3.3 released

Düsseldorf on Rails with Gregg Pollack

Düsseldorf on Rails with Gregg Pollack

Rails internationalization - removing validation message prefix

Rails internationalization - removing validation message prefix

Short and full error messages

String interpolation everywhere

Non-programmer capable translation files

ActiveRecord validations and internationalization

ActiveRecord validations and internationalization

New blog - blogging like a geek

New blog - blogging like a geek

impress!ve - presentation for geeks

impress!ve - presentation for geeks

Markdown plus latex

Motivation for the impressive fork

Translator hints with gettext

Translator hints with gettext

`acts_as_state_machine` successor `workflow`

acts_as_state_machine successor workflow

Rails internationalization - locale-selector released

Rails internationalization - locale-selector released

`acts_as_state_machine` successor `workflow`