Jonathan Weiss on Rails Patterns

These are my unedited notes from Jonathan Weiss's talk "Rails Patterns".

Knowledge gained in large-scale Rails projects
Typical issues: mdeia conversion, user data, lots of plugins, external dependencies, slow requests* LOVDbyLESS example
Using Stephen Bristol's LOVDbyLESS as an example
Pattern approach: context, problem, solution

Scenario: Users want to upload some type of data
Typical solution: do conversion inline (esp. with attachment_fu) or system call to ffmpeg
conversion happens within HTTP req/resp cycle (bad)
Problem: conversion takes time: rails process blocked, risk of connections dropped, user sees blank page, all resources consumed on app server; risk of both intentional as well as unintentional DOS attacks
Solution: process media files asynchronously
Pattern: Separate long-running and resource-hungry tasks
options: use of message bus, fork process; often best b/c easiest solution: poll a DB containing jobs [ed.: Don't want to copy all of Jonathan's presentation – sticking to context, problem, solution only now]

Scenario/Context: users upload lots of data; usual approach: storing data locally, or with NFS, or on S3
Problem: NFS not scalable, S3 upload too slow, background jobs need to read/write from S3 (S3 problems now reduced because of local approaches)
Solution: save locally (fast); schedule asynchronous S3 upload (e.g. via message bus)
Audience question: S3 in Europe now? Jonathan: yes, but AWS:S3 doesn't work with S3 Europe (attachment_fu w/ S3 backend doesn't work at the moment)

Scenario/Context: You want to extract data, search for a plug-in, pick the first one
Problem: most plugins aren't well maintained, have poor quality, are hard to adjust.
Even popular plugins have problems: Backgroundrb, acts_as_ferret, restful_authentication
Solution: Do not blindly use plugins
Pattern: Choose your dependencies wisely
Often, adding the missing 10% to a plugin is harder than coding yourself
Re-use is overrated :-)

Scenario/Context: Your applications have several external dependencies, in specific versions, installed by different means (package management, source compile, gem, plugin)
Problem: deployment becomes brittle
Solution: "vendor everything" (all libs and dependencies in /vendor)
Pattern: Freeze your dependencies

Scenario/Context: limited number of mongrels (typical: 500,000 PIs per day/use 10-20 CPUs, 6-8 mongrels each)
Problem: Denial of Service Attacks very easy: simply start many down/uploads over a slow/throttled line
Solution: contaminate slow requests: serve static files through webserver, define several clusters for different tasks, redirect depending on URL
Pattern: Use the right tool for the job - consider e.g. using Merb, Pure CGI, C, Perl to handle uploads/downloads
new version of mod_rails/Passenger buffers uploads

Another very good talk.