In this full-day workshop, you’ll learn the theory, practical applications, and support for advanced scenarios of REST, the architectural style of the Web.

While we’ll start with a solid theoretical foundation first, we’ll do so without boring everyone (including the instructor) to death. Based on the general concepts, we’ll dive into their implementation in the actual architecture of web protocols in theory and practice.

In the second part, we’ll apply the architectural model to real world scenarios, for RESTful web services scenarios, applications intended for human consumption via a browser, and combinations of both. We’ll cover these for both the enterprise-internal as well as Internet use cases.

Finally, we’ll address common pitfalls, patterns and antipatterns, and advanced topics that come up when adopting RESTful HTTP in the real world.

A siginificant chunk of the workshop will be reserved for Q&A and discussions of use cases brought up by the attendees, as well as some re-engineering of well-known web APIs.

Topics covered will include:

• How to make sense of the basic principles - and when to ignore them
• Basic and advanced caching concepts
• Choosing the right representation format
• Using Hypermedia in real life
• How to document APIs and why you may not want to
• What to look for in implementation tool kits
• Security aspects in RESTful HTTP
• Advanced HTTP 1.x usage, SPDY and HTTP 2.0

The workshop is aimed at experienced developers and architects, with some knowledge of distributed systems design and/or web development. Prior knowledge of REST basics is useful, but not absolutely required.

If you speak German, the easiest way to register is through our website.

You can also register via this form:

Drop me a quick email if you have any questions. Oh yes, we’ve priced it at 790 EUR + VAT, and the number of seats is limited to 20.

First, a major effect of using “schemaless”, simple data structures is loose(r) coupling between pieces of logic (functions) operating on them. If I pass a map to a piece of code, that code will extract what it’s interested in, possibly transforming the data in the process (ideally into a new data structure using efficient copy strategies). It will thus only depend on what it actually uses. Take the following Clojure code as an example (this would be doable similarly in Python, Ruby, Perl, Scala and even Java, although with way more boilerplate in it).

;; "projects" is a Clojure set containing three maps
(def projects #{
                {:id "1",
                 :kind :time-material,
                 :description "Consulting for BigCo",
                 :budget 25000,
                 :team [:joe, :chuck, :james]}
                {:id "2",
                 :kind :fixed-price,
                 :description "Development for Startup",
                 :budget 100000,
                 :team [:john, :chuck, :james, :bill]}
                {:id "3",
                 :kind :fixed-price,
                 :description "Clojure Training",
                 :budget 3000,
                 :team [:joe, :john]}})

;; all-members returns all team members in all projects
(defn all-members
  [projects]
  (reduce conj #{} (flatten (map :team projects))))

;; yields #{:chuck :joe :james :john :bill}

The code and the data structure are coupled just by one map key (:team in the example); I can add other data elements without problems as long as I maintain that contract. In languages such as Clojure, a huge library of useful functions to manipulate data rely on this fact (conj, flatten, map and reduce in this case.)

More importantly, it’s possible (and actually quite common) to use generic data structures at boundaries, such as between modules/namespaces. At their interfaces, these modules accept simple data structures, not complex object graphs. In fact this makes it a lot easier to evolve a single-process program into a distributed one: The kinds of interfaces you use internally resemble what you’d be using remotely (in fact there’s a very nice mapping between a nested Clojure, Ruby or Python data structure and e.g. JSON.)

Some languages, such as Clojure, take this to an extreme: Almost everywhere you’d create a new class in an OO language, you’d just use a simple data structure, such as map, a vector, set or list. In a statically typed limited OO language such as Java, you will almost always end up creating new classes. Of course you can use Map in Java, too, but it would not be idiomatic (and the reverse is true for Clojure as well, which enables you to create “normal” Java classes, too.) Some languages are somewhere in the middle, as shown by the Ruby code in Martin’s example. But I find it not very useful to favor one style over the other by default, unless you consider the language you’re using.

Secondly, the use of custom-built data structures – and that’s what a schema boils down to if viewed from a programming language standpoint – always means additional code. You might consider this irrelevant, but it’s nicely shown in interfaces such as those of WSGI, Rack or Ring when you compare them to their equivalent in Java: A simple map containing a method or response code, headers and a body vs. different concrete classes for requests with tons of specific attributes (and bonus getters and setters). Restricting the use of schemaless design to the cases where you actually need dynamic variability misses this advantage.

In summary, I think of Martin’s points are valid, and he definitely spent more time on articulating his conclusion than I did in writing this comment. But I think those two aspects – coupling and verbosity – are worth considering when you need to decide which approach to use.

• Go: I was thoroughly unimpressed with this language when it came out, and I still fail to see a lot of interesting stuff in it. But I’ve heard many people I respect say only good things about their experience with it, so maybe I should reconsider.
• Rust: At first glance, this seems to be a very nicely designed language (and it has a really excellent tutorial). Even though its language features are very advanced, it seems to be intended for low-level use cases (that I mostly don’t have).
• Fantom: Seems to be interesting, too; I remember I looked at it a long time ago, but never in depth.

What do you think? What else is worth a look?

• Supporting true multiplexing on top of a single TCP connection is great. There is no way anybody can prefer the HTTP 1.0 model, which forces a separate TCP connection per request, nor the HTTP 1.1 model, which allows for persistent connections but still requires serialized request/response interactions (never mind HTTP pipelining as it doesn’t work in practice). Browsers having to open separate connections to the same server to achieve parallelism is not a satisfactory solution.
• Reducing header overhead is also an excellent idea. I’ve heard some criticism about the way this is actually done in SPDY, but it very clearly serves no purpose to have a browser send e.g. the same ridiculously long user agent string with each and every request.
• I used to not care much for the push support, i.e. the opportunity for the server to actively send stuff to the client, for the same reason I’m not a fan of Websockets: I don’t think you actually need this in practice on the application level. But in a session done by Jetty developer Thomas Becker today, I learned about a quite intriguing usage of this in Jetty’s SPDY implementation: On the first request of a page and the subsequent request for the resources that are referenced in that page, Jetty will build a map based on the Referer header – it essentially remembers which secondary resources a page references. When the next request comes along, it can actively send the referenced resources before the client actually asks for them.
• I think the fact that SPDY requires TLS is a mistake. While I totally concede that most traffic on the Net is (and should be) encrypted, there are many use cases e.g. within an organization or for public information where this does not make sense. Besides, it prevents the usage of intermediaries, even though I admit these will be much harder to build for SPDY than for plain HTTP anyway.
• While SPDY proponents point to impressive performance improvements, they are the more impressive the worse the website is implemented. For sites that are already optimized in terms of front end performance, e.g. minimize and compress content, minimize the number of requests, usage proper caching, the effect is going to be much less. That said, some of the things we do in terms of optimization, e.g. combining multiple CSS or JS files into a single one, are not exactly milestones of architectural purity.
• For machine-to-machine communication – i.e. typical RESTful web services – I don’t think SPDY will have the same kind of effect as for Web sites, but I’m willing to let myself be convinced otherwise.
• One of the sad but inevitable things when introducing a binary protocol as opposed to a text-based one is reduced transparency for humans. If SPDY becomes successful – and I have small doubts it will – being able to telnet to a servers port 80 is going to be what I miss most.

SPDY has a very good chance of essentially becoming the new HTTP 2.0, and I’m happy about it: I’m pretty confident the HTTP WG with the formidable Mark Nottingham taking care of things will produce something that will be usable for a long time to come.

So what’s an innoQ event? All of innoQ’s consultants meet at some more or less remote venue and spend two or three days there, discussing projects, technology, methods, in general: anything of interest to our work. Most of the time we use the classical conference format (an innoQ guy presents something, followed by sometimes controversial discussion), but we use other approaches, such as open spaces, pecha kuchas, and lightning talks, too. We occasionally invite guests (we were lucky to have e.g. Steve Vinoski, Michael Hunger, Markus Voelter, Michael Nygard pay us a visit). While the location is mostly somewhere in Germany, we go to Switzerland sometimes, and one event per year is reserved for a trip “far far away” (in the past years we went to e.g. Prague, Barcelona, Paris, Rome, Budapest, and Strasbourg; these are the only events where we actually spend a day just sightseeing). Some of the events focus on project reports, others are programming events, one event per year is dedicated to company strategy.

What is amazing to most people I talk to about this is the frequency we do this with, and the resulting amount of time, effort and money we invest. We do 6-8 events per year, 2 of them three days long, the rest two days. Events are always scheduled during regular workdays, typically Thursdays and Fridays; attendance is mandatory. This adds up to 15-18 days per person per year, with the most significant cost factor being the “lost” revenue. Of course there’s also a lot of effort involved in organizing the whole thing: The colleague who does this essentially organizes 6-8 small conferences (we’re regularly about 50 people these days) per year (no small feat; thanks, Thomas).

It’s worth every single cent.

Company events are among the very best things about innoQ. They serve to help us to spend some quality time discussing various topics, whether it’s company strategy, a new programming language, library, framework or product, a new approach to project management, or some very project-specific problem. We’re also able to invite candidate employees to a setting where they have a great chance to get to know how innoQ works.

Most importantly of all, they’re fun. We spend a lot of time doing geek things, but there’s always time for great food, occasional drinks, and socializing and talking about other important things in life.

So if you see me tweet from Barcelona during the next three days (where I plan to spend some time with the works of one of my favorite artists tomorrow), you know why I’m there.

If you are already convinced that REST is the right architectural style for you, you can probably stop reading and switch over to Mike Amundsen’s excellent “H Factor” discussions. That may be a bit tough to start with, though, so I thought it might make sense to explain some of the ways I use to “pitch” hypermedia. I’ve arrived at a number of explanations and examples of meaningful hypermedia usage, explicitly targeted at people who are not deep into REST yet:

• “Service Document” and “Registries”: Especially for people with an SOA/SOAP/WSDL/WS-* background, the idea of putting at least one level of indirection between a client (consumer) and server (provider) is well established. A link in a server response is a way for a client to not couple itself to a particular server address, and a server response including multiple links to “entry point” resources is somewhat similar to a small registry. If providing links to actual entry point resources is the only purpose of a resource, I’ve become used to calling it a “service document”. Of course these documents themselves can be linked to each other, allowing for hierarchies or other relationships; they can support queries that return a subset of services; they can be provided by different servers themselves; and they are way more dynamic than a typical registry setup. In other words, the very simple approach included in HTTP is far more powerful than what most crazily expensive registries provide.
• Extensible contracts: The merits of being able to link to resources can be exploited to add functionality to existing systems without breaking their contracts. This is most visible in server responses that have one or more places where you can put additional links. As your server or service evolves, you can add links that will be ignored by existing clients, but can be used by those that rely on them. The concept of “a link to a resource” is both sufficiently generic and specific to be meaningful enough, especially if you include a way to specify what they actually mean via a link rel=… approach (but more on that in a separate post).
• Co-Location Independence: What I mean by this slightly weird term is the fact that while resources that are exposed as part of a service interface are sometimes (or maybe even often) designed in a way that requires them to be part of the same implementation, they very often are not, i. e. they could at least in theory be part of a different system. (In fact you can reasonably argue that there should be no assumption about this at all, neither on the server nor the client side for something to be rightfully called “RESTful”, but I simply haven’t found that to be doable in practice.) In those cases where resource don’t need to be hosted by the same implementation, you can and should couple them via links and have clients navigate them instead of relying on common URI paths.

There are quite a few more examples to talk about, but I won’t do that now as I promised to publish something today and don’t want to get into the habit of keeping drafts lying around for too long again. (I know, lenghty this is probably not. Sue me.) So please let me know in the comments what you think of these three if you’re just starting to pick up REST, and what additional ways of explanations you use if you already have done so.

First of all, in an ideal world, the customer understands all of the reasons that led to the Agile movement, e.g. accepts that an upfront specification that’s 100% correct is an unattainable pipe dream, agrees to participate in the actual development process, and most importantly, understands that what needs to be built will only become clear during the actual development project. We do have some customers who understand this very clearly, and they agree to our favorite pricing model: We offer a sprint/iteration for a fixed price or on a T&M basis, and after each sprint the customer can decide to continue or to stop (which will require one final short wrap-up phase). This reduces the customer’s risk, which is often seen as a benefit big enough to outweigh the perceived disadvantage of not knowing what the overall cost will be. It’s great to be able to work in an environment where everybody’s goals are perfectly aligned, and this is the case in this model.

Unfortunately, this ideal model is not always an option. Of course one way for a development organization to ensure that all projects are done this way is to simply refuse doing it in any other fashion. That’s a good option, but whether it’s actually doable strongly depends on internal power distribution or external market forces.

But what do you do when you have to accept a different set of restrictions? For example, the customer/stakeholder might require a fixed-scope, fixed-time, fixed-price offer. My guess is we can all agree that this is bad idea for everyone involved. But how do you approach things if you just have to do things this way? What do you do if, as an additional downside, the developers assigned to the project are not as skilled as you’d like the to be?

As possible answer might be to use a classical waterfall approach, but I think this is never a good choice. At the very least, go with an iterative approach, even if that means you have to game the system to do that.

Of course you have to put up some effort into an initial up-front analysis. You’ll be aware that much of what you find out may actually turn out to be wrong, but it’s still better to make a slightly more informed estimate up front as opposed to a totally bogus one, especially if you’re an external partner that’s supposed to provide a fixed-price quote. Then, make sure that you grow the system in increments – i.e., build a first working system, using a number of interesting use cases; then add functionality in the next iteration, and continue until done.

Typically, this will resemble something like an agile process – but with slightly larger iterations (e.g. maybe 6 weeks instead of two), and with the added amount of documentation required to fulfill the typical waterfall requirements. (If this reminds you of a Unified Process kind of thing, that’s no coincidence.)

In the end, you’ll have created all of the documents and other artefacts required, but simply not in the order they were supposed to be generated (first analysis, then design, then implementation, then test), but with the trimmed-down focus of each iteration.

Is this perfect? Not even remotely. But in my experience, you have a far greater chance to meet your goals than with actually following the waterfall approach, and even more importantly, management is likely to accept it (partially because it’s obvious, partially because you don’t tell them about it).

If you can’t get away with that, you’re really out of luck, and it’s as they say: You need to change the company, and if you can’t, change companies.

I am aware that for many folks, specifically those who are interested in REST and thus likely to read this, a common reaction might be “Duh”. And one of the main things I’d like to stress is that we have not invented a single thing, but rather collected a certain set of rules that we found many people liked, but couldn’t name.

Since we started discussing this, we’ve found strong support, as well as violent opposition. Which is exactly what we were looking for, because in only very very few cases, people didn’t understand what we described, and that’s the whole point of the effort: Give a name to a certain cohesive set of practices so that they may used as a reference both when you agree with them, want to build a system that adheres to them or criticize them because you disagree.

I’m looking forward to comments over at the discussion page. If you believe we should make changes, please fork the GitHub repo and create a pull request.

• To be pedantic, REST vs. … almost never makes sense, as people are rarely talking about REST (the architectural style) in comparison to another architectural style. But never mind, let’s assume that what was meant was actually “RESTful HTTP” vs. “Websockets”, then …
• Websockets is not something “more”, it doesn’t add something, it’s not dynamic, or interactive, or in any way “good” – unless you make the same claim about TCP. Websockets essentially allows you build your own proprietary protocols that may or may not be great, with all the typical advantages and disadvantages these end up having: possibly better performance, possibly better suited to the specific task at hand, less standardized, not widely implemented, etc. It’s not a case of one being better than the other, it’s about being different.
• In the long run, HTTP (used in a way aligned with its architectural goals) will continue to have benefits for loosely coupling systems. If that’s what you want, it makes the most sense. If you’re after the most efficient communication possible, and are willing to sacrifice some of the loose coupling – fine, go ahead, use Websockets. But it’s not as if one will supersede the other.
• Does this mean I claim that HTTP is perfect? Of course not, it most definitely could be improved. But if this improvement comes, it’s definitely going to introduce more, not less constraints.

So what about the change itself? When should you use PUT, POST, PATCH? First of all, these are the truths I base my views on:

• POST can mean anything; its most common use is to create something under a location determined by the server; it’s neither safe nor idempotent nor cachable; it should be used whenever using any of the other methods violates one of their guarantees.
• PUT can mean creation or update; it affects the resource to which it is applied; it’s idempotent; it contains a full representation of the resource (as far as the client is responsible for it). Most importantly, by using PUT the client asks the server to store (in the widest possible sense) the representation under the location provided.
• PATCH, a relatively new verb (at least in it’s standardized form) is intended to address partial updates, i.e. it updates only parts of the resource it’s being applied to; it’s not idempotent; the client asks the server to change parts of a resource.

If you’re a server developer and want to enable your clients to update only parts of a resource – say, a customer’s address –, you basically have three options:

• POST the new address to the resource and have the server decide what to do – in this case, process the address change only – based on the content
• Expose each part you want to be changeable individually as a resource in its own right and use PUT, i.e. make address a resource http://…/customer/:id/address that you can PUT to
• Use PATCH to put information about the intended change to the resource itself, using an appropriate format understood by the server

Using POST is OK, but only because it essentially means nothing. The PUT option is perfectly fine, but requires you to explicitly create resources for this purpose. This is actually the best option in many cases, especially if the resources you create in the process turn out to be meaningful in their own right, support other methods (GET in particular). It often ends up feeling a bit contrived, though, so it’s nice to have the third option: Using PATCH means you are being very clear about the purpose of the request, and don’t need to create new and possibly otherwise unnecessary resources. It’s still fully RESTful because PATCH is an extremely generic method.

Note that while using POST for partial updates is OK, using PUT (as Rails does) is not, because it violates the behavior as defined by the spec. So changing it is a very good idea, and the only two options are POST and PATCH.

Even though PATCH is clearly useful, and has the ultimate REST authority’s blessing, my recommendation in the past has been to avoid it because you can’t count on anyone (or anything) supporting it, and go with PUT or POST instead. With Rails’ influence, I see this changing – and I very much look forward to being able to include PATCH in my RESTful designs in the future. Add PATCH (in addition to PUT and DELETE) to HTML 5, and I’ll be more than happy …

There are a number of different ways to deal with media types when designing a RESTful HTTP system. One school of thought advises to stick with hypermedia formats/media types that are well-defined and widely understood, such as HTML or Atom. In other words: Whatever it is you’re trying to send around as part of an HTTP message, use an existing format, such as HTML, the main reason being that there are many processors that are able to understand it. Use the appropriate MIME identifier (such as text/html) in Content-type headers. One can make a strong case for this option: Hypermedia formats are hard to design, so you should avoid inventing your own.

But let’s assume you’ve decided to define your own hypermedia format, mike amundsen-style, whether by designing a completely new XML vocabulary, your own JSON structure, or some other approach: What MIME type do you use?

You can send content labeling it with the generic identifier, say application/xml. In this case, the MIME identifier will signal the technical format being used, while the semantics are only known to clients who either have some out-of-band knowledge or interpret the content itself. The rationale for this approach is that unless your home-grown hypermedia format is likely to be widely adopted, you’d better stick with a media type that is well-known, even though it doesn’t convey specific semantics. Duncan Cragg wrote a nice post on this a while back.

The second option is to invent your own MIME type, say application/vnd.mycompany-myformat, the argument being that you need to convey the semantics of the content to ensure a client, server or intermediary can actually know whether it’s able to process it.

This begs the question of how many different MIME types you’ll end up with. Instead of creating a new identifier for each type of content, (e.g. a customer, a list of customers, a list of orders), I’ve found that a good approach is to think of a specific context – a domain, if you prefer – that your format covers. I like the similarity of this approach to other hypermedia formats, e.g. HTML or Atom/AtomPub, where you actually end up describing something that applies to a set of collaborating participants, instead of some server-side interface. In my favorite example domain (order processing), you might end up with a MIME type of application/vnd.mycompany-ordermanagement, relate this to a particular XML namespace and define a few different XML root elements (order, order confirmation, cancellation, etc.). The assumption would be that processors can be reasonably expected to able to understand all of the elements in this context, not just one of them. (Of course the same reasoning applies when using JSON or something else, minus the namespace benefits/troubles, depending on your view of XML.)

One final approach that I find very interesting was mentioned by Jan Algermissen a while ago: If your format is based on an existing one, e.g. HTML or XML, your server can actually send the same content with different MIME types, depending on the client’s capabilities. A client that only included application/json in its Accept header would then get the content labeled application/json, while one that includes the specific MIME type application/vnd.whatever would get the same content with this label applied.

Which doesn’t mean one can’t restart, right? We’ll see. For now, this is the one and only blog I’ll maintain. It has zero moving parts, i.e. it’s a static site, (currently) generated using Octopress. If everything works as expected, you’ll be able to subcribe to the an Atom feed that should actually be working for a change.

I’m somewhat undecided with regards to comments and the Twitter integration (both in the sidebar as well as in the bottom of each post). I’ve toyed with the idea of doing away with comments altogether, but I have to say that Disqus seems quite smart.