Planet

A while ago, I gave a talk at QCon about breaking up monoliths (there’s a video up on InfoQ), repeated it in a slightly improved version at JavaZone (see slides and video), and the topic continues to come up in almost every consulting engagement and client workshop I’ve been involved in since then. Like many of the topics I talk about, it’s somewhat unfair that I get the positive feedback and people assume I came up with the ideas all on my own: Most of stuff like this is the result of collaboration, with my colleagues at innoQ (see for example an article I wrote with Phillip Ghadir for ObjektSpektrum if you read German), as well as customer staff. But wherever it originated, I found that it strikes a nerve with many developers and architects, not only in big companies that conduct million-Euro development projects, but also in smaller e-commerce companies and even startups that have started to become successful.

The main idea is this (no surprise for almost everyone, I guess): Nobody wants monoliths, i.e. big systems composed of hundreds of thousands or millions of lines of code (in a language like Java) or tens of thousands (e.g. in Ruby), yet everyone ends up having them. And once you have one, you’re basically stuck with them: They’re incredibly hard to maintain, extend, and modernize; yet they provide value and can’t simply be replaced (something that many organizations attempt but fail at, because it’s awfully hard to create something new that is not only great in terms of architecture, but also can actually function as a full replacement for all of the old system’s features.

So what’s the proposed remedy? To talk about that, we need to take a step back and find out how we actually end up systems that are too big in the first place. My theory is that the number one reason is project scope.

When a project is started, there is an assumption that it’s the goal of a project to create a single system. This typically goes unquestioned, even though the people or person coming up with the project boundaries often don’t decide this consciously. This is most obvious if they’re non-technical people who make decisions on a budget basis.

So the very first thing we should be doing as architects (or lead developers if you don’t like the term) is to find out what it actually is we should be building. Is it really a single system? If our task is to replace an existing system with a new one, it’s very tempting to just accept existing boundaries and go with them. If we’re consolidating two systems, it’s equally tempting to view our scope as the union of the predecessor systems’ scope. In the rare cases where our task is to actually modularize something existing, it’s because of business reasons (such as deregulation). Again, while it might seem like a good idea to just accept the boundaries being suggested to us, it’s not at all clear why this should be a good idea. After all, if whoever came up with those boundaries is not an architect or developer, what makes us think they made a good choice?

In my view, the most important thing to do, then, is to find out how many systems we should be building in the first place. It may be a single one, but it may also be two, five or a dozen (though probably not more) – clearly, the decision should be made very consciously, because whatever system boundaries you pick, you will likely be stuck with them for a very long time.

As “system” is a term that can mean almost anything, I need to define what I mean by it in this context. A system is an independent unit that is developed according to its own rules, and only connected to other systems in an unobstrusive fashion. A system, according to this model, has its own database, business logic, and user interface; it’s deployed separately. It’s likely developed by a different team than other systems. It has its own life cycle, in terms of development as well as deployment. It’s operated autonomously. It has its own test suite. In basically every regard, it’s as different from all the other systems as a piece of commercial off-the-shelf software would be. (In fact, one of the systems may end up being a piece of standard software.) Is that the same as the “Micro Services” idea? If you watch James Lewis’s great talk (here’s a recording, also done at JavaZone; in fact his was scheduled directly after mine), you’ll find a lot of similarities, but the major difference is probably the size of each individual unit. But to me, seeing similar concepts appear in different contexts is a very good sign.

It doesn’t really matter that much whether you get the number and distribution right with the first attempt – in fact, you can reasonably consider that to be highly improbable. But it’s one thing to find out you should have built six or eight systems instead of seven, i.e. get it wrong in one or two places, and a completely different one to notice it should have been seven instead of one.

I’ve rambled on for long enough for a single post, so here’s a preliminary conclusion: How many systems you build should be a very conscious decision. It will affect the life of those tasked with evolving and maintaining it for years to come.

Ah, the joys of “Intellectual Property”. I’ve been a long-time fan of the wonderful demotivational posters offered by despair.com. From time to time, I point people to them, e.g. by tweeting about it. In the past, when there was no Twitter (yes, that time existed), I used this blog to do so – on one occasion not only using a plain <a>, but an <img> element as well, embedding one of their images on this site (this is the post minus the image). After all, why not send a little traffic to these fine folks?

But obviously Despair uses stock photos – a perfect use case if there ever was one –, and the rights to the particularly cheesy one used in this poster apparently belong to Getty images (see? I did it again. Sue me.). Now I’ve received a letter, first from their internal legal department, and – after explaining the misunderstanding on their part – now from their lawyer. In both cases, they insist that we need to license the image to use it.

As I didn’t copy the image of the poster, but only link to it, this seems entirely absurd to me – particularly if Despair properly licensed the image, which I’m quite sure of. (If at all, Despair might have more reason, but I can’t believe they’d be that unreasonable, purely out of their own interest.) But my guess is the legal trolls at Getty believe it won’t be worth the hassle to me. They’re wrong – I don’t believe they deserve a single cent of my (or the company’s) money. If you have any advice, or want to share some of your own experience with these people, please leave a comment.

For the last few months, I’ve been continuously tweaking my laptop settings to optimize my typing speed and efficiency. Here are some of the things I’ve learned, and some of the tools and approaches I’m currently using.

• I finally took up “real” touch typing, i.e. using 10 fingers and a classical system. For years I’d been quite satisfied, even a little proud, of my awesome typing skillz, only to be shot down when I took the first real speed typing test and found myself scoring a ridiculous 30-40 words per minute, the reason being mostly mistakes and the “occasional” peeking if some not-so-common character came along. There are a ton of tools for learning touch typing. The one I spent the most time with is the excellent (and free) Tipp10, which is available in both offline and online versions. There’s also the very nicely done Keys if you’re on a Mac and have a US keyboard. If you like to have some social interaction, both 10 Fast Fingers and Typeracer are also quite nice. Finally, for a programmer, typing.io is a fantastic resource.

• I switched to a US keyboard layout a while ago. I don’t know why I’d put this off for so long; while I originally started out using a German keyboard layout (QWERTZ instead of QWERTY), I easily write half of what I produce in English, so it wouldn’t have hurt to do this earlier. The most important benefits are that almost all of the characters required in programming languages are far easier to type using a US keyboard and all of the keyboard shortcuts, particularly in editors such as my favorite, Emacs, suddenly make a lot more sense. Because I still type a lot of German texts and hated the default input method for umlauts and the German “ß”, I installed the great “USGerman” keyboard layout, which allows me to use Option+a, u, o and s to get the appropriate characters with a single combination. This has turned out to be a very workable solution.

• It also made me use Cmd for Meta in Emacs, which is both good and bad: It’s good because when you use Emacs, you use Meta a lot; it’s bad, because some of the keyboard shortcuts for moving around now need to be done with Cmd (in Emacs) and Option (everywhere else), which can be a bit annoying. Also, I ordered my new laptop with a German keyboard layout by accident, which ended up being great because it means I now have no chance to actually look at the keyboard for those special characters anymore. (It also allowed me to turn the keyboard lighting all the way down.)

• Speaking of Emacs, I wanted to make sure I have the Emacs short cuts available everywhere. This is actually the case to a large degree by default on a Mac, but I wanted to be able to rely on combinations such as Ctrl-M, Ctrl-H, Ctrl-I, etc. Enter the slightly ugly, stupidly named, but incredibly powerful KeyRemap4MacBook, which allowed me to ensure Emacs keys work everywhere. (I hear this is possible for vi users, too.)

Now I find myself not taking my fingers off home row very much, and I’m comfortably moving along at 60-80 wpm. There’s a weird satisfaction in this – I noticed that in a recent company-internal discussion, many of my co-workers did not seem to see much of a point in taking up touch-typing because they don’t see typing speed as the limiting factor. One reason I might see this differenly is because I (sadly) don’t program much these days, but produce a lot of prose instead. And there, at least, I’m absolutely confident that not having your typing get in the way is a huge asset.

Softwarearchitektur

Systeme zu konstruieren, die funktionieren und elegant sind, ist das Größte. Leute, die das tun, können ihren Erfolg im Stillen genießen und brauchen eigentlich niemanden, der ihnen auf die Schulter klopft. Das ist aber auch gut so, denn das passiert sowieso in den aller seltensten Fällen.

Der Weg zu einem tollen System ist häufig …

… versperrt mit Wegelagerern, die hoffen, ihre eigenen Anforderungen ohne viel zutun erfüllt zu bekommen.
… länger als er sein muss, weil zu Beginn die Ausgangslage oder das Ziel nicht genau bekannt sind.
… anstrengend, weil nicht alle mit gleichem Eifer in die gleiche Richtung wollen.
… versteckt in einer Menge von technischen, fachlichen oder organisatorischen Möglichkeiten und Unwägbarkeiten.

Und trotzdem ist das Bauen von Software eines meiner größten Hobbys. Am liebsten mit dem besten verfügbaren Team.

Ich rede auch gern darüber, wie man Software-Systeme ordentlich baut. Deshalb habe ich damals den iSAQB e.V. mitgegründet und deshalb gebe ich auch heute noch gern Architektur-Trainings.

Software Architecture Camp

In diesem Jahr gibt es aber erstmalig Trainings in einem neuen Format: Das Software Architecture Camp geleitet von zwei Trainern!

Dr. Gernot Starke und ich geben gemeinsam ein Training zu Software-Architektur, das konform zum Foundation Level-Lehrplan des iSAQB ist und die besten Ideen enthält, die Gernot und mir für so ein Training eingefallen sind.

Zudem übernimmt die Entwickler Akademie die Schulungsorganisation und spendiert jedem Teilnehmer ein riesiges Paket an Büchern. Gernot und ich durften wünschen. Die Teilnehmer bekommen. Großartig!

Zudem verlangt die Entwickler-Akademie für die Trainings einen Preis, der in meinen Augen ein echtes Schnäppchen ist.

Wer sich für Softwarearchitektur interessiert, dem verspreche ich, dass er nicht nur von dem großen Bücher-Paket sondern auch von den unterschiedlichen Sichtweisen von Gernot und mir sehr profitieren wird.

Ich freue mich auf die Schulungen, in denen der Techniker auf den Methodiker trifft - in denen ich, Chaot, auf Gernot, den Meister der Strukturen, treffe.

Auf der folgenden Seite können Sie sich anmelden, wenn Sie dabei sein wollen: Software-Architecture-Camp bei der Entwickler-Akademie.

Martin Fowler has written an infodeck (which is actually a nice format for these kinds of things) on schemaless databases and in-memory structures. I agree with most of it, at least as far as database design is concerned. But I believe there are two important concerns missing, and that’s the reason why I don’t agree with the conclusion. Note that my concern is not really with the database part, but with the extrapolation to in-memory data structures, so you should read the following with this in mind:

First, a major effect of using “schemaless”, simple data structures is loose(r) coupling between pieces of logic (functions) operating on them. If I pass a map to a piece of code, that code will extract what it’s interested in, possibly transforming the data in the process (ideally into a new data structure using efficient copy strategies). It will thus only depend on what it actually uses. Take the following Clojure code as an example (this would be doable similarly in Python, Ruby, Perl, Scala and even Java, although with way more boilerplate in it).

;; "projects" is a Clojure set containing three maps
(def projects #{
                {:id "1",
                 :kind :time-material,
                 :description "Consulting for BigCo",
                 :budget 25000,
                 :team [:joe, :chuck, :james]}
                {:id "2",
                 :kind :fixed-price,
                 :description "Development for Startup",
                 :budget 100000,
                 :team [:john, :chuck, :james, :bill]}
                {:id "3",
                 :kind :fixed-price,
                 :description "Clojure Training",
                 :budget 3000,
                 :team [:joe, :john]}})

;; all-members returns all team members in all projects
(defn all-members
  [projects]
  (reduce conj #{} (flatten (map :team projects))))

;; yields #{:chuck :joe :james :john :bill}

The code and the data structure are coupled just by one map key (:team in the example); I can add other data elements without problems as long as I maintain that contract. In languages such as Clojure, a huge library of useful functions to manipulate data rely on this fact (conj, flatten, map and reduce in this case.)

More importantly, it’s possible (and actually quite common) to use generic data structures at boundaries, such as between modules/namespaces. At their interfaces, these modules accept simple data structures, not complex object graphs. In fact this makes it a lot easier to evolve a single-process program into a distributed one: The kinds of interfaces you use internally resemble what you’d be using remotely (in fact there’s a very nice mapping between a nested Clojure, Ruby or Python data structure and e.g. JSON.)

Some languages, such as Clojure, take this to an extreme: Almost everywhere you’d create a new class in an OO language, you’d just use a simple data structure, such as map, a vector, set or list. In a statically typed limited OO language such as Java, you will almost always end up creating new classes. Of course you can use Map in Java, too, but it would not be idiomatic (and the reverse is true for Clojure as well, which enables you to create “normal” Java classes, too.) Some languages are somewhere in the middle, as shown by the Ruby code in Martin’s example. But I find it not very useful to favor one style over the other by default, unless you consider the language you’re using.

Secondly, the use of custom-built data structures – and that’s what a schema boils down to if viewed from a programming language standpoint – always means additional code. You might consider this irrelevant, but it’s nicely shown in interfaces such as those of WSGI, Rack or Ring when you compare them to their equivalent in Java: A simple map containing a method or response code, headers and a body vs. different concrete classes for requests with tons of specific attributes (and bonus getters and setters). Restricting the use of schemaless design to the cases where you actually need dynamic variability misses this advantage.

In summary, I think of Martin’s points are valid, and he definitely spent more time on articulating his conclusion than I did in writing this comment. But I think those two aspects – coupling and verbosity – are worth considering when you need to decide which approach to use.

Being a programming language geek, I typically try to use the Christmas vacation to learn (or rather, play with) a programming language I don’t know. This year, I find this very hard, as there are so many interesting languages one could spend time with. Some I have considered are:

• Go: I was thoroughly unimpressed with this language when it came out, and I still fail to see a lot of interesting stuff in it. But I’ve heard many people I respect say only good things about their experience with it, so maybe I should reconsider.
• Rust: At first glance, this seems to be a very nicely designed language (and it has a really excellent tutorial). Even though its language features are very advanced, it seems to be intended for low-level use cases (that I mostly don’t have).
• Fantom: Seems to be interesting, too; I remember I looked at it a long time ago, but never in depth.

What do you think? What else is worth a look?

What follows is a number of claims about SPDY, none of which I can back up in any reasonable way. My hope is that readers who know more will educate me. SPDY, in case you don’t know it, is a replacement for the HTTP wire protocol. It originated with Google and aims to preserve HTTP’s application semantics while improving its efficiency and performance.

• Supporting true multiplexing on top of a single TCP connection is great. There is no way anybody can prefer the HTTP 1.0 model, which forces a separate TCP connection per request, nor the HTTP 1.1 model, which allows for persistent connections but still requires serialized request/response interactions (never mind HTTP pipelining as it doesn’t work in practice). Browsers having to open separate connections to the same server to achieve parallelism is not a satisfactory solution.
• Reducing header overhead is also an excellent idea. I’ve heard some criticism about the way this is actually done in SPDY, but it very clearly serves no purpose to have a browser send e.g. the same ridiculously long user agent string with each and every request.
• I used to not care much for the push support, i.e. the opportunity for the server to actively send stuff to the client, for the same reason I’m not a fan of Websockets: I don’t think you actually need this in practice on the application level. But in a session done by Jetty developer Thomas Becker today, I learned about a quite intriguing usage of this in Jetty’s SPDY implementation: On the first request of a page and the subsequent request for the resources that are referenced in that page, Jetty will build a map based on the Referer header – it essentially remembers which secondary resources a page references. When the next request comes along, it can actively send the referenced resources before the client actually asks for them.
• I think the fact that SPDY requires TLS is a mistake. While I totally concede that most traffic on the Net is (and should be) encrypted, there are many use cases e.g. within an organization or for public information where this does not make sense. Besides, it prevents the usage of intermediaries, even though I admit these will be much harder to build for SPDY than for plain HTTP anyway.
• While SPDY proponents point to impressive performance improvements, they are the more impressive the worse the website is implemented. For sites that are already optimized in terms of front end performance, e.g. minimize and compress content, minimize the number of requests, usage proper caching, the effect is going to be much less. That said, some of the things we do in terms of optimization, e.g. combining multiple CSS or JS files into a single one, are not exactly milestones of architectural purity.
• For machine-to-machine communication – i.e. typical RESTful web services – I don’t think SPDY will have the same kind of effect as for Web sites, but I’m willing to let myself be convinced otherwise.
• One of the sad but inevitable things when introducing a binary protocol as opposed to a text-based one is reduced transparency for humans. If SPDY becomes successful – and I have small doubts it will – being able to telnet to a servers port 80 is going to be what I miss most.

SPDY has a very good chance of essentially becoming the new HTTP 2.0, and I’m happy about it: I’m pretty confident the HTTP WG with the formidable Mark Nottingham taking care of things will produce something that will be usable for a long time to come.

Since the beginning of innoQ’s existence, 13 years ago, we’ve maintained a regular schedule of so-called “company events”. In my opinion, this is one of the really, really great things about innoQ, and it’s also quite different from what others do. Which is a sufficient excuse for me to write this …

So what’s an innoQ event? All of innoQ’s consultants meet at some more or less remote venue and spend two or three days there, discussing projects, technology, methods, in general: anything of interest to our work. Most of the time we use the classical conference format (an innoQ guy presents something, followed by sometimes controversial discussion), but we use other approaches, such as open spaces, pecha kuchas, and lightning talks, too. We occasionally invite guests (we were lucky to have e.g. Steve Vinoski, Michael Hunger, Markus Voelter, Michael Nygard pay us a visit). While the location is mostly somewhere in Germany, we go to Switzerland sometimes, and one event per year is reserved for a trip “far far away” (in the past years we went to e.g. Prague, Barcelona, Paris, Rome, Budapest, and Strasbourg; these are the only events where we actually spend a day just sightseeing). Some of the events focus on project reports, others are programming events, one event per year is dedicated to company strategy.

What is amazing to most people I talk to about this is the frequency we do this with, and the resulting amount of time, effort and money we invest. We do 6-8 events per year, 2 of them three days long, the rest two days. Events are always scheduled during regular workdays, typically Thursdays and Fridays; attendance is mandatory. This adds up to 15-18 days per person per year, with the most significant cost factor being the “lost” revenue. Of course there’s also a lot of effort involved in organizing the whole thing: The colleague who does this essentially organizes 6-8 small conferences (we’re regularly about 50 people these days) per year (no small feat; thanks, Thomas).

It’s worth every single cent.

Company events are among the very best things about innoQ. They serve to help us to spend some quality time discussing various topics, whether it’s company strategy, a new programming language, library, framework or product, a new approach to project management, or some very project-specific problem. We’re also able to invite candidate employees to a setting where they have a great chance to get to know how innoQ works.

Most importantly of all, they’re fun. We spend a lot of time doing geek things, but there’s always time for great food, occasional drinks, and socializing and talking about other important things in life.

So if you see me tweet from Barcelona during the next three days (where I plan to spend some time with the works of one of my favorite artists tomorrow), you know why I’m there.

Hypermedia is the topic that generates the most intensive discussions in any REST training, workshop, or project. While the other aspects of RESTful HTTP usage are pretty straightforward to grasp and hard to argue with, the use of hypermedia in machine-to-machine communication is a bit tougher to explain.

If you are already convinced that REST is the right architectural style for you, you can probably stop reading and switch over to Mike Amundsen’s excellent “H Factor” discussions. That may be a bit tough to start with, though, so I thought it might make sense to explain some of the ways I use to “pitch” hypermedia. I’ve arrived at a number of explanations and examples of meaningful hypermedia usage, explicitly targeted at people who are not deep into REST yet:

• “Service Document” and “Registries”: Especially for people with an SOA/SOAP/WSDL/WS-* background, the idea of putting at least one level of indirection between a client (consumer) and server (provider) is well established. A link in a server response is a way for a client to not couple itself to a particular server address, and a server response including multiple links to “entry point” resources is somewhat similar to a small registry. If providing links to actual entry point resources is the only purpose of a resource, I’ve become used to calling it a “service document”. Of course these documents themselves can be linked to each other, allowing for hierarchies or other relationships; they can support queries that return a subset of services; they can be provided by different servers themselves; and they are way more dynamic than a typical registry setup. In other words, the very simple approach included in HTTP is far more powerful than what most crazily expensive registries provide.
• Extensible contracts: The merits of being able to link to resources can be exploited to add functionality to existing systems without breaking their contracts. This is most visible in server responses that have one or more places where you can put additional links. As your server or service evolves, you can add links that will be ignored by existing clients, but can be used by those that rely on them. The concept of “a link to a resource” is both sufficiently generic and specific to be meaningful enough, especially if you include a way to specify what they actually mean via a link rel=… approach (but more on that in a separate post).
• Co-Location Independence: What I mean by this slightly weird term is the fact that while resources that are exposed as part of a service interface are sometimes (or maybe even often) designed in a way that requires them to be part of the same implementation, they very often are not, i. e. they could at least in theory be part of a different system. (In fact you can reasonably argue that there should be no assumption about this at all, neither on the server nor the client side for something to be rightfully called “RESTful”, but I simply haven’t found that to be doable in practice.) In those cases where resource don’t need to be hosted by the same implementation, you can and should couple them via links and have clients navigate them instead of relying on common URI paths.

There are quite a few more examples to talk about, but I won’t do that now as I promised to publish something today and don’t want to get into the habit of keeping drafts lying around for too long again. (I know, lenghty this is probably not. Sue me.) So please let me know in the comments what you think of these three if you’re just starting to pick up REST, and what additional ways of explanations you use if you already have done so.

Today, I had an interesting discussion over Twitter related to project organization in restricted environments. (Update: I’ve removed all references to the actual discussion because the person I interacted with felt mis-quoted, and I don’t think that it was actually that important with regards to what I actually wanted to get across.) This prompted me to take a break from my usual topics and elaborate a bit on my thoughts with more than 140 characters. All this assumes you’re in the role of not only having to actually deliver a piece of software, but also to get the necessary funding – regardless of whether you’re part of an internal organization that requires approval from top management or you’re an external partner that needs to sell to its customer. That said, I’ll focus on the second scenario as that is my primary focus at innoQ.

First of all, in an ideal world, the customer understands all of the reasons that led to the Agile movement, e.g. accepts that an upfront specification that’s 100% correct is an unattainable pipe dream, agrees to participate in the actual development process, and most importantly, understands that what needs to be built will only become clear during the actual development project. We do have some customers who understand this very clearly, and they agree to our favorite pricing model: We offer a sprint/iteration for a fixed price or on a T&M basis, and after each sprint the customer can decide to continue or to stop (which will require one final short wrap-up phase). This reduces the customer’s risk, which is often seen as a benefit big enough to outweigh the perceived disadvantage of not knowing what the overall cost will be. It’s great to be able to work in an environment where everybody’s goals are perfectly aligned, and this is the case in this model.

Unfortunately, this ideal model is not always an option. Of course one way for a development organization to ensure that all projects are done this way is to simply refuse doing it in any other fashion. That’s a good option, but whether it’s actually doable strongly depends on internal power distribution or external market forces.

But what do you do when you have to accept a different set of restrictions? For example, the customer/stakeholder might require a fixed-scope, fixed-time, fixed-price offer. My guess is we can all agree that this is bad idea for everyone involved. But how do you approach things if you just have to do things this way? What do you do if, as an additional downside, the developers assigned to the project are not as skilled as you’d like the to be?

As possible answer might be to use a classical waterfall approach, but I think this is never a good choice. At the very least, go with an iterative approach, even if that means you have to game the system to do that.

Of course you have to put up some effort into an initial up-front analysis. You’ll be aware that much of what you find out may actually turn out to be wrong, but it’s still better to make a slightly more informed estimate up front as opposed to a totally bogus one, especially if you’re an external partner that’s supposed to provide a fixed-price quote. Then, make sure that you grow the system in increments – i.e., build a first working system, using a number of interesting use cases; then add functionality in the next iteration, and continue until done.

Typically, this will resemble something like an agile process – but with slightly larger iterations (e.g. maybe 6 weeks instead of two), and with the added amount of documentation required to fulfill the typical waterfall requirements. (If this reminds you of a Unified Process kind of thing, that’s no coincidence.)

In the end, you’ll have created all of the documents and other artefacts required, but simply not in the order they were supposed to be generated (first analysis, then design, then implementation, then test), but with the trimmed-down focus of each iteration.

Is this perfect? Not even remotely. But in my experience, you have a far greater chance to meet your goals than with actually following the waterfall approach, and even more importantly, management is likely to accept it (partially because it’s obvious, partially because you don’t tell them about it).

If you can’t get away with that, you’re really out of luck, and it’s as they say: You need to change the company, and if you can’t, change companies.

In the past few days, we finally managed to write down a few ideas on Web frontend design, basically a set of rules to apply if the goal is to come up with a Web app that is actually on the Web as opposed to be tunnelled through the Web. We tried to come up with a catchy name, and finally arrived at ”ROCA”, a conveniently pronouncable acronym for “Resource-oriented client architecture”.

I am aware that for many folks, specifically those who are interested in REST and thus likely to read this, a common reaction might be “Duh”. And one of the main things I’d like to stress is that we have not invented a single thing, but rather collected a certain set of rules that we found many people liked, but couldn’t name.

Since we started discussing this, we’ve found strong support, as well as violent opposition. Which is exactly what we were looking for, because in only very very few cases, people didn’t understand what we described, and that’s the whole point of the effort: Give a name to a certain cohesive set of practices so that they may used as a reference both when you agree with them, want to build a system that adheres to them or criticize them because you disagree.

I’m looking forward to comments over at the discussion page. If you believe we should make changes, please fork the GitHub repo and create a pull request.

One of the design goals of RESTful services is loose coupling. When using JSON as data format one implication is that both a REST Server and Client should not fail on additional fields. This way smaller changes can happen to the API without breaking existing services and clients. Implementing this happens automatically in dynamic languages where a JSON Object is just some type of map. But it is easy to get wrong using Java.

A naive databinding approach would not accept additional data. However the other extrem - unmarshalling json to maps and lists - does not fit Javas static typed world. Luckily there is an approach in the middle: semistatic Binding. Bind those attributes you expect to static java types and put everything else in a Map. Here es an example how to achieve this with Jackson¹ and Svenson².

Example for Jackson:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

class SomeEntity {

    private String staticProperty;

    @JsonIgnore
    private Map<String,Object> dynamicProperties = new HashMap<String, Object>();

    @JsonAnyGetter
    public Map<String,Object> getDynamicProperties() {
        return dynamicProperties;
    }

    // Boilerplate getter and setter for staticProperty go here...
}

This Class can be mapped to from arbitrary JSON Objects. e.g:

1
2
3
4
5

{
    "foo" : "bar",
    "staticProperty" : "A Value",
    "TheAnswer" : 42
}

Attributes matching a (bean-)property (“staticProperty” here) go into that property. Everything else goes into the map.

The very same can be done in svenson by implementing DynamicProperties (or subclassing AbstractDynamicProperties, but that would hide the fun of it…):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

public class SomeEntity implements DynamicProperties

{
   private String staticProperty;
   private Map<String,Object> dynamicProperties = new HashMap<String, Object>();

   public Object getProperty(String name)
   {
       return dynamicProperties.get(name);
   }

   public void setProperty(String name, Object value)
   {
       dynamicProperties.put(name,value);
   }

   public Set<String> propertyNames()
   {
       return dynamicProperties.keySet();
   }

    // Boilerplate getter and setter for staticProperty go here...
}

There is an entirely absurd discussion going on about “REST vs. Websockets”, predictably claiming that in the long term, REST will be succeeded by Websockets. This is stupid on many levels. I’ll try to be brief:

• To be pedantic, REST vs. … almost never makes sense, as people are rarely talking about REST (the architectural style) in comparison to another architectural style. But never mind, let’s assume that what was meant was actually “RESTful HTTP” vs. “Websockets”, then …
• Websockets is not something “more”, it doesn’t add something, it’s not dynamic, or interactive, or in any way “good” – unless you make the same claim about TCP. Websockets essentially allows you build your own proprietary protocols that may or may not be great, with all the typical advantages and disadvantages these end up having: possibly better performance, possibly better suited to the specific task at hand, less standardized, not widely implemented, etc. It’s not a case of one being better than the other, it’s about being different.
• In the long run, HTTP (used in a way aligned with its architectural goals) will continue to have benefits for loosely coupling systems. If that’s what you want, it makes the most sense. If you’re after the most efficient communication possible, and are willing to sacrifice some of the loose coupling – fine, go ahead, use Websockets. But it’s not as if one will supersede the other.
• Does this mean I claim that HTTP is perfect? Of course not, it most definitely could be improved. But if this improvement comes, it’s definitely going to introduce more, not less constraints.

The other day I tried to run Jersey (Version 1.11) and Weld (The CDI Implementation) in Tomcat. It didn’t work out of the box. Some googling revealed the problem: To support CDI Jersey needs access to the BeanManager, the main interface for frameworks that want to interact with CDI. According to the JAX-RS spec the BeanManager should be bound to java:comp/BeanManager in JNDI where Jersey tries to look it up.

Unfortunately Tomcat does not allow putting it there. However it can be obtained via JNDI from java:comp/env/BeanManager. This Problem is mentioned by the WELD Documentation. So how can you make Jersey find it here? There is a thread on the jersey Mailinglist which provides a solution: Subclass Jerseys Servlet/Filter implementation and lookup the BeanManager from java:comp/env/BeanManager. I tried this and it works fine for me.

However while this blogpost was siting in my draft folder there are even better news: Within a Servlet Container Weld also binds the BeanManager into the ServletContext. Starting from 1.12 (Which has been released on Feb. 22) Jersey will lookup the BeanManager from ServletContext and everything works out of the box. If you need to backport the change you can have a look at the patch attached to JERSEY-883.

As mentioned on the Ruby on Rails weblog, Rails 4.0 will include (optional) support for partial updates via PATCH, a change included to better comply to the HTTP spec and the REST architectural style. As you can guess, I really like this motivation, even though I think it’s insufficient to justify major changes – being “RESTful” should not be the goal, building a better system should be. It seems the Rails team has found a good way to do this, as the change is made in a backwards-compatible fashion (so if you don’t care, you can simply ignore it). But it highlights one of the things I really, really like about Rails: It tries to make it a lot easier to build something that’s RESTful than something that isn’t, and its reach means many more people will be exposed to this as the way it’s being done.

So what about the change itself? When should you use PUT, POST, PATCH? First of all, these are the truths I base my views on:

• POST can mean anything; its most common use is to create something under a location determined by the server; it’s neither safe nor idempotent nor cachable; it should be used whenever using any of the other methods violates one of their guarantees.
• PUT can mean creation or update; it affects the resource to which it is applied; it’s idempotent; it contains a full representation of the resource (as far as the client is responsible for it). Most importantly, by using PUT the client asks the server to store (in the widest possible sense) the representation under the location provided.
• PATCH, a relatively new verb (at least in it’s standardized form) is intended to address partial updates, i.e. it updates only parts of the resource it’s being applied to; it’s not idempotent; the client asks the server to change parts of a resource.

If you’re a server developer and want to enable your clients to update only parts of a resource – say, a customer’s address –, you basically have three options:

• POST the new address to the resource and have the server decide what to do – in this case, process the address change only – based on the content
• Expose each part you want to be changeable individually as a resource in its own right and use PUT, i.e. make address a resource http://…/customer/:id/address that you can PUT to
• Use PATCH to put information about the intended change to the resource itself, using an appropriate format understood by the server

Using POST is OK, but only because it essentially means nothing. The PUT option is perfectly fine, but requires you to explicitly create resources for this purpose. This is actually the best option in many cases, especially if the resources you create in the process turn out to be meaningful in their own right, support other methods (GET in particular). It often ends up feeling a bit contrived, though, so it’s nice to have the third option: Using PATCH means you are being very clear about the purpose of the request, and don’t need to create new and possibly otherwise unnecessary resources. It’s still fully RESTful because PATCH is an extremely generic method.

Note that while using POST for partial updates is OK, using PUT (as Rails does) is not, because it violates the behavior as defined by the spec. So changing it is a very good idea, and the only two options are POST and PATCH.

Even though PATCH is clearly useful, and has the ultimate REST authority’s blessing, my recommendation in the past has been to avoid it because you can’t count on anyone (or anything) supporting it, and go with PUT or POST instead. With Rails’ influence, I see this changing – and I very much look forward to being able to include PATCH in my RESTful designs in the future. Add PATCH (in addition to PUT and DELETE) to HTML 5, and I’ll be more than happy …

I took the opportunity of a 10-hour flight to finally re-import all of my blog’s old entries into Octopress, fiddling around with a semi-automated process of Ruby and shell scripts and some creative Emacs macros (check out the archives if you care, even though I have no idea why you would). So if this worked, all of the old stuff should be preserved, even though most of it now only has historic value (and prevents Google searches to hit a 404). I’ve also taken the risk of moving the whole stuff back to its old location, and will redirect the temporary one back once I’m sure everything works. One less excuse to not take up blogging again …

Aristotle Pagaltzis has written a very nice and concise XPath intro. Of all the various standards in the XML ecosystem, I like XPath best, most of all because it enables interaction with an XML document or message in a way that matches Postel’s law: It’s a great way to implement code that doesn’t break each time a minor change is made to an XML format. In fact I’d say it’s one the few very good reasons to stick with XML instead of adopting JSON – even though there are things like JSONPath, they don’t have the tool support and standardization you get from XPath.

A topic that comes up again and again in discussions about RESTful design is what kinds of media type to use. I’ve changed my mind about this a few times, often enough so that I believe it makes sense to write a bit about my current thinking on this. (A “media type”, should you happen to wonder what I’m talking about, is a specific format with a well-defined name, in this context represented in the form of a media type identifier such as application/xml or text/html or application/json. [That’s not 100% correct, but that doesn’t really matter here.] A “hypermedia format” is one that includes links or other hypermedia controls.)

There are a number of different ways to deal with media types when designing a RESTful HTTP system. One school of thought advises to stick with hypermedia formats/media types that are well-defined and widely understood, such as HTML or Atom. In other words: Whatever it is you’re trying to send around as part of an HTTP message, use an existing format, such as HTML, the main reason being that there are many processors that are able to understand it. Use the appropriate MIME identifier (such as text/html) in Content-type headers. One can make a strong case for this option: Hypermedia formats are hard to design, so you should avoid inventing your own.

But let’s assume you’ve decided to define your own hypermedia format, mike amundsen-style, whether by designing a completely new XML vocabulary, your own JSON structure, or some other approach: What MIME type do you use?

You can send content labeling it with the generic identifier, say application/xml. In this case, the MIME identifier will signal the technical format being used, while the semantics are only known to clients who either have some out-of-band knowledge or interpret the content itself. The rationale for this approach is that unless your home-grown hypermedia format is likely to be widely adopted, you’d better stick with a media type that is well-known, even though it doesn’t convey specific semantics. Duncan Cragg wrote a nice post on this a while back.

The second option is to invent your own MIME type, say application/vnd.mycompany-myformat, the argument being that you need to convey the semantics of the content to ensure a client, server or intermediary can actually know whether it’s able to process it.

This begs the question of how many different MIME types you’ll end up with. Instead of creating a new identifier for each type of content, (e.g. a customer, a list of customers, a list of orders), I’ve found that a good approach is to think of a specific context – a domain, if you prefer – that your format covers. I like the similarity of this approach to other hypermedia formats, e.g. HTML or Atom/AtomPub, where you actually end up describing something that applies to a set of collaborating participants, instead of some server-side interface. In my favorite example domain (order processing), you might end up with a MIME type of application/vnd.mycompany-ordermanagement, relate this to a particular XML namespace and define a few different XML root elements (order, order confirmation, cancellation, etc.). The assumption would be that processors can be reasonably expected to able to understand all of the elements in this context, not just one of them. (Of course the same reasoning applies when using JSON or something else, minus the namespace benefits/troubles, depending on your view of XML.)

One final approach that I find very interesting was mentioned by Jan Algermissen a while ago: If your format is based on an existing one, e.g. HTML or XML, your server can actually send the same content with different MIME types, depending on the client’s capabilities. A client that only included application/json in its Accept header would then get the content labeled application/json, while one that includes the specific MIME type application/vnd.whatever would get the same content with this label applied.

Google Reader’s extremely weird behavior after a first attempt to move my blog to a new location has made me try yet another different blog setup. This time, there should be nothing at all left from the old installation, no redirects, different URIs for everything, a new blog system – clearly, there’s no way for Google Reader to mess things up this time. Let that be the last thing said about this mess; even though it’s very sad to see all of those blog followers go who only occasionally drop by whenever something new appears in their Google Reader subscription, many of them may not have been following anymore, anyway. Who knows. It’s been a long time since I spent some serious time blogging.

Which doesn’t mean one can’t restart, right? We’ll see. For now, this is the one and only blog I’ll maintain. It has zero moving parts, i.e. it’s a static site, (currently) generated using Octopress. If everything works as expected, you’ll be able to subcribe to the an Atom feed that should actually be working for a change.

I’m somewhat undecided with regards to comments and the Twitter integration (both in the sidebar as well as in the bottom of each post). I’ve toyed with the idea of doing away with comments altogether, but I have to say that Disqus seems quite smart.

	"Library" = "Bibliothek";
	"Shop" = "Kiosk";

	UIA.Localizables = {
	"de":{
	...
	"Library" : "Bibliothek",
	"Shop" : "Kiosk",
	...
	},
	"English":{
	}
	...
	}

	UIA.getCurrentLang = function(){
		if(application.preferencesValueForKey("AppleLanguages")[0]  == "en")
			return "English";
		else
			return application.preferencesValueForKey("AppleLanguages")[0];
	}
	UIA.getCurrentLocalizables = function(){
		return UIA.Localizables[UIA.getCurrentLang()];
	}

	var Localizable = UIA.getCurrentLocalizables();

#import "lib/Localizables.js"

function	delay(seconds){
	UIATarget.localTarget().delay(seconds);
}

function tapTab(name){
	var window = UIATarget.localTarget().frontMostApp().mainWindow();
	window.tabBar().buttons()[name].tap();
}

var window = UIATarget.localTarget().frontMostApp().mainWindow();
tapTab(Localizable['Library']);
delay(1);
tapTab(Localizable['Shop']);
delay(7);

Picture 1: Example Application About Dialog

	#define APP_VERSION   \ 
	[[[NSBundle mainBundle] infoDictionary]   \  
	objectForKey:@"CFBundleVersion"]
	#define APP_EXECUTABLE   \ 
	[[[NSBundle mainBundle] infoDictionary]   \  
	objectForKey:@"CFBundleExecutable"]
	#define APP_NAME   \ 
	[[[NSBundle mainBundle] infoDictionary]   \  
	objectForKey:@"CFBundleName"]
	#define APP_BUILD_REVISION @"$Rev$"
	#define APP_BUILD_DATE @"$Date$"
	#define APP_LAST_AUTHOR @"$Author$" 
	

	svn propset svn:keywords 'Revision Author Date' version.h

	#!/bin/sh

	DATE=`date`
	HOST=`hostname`

	svn propset build "$HOST $DATE" Version.h

Picture 2: Project Target Settings

Picture 3: Insert Script Call

	#define APP_VERSION   \ 
	[[[NSBundle mainBundle] infoDictionary]   \  
	objectForKey:@"CFBundleVersion"]
	#define APP_EXECUTABLE   \ 
	[[[NSBundle mainBundle] infoDictionary]   \  
	objectForKey:@"CFBundleExecutable"]
	#define APP_NAME   \
	[[[NSBundle mainBundle] infoDictionary]   \  
	objectForKey:@"CFBundleName"]
	#define APP_BUILD_REVISION @"$Rev: 1047 $"
	#define APP_BUILD_DATE @"$Date: 2011-01-21 18:53:38 +0100 (Fri, 21 Jan 2011) $"
	#define APP_LAST_AUTHOR @"$Author: phaus $"

Kommt es zur Technik gelten in der Softwaretechnik vermutlich zwei Maxime:

Es gibt nichts Wichtigeres als die Auswahl einer stabilen technologischen Basis für die Entwicklung. Der Einsatz neuer Technologien steht dem entgegen. Gleichzeitig gilt jedoch: Es gibt nichts Wichtigeres als den Erhalt von Anpassbarkeit und Erweiterbarkeit eines Systems. Dies beinhaltet die Erneuerung oder den Austausch von eingesetzten Technologien und destabilisiert jede stabile technologische Basis.

Jeder Techniker trägt diesen Kampf in sich aus. Ständig suchen wir aufs Neue die Balance zwischen dem Wunsch nach stabiler, vorhersehbarer Softwareentwicklung und der Veränderung von Umgebung, Anforderungen und Rahmenbedingungen.

<?php
header('Content-type: application/json');
echo'{';
echo '"value":"' . rand(0, 60) . '"';
echo '}';
?>

{"value":"34"}

	function init(){
		for(var i=0; i < 300; i++){
			randomValues[i] = 0;
		}
	}

This step is optional, but then you have a nice "zero line" at the beginning.

Secondly we have a method, that pushes a new value to the existing array, and drops the first entry, if the length of the array is greater than 300.

	function addValue(arr, value){
		if(arr.push(value) > 300){
			arr.shift();
		}
	}

The next two methods are necessary for sending our ajax-request and for handling the response in a callback method. Basically the callback method just calls the addValue method.

The timeout variable is set to 200 ms. So the script calls our backend periodically every 200 ms and then adds a new value to our array.

	function pullValue(){
		sendRequest('random.php',handleRandomRequest);
		setTimeout(pullValue, timeout);
	}

	function handleRandomRequest(req) {
		var text = JSON.parse(req.responseText);
		addValue(randomValues, text.value);
	}

The last method is for the drawing functionality:

	function draw(){
		ctx.clearRect(0, 0, 300, 60);
		ctx.fillStyle = "rgba(101,101,101, 0.5)";
		ctx.fillRect (0, 0, 300, 60);
		ctx.lineWidth = 1;
		ctx.strokeStyle = 'blue';
		ctx.beginPath();
		ctx.moveTo(1, 60-parseInt(randomValues[0]));
		for (var i=1; i<randomValues.length; i++){
			value = 60-parseInt(randomValues[i]);
			ctx.lineTo(i,value);
		}
		ctx.stroke();
		setTimeout(draw, timeout);
	}

ctx is a 2d context of the canvas element. On every call of the draw method, all elements of the array are painted. The first element is always the start point. Because the canvas coordinate system has the point 0,0 in the upper left corner but the 0,0 point of our diagram should be in the lower left corner, you have to subtract the array-values from 60 to get the right drawing coordinate. This method also runs periodically every 200 ms. But it also works for two times for pulling the data an drawing it.

Here you can see the script in action

Recently I was in urged to do a web project with the latest and coolest web framework Ruby on Rails.

That's just one line of code grabbed from somewhere:

      next if entry =~ /^\./

Ruby is said to read like a natural language, but for my eyes this is just a cryptic sequence. Maybe I have the wrong natural language. So to clarify what it means I translated it to some assembly language:

      cmp (entryptr),'.'
      beq next

Yes: if the entry begins with '.', skip it.

The latter code might take 2 CPU cycles to execute.

How much will the Ruby code take? There is the line to be interpreted (syntax check, AST creation,... no idea), than a regexp engine is started which starts parsing the expression before it tries to find a match. All this is accompanied by lots of heap activity. My guess: 100.000 cycles. At least.

So it's 50.000 times slower than it needs to be to get the work done. Who needs a cluster of 50.000 machines to serve his application? Well, all others should be well of with a single server, even a poor one, if it only was programmed with care.

I know it's unfair and untrue, so please don't comment on that. But ... to some respect it is true.

How often have I read about cool new technologies where, after all, the programmer can now 'fully concentrate on the business logic'.

This meanwhile happens since at least 20 years so I wonder why still anybody does something else than concentrating on the 'business logic'?

One reason might be that it always has been a lie. Using any web framework still ends with tracking and analyzing HTTP requests or reading server logs, using persistence layers ends with monitoring the database, network traffic and such and so on. The promised abstraction just doesn't hold.

But - the main question is: Who - as a software developer - really wants to deal with business logic? Isn't business logic the most boring thing after all? What is the business logic of a webshop e.g.? Displaying a list, maybe filtered, putting things of that list onto another list (aka 'shopping cart'), sending an email, telling the backend. +1 here, -1 there. Nobody studied computer science for that. Business is all about lists and simple calculations. Let's face it: business logic is boring. Period.

I cannot imagine that our brightest minds cannot come up with a suitable solution for a shop in 20 years. Each year we have at least dozens of new-and-cool frameworks to solve that list-thing. +1. -1. Once again. How stupid.

Nevertheless programming computers can be exciting. That's why people try to get rid of doing business logic and do frameworks and technology. That's by far more appealing. Just to get that funded, they claim that later some poor guy can 'fully concentrate on the business logic'.

When reading http://www.eclipsezone.com/articles/what-is-iadaptable/ I got the impression that certain solutions always come back, are quite universal and independant of domain, language or environment.
The question here is about getting certain aspects of an object that are not statically declared in a yet typesafe manner. The Eclipse world came up with IAdaptable, as it seems.

public interface IAdaptable {
  Object getAdapter(Class clazz);
}

public class HashMap implements IAdaptable {
  public Object getAdapter(Class clazz) {
    if (clazz == java.util.List.class) {
      List list = new ArrayList(this.size());
      list.addAll(this.values());
      return list;
    }
    return null;
  }
// ...
}

To be used like that:

IAdaptable adaptable = new HashMap();
List list = (List)adaptable.getAdapter(java.util.List.class);

Well, this immediately brought back good-old COM, Microsofts Component Object Model back to my mind. I alway felt comfortable with COM because COM objects are just as plain C++ objects and thus very fast.
The basics are the same. Every object is just a IUnknown (which transforms into an IAdaptable)

[
  object,
  uuid(00000000-0000-0000-C000-000000000046)
]
interface IUnknown {
  [restricted]
  HRESULT _stdcall QueryInterface([in] GUID* rrid, [out] void** ppvObj);
  [restricted]
  unsigned long _stdcall AddRef();
  [restricted]
  unsigned long _stdcall Release();
}

For this is nothing useful, derived interfaces can be defined.

class IStandardMathFunctions : public IUnknown
{
  public:
  STDMETHOD(Add)(long, long, long*)
  STDMETHOD(Sub)(long, long, long*)
};

class IAdvancedMathFunctions : public IUnknown
{
  public:
  STDMETHOD(Fibonacci)(short, long*)
}

But despite static inheritance, every COM object can deliver all of its capabilities (which is, all the interfaces it implements) at runtime from the object using QueryInterface() which is pretty much the same approach as with IAdaptable.

Interesting enough, when I designed an abstraction layer for my pet project http://www.flyingguns.com describing the static 3D world, my solution was similar again.

public interface Feature
{
}

public interface SceneObject extends Disposable
{
  /**
   * Provide a certain feature if available.
   */
  <T extends Feature> T getFeature( Class<T> clazz );
}

public interface Movable extends Feature
{
  void set( Vector3f position, Quat4f orientation );
}

public interface Damageable extends Feature
{
  void setDamage( float damage );
  void destroy();
}

public interface Aircraft extends Feature
{
  SceneObject[] getEngines();

  void setRudder();
  void setAileron();
  void setElevator();
}

The approach has a lot of advantages.

• 100% typesafe, no casts due to generics
• features can be combined deliberately per object
• features may even change over time
• makes delegation easy, good for code reuse
• no runtime overhead
• small footprint: resources for unsused features don't need to be loaded e.g. no fire and smoke if nobody wants to damage an object.

Sample code looks like that:

SceneObject fokkerobject = scene.create( DefaultDatabaseSetup.FOKKER_DR1 );

Movable fokkermover = fokkerobject.getFeature( Movable.class );
fokkermover.set( new Vector3f( 0f, 700, 0f ), new Quat4f( 0,0,0,1 ) );

WeaponSystem fokkerweapons = fokkerobject.getFeature( WeaponSystem.class );
SustainedFire machinegun = fokkerweapons.getWeapons()[0].getFeature(SustainedFire.class);

machinegun.startFire();

SceneObject bullet = machinegun.createProjectile();
Movable bulletmove = bullet.getFeature( Movable.class );
bulletmove.set( machinegun.getNuzzlePosition(), new Quat4f( 0,0,0,1 ) );

Projectile hitter = bullet.getFeature( Projectile.class );
Projectile.Hit hit = hitter.findHit( 50 );
if( null != hit )
  	System.out.println( hit.getTarget() + " at " + hit.getLocation() );

I recently got pointed to that link: http://java.dzone.com/news/ant-18-scanning-leaves-171

I read the news with some pleasure reminding me that I still like Ant based builds very much over Maven in many cases.

Of course there are a lot of well maintained projects on the web that work very well with Maven. You never know how many enthusiasts-hours have been spent to make that happen. However, in smaller business projects I experienced the situation to be slightly different. Not a single one I came across ran out-of-the-box. Some actions had to be taken upfront, jars needed to be downloaded seperately, some repository got closed or moved away … and so on.

For the latest project of that kind I created an Ant based build which was really fun to do. It was so simple, fast, predictable, transparent. Together with the NetBeans' Ant environment including Ant debugger - fun!

Looking at the result, the build was fully canned, way faster (even with Ant 1.7 which is known to be slow now :)), the artifacts smaller. Everything is built right in place, no redundant copying of files which feels good from an esthetic point of view. The build.xml files are just a few lines and so much smaller than any pom.xml could ever be.

Interesting enough when monitoring the process: creating the Ant build from scratch took approximately the same time as bringing the Maven stuff to work.

I like the simple things.

In my famous company innoQ I currently have the opportunity to work on a real cool tool: Bundle-Bee. It claims to be able to take any OSGi bundle and distribute the computational load to an ad-hoc grid (e.g. all machines in an office) without special setup or configuration.

We just released version 0.5.3 which is still very restricted and far from feature complete - we don't even know what feature completeness might mean - but is already quite useful when it comes to doing computationally intensive things. Like fractals ... for example. Fractal computing is known to be computationally intensive and it may take long times to render the fascinating images. On the other hand, the nature of the problem gives a straight forward way to compute the 2D area in separated tiles and thus - in parallel. 

Bundle-Bee is about taking any OSGi bundle an distributing parts of that bundles computational code over a grid with (close-to) zero configuration. We did so e.g. on one of our company's meeting, where we ran the fractal computation on the attendees notebooks in parallel without any hassle. Just let them all start the Bundle-Bee equipped OSGi container (not a big deal) and they immediately make up an ad-hoc grid. Even one iPhone has been part of the game. The fractals jar is automagically flooded to the grid and execution load is split among all nodes.

Hence Bundle-Bee does the grid distribution transparently, there is no special notion of grid computing in the code at all. The brute-force Mandelbrot implementation is just like that:

public class MandelbrotAlgorithm {

   private static int iterate(Complex z0, int maxdepth) {
      Complex z = z0;
      for (int t = 0; t <= maxdepth; t++) {
         if (z.absSquared() >= 4.0)   return t;
         z = z.times(z).plus(z0);
      }
      return maxdepth;
   }

   public int[][] computeArea( Complex z0, Complex dz, int width, int height, int maxdepth ) {
      int[][] res = new int[width][height];
      
      for( int i=0; i
         for( int j=0; j
            Complex z = new Complex( z0.re()+i*dz.re(), z0.im()+j*dz.im() );
            res[i][j] = iterate(z, maxdepth);
         }
      }
      return res;
   }
} 

Just the main loop needs to go multithreaded which is always a bit tricky when dealing with an UI, but the pseudocode looks easy like that:

MandelbrotAlgorithm algo = new MandelbrotAlgorithm();

onUpdateRequest(area) {
  Collection tiles = divideToTiles(area);
  foreach(tile : tiles) {
    runAsThread() {
      int[][] result = algo.computeArea(tile);
      tile.paint(result);
    }
  }
}

Letting Java2D fly a bit, a kind-of-nice UI is not far away:

Now, by just packing that code into an OSGi bundle (which basically means to augment it with a handful of entries in the MANIFEST.MF), it is ready to be run on a Bundle-Bee grid with nearly unlimited computing power. 

Sounds too easy, eh? But it actually is....

Happy New Year folks!

In 2002, I wrote a highly specialized, very small Q&D tool for my brother to support him in his oncological doctoral's practice. They are three physicians sharing a common room with some unique ultrasonic device. They needed something showing them at their desk wether the room is available or is currently occupied by somebody else. The name 'dokma' is a German pun.

Admittedly not very pretty and I definately need to give it an UI overhaul.

Those days, I used RMI (still with 'rmic' to generate the stubs, supported by a NetBeans RMI plugin). The tool worked flawlessly since then. Yesterday, I did a small extension (allow for more physicians) and thereby upgraded to the new, stub-less RMI scheme.

While doing so, I remembered how cool RMI is. Today people tend to think of web technologies for everything that sounds like networked operation, starting with a tomcat as a bare minimum. But in this case, RMI is way easier and way cooler, for it allows to transparantly use callbacks for 'room-occupied' notifications and a tiny, integrated chat function.

Very hard and cumbersome to achieve with web techniques. And RMI comes with the JRE, so the whole application still fits into a 50kb jar file and zero installation and configuration effort.

A point worth discussing: is RMI still the way to go today? What do the experts say?

Do you know that feeling? Something annoys you and you have to tell somebody? This is the bare reason for this blog entry. Don't read it. It is just for me to make me feel better.

When I joined my new company in January, they let my choose my weapons freely. For I've heard so much about these Apple machines and how cool they are and so far ahead of the Windows world, I was curious and ordered one for myself. MacBook Pro 17". Very new, very expensive. Must be great.

As my grandma used to say (or was it Forrest Gump?): if you say something, try to say something kind.

The display, the battery and the touchpad are great. Outstanding, excellent.

The rest was quite disappointing. First, besides the lack of some keys (PgUp/Dn....), many very important key labels are just missing (at least on the German keyboard). []{}~|\ ... try to find them. Very annoying on a coders machine. And why? My old, cheap discounter notebook had them.

The aluminium case looks very stylish but unfortunately is quite thin-skinned and soon had a first bump which no longer looks so cool. But worse, the bottom edge is very sharp and tries to shave my wrist. Which might be part of the plan, for it often gets quite hot on the lower left side which might be easier to bear with if your skin is properly shaved.

Software ... well. The darkest part maybe (I'm an UI guy, so maybe I'm too sensitive here). Try to find a svn client. You have to buy it or go back to the commandline which always gives me that nice feeling of the 80th ... spooky. The Finder needs to go back to usability lab. Even Mac-enthusiasts do say that. Explorer is far ahead! The basic desktop handling is clearty motivated by technical reasoning instead of by a users perspective. Use cmd-tab to switch process, than use cmd-shift-' (German keyboard) to switch applications windows ... what the f**k? Users deal with windows, they don't care for processes.
What is the difference between closing, hiding and minimizing an applications window? Very subtle. I never found out....
And so on and so on ...

Anyway, often it is very hard to get that far. Instead, I admire the spinning beach ball of death. This always seems to happen if MacOS runs out of physical memory. Which is ... always. Or the camera doesn't work any more. For that, Apple very frequently publishes system updates that require a reboot. I assume this is a trick...

Finally, Java applications are somewhat different than they used to be on windows. They feel sluggish and less responsive. I use VirtualBox with Windows if I want to work with NetBeans e.g. (not a joke!).

I don't say its a bad thing overall, it's ok, some things are really nice. The blog just emphasizes on the annoying parts. But next time I'll save my companies money, and will buy a common notebook for less than half. I have no idea where Macs reputation comes from but I suspect it is kind of reality distortion and wishful thinking. It's a linux for the rich.

Did I mention that I also ordered an iPhone? :)

Hiermit komme ich der Bitte nach, ein paar Buchempfehlungen zu geben, die man als praktischer Informatiker gelesen haben sollte. Es gibt tolle theoretische Bücher oder Grundlagenwerke. Es gibt Bücher, die sich eher auf Projektmanagement beziehen. Und es gibt Bücher, die sich auf das Selbstverständnis und das Handwerkszeug beziehen.

Es fällt mir schwer, meine persönliche Top 10 zusammenzustellen. Bemerkenswert ist, wie alt die wertvollen Bücher im Schnitt sind… Hier ist die Liste der nicht spezifischen Bücher, die sich eher mit zeitlosen Themen befassen:

* Code Complete
* Pragmatic Programmer
* Mythos des Mann-Monats
* Objektorientierte Softwareentwicklung Analyse und Design mit UML 2.0
* Refactoring
* Extreme Programming
* Business Components Factory
* Analysis Patterns
* Programming Pearls
* Release It

Eigentlich müsste man natürlich noch The Art of Computer Programming aufführen, allerdings kenne ich nur einen der das Buch lesen und durcharbeiten kann. Die anderen Menschen, die ich kenne, sind nicht bereit dazu, sich mit der hohen Informationsdichte und Komplexität auseinander zu setzen.

Darüber hinaus könnte man noch ein paar hervorragende Programmiersprachen-spezifische Bücher aufführen, die sicherlich einen Platz in der Top 10 einnehmen könnten. Meine Liste der tollen Management-Bücher habe ich vor gut zwei Jahren vorgestellt: Empfehlung zu Management-Büchern

Preisfrage: Welches der obigen Bücher sollte man mit ganz großer Vorsicht lesen und besser noch doppelt und dreifach darüber sinnieren?

Das Internet, das wir alle kennen und täglich nutzen, durchdringt nicht nur den Alltag immer stärker sondern findet durch eine Bewegung anerkannter IT-Experten immer größere Beachtung. Bis vor kurzem wurde stark bezweifelt, dass die für das Internet verwendeten Dienste und Anwendungen ausreichen könnten, um ernsthafte Geschäftsanwendungen zu realisieren. Selbstverständlich sah man lange Zeit in Amazon, Google und Co. die obligatorischen Ausnahmen, die für einige aus diesem Zweifel eine Regel machten.

Dank technisch versierter Experten wie Fielding, Tilkov oder Vinoski wird mittlerweile auch den Anhängern der klassischen Unternehmens-IT-Disziplinen klar, welche Vorzüge in dem Architektur-Stil des weltweiten Datennetzes (des World Wide Webs) stecken. Rhethorischen Fragen a la “Warum findet man mit einer Internet-Recherche mehr Informationen über einen bestehenden Vertrag mit einem Vertragspartner als in Eurer Bestandsführung?” treffen einen Nerv und decken architekturelle Schwächen gängiger Unternehmensanwendungen auf.

Dennoch: Gerade in einer Zeit, die das Zusammenbrechen technischer Barrieren verspricht, gefährdet eine aktuelle Entdeckung den Fortbestand des Internets, wie wir es kennen. Die IP-Adressen - sozusagen die Telefonnummern des Internets - sind aus.

Ohne eine flächendeckende Umstellung auf das neue Protokoll IPv6 wird das Internet nicht mehr lang funktionieren. Der selbst ernannte IT-Experte und System-Analytiker, Max Mustermann, prognostiziert einen Zusammenbruch innerhalb der nächsten 2 hoch 128 Tage. Es bleibt also weniger Zeit als gedacht.

Darüber hinaus führt eine kürzlich entdeckte Lücke in der Spezifikation des Internets (nachzulesen unter ietf.org) in letzter Konsequenz dazu, dass jeder Nutzer eines herkömmlichen Web-Browsers sich angreifbar macht und was noch schlimmer ist, selbst unbeabsichtigt zu einem Angreifer für Internet-Angebote werden kann. Die Angriffsform ist auf Neu-Deutsch unter dem Akronym CSRF (wie sea-surf) bekannt. ZDNet sagt dazu unter Anderem:

In either case, the browser executes a malicious transaction such as a wire transfer to the cybercrook’s bank.

Frei übersetzt bedeutet dies: Man kann hier von einem immer schnelleren Verbrauch der jetzt schon kaum mehr verfügbaren IP-Adressen ausgehen. Schuld daran ist die Architektur des Webs - genannt REpresentational State Transfer - oder kurz: REST.

1. SAX-basiertes Parsen der Datei und einlesen in eine Datenbank (aus Performance-Gründen wird hier H2 als inMemory Datenbank genutzt).
2. Um schnelle Vergleiche zu ermöglichen wird ein Modell benutzt, welches u.a. auch in ZFS angewendet wird: Erkennen von Veränderungen anhand von Prüfsummen. Was bei ZFS dazu benutzt wird, um Änderungen innerhalb des Dateisystems zu erkennen, soll hier dazu dienen, Unterschieder zwischen zwei XML-Modellen schnell und zuverlässig zu erkennen. Hierzu wird für jeden Knoten eine Prüfsumme berechnet. Diese leitet sich jeweils aus den Prüfsummen seiner Kindsknoten, dem Inhalt seiner Attribute und dem Wert des Knotens ab. Momentan wird über diesen Gesamt-String ein SHA1-Hash gebildet. Eine weitere Prüfsumme wird benötigt, um den Knoten innerhalb des Modells zu lokalisieren (wir verwenden hier den XML-Path+Knotennummer):
   1. <--                   Hash des Pfades                   --> 3a52ce780950d4d969792a2559cd519d7ee8c727  ./items/item/value           
   2. <--             Hash des Knotens item              --> 481e6ff69a8402231a3b9c6e46a7fef4f09adbb3 hash von: "item", attribute "name=b", hash von "value"
3. Da sowohl eine Aussage über Unterschied im sortierten Zustand - die Reihenfolge der (Kinder-)Knoten ist wichtig - als auch im unsortierten Zustand (Die Reihenfolge der Kinder-)Knoten ist egal, wird vor dem Berechnen des Hashes des Kindes eines Knotens, die Kinder einmal unsortiert und einmal sortiert als Basis für den SHA1-Hash genommen.

import javax.swing.JOptionPane;

import org.gpg.java.GnuPG;

public class Loader {

public static void main (String args[]){
	GnuPG pgp = new GnuPG ();
		
	String toolChain[] = {"sign", "clearsign", "signAndEncrypt", "encrypt", "decrypt"};
	String message = JOptionPane.showInputDialog(null, "Message you want to encrypt?", "Enter your message", JOptionPane.QUESTION_MESSAGE);
	String keyID = "0x56B69D6B";
	System.out.println("using message: "+message);
	System.out.println("using key ID: "+keyID);
	for(String tool : toolChain){
		System.out.println("running: "+tool);
		if(tool.equals("sign")){
			String passPhrase = enterPassPhrase(tool);
			pgp.sign (message, passPhrase);				
		}

		if(tool.equals("clearsign")){
			String passPhrase = enterPassPhrase(tool);
			pgp.clearSign (message, passPhrase);				
		}			
		if(tool.equals("signAndEncrypt")){
			String passPhrase = enterPassPhrase(tool);
			pgp.signAndEncrypt (message, keyID, passPhrase);				
		}
		if(tool.equals("encrypt")){
			pgp.encrypt (message, keyID);				
		}	
		if(tool.equals("decrypt")){
			String passPhrase = enterPassPhrase(tool);
			pgp.decrypt (message, passPhrase);				
		}				
		System.out.println("result: " + pgp.getGpg_result() + "\n\n");
		System.out.println("error: " + pgp.getGpg_err() + "\n\n");
		System.out.println("exit: " + pgp.getGpg_exitCode() + "\n\n");
	}
}
    
    public static String enterPassPhrase(String usage){
    	return JOptionPane.showInputDialog(null, "Please enter the Passphrase of your private Key for "+usage, "Passphrase", JOptionPane.QUESTION_MESSAGE);
    }

}

	public synchronized void store(OutputStream out, String comments)
			throws IOException {
		
		TreeMap propTree = new TreeMap();
		
		for (Enumeration e = keys(); e.hasMoreElements();) {
			String key = (String) e.nextElement();
			String value = (String) get(key);
			key = saveConvert(key, true);
			value = saveConvert(value, false);
			propTree.put(key, value);
		}
		BufferedWriter awriter;
		awriter = new BufferedWriter(new OutputStreamWriter(out, "8859_1"));
		if (comments != null)
			writeln(awriter, "#" + comments);
		writeln(awriter, "#" + new Date().toString());		
		Set keys = propTree.keySet();
		for (Iterator iterator = keys.iterator(); iterator.hasNext();) {
			String key = (String) iterator.next();
			writeln(awriter, key + "=" + propTree.get(key));			
		}	
		awriter.flush();
	}

• Logs werden nun gzip komprimiert
• wöchentliche Backups aus der vorherigen Woche werden tar.gz komprimiert

    development:
      adapter: jdbc
      driver: org.apache.derby.jdbc.ClientDriver
      url: jdbc:derby://localhost/microblog_development;create=true
      encoding: utf8
      pool: 5
      username: microblog
      password: microblog
      host: localhost

namespace :microblog do
  desc 'Import old SQL Data'
  task :sqlimport => :environment do
  dbConn = ActiveRecord::Base.establish_connection :development
  sql = File.open("db/microblog.sql").read
  sql.split(';').each do |sql_statement|
    dbConn.connection.execute(sql_statement)
  end
  puts "imported user data '#{Time.now}' "  
  end
end

#!/bin/sh

BASE_DIR=`pwd`
CP=".:$BASE_DIR/jruby/lib/*:$BASE_DIR/derby/lib/derbyclient.jar"
JAVA_OPTS="-Djdbc.drivers=org.apache.derby.jdbc.EmbeddedDriver"
JRUBY="$BASE_DIR/jruby/bin/jruby"

DERBY_HOME=`cd derby && pwd`
export DERBY_HOME=$DERBY_HOME
cd $BASE_DIR

echo "setting up jgems..."
$BASE_DIR/jruby/bin/jruby $BASE_DIR/jruby/bin/jgem update --system
$BASE_DIR/jruby/bin/jruby $BASE_DIR/jruby/bin/jgem install jruby-openssl --no-rdoc --no-ri
$BASE_DIR/jruby/bin/jruby $BASE_DIR/jruby/bin/jgem install -v=2.2.2 rails --no-rdoc --no-ri
$BASE_DIR/jruby/bin/jruby $BASE_DIR/jruby/bin/jgem install activerecord-jdbc-adapter activerecord-jdbcderby-adapter --no-rdoc --no-ri

echo "starting derby..."
$BASE_DIR/derby/bin/startNetworkServer &

echo "setting up derby..."
cd microblog
$BASE_DIR/jruby/bin/jruby $BASE_DIR/jruby/bin/rake db:migrate
$BASE_DIR/jruby/bin/jruby $BASE_DIR/jruby/bin/rake microblog:sqlimport
cd $BASE_DIR

echo "stopping derby..."
$BASE_DIR/derby/bin/stopNetworkServer &

#!/bin/sh
BASE_DIR=`pwd`
CP=".:$BASE_DIR/jruby/lib/*:$BASE_DIR/derby/lib/derbyclient.jar"
JAVA_OPTS="-Djdbc.drivers=org.apache.derby.jdbc.EmbeddedDriver"
JRUBY="$BASE_DIR/jruby/bin/jruby"
export BASE_DIR=$BASE_DIR
export JRUBY=$JRUBY

DERBY_HOME=`cd derby && pwd`
export DERBY_HOME=$DERBY_HOME
cd $BASE_DIR


echo "starting derby..."
$BASE_DIR/derby/bin/startNetworkServer &

echo "setting up derby..."
cd microblog
$BASE_DIR/jruby/bin/jruby $BASE_DIR/jruby/bin/rake db:migrate

echo "starting microblog"
$BASE_DIR/jruby/bin/jruby $BASE_DIR/microblog/script/server

echo "stopping derby..."
$BASE_DIR/derby/bin/stopNetworkServer &

#!/bin/sh
find . -name '*.DS_Store' -type f -delete
jar -cvf statusQ-runtime.jar derby/ jruby/ run.sh pack.sh microblog/db/microblog.sql microblog/lib/tasks/sql-import.rake
rm -R jruby
rm -R derby
rm run.sh
rm microblog/db/microblog.sql
rm microblog/lib/tasks/sql-import.rake
rm pack.sh

#!/bin/sh
jar -xvf statusQ-runtime.jar
rm -R META-INF
chmod +x run.sh
chmod +x setup.sh
chmod +x pack.sh
chmod +x jruby/bin/jruby
chmod +x derby/bin/startNetworkServer
chmod +x derby/bin/stopNetworkServer

BASE_DIR=`pwd`
CP=".:$BASE_DIR/jruby/lib/*:$BASE_DIR/derby/lib/derbyclient.jar"
JAVA_OPTS="-Djdbc.drivers=org.apache.derby.jdbc.EmbeddedDriver"
JRUBY="$BASE_DIR/jruby/bin/jruby"

DERBY_HOME=`cd derby && pwd`
export DERBY_HOME=$DERBY_HOME
cd $BASE_DIR

echo "setting up jgems..."
$BASE_DIR/jruby/bin/jruby $BASE_DIR/jruby/bin/jgem update --system
$BASE_DIR/jruby/bin/jruby $BASE_DIR/jruby/bin/jgem install jruby-openssl --no-rdoc --no-ri
$BASE_DIR/jruby/bin/jruby $BASE_DIR/jruby/bin/jgem install -v=2.2.2 rails --no-rdoc --no-ri
$BASE_DIR/jruby/bin/jruby $BASE_DIR/jruby/bin/jgem install activerecord-jdbc-adapter activerecord-jdbcderby-adapter --no-rdoc --no-ri

echo "starting derby..."
$BASE_DIR/derby/bin/startNetworkServer &

echo "setting up derby..."
cd microblog
$BASE_DIR/jruby/bin/jruby $BASE_DIR/jruby/bin/rake db:migrate
$BASE_DIR/jruby/bin/jruby $BASE_DIR/jruby/bin/rake microblog:sqlimport
cd $BASE_DIR

echo "stopping derby..."
$BASE_DIR/derby/bin/stopNetworkServer &

… dann wäre ich wohl ein 3D-Avatar.

Denn dieses Bild wurde mit Terragen gerendert.

Da habe ich am Mittwoch auf der OOP-Konferenz Tom DeMarco gehört. Er hat allen Ernstes gesagt, dass er die Texte eines Kollegen dadurch verbessert, indem er sinnlose Füllsätze einfügt. Dessen Texte seien sonst immer so dicht mit Informationen gepackt, dass man beim Lesen zu viele davon überliest. Durch die Füllsätze bekomme das Gehirn Zeit, alle Informationen zu verdauen.

Meiner Ansicht nach sollte man auf Füllsätze verzichten und Informationen kurz und präzise darstellen. Die Form eines Textes so anzupassen, dass auch einem unaufmerksamen Leser darin enthaltene Kernpunkte auffallen, sollte meiner Meinung nach durch sinnvolle Übergänge und Erläuterungen erfolgen. Sinnlose Füllsätze haben da nichts zu suchen. Frau Minto beschreibt in ihrem Buch Das Prinzip der Pyramide, wie man seine Informationen klar strukturieren kann.