April 14, 2004

iQgen 2.0

My company, innoQ, has just released version 2 of its model driven software generator, iQgen. I have mentioned the product a few times before; if you are considering to try out MDA in a manner that leaves architectural choices to you, give it a try (it's commercial, but free for personal and non-commercial use).

Expect some related posts within the next few weeks.

April 12, 2004

Web Services Integration Scenarios and UML

Eric Newcomer follows up on our discussion with regards to UML's suitability for Web services.

First of all, Eric states that UML is not suited very well for Web services, since they are typically used for integration scenarios across application domains. The easy, MDA-zealot answer would be that of course you can UML for integration purposes. In fact there is a two-year-old, 304-page specification, the UML for EAI profile, that is explicitly aimed at this (it's a bit out of date, with mappings just to MQ Series and JMS, but so what). I don't have any experience with it, though, and I'm highly skeptical about publicly adopted UML profiles anyway; so I'll avoid that easy answer.

I can follow Eric's argument that basically, UML (not MDA ;-) ) has been designed to build new systems, not to integrate existing ones. Still, often you build a new system and need to integrate several existing ones into it. When you're building a system today, you'll also equip it with a Web services-enabled interface right from the start, and not add one later. In these cases, you need to do both "classical" as well as "service-based" development, and that is where I believe UML-driven Web service creation makes a lot of sense.

Even if this is not your use case, I'm not sure I agree that learning UML in order to use Web services is overkill. I'll try to back that up with a nice example, stay tuned (I'll need some spare time to do this, as well as to publish some information about the project I was referring to). It basically seems to boil down to the question whether you know one (Web Services or UML) already. Picture yourself not knowing either; what would be easier to understand — a UML class diagram, or an XSD/WSDL file?

When I'm not constrained by the borders of reality — i.e. when I stop thinking about what is actually usable today, and allow myself to think about what I believe would be ideal — I feel that it's actually MOF that I would really love to be able to use, instead of "just" UML. With a MOF-based meta-CASE tool, the Web services world would have its own Service Modeling Language (SML™), perfectly suited to the task at hand, and easy to integrate with the rest.

April 05, 2004

Metamodel standards: EMF and MOF

An interesting presentation by David Frankel on Eclipse's EMF vs. OMG's MOF, and the associated standards.

April 04, 2004

UML to model Web Services?

Eric Newcomer answers some of my criticisms of his recent posting on MDA.

Great to see that we're not that far apart, and kudos to Eric for openly acknowledging a slight change of mind — something you don't find very often. Interestingly, many opponents of code generation tend to compromise on the viability of doing one-shot generation without later on going to back to the model, while I believe that this is even worse than not doing code generation at all.

One interesting point remains: Eric believes UML to unsuited for modeling Web services. I see no reason why this should be the case. I strongly agree that Web services are not distributed objects, but of course UML can be used to model non-OO concepts as well. In fact, we are using UML in one of our projects to model Web services, and we definitely do not take a distributed objects approach.

Maybe I should illustrate this using an example. Anyone got a nice Web service I could use? ;-)

March 28, 2004

More MDA Critique

IONA's Eric Newcomer has some thoughts about MDA as well. Sigh.

I wonder when people will start to understand that MDA is not about pictures, but about meta-information, and that an MDA approach does not mean that programming languages are obsolete. The key is to use model and PL where they are appropriate — instead of forcing everything into one pattern.

March 18, 2004

Responding to Dave Orchard's Take on MDA

Dave Orchard had some thoughts about MDA, and I commented on his blog entry very briefly. I'll try to address each of his claims in more detail below. I have elaborated many of the points in an older posting; it might be a good idea to read it if you're looking for some initial discussion.

Dave says:

The first lesson to learn for MDA is that "abstractions", like a logical model, typically worsen performance and there is a need for selectively optimizing performance.

First of all, I think a logical model is something that you just can't avoid having — if you design any sort of application, you'll have a conceptual model of your entities, their relationship, your processes, and so on, regardless of whether you actually draw this as a UML model in some fancy CASE tool, on a napkin, or just keep it all in your mind. You will think of he Customer having n Contracts, and each Contract referring to n Articles, or something like that; it's highly unlikely that you will think of your application's concepts in terms of the underlying technology.

The point Dave is trying to make is, if I guess correctly, that having a uniform way for mapping this logical model to a specific implementation strategy worsens performance. That's true, and it is a common trade-off: Uniformity will make your system easier to understand because things look more or less the same; it will not offer the same performance as the manually tuned solution that can take all of the specifics into account.

While I agree in principle, I believe that in every (complex) application, you will have some guidelines that describe the default "mapping". In most J2EE applications, for example, you will decide on one general strategy - such as "We will use EJB CMP" or "we'll use Hibernate" - and stick with it, except for the places where it's justified to deviate from your own standards.

As you will always have a standard way of doing things, an MDA approach will automate this, saving you time; it will localize the concept in a single place, so that it's easy to understand; it will increase quality, since you'll be doing it the same way automatically; and it will allow you to evolve your strategy should you find out that it can be improved.

Dave continues:

We had a bigger problem though: The developers hated it. Somebody would create the logical model, they'd push the "generate" code button, and then run the software. But guess what, they always got the model wrong. Maybe they forgot about the zip code in the address, or the middle name in the name structure. So the model needed to be updated.

So far, I see no problem — you of course don't get everything right the first time, so you'll have to update and change your model. I fail to see anything particularly tied to an MDA approach so far. The actual problem seems to be described in the next two sentences:

To make a simple change in the model and then generate took way too long. It could take up to half an hour before the system could be retested. The devs simply would make the change they needed in the place they needed it. For example, the SQL already had the zip code so they only needed to add the zip code in the Java and in the SQL select.

If it takes half an hour to do this, something is wrong with your tool set. There are generators that can generate hundreds of files in under a minute; depending on the CASE tool, the XMI export itself may sometimes take longer than the actual code generation step.

Even more importantly, in my experience the example given ("they only needed to add the zip code in the Java and in the SQL select") is an argument that clearly shows where the strengths, not the weaknesses of MDA lie: In a real-world architecture, you'll have to change the zip code in the SQL DDL, in the query string (as well as in the update and create statements); you'll have to add it in the place where you extract the data; you'll have to add it to some Java object, create getters and setters; extend some interface (e.g. in one of your EJBs); might have to add it to some FormBean, modify the code that copies it from some back-end object to the front-end object, etc. etc.

Saving time by doing this manually? I don't think so.

Next quote:

I assert that MDA systems almost invariably suffer from the "design documents collect dust on bookshelves" problem, despite best attempts of the tools and organization to stop the natural entropy

I assert that anytime you use modeling, UML or not, and you don't use an MDA approach, you will suffer from this problem (since the model just is part of the initial design documentation). With MDA, you make your model part of the development artifacts; it can't get out of sync (that would be like part of your source code getting out of sync). Of course you have to enforce this, but this is no different than disallowing people from modifying the byte code emitted by javac.

The technology simply introduces an artificial layer of abstraction that is too difficult to modify and build high performance systems.

By the same logic, I could say that EJB CMP introduces a layer that makes it impossible to build high performance systems. Or Java. Or maybe even an RDBMS. After all, not being tied to the relational abstraction will allow me to build a faster system - right?

Stripping down the rest of Dave's posting, I believe the key statements are

A final problem with MDA systems is that they don't really solve the hard problem. [...] A final problem with MDA systems is that they don't really solve the hard problem.

I totally agree; anybody who claims that MDA solves all problems, and turns it into a silver bullet, is clearly wrong. I believe that software development is hard; but it's hard enough to deal with even with an MDA approach. Without, it becomes unnecessarily harder - I see no reason to do this.

Dave's summary is:

In conclusion, I believe that MDA systems solve a small portion of systems development and will typically suffer from the "stale design" and performance problems. I think the path forward for software development is perhaps to use MDA as a prototyping exercise, but the real productivity gains will come from ever increasing productivity tools (like better GUIs, APIs and programming models) and increasing metadata.

As I hope to have made clear, I strongly disagree with the first part of this statement. Using MDA for prototyping only clearly misses the real benefits — and in my view, MDA is only about increasing productivity, nothing else.

I'm not a fan of statistics, and I don't particularly believe in specific case studies being generalized to an overall productivity gain. But I know from experience that especially in scenarios where you have a big code base, a strong set of architectural guidelines, and an application that is more data-centric then process-centric, the productivity gains are significant. Especially with EJB/CMP-based backends, the percentage of generated code (for this layer) can be up to 100%.

March 11, 2004

DSM vs. MDA

Michael Platt has a well-written explanation about the differences between MDA and the DSM approach favored by Microsoft. I agree with some of the criticisms, while I still hate to think that we'll be left with a new Microsoft-vs.-Rest-of-the-World battle.

March 10, 2004

Whitehorse - Microsoft's Modeling Tool

Microsoft's Keith Short has some information, including screenshots, about Microsoft's new modeling tool, Whitehorse (some more information here, here and here). Very interesting, although the comments about keeping code and model in sync remind me of other flawed efforts.

March 09, 2004

Canvas: Groovy Templating

Cedric Beust has created a prototype of a templating engine based on Groovy. Very cool.

February 17, 2004

MDA blog

Mike Burba, product manager for Compuware's OptimalJ, has a blog [via Charlie Betz]

February 08, 2004

Platform Independent Models, Platform Dependent Models, and Code Generation

I got into a very interesting discussion a few days ago: Where exactly can you find a PIM and a PSM in both in the MDA vision as such as well as in existing tools such as iQgen. My view on this issue is:

In MDA:

• A Platform Independent Model (PIM) is the starting point.
• The PIM is turned into a Platform Specific Model (PSM) by means of a transformation. This can be done either manually or automatically.
• The PSM can be transformed again, yielding an even more platform specific model - other words, a new PSM, targeting another platform.
• The final PSM is turned into code.

In "Pragmatic MDA" or "MDA light" or "MDA as it exists today" ...

• A Platform Independent Model (PIM) is the starting point, although it might be called a PSM with some justification (as it will usually contain at least information to target an "abstract" platform)
• The PIM is turned into code by means of an automatic transformation.

So is there a PSM in the second approach? If so, where?

In my opinion, the PSM, if you insist on finding it, is the code itself. How can the code be a model? A model, in general, is a simulation of some concept or aspect from a specific point of view, intended to enable reasoning about it for a particular purpose. If I take the resulting code in the second approach from above, I can look at it in an IDE, the purpose being to edit in its textual form. If I read the same code into a 1:1 CASE tool such as Together, I see a visual rendering; the purpose of which might be to manipulate it visually or explain it to somebody else.

February 06, 2004

Michael Guttmann vs. Steve Cook

Michael Guttmann's response to Steve Cook's article (something I commented on before) is now online, and right on the mark.

February 03, 2004

A critique of Fowler's MDA critique

I promised to follow up with some more thoughts on Fowler's posting, so here we go ...

First of all, I really like Martin's work - I'm a long time admirer of his books and own about three of them. In this particular case, though, I believe that he is wrong. That would be OK if he were some unknown guy nobody cares about (like me ;-)), but because he is not, and because our industry as a whole tends to attach guru status to people and believe everything they utter once they have it, I believe that he is actually causing harm.

I am a strong proponent of what I call Pragmatic MDA. Pragmatic, because I don't believe any technology can be a silver bullet, and not every thing that is mentioned in the context of the current MDA hype is worth the paper it's written on (or the bits it's encoded in. But I digress.) So let me state some of my own beliefs about Pragmatic MDA up front:

• Pragmatic MDA is not about expressing everything in the model, and most definitely not in graphical notation. I don't believe in executable UML unless it's applied to very restricted domains, such as embedded systems (and I state this only because I know next to nothing about that domain).
• A tool that generates code according to some internal, unmodifiable, hidden set of rules, is not an MDA tool. To me, the ability to modify every aspect of model and/or code generation is essential for the approach to make sense.
• Being a proponent of MDA does not mean that one considers UML, especially 2.0, or XMI to be great. In fact, I totally agree that they have their own set of problems.

MDA, in my book, is about providing the developer with a tool set that enables him (or her) to build and modify the system in a way that's a lot easier than the traditional approach. Instead of spreading knowledge about design patterns, programming language idioms and platform details across a multitude of different artifacts, where they are mixed with domain/business knowledge, this technical knowledge is localized in code generation rules. This works to a certain degree, not perfectly; it still beats the approach to code everything by hand by a very significant percentage.

Let me address some of Martin's criticisms in more detail.

• First of all, Martin gives the example of activity diagrams that are, in his opinion, not better than writing code in a modern language. That's absolutely true! It's also absolutely irrelevant. If something is more efficient to do in a programming language, by all means use that. That does not mean that everything is easier to write in code; if it were, nobody would have come up with the MDA idea in the first place. (As an aside, our own tool, iQgen, supports code generation from activity diagrams. And guess what? We have never used this feature (although we plan investigate whether it might be appropriate e.g. for generating a Struts configuration file).) This is clearly a side-effect of some silver-bullet promises that have been made about UML and MDA, which backfire now.
• If there were a modeling standard that focused not so heavily on graphical notations, were simpler, and as accepted as UML, I would happily advocate that. Sadly, there is none; and while coming up with your own XML language for marking up software structure is pretty easy, there would be no tools to support it. So UML, while not that great, is very clearly the dominating standard. If the alternative is having to write boiler-plate code by hand again, I'll opt for UML anytime.
• Martin criticizes the platform independence, and he is right in doubting the .NET/Java platform claim made by the OMG. Again - I agree. But he misses the point again; in our projects, a platform is not lower-level than .NET or Java, but higher-level. For example, your platform might in one project be AIX/Weblogic/Entity Beans/Session Beans/Struts/JSP, and Solaris/Jboss/Hibernate/Session Beans/Systinet WASP/Tapestry in another. (And in both scenarios, you are likely to have built your own framework layer on top.) In these scenarios, it' actually pretty easy to hide a lot, if not everything, of those platform decisions from your model.
• Generating code, some say, is a code smell, and to a certain degree, this is true. But it's sad fact of life that most mission-critical systems - which are those my company focuses on - are composed to large degree of boring, repetitive, monkey-level code. I have yet to see a mainstream application that does this differently. Using the MDA approach to automate this is not something to be done instead of using frameworks, it's something to be done in addition. The parts of the system that are generated are not source code anymore; the source code is the model + the code generation templates. (In fact, there is some similarity to the macro approach in languages such as Lisp or Scheme. I'll accept criticism from Lisp hackers any day - yes, we are re-inventing things. Such is life.)
• Pictures are not better than text - at least not generally. But MDA is not about pictures, it's about expressing structural (and sometimes behavioral) information in a standardized way.
• Martin's final point about model driven tools that don't use MDA standards is well taken. In fact, I have argued for alternative, more programmer-oriented input formats in the past. Still, the MDA standard are what is currently available, and I much prefer a less-than-perfect standard to none at all.

Comments and feedback are very welcome!

January 30, 2004

Ambler on UML 2.0

Quicklink: Scott Ambler on UML 2.0 (free registration required). [found via Connections]

January 23, 2004

Fowler on Architecture

Hendrik Schreiber (who, BTW, definitely should have a weblog) sent me an excellent article by Martin Fowler, Who needs an architect. Not exactly new anymore, but very worth reading. Unsurprisingly, I have my own thoughts on making software architecture changeable.

January 22, 2004

Microsoft and MDA, cont.

An interesting article was mentioned to me today: Microsoft Zeros in on Model-Based Programming. It's pretty interesting to compare this to Steve Cooke's take.

January 20, 2004

Microsoft FUD against MDA

Charles Betz has linked to an article by Microsoft's Steve Cook on MDA. Read it, the come back.
The article contains an important lesson: how to create FUD in the most sophisticated way. These two paragraphs are a good example:

Nevertheless, an approach like MDA can indeed be helpful for some application areas. Some vendors have shown that building a Web application to run within the J2EE (Java 2 Platform, Enterprise Edition) environment can be simplified by building UML models of the data entities and components involved, and mapping these models into the various J2EE artifacts. However, as we mentioned earlier, for developers to adopt such an approach it is vital for the overall process to be agile and to not introduce unnecessary transitions and barriers such as irreversible code-generation and debugging problems.

Building J2EE apps with an MDA approach is not something "some vendors have shown", it's very quickly becoming the standard way of building sophisticated enterprise applications. Trying to create some sort of lock-in between MDA and J2EE is totally misleading. We have used our own and other MDA products, as have our competitors, to aid customers in building applications based on a wide range of platforms, which of course include, but are not limited, to J2EE - ranging from CORBA to COBOL. It's definitely true, though, that J2EE is the most important platform to be targeted by MDA tool vendors, the reason for that being, in my experience, that most enterprise applications today are built on J2EE.
Also, pretty cleverly hidden between the lines, is the notion that somehow current MDA tools (you know, those that are tied to J2EE, hah) suffer from what the author calls the "agility problem". This is also very misleading — no MDA tool worth the label has this problem. FUD, pure and simple.

Extending the reach of MDA into other areas will be difficult. The OMG’s definition of “platform” is vague, and the only real example is J2EE. Using models to assist with creation of J2EE applications is a valid pragmatic step on the road towards using models successfully in software development. In fact, there are very few standardized mappings between platform-independent and platformspecific models; the only ones that exist target the Java platform, although there are non-standard commercial implementations that claim to be MDA compliant and target other platforms.

First of all, it seems illogical for the author to make a case for domain-specific modeling languages, and then demand that there should be more standardized mappings (and following from that, standardized domain languages) inside MDA. To target other platforms, i.e. those for which no standardized mapping exists (which is essentially everything, since nobody uses the low-level EJB profiles because they are useless), is absolutely within the standards. This is another good example of twisting the truth — somehow doing things that are not standardized at all becomes doing non-standard (with every casual reader will read as violating some existing standard).
(As an aside, the section's summary does not summarize what is claimed in the previous paragraphs; while I agree that the name "MDA" itself is rather dumb, the rest seems to be just a bit of mud thrown around).
Next, the author starts to beat up the UML:

UMLAsBlueprint raises the bar for UML. Here the objective is to integrate one or more UML models as first-class artifacts within a software development process. [...] When we try to do this, we immediately find problems with UML, because it does not translate very directly into the technologies that we use. For example, a UML class cannot be used directly to depict a C# class because a UML class does not support the notion of properties while a C# class does. Similarly, a UML interface cannot be used directly to depict a Java interface becauinterface does not contain static fields while a Java interface does.

It is astoundingly trivial to add support for properties or static fields in Java interfaces to UML models by using stereotypes and/or tagged values appropriately. In case there is any doubt, I could easily draw up a valid UML model and use it to generate C# including properties or Java interfaces with static fields. IMO, though, platform-specific details like these don't have any value in MDA/UML models anyway ... but that is another story.

The technology of “UML profiles” is intended to offer flexibility in the blueprinting world by permitting a limited level of extensibility to the language, but practice has found that the very limited level of extensibility provided by profiles does not permit a seamless mapping from UML into popular target technologies.

That is an outrageous and unfounded claim, as lots of successful MDA project that have done just that clearly demonstrate.
In the next section, the author explains that Microsoft will add its own, proprietary views to the UML. Worried about standards? No need, just read this:

Although the diagrams are not legal UML by the letter of the standard, the meaning of the diagrams is readily apparent to anybody that understands the UML conventions.

Oh, great, so it's obvious to a human who looks at the diagrams. It's also obvious that these non-standard extensions will kill any desired interoperability.
While I like MOF and XMI conceptually, I dislike the implementation of those concepts, which I deem to be too complex and rather academic in many places. I disagree that being able to store model information in a standardized way is not important, though; I think that a sound model for representing the information is a lot more important than a nice UI on top of it. Then again, I can understand why a company like Microsoft would view this differently. Concerning XMI interoperability, there definitely are problems; still, we were able to build a product that can import XMI written by half a dozen different CASE tools.
Whether, conceptually, it's better to have domain-specific exchange formats that a single format like XMI reminds me of the XML/RDF debates; there are valid arguments for both views.
To summarize my personal view on this: Unless follow-up articles provide more value, this is a lot less than what one would have expected from somebody as Steve Cook.

November 16, 2003

MDA and Eclipse

vowe is interested in using Eclipse to do MDA? Maybe he should try out the iQgen Eclipse Plug-In Beta.

October 24, 2003

Repository implementations

Charles Betz has nice list of Major players in the current repository market.

October 19, 2003

MDA and Offshore Development

While I'm in a blogging mood, I might as well finally write about something I found quite interesting: A long term customer of ours who has been using MDA for quite some time, was recently more or less forced to outsource part of its development work to an offshore IT shop. The approach they have taken is quite interesting.

Their development infrastructure relies heavily on modeling, in their case using Innovator, a modeling tool very popular here in Germany. Innovator is very powerful, but also pretty complex. In addition to this, they use our product, iQgen, with a huge set of custom templates that drive the code generation. While they have achieved a high degree of automation — they can generate close to 100% of their backend code in most cases — it's not an environment that is well known, nor is it easy to learn in a short time.

So the problem was: How does one address this in the context of offshore development? Should they train the offshore developers in the usage of the toolset?

My recommendation was different: Continue to do the modeling on site, and ship the generated skeleton code to the offshore IT shop. This has the major advantage of agreeing on a specification that is as close to code as you can get, because it is code. In case you're wondering what the offshore IT developers have to do then, it's about implementing those parts of the code that cannot (or rather: are not) generated, which in this case includes the Swing client UI and those business logic elements that don't lend themselves to be modeled in UML easily.

The customer has taken this approach and exercised it for 3-4 months now, and it seems to work very well.

ERP for IT

Charles Betz has commented on an article of mine. Looking at his site, in turns out the topic is ERP for IT — an idea that I have been thinking about for quite some time:

This site is dedicated to the emerging subject of enterprise IT automation, otherwise called Enterprise Resource Planning for IT. This is not the automation of business process by IT; rather it is IT automating itself. If you are concerned with enterprise architecture, metadata, or configuration management - read on!

And indeed, read on you should.

July 16, 2003

Monkey Code

There's another MDA discussion at TSS, and it has prompted this longer-than-usual post on Monkey Code.

When I wrote this article, my main motivation was to cut the abstract, high-level, model-to-model transformation, meta-meta-model stuff and try to present the approach in a manner that would cause interest in the not-so-average, knowledgeable developer. Maybe I succeeded to some degree; but still I find it amazing that what is hardest is to convince people who usually "get it", not the managers or analysts.

Take a look at your usual J2EE application (just as an example - you can pick .NET or C++/CORBA or whatever you like as well). You have probably built it based on common knowledge about best practices, such as standard design patterns like those mentioned e.g. here or in books like this. In all of those patterns, you will find a description of how to take a business problem and turn it into an implementation.

The basic idea behind a pattern is to factor out common structural or behavioral aspects from a working solution so that they can be applied to similar problems. If you look at the code of your application, you will find that you do this, the degree depending on how strictly you follow a certain approach. This leads to an amazing level of redundancy, the reason being that you have flattened the patterns into an implementation that contains the pattern over and over again.

The easiest example is the get/set JavaBean pattern that we have become so used to we don't even recognize it. If you have a get and a set method and an attribute to hold the value, the semantic information is minimal - an attribute with a type, and possibly with a read/write option. This is a very small example, but the same is true on higher levels as well. Think about a conceptual entity in your business domain, e.g. a customer. The customer concept might be as simple as having a number of attributes, like name and basic address information, as illustrated by this pseudo-code:

  concept customer {
    attributes  {
        name: String;
        address: String;
        zipcode: String;
        country: String;
     }
  } 

Now think about how much code that simple concept is turned into. A standard architecture might have an entity bean implementation, local interface, local home interface, deployment descriptor; session facade bean, remote home, remote interface, value object and HTML form. All of these contain information that is dependent on the information of the customer concept.

That's what I call monkey code.

That is not to say that the implementation patterns - such as the standard J2EE design sketched out above - are simple, or even that such a thing as a default exists. They are a very important part of your architecture. But they are important only once.

Basically they are two things that are very interesting to spend time on - the business domain analysis and design, and the specification of how to map a business model to a specific platform. What's definitely not worth spending time on is doing the same thing over and over again. After all, that's what we have a computer for, isn't it?

July 07, 2003

Code Generation Network

The Code Generation Network is an excellent site with lots of information about code generation, a list of tools (sadly omitting iQgen as of yet), a very nice FAQ, and a useful classification of different code generation approaches.
One thing definitely missing (or was I just too dumb to find it?): an RSS feed.

July 01, 2003

MDA from a Developer's Perspective

Back in December, I wrote an article for TSS, trying to introduce concepts from OMG's Model Driven Architecture (MDA) to developers. I'm reproducing it here (with some minor editorial corrections), with the goal of elaborating on some of the things in more detail, as well as addressing some of the comments posted in the the TSS discussion, as well as in other places around the web.

MDA from a Developer's perspective

You've probably heard about Model Driven Architecture (MDA), the Object Management Group's brand-new approach to software development. It's likely you have also seen lots of product announcements, where new "MDA tools" appear in the market and existing ones, such as CASE tools or IDEs, suddenly become "MDA enabled". In this article I'll try to explain what MDA is all about, and do so in a bottom-up way. By this I mean that I'm going to focus on the benefits a developer (as opposed to a manager or someone with architectural responsibility in an enterprise) can gain from applying an MDA approach.

One thing that never ceases to surprise me is the way some developers are willing to implement the same thing over and over again. I'm not talking about writing a software system for the third of fourth time, each time applying experience and lessons learned from the last iteration. That's perfectly fine with me, and in fact I believe that most good software is based on something that has been thrown away at least once. What I'm talking about is doing the same thing again and again on a different level. Think, for example, about the patterns you apply when you develop a J2EE system. If you're following decent architectural guidelines, you might define operations in your EJBs to take value objects as parameters. These value objects are simple, plain old data types that hold some state. Typically, each of those attributes has a read and a write accessor, and perhaps also something like "isNull" to check for null values:

public class ExampleValueObject {
  private long value;
  private boolean valueNull;

  public boolean isValueNull() {
    return valueNull;
  }

  public long getValue() {
    return value;
  }

  public void setValue(long value) {
    this.value = value;
  }
}

(I'm not surprising anyone yet, I know. Please bear with me for a few seconds more.) Chances are you have implemented this pattern, probably better called an idiom, about a thousand times at least. If you're like me, you hate doing things that can be automated by hand, so you are likely using some sort of IDE like JBuilder, Eclipse, IDEA, or even Emacs, all of which allow you to automatically generate accessor methods for attributes. (Thinking about it - how many IDEs do you know that allow you to generate not only the bean-like get and set methods, but an isNull method as well?). This is a very simple case, but if you consider it for a while, you'll find that your code is full of patterns like these, from very simple to very complicated. Some examples of patterns are:

• classes used for communicating with servers remotely;
• the combination of classes, interfaces, and deployment descriptors used to describe and implement how you persist objects using J2EE CMP;
• classes implementing alternative persistence strategies, such as session beans managing simple Java objects in combination with a persistence layer
• Java Beans used to allow for communication between the web tier and the EJB tier;
• and lots more.

If you look at all those occurrences, you'll notice that in most of these cases, a very simple structure is behind all of those implementation details, together with a specific mapping of those structures to an underlying platform.

Let me elaborate on that a bit. Assume that you have a simple object model, described in a UML class diagram:

Figure 1

If you are using a pre-EJB 2.0 application server, you might have made the decision to use BMP entity beans only and persist objects using some sort of DAO pattern. For each persistent object, you decide whether it's worth being implemented as a bean or an object only, following what used to be called a "coarse grained object pattern". Regardless of whether you're using UML or not, the result you get when you implement your object model following those rules might look like this:

Figure 2

So where is this leading us? Imagine that you introduce a few new attributes to your customer object. You will have to make modifications to the customer bean's remote interface because of the new getters and setters (or, if you're following a value object pattern, to the value object associated with it); you'll need to apply the same change to your DAO, as well as to any other class somehow affected by that change. Adding an attribute is not what I would term a complex change. So why would I have to edit 3-5 different files for a change as simple as that?

Now think about what happens when you switch application servers, or even upgrade your app server to a new version. Let's say it supports EJB 2.0, including CMP, and it does so very efficiently. Let's also assume that you want to take advantage from that fact, resulting in a model that might roughly look like this:

Figure 3

Where does that leave you? It's not your logical object model that changes. A customer is still associated with an order in the same way as before. It's the rules, the mapping from logical to physical model that change, forcing you to touch each and every single software artefact related to persistence. Do you really think it's worth your time doing that? Do you call that creative work?

Of course, a real-world architecture will be a lot more complicated. The same principles apply, though. After all, what is architecture if not a set of rules stating how to solve common problems?

So what's the point? You'll remember that I started by describing that I wanted to explain MDA bottom-up, and hopefully I have convinced you by now of some facts:

1. When you implement a solution on a powerful platform such as J2EE, you follow certain rules to map your logical model to a physical representation.
2. If this transformation could be automated, you'll be able to save a lot of time required for tedious, uncreative work.
3. Separating a logical model from a technical mapping will isolate changes made to the two from each other.

So let me now step a bit more into MDA terminology and explain what the OMG suggests as a solution.

First of all, the OMG introduces the concept of platform independent and platform specific models (abbreviated as PIM and PSM, respectively). An example of a platform independent model is the one given in Figure 1 (although one might argue whether this is really platform independent, but more on that later). The idea is to have a model that represents business knowledge without being cluttered with technical detail. Figure 2 and Figure 3 clearly represent platform specific models, being tightly coupled to the underlying platform. When using MDA, PIMs are transformed into PSMs by applying a mapping or transformation. PIMs and PSMs can exist on different levels; basically, your PIM might be somebody else's PSM, depending on their perspective. Thus, a transformation might include several steps, from highly independent to more and more specific ones. The final result of the transformation will (hopefully) be the code. In my view, the code is also a model - but your mileage may vary here.

To put it very simply, MDA is thus exactly concerned with solving the problems mentioned above by allowing you to look at and modify your logical model (the PIM) independently from the underlying platform, automate the transformation into a PSM, and thus be able to change the two of them independently of each other.

To do this, MDA relies on some technologies that are controlled by the OMG, but are pretty much ubiquitous:

UML - the Unified Modeling Language

Probably everybody knows UML, so I'll not go into too much detail here. UML has become the one and only wide-spread standard modeling language, so that if anybody uses a modeling tool or simply draws something on a whiteboard, it'll most likely follow UML conventions. UML defines a lot of different diagram types, the best-known of which are the class diagram and the interaction diagram.

XMI - XML Metadata Interchange

XMI is a standard, XML-based language to exchange model information between tools. All of the leading CASE tools, such as Rose, Together, Innovator, ArgoUML, Enterprise Architect and so on support XMI as their standard import and export format. XMI is not as standard as one could hope, but it's getting there.

MOF - the Meta-Object Facility

This is probably the least known of the OMG standards, although I personally consider it to be the most important one. MOF is a conceptual framework for describing meta-data. Meta-data is, as the names suggests, data about data. To explain this, consider the following layers:

0. A customer, ACME Corp., has placed an order for 32 boxes of candy on the 23rd of April.
1. A customer has a name and can place zero to n orders, each order having a date and referencing a specfic item that is being sold.
2. Classes have attributes which have a type, and classes can be related to each other.
3. There are things that describe something, and there are connections between those things.

With numbers increasing, we move away from pure information (level 0) to a model layer (1), to a meta-model layer roughly resembling UML (2), to a meta-meta-model layer representing a way to describe meta-models.

MOF is mainly concerned with level 3, for which it contains a hard-coded set of "things". (If you're not confused enough already: in fact this layer is a subset of UML. Don't worry, things will hopefully become clearer soon).

So what is MOF? MOF is way to describe meta-models, which in turn describe a specific way to model something. One example of a well-known meta-model is UML. But the separation allows MOF to be independent of UML and support other meta-models as well. MOF is thus a very generic and very powerful framework for expressing model and meta data information.

Apart from being a conceptual framework, MOF also defines a CORBA IDL mapping that specifies how MOF data can be accessed from CORBA applications. With JMI, the Java Metadata Interface, the JCP has just finalized a mapping of these interfaces for the Java language.

XMI - again

Huh? Didn't we already discuss XMI? Yes, we did - but as we now know what MOF is about, we'll have to take a closer look. In fact, XMI cannot only be used as an interchange format for UML, but for any model that can be described by a MOF meta-model.

Let me give you an example. Let's say that your organisation defines a meta-model to express certain concepts, for example the concept of a business service. Each business service might be an aggregation of interfaces, each of them consisting of operation signatures. Each business service might also have an owner and a version it is available in.

MOF-compliant tools would allow you to define this meta-model, import it into e.g. repositories and CASE tools, and start drawing or editing model information - i.e., concrete instances of business services. This model information could then be exchanged with other MOF-compliant tools via XMI.

An existing example of a MOF-compliant meta-model apart from UML is the Common Warehouse Meta-model, CWM, which is defined by OMG and used to describe meta-data for use in data warehousing scenarios. It includes well-known model elements such as tables, columns, views, and indices.

Back to the beginning ...

So how does all of this relate? MDA is just starting to become popular, and while user-defined meta-models will definitely become, IMHO, extremely popular in the future, for now they are concerned mainly with automating the transformation mentioned quite a few times.

So what type of tools should you take a look at? There are quite a few MDA-compliant software generators out there, one of them being iQgen (iQgen is freely downloadable at http://www.innoq.com/iqgen), the one the company I happen to work for produces. iQgen is able to read a logical model, a PIM, in the form of an XMI file as exported from a CASE tool. This PIM is then transformed into the ultimate PSM, the code, based on a set of rules defined as templates.

The issue of transformation is the one the least specified by the OMG at the moment. Therefore, available tools use different strategies. iQgen uses Java Server Pages, which allows Java developers to start developing templates (or rather, refactor existing code into templates) without having to learn something new. Other tools rely e.g. on Jython or Jakarta's Velocity template engine. All of them preserve code that has already been modified, usually by enclosing it in some form of special comment.

All of these tools enable you to code with different perspectives. While you make structural changes in a UML model, you define the transformation or mapping to your selected software architecture in a different, localized place. Moving from e.g. EJB 1.1 to EJB 2.0 (or the upcoming 2.1, for that matter), is simply done by changing templates. This dramatically reduces monkey-work, saves a lot of development time, and enforces adherence to architectural principles in your application.

For the future I expect three major things to emerge:

1. CASE tools will become MOF-compliant. This will allow users to define their own meta-models, import them into the CASE tool, and start drawing diagrams conforming to them.
2. CASE tool vendors will have to shift their focus from documentation to efficient maintenance of model information.
3. "Real" model-to-model transformation, based on MOF, JMI, and the result of the MOF 2.0 Query/View/Transf. RFP issued by OMG, will become commonplace, pushing programming tasks to another meta-level.

But that's just an expectation of what might become mainstream in the future. To stick with the goal of introducing things in a bottom-up approach, I'll also try to address some of the concerns people have about automating software development, and about code generation in particular. When you talk to people about the use of code generation, as I very often do, you will immediately be confronted by either great enthusiasm or equally great skepticism. There are, of course, valid points that can be made against using a generative approach. But in my experience, I have found that most of these only apply to specific products, can be circumvented easily, or are based on a plain misunderstanding of the way the technology can and should be used.

In the following paragraphs, I will try to list some of the most common reasons against code generation, as well as describe how they can be addressed.

10 Myths About Code Generation

1. Code generation is a great way to impress managers, but once you hit the 80% wall, you need 800% more effort to achieve the final result.

This is very true for tools that force you to accept what they deem to be "the right way" to build your software. It's also the reason why a lot of people don't like to work with 4GL tools like PowerBuilder. Basically, you get a lot of benefits from the fact that they hide the complexity, but you run into severe trouble once you need to solve a problem the tool designers did not expect.

A good tool will enable you to automate what you would otherwise do by hand. The code generated should be - at least in principle -no different from what you would have written yourself. It's also of critical importance that you can still implement things without using the generator, because for some things, you need to be able to put the full power and flexibility of the underlying platform to use.

If your tool environment enables you to use code generation where appropriate and mix the result with hand-written code as needed, you will get the best of both worlds.

2. People will soon start to modify the generated sources, making repetitive generation impossible. So why bother?

A one-shot code generation approach, i.e. one where you generate an initial skeleton implementation and then do the elaboration by hand without being able to regenerate, is of no use at all. This approach reminds me of the first "wizards" that became available in popular IDEs some time ago. So you really need the ability to generate code, enhance it, and regenerate after you have made changes to the model.

But if people start modifying the generated code and break the rules, e.g. by inserting code outside of "protected sections", you will have a problem. I believe that there are two ways to circumvent this:

One is the brute force way that you can apply in certain environments. With this approach, you include the code generation step in your daily or weekly builds by checking out the model and the code and regenerating before you compile. Anything that does not survive this project is not included in the build result, and this is likely to make people break the code generation rules only once. While effective, this approach smells of centralized control, and thus I don't really like it.

The far better way is to use tools that provide a clear and measurable benefit to the developer, so that they will not even think about not using them. Which approach is better depends on your environment.

3. Code will never be generated in a way that meets my standards, coding conventions, works with my architecture and frameworks, etc.

True for a lot of environments, and a very good reason against using them. I am strongly opposed to any tool vendor trying to force their view of doing software development "the right way" on me. Fortunately, a lot of tools in the MDA market allow you to adapt the way they transform and generate code to your own needs. From simple things like allowing you to decide where to put your curly braces, to naming conventions, attribute access, copyright notices, and persistence strategies - everything must be customizable by the end user. And it is - at least in tools that I consider being worth their money.

4. Code generation creates too much of a dependency on proprietary tools.

This is probably the strongest argument one can make as long as MDA standards are still weak. While using XMI as the language to describe your model in should make you independent of any particular CASE tool vendor, it's not as strict as one would like. Every XMI export is a little bit different, so support for multiple CASE tools is a crucial factor in assuring that you can switch to another one, if need be. JMI, the Java Metadata Interface, standardizes the access to model information. In most tools, the way to describe transformations is proprietary, although languages used in specifying templates can include open standards like JSP or Jython. This will hopefully change once the RFP ... is finalized and implementations emerge. At the moment, though, some dependence on a tool implementation will exist in all solutions I am aware of.

5. Code generation has been tried in the past - and failed.

True. Lots of concepts have been tried in the past, and failed miserably. I do believe that with the current hype surrounding the MDA approach, lots of tools appear in the market, which creates a lot of competition, which fuels innovation ... you get the picture. With standard formats like XMI and the rising need for improving productivity, things look a lot different today than they did only a year ago. Don't let past experience make you believe that things can't improve.

6. My CASE tool already has a code generator that I've tried without being impressed.

I believe this is due to the fact that historically, code generation has been added to CASE tools as an afterthought, and I often have the impression that it has mostly been tested during presentation where the main goal was to impress management. Most CASE tools' code generators are very hard to modify, written in some proprietary language (or even Visual Basic, which some might consider even worse), and perform very badly. One should not assume from experience with bad implementations that the scenario itself is flawed - it's more productive to look for a tool that does the job better.

7. I'll just use XML/XSLT and be done with it.

XML and XSL(T) are fine tools, and I use them quite a lot. For example, our product's web site is based on the excellent Cocoon framework, which does a great job of coordinating and integrating transformation pipelines.

There are quite a lot of problems with using XSLT for code generation, though. The most obvious one is that preserving existing code requires quite a bit of effort, e.g. by writing a custom merger task and integrating it with the XSLT engine. When XMI is used as the input format, stylesheets become extremely complicated - because XMI is really, really hard to read and understand. Finally, not everyone knows how to use XSL, and even developers used to it tend to get into problems once stylesheets become complicated.

8. I don't need a code generator, I'm using XDoclet.

XDoclet is cool, and I think that in a lot of cases it's a sensible alternative to a full-fledged MDA approach. If you're still writing (and updating!) deployment descriptors by hand, XDoclet will definitely reduce the work you have to do tremendously. The one problem I have with it is that the basis for all of the automatic code generation is the Java code, which is nothing I can show to my customers. While Java code is arguably an easy means of communication between Java developers, it's probably rather hard to discuss an entity/relationship model with your local DB admin, or the implementation of a use case with a business analyst.

9. It's faster to code something in Java than to draw diagrams in some GUI tool.

If this is the case, by all means use Java. I'm not advocating to do everything with the graphical representation. In my experience, what works best is to make structural designs (and updates to them) in the CASE tool, and implement the generated methods in the code. It's true though that a lot of CASE tools don't easily allow the developer to e.g. create a bunch of attributes efficiently. I think this is something where CASE tools yet have to match the support offered by IDEs. Once they do so, the main argument against their usage - which is again tied to the implementation, not the general concept - disappears.

10. If I use code generation to automate my work, I'll soon be out of a job.

You can easily use that as a justification to program in Assembler. I tend to believe that developers like to be creative, so people justifying their role by doing lots of monkey-work by hand is something I am rarely confronted with.

Conclusion

So what's so great about MDA? If you think that there's nothing so new about it, I'll agree whole-heartedly. MDA is not a new invention; it is a bunch of techniques integrated under a single name. What's good news is that this leads to standards that are supported by different vendors, which is in general considered to be a good thing. MDA is thus an approach that is intended to solve problems common to application development in a standard way, freeing developers and architects from restrictions imposed on them by the complexity of today's technologies.

It's a lot easier to talk about MDA in more abstract terms to managers than it is to convince developers actually implementing systems on a day-to-day basis. In this article, I have tried to point out MDA's benefits from a developer's perspective. Whether I did so successfully is up to you to decide.

June 12, 2003

MDA Reverse Engineering

Reverse Engineering from Existing Codebases is kind of the other side of the MDA coin - and probably one of the more interesting issues in applying an MDA approach. Phillip's approach works nicely in a lot of cases; but I feel that it's kind of a perversion that I need to do things this way.