Introduction
Although more and more tools exist to assist software professionals in their work, one cannot say that software development has become less complex. Rather, using modern development and runtime environments, systems can be developed that were impossible or at least very hard to do just a few years ago. Since we are doing more and more complex things, we don't get the benefit of using better tools for the same problems: the problems become more complex while the tool support becomes better. The multitude of tools and technologies helps a lot, but adds its own set of issues. To make matters worse, technology has not come to a halt - instead, it seems to change faster and faster. This whitepaper describes a generative software development approach, discusses how and why this will aid in both software development and, more importantly, architecture management, and introduces a suite of tools developed by innoQ that implement these ideas. Although you might have your own tools or your own trusted vendor who offers similar products, we believe that the ideas discussed herein are valuable in their own right.
Architecture as a Set of Rules
If you look for definitions of the term software architecture, you will find that there are at least as many definitions as there are software architects. innoQ, whose main focus is software architecture consulting, has found that one definition given by Charles Martin in response to a discussion hosted at the SEI's (Software Engineering Institute) web site works pretty well:
A software architecture is the assignment of specific transformations that are applied to convert a purely logical model of a system that satisfies all functional requirements into a model of a system that satisfies both functional and nonfunctional requirements.
We think that the notion of "transforming" one model into another really is a very good way to describe what architecture is all about. Let's look at an example. Assume that you have an extremely simple domain model, consisting of three classes. If you refined that model so that it reflects all the attributes and - if you want to do that at this point in time - operations of the classes, you have a nice, clean, and purely logical business model of your system. To turn that into a system that really works, i.e. a complete program representation in Java, C++, C# or whatever language you prefer, you apply a set of rules and patterns. For example, your architecture might define different types of classes: some may represent persistent entities, some may be processes or process steps. Whether one specific class in your model is an entity or a process is a logical, domain-specific issue. But the way that you transform a logical entity into a physical representation presumably is the same for all your entities. You basically solve the same problem in the same way, as defined by your architecture.
Let's take a more detailed look at some examples. If we re-use the example from above, we might map the logical model to a standard J2EE (EJB 1.1) representation, where each logical entity becomes a CMP entity bean. (Depending on the application server you use, this may or may not be a good idea, but let's for a moment assume that it is.) For each entity, you thus have to create a home interface, a remote interface, an implementation class, both a standard and an application server specific deployment descriptor and (possibly) the SQL DDL to create a table in your relational database. For each attribute, there will be an appropriate instance variable in the implementation class and an entry in the mapping information of your app server's deployment descriptor. For each operation, you will have the method declaration in the remote interface and the implementation in the bean class.
So to transform your logical model into its physical representation, you need to create 5 or 6 files for each entity which contain a lot of redundant information. Let's assume that you specified that your component has a method called doSomething() when you modeled it. In general, you will want to have a declaration for this method in the remote interface (including the required throws RemoteException clause), an implementation in your bean class and a method entry in the deployment descriptor with a default transaction attribute of Supports. This can all be done automatically, and - as always - if it can be done by the machine, let the machine do it. If you have developed software without tool support in this area, you know how easy it is to get something wrong and how time consuming it is to get rid of all the problems introduced by having to do all of this work by hand. And we did not even look at the problems that arise when you need to change something. A simple change in the model might force you to touch several files, making it a lot more expensive than it needs to be.
Why IDEs and Current CASE Tools Don't Solve the Problem
Why is this problem not solved by IDEs and their fancy wizards? The problem is that they can only support what is either defined by the standard or based on some proprietary notion of how to architect a system. But if you have ever developed a complex, real-world system, you know that there is no "standard architecture" fit for all purposes (nor can there be one). In every project, you will have specific issues that need to be addressed and that can have a huge impact on the architecture. As an application developer following an architectural style guide, you follow a lot of conventions that are project specific. The point is: Intellectually, you work with the logical model if you are dealing with the business problem that you want to solve. But technology forces you to switch back and forth between the logical and the physical model. Since the latter consists of a lot more artifacts than the former, this is a significant complication.
If you have used the generation mechanism built into your CASE tool, you probably know how weak it generally is. What it will do is this: From a model that contains n classes, it will generate (if you happen to use Java) n files, each of them containing a class with its attributes and operations. If there are associations between your classes, a simple data structure will be used to hold the information, e.g. a java.util.Vector in case of a 1:n relationship. Basically, your logical model is translated into your physical implementation in 1:1 way - no real transformation is happening. Of course, this simple translation allows the tool to also do reverse engineering, at least in a limited way: Because of the 1:1 correspondence, the translation can be done in both directions. (This is limited because CASE tools are not able to analyze the behavioral aspects of e.g. Java code, only the static aspects. From a member typed as java.util.Vector, it is thus impossible to find out what kind of objects (which type) the vector contains, so the association is not recognized).
What CASE tools are very good at (at least generally speaking) is round trip engineering: If you have generated the code from inside the CASE tool, you can modify it outside of it (with your favorite IDE) and then import it back into your CASE tool and so forth. The best example of a tool that supports this very well is TogetherSoft's Together that even uses the source code as the place where the model is stored. While this is very nice, it also forces you to model all the physical elements of your software in your modeling tool. While round trip engineering becomes very easy, you also lose a lot: You can no longer reduce your model to the logical aspects and specify architectural aspects separately. Basically, a CASE tool (or rather, a modeling strategy) that forces you to go to a modeling level that is as low as this is nothing more than an IDE. That doesn't mean that it's necessarily bad, just that it's not what we consider to be a high-level, architectural modeling tool. A CASE tool used like this is similar to an IDE that knows how to generate a set and a get method for each attribute of your class.
How can we exploit the fact that architecture can be viewed as a set of transformation rules? Basically, we need to define the transformations, specify when to apply them, and annotate our model with some information that enables us to do the transformation automatically. This is what the next few sections are about.
Stereotypes and Tagged Values - The UML Extension Mechanism
UML (the Unified Modeling Language) allows you to extend the notation itself by means of stereotypes and tagged values. In addition to being able to separate e.g. classes from interfaces, you can distinguish different kinds of classes by specifying a stereotype. Stereotypes can be specified with a notation like <<stereotypename>> or - depending on the CASE tool - with a different graphical symbol. Examples of stereotypes you might use are Entity, Process, Activity, HostObject, ViewableObject, etc. You can also specify attributes and values by means of tagged values. For instance, you might specify a key/value pair persistence_type/CMP, where CMP is the valued, tagged with persistence_type.
A set of UML extensions for a particular architecture (or other) domain is called a UML profile. The OMG has defined profiles for CORBA application, Sun is (in the Java Community Process) involved in creating a profile for EJB applications. If you use one of these standard profiles, you follow a standard architecture that offers you a lot more than the basic UML supported by standard CASE tools. Still, what is not being taken into account is the fact that there are no two projects that can live with exactly the same architecture. There will always be differences due to different integration requirements, tools and languages used, and skill of the project's programmers and architects.
Transformation Rules
To specify a transformation (sometimes also called a mapping or projection), you need to define two different things: templates for the different physical artifacts that need to be created during the generation process, consisting of a static part, placeholders for values derived from model information, control logic, and a transformation configuration that specifies how the stereotypes and/or tagged values in the model are related to the templates, i.e. which template(s) to use for a particular meta-type of model element. If you look at the way most generators that are developed as part of an application development project interpret the necessary rules, you will find that they either (a) implement the generation engine together with the code generation rules in a script or other programming language or (b) if they use a template approach, have only a simple place holder mechanism (e.g. "${class}" is replaced by the current class in the model. Both approaches have advantages and disadvantages. Using a generation engine that supports templates with a feature-rich control language combines the best of both worlds (more on that later).
Let's look at the first few rules of an (oversimplified) architecture for a business application, expressed as a transformation (in prose):
There are two basic stereotypes for classes: <<entity>> and <<process>> For each <<entity>> class, there is an EJB Entity Bean, consisting of a home and remote interface, (called <Name>Home.java and <Name>.java, respectively) a bean implementation called <Name>Bean.java, a primary key class called <Name>PK.java, a standard deployment descriptor called ejb-jar.xml, a Weblogic specific descriptor called weblogic-ejb-jar.xml, and an entry in the shared file schema.sql containing the create statement for a table, containg all the Entity Bean's attributes, with the attributes of stereotype <<pk_field>> as part of the primary key, an index on the primary key, and a view on the attributes with stereotype <<id_field>>. For each <<process>> class, there is an EJB Session Bean, stateless unless the tagged valued state_type is set to stateful.
...
One can easily imagine how this list would continue: By means of stereotypes, tagged values and the relation they have to a specific technical construct, you define how modeling is done and what the end result of the transformation will be. We have defined what kind of model we would like to have and how we would annotate it with appropriate information to enable an automatic transformation according to the configuration and rules. One question remains: How can we support this process in the context of a standard development process? If we generate code, how can we ensure that we don't have to start from scratch each time we do this? This question is answered in the next section.
Code Integration
The answer is neither new nor surprising: Since not everything can be defined in the model (nor should it) the remaining, behavioral aspects need to be defined by programming inside of the generated constructs. The sections in which the developer writes his or her code need to be protected from the next run of the code generator - not amazingly, they are thus called protected sections. As long as developers write their code inside these sections (represented by appropriate tags in comments around the section), it will remain unchanged. Of course, the code needs to survive some changes in the model - e.g. if the name or other parts of a method signature change, the code should still remain in the same logical place (although it might not compile anymore without being changed appropriately).
iQgen - innoQ's Generator Implementation
innoQ is a consulting company with its main focus on software architecture. In most of our customers' projects, a generative approach has been used, and most of the time, this was due to our insistence on the benefits of this model. We support this claim regardless of the tools that are being used. Most of the time, the tools - most importantly, the generator itself - was developed from scratch to reflect the project specific issues. In one particular project, we tried to use a well-known commercial product that - apart from a lot of other stuff - supports a template driven generator. When we used it, we found out that it worked pretty well. Some of the disadvantages were: We paid a lot for functionality we did not need, a more or less exotic language was used to write the template control logic, the vendor was focused on providing pre-defined standard architecture transformations instead of providing us with the flexibility of being able to define our own, the tool was closely tied to a particular CASE tool's output format, it was pretty slow - in short, while we made the decision to use the product, we just did so because it was the best of all the bad ones in the market. And so we decided to develop the product that we would have liked to buy ourselves. The result is innoQ's iQgen, and the following sections contain an overview of how iQgen implements what we consider to be the best way to deal with the challenge. Experience with early versions that are in production in real life projects suggest that we seem to have met our goals.
A Clear Separation of Engine and Transformations
Very importantly, we are strongly suspicious of vendors that claim to have defined "a standard architecture for e-business applications" (replace "e-business" with whatever you like). Our experience shows that the re-use of a complete architecture without making at least some changes is rarely, if ever, possible. The likelihood decreases as the project's complexity grows: There are so many factors to consider that we just don't believe in the one and only architecture. For this reason, iQgen consists of a generation engine that is driven by transformation configuration and templates. We don't tell you which architecture is best for you, or at least we don't do that until we know about your non-functional and functional requirements, your strategic decisions, the stake holder's needs etc. iQgen comes with a set of predefined transformations that work pretty well, but are intended as a starting point from which to redefine them according to your needs.
Input and Output Formats
The first input to iQgen is - of course - the model that it operates on. This has to be in XMI format. If you know XMI, you also know that although it's supposed to be a standard, different CASE tools have different interpretations of what this standard means. Currently, iQgen supports XMI output from Rational's Rose, TogehtherSoft's Together, MID's Innovator, MicroTool's ObjectiF, and the open source CASE tool ArgoUML. For details, see our platform matrix.
The second input to the generation process is the transformation. This consists of an XML file specifying the configuration and a set of templates in JSP (Java Server Pages) format. Why did we choose JSP? First of all, it allows us to write the template as a mixture of the code to be generated, interleaved with control instructions in the world's favorite language - Java. Since most projects we work on use Java at least in some areas, this seemed to a better option than for example TCL, Python or other script languages. Secondly, the JSP specification defines an extremely powerful extension mechanism, TagLibs, that allow us (and iQgen's users, of course) to implement complex model interaction logic and hide the complexity from the template developers. The third, not so obvious input is the output itself. Since iQgen is supposed to be used iteratively as part of the standard development process, it preservers everything that it finds between its protection tags. So the model and the existing implementation are combined, based on the rules defined in the transformation.
iQgen's output can be almost anything you can imagine: Java, C++, C, C# or COBOL source files, SQL/DDL scripts, Makefiles, Shell, Perl or Ant build scripts, project files for your IDE, deployment descriptors ... anything that you can generate from Java. And if you need to create binary files, this is also possible - as long as you know how to create the appropriate content from Java. You can easily use any available Java API from inside the templates, so the options are without limits.
Writing iQgen templates
To write a template, all you need is a text editor of your choice, preferably with support for editing Java Server Pages. You know best what you want to generate, and you have access to the details of your UML model via a simple Java API. Let's take a look at the way you would go about generating code for an EJB application. First of all, you need to specify the main configuration for your transformation. This is done in a special JSP file called main.jsp. It doesn't necessarily have to be called that, but it is the entry point of your set of templates.
| <%@ page extends="com.innoq.generator.jsp.JspMain" %> <%@ page import="ru.novosoft.uml.MBase" %> <%@ page import="com.innoq.generator.MetaModel" %> <%@ page import="java.util.Collection" %> <%-- Main template for EJB --%> <%! public String[] getTemplates(MBase element) { String s = MetaModel.getStereotype(element).trim(); String name = MetaModel.getName(element); if ("EntityClass".equals(s)) { addToPart("ENTITY",element); return new String[]{"entity.jsp","home.jsp", "remote.jsp","drop_and_create.jsp", "webform.jsp"}; } else if ("Exception".equals(s)) { return new String[]{"exception.jsp"}; } else if ("SessionClass".equals(s)) { addToPart("SESSION",element); return new String[]{"session.jsp", "home.jsp","remote.jsp"}; } else if (MetaModel.isInterface(element)) { return new String[]{"interface.jsp"}; } else if (MetaModel.isActor(element)) { return new String[]{""}; } else { Collection c = MetaModel.getAttributes(element); if (c==null || c.size()==0) { return new String[]{"class.jsp"}; } else { return new String[]{"class.jsp", "drop_and_create.jsp"}; } } } %> <%! public String getName() { return "Tutorial 1"; } %> <%! public String[] getPostprocessTemplates() { return new String[] {"jndi_names.jsp", "build.jsp"}; } %> |
As you can see, we have implemented three methods: getTemplates(), getName(), and getPostProcessTemplates(). In getTemplates(), we specify which set of templates iQgen should execute for a particular stereotype of class. Not surprisingly, for an EJB 1.1 entity bean, a template for each of the remote and home interface, as well as one for the implementation class are defined. Additionally, we have specified a template for creating the necessary SQL DDL and one to generate a simple HTML form to edit the entity bean's values.
To completely describe the sample mentioned here is beyond the scope of this paper, so let's just take a look at the remote.jsp template:
| <%@include file="params.jsp" %> <%@include file="copyright.jsp" %> * * (#)<%=classname%>.java * Stereotype: <%=clazz.stereotype%> (Remote Interface for Bean <%=classname%>) */ package <%=getPath()%>; <%@include file="imports.jsp" %> import javax.ejb.EJBObject; import java.rmi.RemoteException; import java.util.Collection; /** * Remote Interface <%=classname%> * <%=clazz.documentation%> */ public interface <%=classname%> extends EJBObject { <%out.beginUserCode("home");%> // Add your own declarations here <%out.endUserCode("home");%> // Remote methods <% MBase mo; Collection ops = getMetaModel().getOperations(clazz); Iterator ops_it = ops.iterator(); while (ops_it.hasNext()) { mo= (MBase)ops_it.next(); String st = getMetaModel().getStereotype(mo); if (!"RemoteMethode".equals(st)) continue; String retTypeName = getMetaModel().getReturnTypename(mo); Object retType = getMetaModel().getReturnType(mo); String parList = getMetaModel().getParameterListAsString(mo); Collection pars = getMetaModel().getParameterList(mo); Iterator itp = null; if (pars!=null) { itp = pars.iterator(); } String exList = getMetaModel().getExceptionListAsString(mo); if (exList.length()==0) { exList="throws RemoteException"; } else { exList+=", RemoteException"; } String name = getMetaModel().getName(mo); String lexception_string = getMetaModel().getExceptionListAsString(mo); if (lexception_string.length()==0) { lexception_string = "throws RemoteException"; } else { lexception_string = lexception_string + ", RemoteException"; } %> /** * Method <%=name%><%while (itp!=null && itp.hasNext()) {%> * @param <%=getMetaModel().getName((MBase)itp.next())%> Description...<%}%><%if (retType!=null) {%> * @return <%=retTypeName%><%}%> */ public <%=retTypeName%> <%=name%>(<%=parList%>) <%=lexception_string%>; <% }// while (ops...) %> } <%@include file="../utilities.jsp" %> |
Implementation Details
iQgen is written in 100% pure Java. As such, it runs on every platform that supports a JRE version 1.3 or higher. Initially skeptical, we have found it to work not only well, but also very fast - its speed is clearly good enough to integrate it into a standard development process. JSP handling is done via a standard servlet/JSP engine. Every JSP1.2 compliant web container can be used together with iQgen, although it comes bundled with a small and very efficient engine called jo! that is embedded and runs in process with the generation engine.
iQgen runs on a variety of platforms due to the fact that its basic requirement is a Java Development Kit (JDK) version 1.3 or higher. It includes both a batch mode suitable for running it from the command line and a graphical user interface to aid during template development. Also included is an integration with Jakarta Ant.
Conclusion
The benefits of applying an architecture-centric, model-driven approach to software development are widely known. OMG's MDA initiative is a direct consequence of the knowledge gained using generative technology. Regardless of the architecture you chose, the language you program in, or the generation toolset you use, we strongly believe that for any project that exceeds a trivial level of complexity, the generative approach is the one that should be used.
With iQgen, we offer a tool that is aimed at easing your transition from a classical to a generative software development approach, thus saving you time and money while at the same time increasing the likelihood of your project's success.