How to design? - the data

How to design? This is the basic question for creating reliable and well-working OSB components. What are those repetitive steps which the designers or the developers must do?

OSB is a communication platform so the data (requests / responses) are very important. The main task of  the OSB is to deal with the data and not with the business logic.
Usually we have to process the incoming data before sending to the other systems or third parties. These process steps are called the message flow. I emphasize the purpose of these processes are to handle the data and not to implement business workflows.
We have to know the answer of the next question as well: what will we do if an error occurs? We have got lots of choices what to do in these cases but the circumstances (business requirements, used protocols, ...) are different and limit our possibilities.
OSB is a gateway among systems it might slow down the communication and cause bottlenecks. You should be concerned with the performance issues as well.
It is very important to check what is happening inside the OSB. The technical and business users want to monitor the events or just become aware of the errors or the more important happenings.

These are the key factors what You should be concerned with:

  - input / output data
  - message flows
  - error handling
  - performance
  - monitoring

Input / output data (requests / responses)

The handling of the requests or responses is different in case of
  - business services
  - proxy services.

Usually the data structure of requests of the business services is determined as the services which must be connected to OSB have been working already. So You might have less tasks to do. You have to create the business service, set the protocol and the structure of input / output data (in case of webservice just use the XSD in the WSDL).
In some cases (e.g. attachment of email) You can't set exactly the data structure for the business service (anything can be sent in attachment), You have to find out and validate it in the message flow (e.g. XSD validation). In these cases You must check the data because You mustn't encumber the source system unnecessarily.
Sometimes You need to create a request or a response message. If You want to convert an asynchronous service to a synchronous one inside the OSB then You need to create a virtual response because the source system won't send a response back. (e.g. webservice → email)

If You might receive unstructured data then You can't validate it simply. Why? The best example is the attachment of emails for this. You can receive ANYTHING in an email. One part of the incoming email is incorrect (for example a JPG file was sent instead of XML) and the other part seems to be good.
If the same data is being sent to an email address then there is no problem. But what if the partners send orders and complaints to the same email address? And what's more there are more versions of the orders and one of the partners send version 1.0, the other partner sends version 2.0 and so on. In these cases You can't validate so simply.
To consider the different business data (orders, complaints) You have to validate the attachment using one of the XSD's and if an error happens You have to use an other schema and so on. If there are more versions of the incoming data and there is version information in the XML (e.g. <order version="1.0">) then You are able to choose the correct schema to validate the XML. If You haven't got version information then You have to use the strictest schema to validate and the second strictest one and so on.

Proxy service is a new service created by You. In this sense You have got freedom to define the input / output data. But this is not true entirely. If You implement an atomic service usually You have to use the requests / responses format of the business service. In the case of the domain proxy services the input / output data are similar to the data of business service but this is not certain. You have to process the input data and to transform them so the difference between the requests / responses  of the business and proxy service may be huge. This is determined by the business requirements. And if we are talking about enterprise services we have to work out the structure completely ourselves.

To allow more request format for an operation is a bad design pattern. I mean You can use the 'choice' XSD element which let You use different request formats. For example:

<xss:choice>
   <xs:element name="mouse" type="mouse"/>
   <xs:element name="cat" type="cat"/>
</xs:choice>

In this case the request may be this one:

<soap:Envelope ...>
   <soap:Body>
       <mouse>
           ...
       </mouse>
   </soap:Body>
</soap:Envelope>

and this one as well:

<soap:Envelope ...>
   <soap:Body>
       <cat>
           ...
       </cat>
   </soap:Body>
</soap:Envelope>

It seems to be simpler to use the 'choice' XSD element but it is a better and cleaner solution to create an operation for the cats and an other one for mice.

I suggest to put the generally used schemas to a separate project which is for common components. You mustn't modify these schemas even if You should do it because of a proxy service. If You need to modify one of them then copy it to your project and modify that one not the original.

What kind of tools can we use to design?

The fact that we have to design the OSB components seems to be pretty clear. What we have to know is the input/output data structure, how to process the input data (message flow) and what protocol we need to use (Ok, there are a lot of other factors but these are the most important ones).

But what I am thinking of what tools we can use to design OSB components. I bet the quickest answer is the OSB IDE. But this is a developer tool not a designer one. So the real question is do we really need a desinger software to model the components? The answer is not easy. I have to admit I did not have to use always this kind of software when developing OSB. And I can't state whether You really need it or You don't.

It is always good to use a designer tool but I think the main reason to use it is to generate code from model.

The proxy services don't contain deep business logic (except maybe the newly created enterprise services) so it is unnecessary to model it.
It is hard to model the input / output data without so much as not to implement them. I think to model the detailed input / output data and message flow is not reasonable. So we should model the services (proxy, business) and their structures (see the previous blog). In some cases there might be justified to design the data structures but You need to make up Your mind the resolution level.

I have already told the magic word: code generation. It means code can be generated from model. You are creating the model adding stereotypes and properties to the components and the code generator will create folders, XML files (for proxy / business services), project descriptors files, etc.
This is a sample model which I created in the designer tool (this is the model of the topic of previous blog ):

You can assign properties to the stereotype and set those ones when creating the model. In this case I have defined if the proxy service is atomic or it isn't.

After the generator created the code You have to implement the message flows, to detail the requests / responses and so on. The generator can create just the frame You must fill it with content.

What are the advantages? You have got the model of the OSB components. You can review the model easier than the real services in the OSB IDE or console. But You can say now 'a model is never static it is changing often!' Yes, You  are right it is not. If your model is being changed and You have to regenerate the source code then the code which You implemented after the previous generation will be lost (because we have got a source code on the computer which we maintain).
However the good news is this is not an obstacle for our generator. It can merge the old source code (the result of the earlier generation) and the new one! So our work can't disappear. The trick is You have to define read only and modifiable parts in your template and the developers can change the modifiable ones (check the //<%startUserCode%> code in the sample below). If they will write into the read only parts of generated code then this new code will really be lost at next code generation event.

The above ones were the advantages but there are some drawbacks as well. You have to write the code generation algorythms. It might be huge work and depends what exactly You want to generate from model. If You want to create more code from model (e.g. not just services but input / output data as well) then You have to deal more time with the code generator. This effort is worth if You have got a lot of services / components on the OSB.
This is a sample for OSB proxy service template:

<%
metamodel http://www.eclipse.org/uml2/2.1.0/UML

import com.astron.uml2.java.services.Uml2Services
import com.astron.uml2.java.services.StringServices
%>

<%script type="Class" name="fullFilePath"%>
<%if (hasStereotype("ProxyService")) {%>
/<%name.toU1Case()%>PS.xml
<%}%>

<%script type="Class" name="generate" file="<%fullFilePath%>"%>
<?xml version="1.0" encoding="UTF-8"?>
<xml-fragment xmlns:ser="http://www.bea.com/wli/sb/services" xmlns:tran="http://www.bea.com/wli/sb/transports" xmlns:env="http://www.bea.com/wli/config/env" xmlns:http="http://www.bea.com/wli/sb/transports/http">
  <ser:coreEntry isProxy="true" isEnabled="true">
    <ser:binding type="abstract XML"/>
    <ser:monitoring isEnabled="false">
      <ser:aggregationInterval>10</ser:aggregationInterval>
      <ser:pipelineMonitoringLevel>Pipeline</ser:pipelineMonitoringLevel>
    </ser:monitoring>
    <ser:reporting>true</ser:reporting>
    <ser:logging isEnabled="true">
      <ser:logLevel>debug</ser:logLevel>
    </ser:logging>
    <ser:sla-alerting isEnabled="true">
      <ser:alertLevel>normal</ser:alertLevel>
    </ser:sla-alerting>
    <ser:pipeline-alerting isEnabled="true">
      <ser:alertLevel>normal</ser:alertLevel>
    </ser:pipeline-alerting>
  </ser:coreEntry>
  <ser:endpointConfig>
    <tran:provider-id>http</tran:provider-id>
    <tran:inbound>true</tran:inbound>
    <tran:URI>
      <env:value>/OSB/<%name.toU1Case()%>PS</env:value>
    </tran:URI>
    <tran:inbound-properties/>
    <tran:provider-specific>
      <http:inbound-properties/>
    </tran:provider-specific>
  </ser:endpointConfig>
  <ser:router>
  //<%startUserCode%> for messageFlow
  //<%endUserCode%> for messageFlow
  </ser:router>
</xml-fragment>

And here it is a sample for Java template as well:

<%
metamodel http://www.eclipse.org/uml2/2.1.0/UML

import com.astron.uml2.java.services.Uml2Services
import com.astron.uml2.java.services.StringServices
%>

<%script type="Class" name="fullFilePath"%>
<%if (hasStereotype("entity")) {%>
/<%getProperty("package.server.hibernate").toPath()%>/<%name.toU1Case()%>Dao.java
<%}%>

<%script type="Class" name="generate" file="<%fullFilePath%>"%>

package <%getProperty("package.server.hibernate")%>;


import java.util.List;

import org.hibernate.HibernateException;
import org.hibernate.Session;
import org.hibernate.Transaction;

import com.astron.common.exception.DaoException;
import com.astron.common.hibernate.SessionFactoryUtil;
//<%startUserCode%> for imports
//<%endUserCode%> for imports


/**
 * Data Access Object
 * Generated by Astron MDA tools
 */
public class <%name.toU1Case()%>Dao {


    /*
     * Get all <%name%> entities
     * 
     * @return a list which contains <%name%> objects
     */
    public static List<<%name%>> getAll<%name.toU1Case()%>s() {
        Transaction tx = null;
        Session session = SessionFactoryUtil.getInstance().getCurrentSession();
        try {
              tx = session.beginTransaction();
              List<<%name%>> <%name.toL1Case()%>s = 
                  session.createQuery("select h from <%name.toU1Case()%> as h").list();
            //<%startUserCode%> for getAll
            //<%endUserCode%> for getAll
              tx.commit();
              return <%name.toL1Case()%>s;
        }
        catch (RuntimeException re) {
              if (tx != null && tx.isActive()) {
                try {
                      tx.rollback();
                }
                catch (HibernateException he) {
                    throw new DaoException("Hibernate: error at rollback!", he);
                }
                throw new DaoException("Hibernate: error at list<%name.toU1Case()%>()!", re);
              }
        }
        return null;
      }

      ...

I recommend two open source products to generate code:
  - Topcased (http://www.topcased.org/) - for designing
  - Obeo (http://obeo.fr/pages/acceleo/en) - for code generating

To use this technique You have to create some roles in Your IT department. Have a look at the next figure:

As You can see on the figure above at least three different roles must exist:

1. MDA architect
   The responsibility of this role is to maintain the Source Code Generator Framework (SCGF). And what does the SCGF mean? It contains:
   - templates: Each artifacts which You want to create from the model need a template (these artifacts for eample: proxy / business service XML files, XSD's). The generated code will be created from these templates.
   - SCGF package for developers: Usually the developers don't need to know anything about the code generation. Their task is to implement the business logic so they have to put the least effort to generate the code. Due to this fact the MDA architect must create a generator package for developers which they just download from the version control system configure at the beginning of the project (e.g. what is the path of thier sour code or model) and use it simply (e.g. they have to execute a batch file on windows systems)
   
2. designer
   The designer will create the initial model and modify it according to the requirements. They have to upload the modified model to the version control system from where the developers can download it to (re)generate their code.
   
3. developers
   The developers maintain the source code. They download the model and the SCGF package from version control system and generate the first version of the code fill the empty places of code and regenerate it if necessary.

The MDA architect role is project independent but the designer and developers roles are not.

The MDA based code generation is not a magic wand to build something from nothing. You have to create the SCFG Yourself and this might be a huge work. But it could be an enormous help to create the frame of the code easily and quickly.