CSV to XML Marshaler for Apache ServiceMix

Reading CSV files, can cause a problem for developers of Enterprise Service Bus applications. This article describes a simple solution for Apache ServiceMix.

If you want to process messages in the Comma Separated Values file format in an Enterprise Service Bus you will be confronted with a problem. The problem arises because a JBI conformal ESB accepts only normalized messages with XML content. Therefore a file stored in CSV file format has to be converted to XML before entering the bus. This task could be done by a special adaptor, a CSV Binding Component. But there's another alternative for the ServiceMix ESB. The file component which is used for reading files can delegate the normalization of the content, more precisely the transformation to the XML format, to a Marshaler. This article describes such a Marshaler and also how to use it for your own Integration Application.

The CSV2XML Marshaler

The source code and a binary Shared Library of the CSV2XML-Marshaler of this article are available. The Marshaler can be used in any commercial and non-commercial project. Please read the license at the end of this article.

Prerequisite

Apache ServiceMix Version 3.2.1 or newer.

Importing the Shared Library into ServiceMix

The archive csv2xml-marshaler-1.0.zip contains the needed libraries for the marshaler. This archive has to be pasted into the folder hotdeploy of the ServiceMix engine. Afterwards the library is available for several components. To gain access to the library from a Service Unit, it has to be included into the classpath of the SU.

Integrating the Marshaler into a file-Service-Unit

The Marshaler is added, in the form of a property, into a file-poller. Therefore a new bean is declared in the configuration descriptor file xbean.xml of the file-SU. The bean refers to the class of the marshaler and may contain several properties for configuration (see chapter properties). Additionally, the Shared Library of the Marshaler has to be included into the classpath.

<file:poller service="csv2xml:poller"
      endpoint="e1"
      targetService="csv2xml:sender"
      file="file:.\inbound">
      
  <property name="marshaler">
     <bean class="de.predic8.CSV2XMLMarshaler">
       <property name="delimiter" value=';' />
     </bean>
  </property> 
</file:poller>
<classpath>
  <library>csv2xml-marshaler</library>
</classpath>

Listing 1: Example of a xbean.xml configuration file including the Marshaler

Properties

Following properties can be committed to the Marshaler:

quantity;price;good
    5;1.45;Milk

<quantity>5</quantity>
    <price>1.45</price>
    <good>Milk</good>

Table 1: Description of the properties of the Marshaler

Functionality of the Marshaler

The tool is based on the CSV to XML converter from OIO. The Marshaler is integrated into a file Service Unit and it receives the InputStream of the file poller and tries to convert it to generic XML. If the transformation has been finished successfully, the generated XML is put onto the bus. Parameters, as the delimiter between several entries of the file, can be configured with the declaration of properties. In the output, lines of the CSV data are separated with a <row> element from each other. You can find a complete output of a transformed CSV file at the end of this article.

Example of a transformed file

Subsequent the result of a transformation is shown on the basis of a simple CSV dataset.

quantity;price;good
5;1.45;Milk
2;0.99;Butter
10;1.19;Muffin

Listing 2: Content of the CSV file

<?xml version="1.0" encoding="UTF-8"?>
<csvimport>
  <row>
    <quantity>5</quantity>
    <price>1.45</price>
    <good>Milk</good>
  </row>
  <row>
    <quantity>2</quantity>
    <price>0.99</price>
    <good>Butter</good>
  </row>
  <row>
    <quantity>10</quantity>
    <price>1.19</price>
    <good>Muffin</good>
  </row>
</csvimport>

Listing 3: XML output of the Marshalers

Source code of the Marshalers

/**
 * This source code can be used in any order. This software is provided "AS IS" without warranty of any kind
 *
 * @author Thomas Bayer, Marco Hippler
 * predic8 GmbH
 * http://www.predic8.com
 *
 */


package de.predic8;

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;

import javax.jbi.JBIException;
import javax.jbi.messaging.MessageExchange;
import javax.jbi.messaging.MessagingException;
import javax.jbi.messaging.NormalizedMessage;

import org.apache.log4j.Logger;
import org.apache.servicemix.components.util.FileMarshaler;
import org.apache.servicemix.jbi.jaxp.BytesSource;
import org.apache.servicemix.jbi.util.StreamDataSource;
import org.xml.sax.ContentHandler;

import de.oio.util.xml.CSVReader;
import de.oio.util.xml.SAX2Writer;



public class CSV2XMLMarshaler implements FileMarshaler{

  private static final Logger log = Logger.getLogger(CSV2XMLMarshaler.class);
  private char delimiter=';';
  private boolean firstLine=true;
  private boolean containsDummyL=false;
  
  public String getOutputName(MessageExchange arg0, NormalizedMessage arg1)
      throws MessagingException {
    return null;
  }

  public void readMessage(MessageExchange arg0, NormalizedMessage out,
      InputStream in, String arg3) throws IOException, JBIException {
    StreamDataSource inputData=new StreamDataSource(in);
    ByteArrayOutputStream bStream=new ByteArrayOutputStream();
    ContentHandler contentHandler = new SAX2Writer(bStream);
    CSVReader reader = new CSVReader(inputData.getInputStream(), "Cp1252",  
contentHandler);
    reader.setDelimiter(delimiter);
    reader.setFirstLineAreColumnNames(firstLine);
    reader.setContainsDummyLine(containsDummyL);
    reader.parse();
    
    log.info("InputStream processed");
      
    out.setContent(new BytesSource(bStream.toByteArray()));
    
    
  }

  public void writeMessage(MessageExchange arg0, NormalizedMessage arg1,
      OutputStream arg2, String arg3) throws IOException, JBIException {
 
    
  }

   public void setDelimiter(char delimiter) {
    this.delimiter = delimiter;
  }

  public char getDelimiter() {
    return delimiter;
  }

  public void setFirstLine(boolean firstLine) {
    this.firstLine = firstLine;
  }

  public boolean isFirstLine() {
    return firstLine;
  }

  public void setContainsDummyL(boolean containsDummyL) {
    this.containsDummyL = containsDummyL;
  }

  public boolean isContainsDummyL() {
    return containsDummyL;
  }
}

Listing 4: Source code of the CSV2XML-Marshaler

You can find the complete Maven project to create the jar archive needed for the hotdeployment in ServiceMix here.

License

Use of this software is free of charge for both personal and commercial purposes.

Disclaimer

THE predic8 GMBH HEREBY DISCLAIMS ANY AND ALL WARRANTIES, EXPRESS OR IMPLIED, RELATIVE TO THE SOFTWARE, INCLUDING BUT NOT LIMITED TO ANY WARRANTY OF FITNESS FOR A PARTICULAR PURPOSE OR MERCHANTIBILITY. THE predic8 GMBH SHALL NOT BE LIABLE OR RESPONSIBLE FOR ANY DAMAGES, INJURIES OR LIABILITIES CAUSED DIRECTLY OR INDIRECTLY FROM THE USE OF THE SOFTWARE, INCLUDING BUT NOT LIMITED TO INCIDENTAL, CONSEQUENTIAL OR SPECIAL DAMAGES.