Spring Framework Property Configuration

Architecting software development projects often requires a mechanism to manage configuration properties in such a way that they can be defined and overridden depending on the environment in which they’re being used. This requirement often driven by a need to use different resources at each stage of the development life-cycle, i.e. development, test and production. This article describes a scheme that allows properties to be defined and overridden in a simple way within properties @Configuration classes.

Using the following class as an example, the @PropertySource annotation triggers an attempt to load property values from two properties files. One is a simple literal classpath name entry, the second is also a classpath name entry but uses an embedded expression to allow selection of the appropriate classpath locatable file at runtime:

  1. “classpath:properties/app.properties”
  2. “classpath:properties/app-${spring.profiles.active:default}.properties”.

The @PropertySource annotation triggers property file loading in definition order – consequently “classpath:properties/app.properties” is loaded first and “classpath:properties/app-${spring.profiles.active:default}.properties” second.

package com.greendot.properties;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.PropertySource;
import org.springframework.context.support.PropertySourcesPlaceholderConfigurer;

/**
 * Created on behalf of GreenDot Software Ltd.
 *
 * @author matt.d.vickery@greendotsoftware.co.uk
 * @since 08/07/2013
 */
@Configuration
@PropertySource(value = {
        "classpath:properties/app.properties",
        "classpath:properties/app-${spring.profiles.active:default}.properties"
})
public class PropertiesConfiguration {

    private final Logger logger = LoggerFactory.getLogger(this.getClass());

    @Bean
    public PropertySourcesPlaceholderConfigurer getProperties() {
        return new PropertySourcesPlaceholderConfigurer();
    }
}

 

Overriding is supported using this scheme as any property values loaded from the properties file ‘app.properties’ will be overridden by any property values loaded from ‘app-${spring.profiles.active:default}.properties’ that use the same properties key.

As for expansion of the variable expression (${spring.profiles.active:default}), the variable value will be populated at runtime according to the value set for the relevant Java System Property (i.e. -Dspring.profiles.active=test). You may observe that any value can be used for this property, the following code examples use default, test and production as possible values that make sense for the problem. If no value is set, then default will be used, determined by the definition ‘..:default}.properties’. This example uses the property key ‘spring.profiles.active’ specifically in order that Spring Framework Profiles can be used through the same configuration mechanism.

Notice that the example also uses a PropertySourcesPlaceholderConfigurer @Bean, this is made available in order that @Value annotations can be used in other Spring bean classes – an example follows.

A suitable mechanism to demonstrate properties configuration is using a Unit Test, the following properties will be used to exercise the PropertiesConfiguration class.

The content of app.properties is:

1
2
3
4
mongo.db.port=27017
mongo.db.name=catalogue
mongo.db.logon=mvickery
mongo.db.password=sugar

The content of app-default.properties is:

1
mongo.db.server=localhost

The content of app-test.properties is:

1
2
3
mongo.db.server=testhost.greendotsoftware.co.uk
mongo.db.logon=tester
mongo.db.password=tpassword

The content of app-production.properties is:

1
2
3
mongo.db.server=prodhost.greendotsoftware.co.uk
mongo.db.logon=operations
mongo.db.password=opassword

 

As an example of how properties management works with this scheme, the following test class loads properties files through the @ContextConfiguration loading of the PropertiesConfiguration class we defined above. The class is then run by the JUnit class runner utility SpringJUnit4ClassRunner, this means that the test can be run with a Spring context with beans loaded from any referenced @Configuration classes, e.g. PropertiesConfiguration.class. Furthermore, the @Value annotation triggers autowiring of property values into annotated variables such as dbName, dbServer etc.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
package com.greendot.properties;

import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.AnnotationConfigApplicationContext;
import org.springframework.core.env.Environment;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;

import static org.hamcrest.CoreMatchers.is;
import static org.springframework.test.util.MatcherAssertionErrors.assertThat;

/**
 * Created on behalf of GreenDot Software Ltd.
 *
 * @author matt.d.vickery@greendotsoftware.co.uk
 * @since 08/07/2013
 */
@ContextConfiguration(classes = {
        PropertiesConfiguration.class
})
@RunWith(SpringJUnit4ClassRunner.class)
public class PropertiesConfigurationTest {

    private static final String MONGO_DB_SERVER = "mongo.db.server";
    private static final String MONGO_DB_NAME = "mongo.db.name";
    private static final String MONGO_DB_LOGON = "mongo.db.logon";
    private static final String MONGO_DB_PASSWORD = "mongo.db.password";

    @Value("${"+MONGO_DB_NAME+"}")
    private String dbName;
    @Value("${"+MONGO_DB_SERVER+"}")
    private String dbServer;
    @Value("${"+MONGO_DB_LOGON+"}")
    private String dbLogon;
    @Value("${"+MONGO_DB_PASSWORD+"}")
    private String dbPassword;

    @Test
    public void defaultProfile() {
        assertThat(dbName, is("catalogue"));
        assertThat(dbServer, is("localhost"));
        assertThat(dbLogon, is("mvickery"));
        assertThat(dbPassword, is("sugar"));
    }

    @Test
    public void productionProfile() {

        System.setProperty("spring.profiles.active", "production");

        AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext();
        context.register(PropertiesConfiguration.class);
        context.refresh();
        assertThat(getProperty(context, MONGO_DB_SERVER), is("prodhost.greendotsoftware.co.uk"));
        assertThat(getProperty(context, MONGO_DB_NAME), is("catalogue"));
        assertThat(getProperty(context, MONGO_DB_LOGON), is("operations"));
        assertThat(getProperty(context, MONGO_DB_PASSWORD), is("opassword"));
    }

    @Test
    public void testProfile() {

        System.setProperty("spring.profiles.active", "test");

        AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext();
        context.register(PropertiesConfiguration.class);
        context.refresh();
        assertThat(getProperty(context, MONGO_DB_SERVER), is("testhost.greendotsoftware.co.uk"));
        assertThat(getProperty(context, MONGO_DB_NAME), is("catalogue"));
        assertThat(getProperty(context, MONGO_DB_LOGON), is("tester"));
        assertThat(getProperty(context, MONGO_DB_PASSWORD), is("tpassword"));
    }

    private String getProperty(final AnnotationConfigApplicationContext context, final String property) {
        return context.getBean(Environment.class).getProperty(property);
    }
}

 

There are three test methods in this class. The first method tests loading of ‘default‘ properties – as the value of ‘spring.profiles.properties‘ is null as runtime. We expect properties found in ‘app.properties‘ to be loaded first along with a single additional property loaded from ‘app-default.properties‘ immediately aftwards.

Spring Integration – Message Gateway Adapters

Introduction

This article is going to present a technique for handling response types from message sub-systems in a service-based Message Driven Architecture (MDA). It will provide some theory, experience of message sub-system development and some robust working code that’s running in several financial  institution production deployments.

As Spring Integration is underpinned by a well established set of Spring APIs familiar to most developers, I have chosen this as a basis for this article.

If developers only ever had to build integration flows to handle positive business responses, this article would not be necessary. However, real world development, particularly within service integration environments, results in developers having to deal not just with positive business responses but business exceptions, technical exceptions and services that become unresponsive.

Spring Integration Gateways

Spring Integration (SI) Gateways are entry points for message sub-systems. Specify a gateway using a Java interface and a dynamic proxy implementation is automatically generated by the SI framework at runtime such that it can be used by any beans in which it’s injected. Incidentally, SI gateways also offer a number of other benefits such as request & reply timeout behaviour.

Typical Message Driven Architecture (MDA) Applications

As with Java design patterns for application development, similar patterns exist for integration applications. One such common integration pattern is service-chaining – a number of services are connected in series or parallel that together perform a business function. If you’ve built and deployed services using MuleSoft’s Mule, SpringSource’s Spring Integration, FuseSource’s ServiceMix (Camel), Oracle’s Service Bus or IBMs WebSphere ESB the chances are that you’ve already built an application using chained-services. This pattern will become ever more widespread as the software industry moves away from client-server topologies towards service based architectures.

A recent engagement in which I provided architectural consultancy will be used as an example  implementation of the service-chain application pattern. The engagement required the team to build a solution that would transition financial messages (SWIFT FIN) from a raw (ISO15022) state through binding, validation and transformation (into XML) and ultimately added to a data store for further processing or business exception management.

Using EIP graph notation, the first phase service composition could be represented as follows. A document message arrives into the application domain, it’s parsed and bound to a Java object, semantically validated as a SWIFT message, transformed to XML and then stored in a datastore. As a side note, the binding, semantic validation and transformation are all performed by C24s iO product. Furthermore, the full solution sample that can be found in GitHub for this project contains dispatcher configurations in order that thread pools can be used for each message sub-system, for the sake of brevity and clarity, such details will be omitted from this article.

Using EIP graph notation, the first phase service composition could be represented as follows. A document message arrives into the application domain, it’s parsed and bound to a Java object, semantically validated as a SWIFT message, transformed to XML and then stored in a datastore. As a side note, the binding, semantic validation and transformation are all performed by C24s iO product. Furthermore, the full solution sample that can be found in GitHub for this project contains dispatcher configurations in order that thread pools can be used for each message sub-system, for the sake of brevity and clarity, such details will be omitted from this article.

Although this configuration specifies a chain of services it’s not adequate to form the bases of a robust production deployment. An exception thrown by any of the services would be thrown straight back to the entry gateway thus loosing context, i.e. which service threw the exception, any non-response code invoked by the service may result in its Java thread getting parked and null values returned by a service may cause unexpected problems or even for the entry gateway to hang; if an entry gateway is used unlike in this diagram.

Unresponsive Service Invocation

The Spring Integration specific construct for accessing a message sub-system is the gateway. Although apparently simple, the SI gateway is a powerful feature that results in generation of a dynamic proxy generated by Spring’s GatewayProxyFactoryBean. This bean can be injected into new or existing code or services as an implementation of the interface. The SI gateway also provides facilities to deal with timeouts and provides some error handling facility. A typical Spring Integration namespace XML configuration is as follows:

<int:channel id="parsing-gw-request-channel" datatype="java.lang.String">
  <int:queue capacity="${gateway.parse.queue.capacity}"/>
</int:channel>

<int:gateway id="parseGateway"
             service-interface=
                 "com.c24.solution.swift.flatten.gateway.ParseGateway"
             default-request-channel="parsing-gw-request-channel"
             default-reply-channel="parsing-gw-reply-channel"
             default-reply-timeout="${gateway.parse.timeout}"/>
<int:channel id="parsing-gw-reply-channel" 
             datatype="biz.c24.io.api.data.ComplexDataObject"/>

Evolving the design, the next phase application architecture design needs to take advantage of these gateways, the Spring Integration context can be extended quite simply by creating a Java interface for each service. Building message sub-system gateways leads us towards a design model like the following:

A small amount of additional configuration and some very simple Java interfaces means that developers can now configure request/reply timeouts on the gateway and avoid any unresponsive code – assuming (as we always should) that code written locally or supplied by 3rd parties is capable of misbehaving.

Additionally, the SI gateway allows specification of an error handling channel so that you have the opportunity to handle errors. This design I am presenting here is not going to use error channels but handle them in a different way – hopefully that will become more obvious as the design evolves.

Specific semantics and configuration examples for gateways and gateway timeouts can be found in reference material provided by SpringSource and other blogs.

A significant benefit to the integration solution has been added with the use of Spring Integration gateways. However, further improvements need to be made for the following reasons:

  1. A gateway timeout will result in a null value being returned to the calling, or outer, flow. This is a business exception condition that means that the payload that was undergoing processing needs to be pushed into a business exception management process in order that it can be progressed through to conclusion – normally dictated by the business. If a transient technical issue caused this problem, resubmission may solve the exception, otherwise further investigation will be required. Whatever the eventual outcome, context about the failure and it’s location need to be recorded and made available to the business exception management operative. The context must contain information about the message sub-system that failed to process the message and the message itself.
  2. Exceptions generated by message sub-systems can be handled in a few different ways, through the error channel inside the gateway undergoing message processing failure or by the calling flow. Again, context needs to be recorded. In this case, the technical exception needs to be added to the context, along with the gateway processing the message undergoing failure and any additional information that may be used within the business exception management process for resolution.

If transactions were involved in this flow, for example a transactional JMS message consumption flow trigger, it would be possible to rollback the transaction in order that it can be re-queued to a DLQ. However, the design in this software compensates for failures by pushing failing messages to a destination configured by the developer directly; and this may of course be a JMS queue if that’s required. This avoids transaction rollback and adds the benefit of exception context directly rather than operations staff and developers having to scour logs to locate exception details.

The Gateway Adapter

In order to all handle exceptions taking place within a message sub-system and also handle null values a Gateway Adapter class can be used. The reason that the SI gateway error channel is not used for this purpose is that it would be complex to define and would have to be done in several places. The gateway adapter allows all business exception conditions to be handled in one place and treat them in the same way. The Gateway Adapter is a custom written Spring bean that invokes the injected gateway directly and manages null and exception responses before allowing the invocation request to return to the caller (or outer flow).

The architectural design diagram evolved from the previous design phase includes a service activator backed by a Gateway Adapter bean, this calls the injected gateway (dynamic proxy) which calls the business service.

Nuts and Bolts

The design diagrams are a useful guide for making the point but maybe the more interesting part is the configuration and code itself. As with the entire solution, the outer or calling flow can be seen in the project located in GitHub, however a useful snippet to view is one of the gateway adapter namespace configurations, the gateway, the gateway adapter and the gateway service configuration. As a number of gateways exist in this application, and they all follow the same pattern, the following configuration and code will merely demonstrate one of them.

Gateway Adapter Namespace Configuration

 
<int:channel id="message-parse-channel" datatype="java.lang.String"/>
<int:chain input-channel="message-parse-channel" 
           output-channel="message-validate-channel">
    <int:service-activator ref="parseGatewayService" method="service"/>
</int:chain>

Gateway

package com.c24.solution.swift.flatten.gateway;

import org.springframework.integration.Message;

/**
 * @author Matt Vickery - matt.vickery@incept5.com
 * @since 17/05/2012
 */
public interface ParseGateway {
    public Message<?> send(Message<String> message);
}

GatewayAdapter

package com.c24.solution.swift.flatten.gateway.adapter;

import biz.c24.io.api.data.ComplexDataObject;
import com.c24.solution.swift.flatten.exception.ExceptionContext;
import com.c24.solution.swift.flatten.exception.ExceptionSubContext;
import com.c24.solution.swift.flatten.gateway.ExceptionGatewayService;
import com.c24.solution.swift.flatten.gateway.ParseGateway;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.integration.Message;
import org.springframework.util.Assert;

/**
 * @author Matt Vickery - matt.vickery@incept5.com
 * @since 17/05/2012
 */
public class ParseGatewayService extends AbstractGatewayService {

  private static final Logger LOG =
      LoggerFactory.getLogger(ParseGatewayService.class);
  private final ParseGateway parseGateway;
  private final ExceptionGatewayService exceptionGatewayService;

  public ParseGatewayService(
         final ParseGateway parseGateway,
         final ExceptionGatewayService exceptionGatewayService) {
      this.parseGateway = parseGateway;
      this.exceptionGatewayService = exceptionGatewayService;
  }

  public Message<ComplexDataObject> service(Message<String> message) {

      Message<?> response;
      try {
          LOG.debug("Entering parse gateway.");
          response = parseGateway.send(message);
      } catch (RuntimeException e) {
          LOG.error("Exception response .. {}, exception: {}", getClass(), e);
          LOG.error("Invoking ... process because: {}.", e.getCause());
          buildExceptionContextAndDispatch(
            message,
            ExceptionContext.PARSE_FAILURE,
            ExceptionSubContext.EXCEPTION_GATEWAY_RESPONSE,
            exceptionGatewayService);
          throw e;
      }

      if (response != null) {
        if (!(response.getPayload() instanceof ComplexDataObject))
            throw new IllegalStateException(INTERRUPTING_..._EXCEPTION);
      } else {
        LOG.info("Null response received ....", getClass());
        buildExceptionContextAndDispatch(
          message,
          ExceptionContext.PARSE_FAILURE,
          ExceptionSubContext.NULL_GATEWAY_RESPONSE,
          exceptionGatewayService);
        throw new GatewayAdapterException(NULL_GATEWAY_RESPONSE_CAUGHT);
      }

      Assert.state(response.getPayload() instanceof ComplexDataObject);
      return (Message<ComplexDataObject>) response;
  }
}

GatewayService

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:c24="http://schema.c24.biz/spring-integration"
       xmlns:int="http://www.springframework.org/schema/integration"
       xsi:schemaLocation="http://www.springframework.org/schema/integration
 http://www..org/schema/integration/spring-integration.xsd  http://schema.c24.biz/spring-integration
 http://schema.c24.biz/spring-integration.xsd
 http://www.springframework.org/schema/beans
 http://www.springframework.org/schema/beans/spring-beans.xsd">

    <int:channel id="parsing-gw-request-channel" 
                 datatype="java.lang.String">
        <int:queue capacity="${gateway.parse.queue.capacity}"/>
    </int:channel>
    <int:gateway
      id="parseGateway"
      service-interface="com.c24.solution.swift.flatten.gateway.ParseGateway"
      default-request-channel="parsing-gw-request-channel"
      default-reply-channel="parsing-gw-reply-channel"
      default-reply-timeout="${gateway.parse.timeout}"/>
    <int:channel id="parsing-gw-reply-channel" 
                 datatype="biz.c24.io.api.data.ComplexDataObject"/>

    <int:chain input-channel="parsing-gw-request-channel" 
               output-channel="parsing-gw-reply-channel">
        <int:poller fixed-delay="50" 
                    task-executor="binding-thread"/>
        <c24:unmarshalling-transformer
          id="c24Mt541UnmarshallingTransformer"
          source-factory-ref="textualSourceFactory"
          model-ref="mt541Model"/>
    </int:chain>

</beans>

Summary

Whatever type of message based integration system you are designing, once you get beyond the use of sample code for prototyping a technology, and let’s face it, there are millions of lines of code out there that are copied from trivial examples, you need to consider invocation in the face of technical exceptions, business exceptions & non-responsive services. Using Spring Integration Gateways to represent entry to message sub-systems means that we must be able to cope with all of these types of behaviour. The Gateway Adapter pattern allows a single, central processing location for such conditions.

The intent for the Gateway Adapter should be clear from the example configuration and code provided but can also be run and tested using the full source located on GitHub. Please leave any feedback or comments as you see fit.

Spring Integration – Payload Storage via Header Enrichment

There’s often a need to temporarily store transient messages during design of Spring Integration flows – several different mechanisms are available in the toolkit.

It’s pretty straight forward to take a message, use an SI header enricher construct and place the message in the header using a SpEL expression – in fact one for the header key name and one for the payload extraction.

The following SI flow demonstrates an example of how to do just that :

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:int="http://www.springframework.org/schema/integration"
       xsi:schemaLocation="http://www.springframework.org/schema/beans 
       http://www.springframework.org/schema/beans/spring-beans.xsd
       http://www.springframework.org/schema/integration 
       http://www.springframework.org/schema/integration/spring-integration-2.1.xsd">
  <int:gateway id="headerManagementGateway"
                 service-interface="com.l8mdv.sample.HeaderManagementGateway"/>
  <int:chain input-channel="request-message-storage-channel"
             output-channel="request-message-retrieval-channel">
    <int:header-enricher>
      <int:header name="#{T(com.l8mdv.sample.HeaderManagementGateway)
                   .REQUEST_PAYLOAD_HEADER_KEY}" expression="getPayload()"/>
    </int:header-enricher>
  </int:chain>

  <int:chain input-channel="request-message-retrieval-channel">
    <int:transformer expression="headers.get(T(com.l8mdv.sample.HeaderManagementGateway)
               .REQUEST_PAYLOAD_HEADER_KEY)"/>
  </int:chain>
</beans>

This example can be executed by implementing a gateway as follows:

package com.l8mdv.sample;

import org.springframework.integration.Message;
import org.springframework.integration.annotation.Gateway;

public interface HeaderManagementGateway {

    public static final String REQUEST_PAYLOAD_HEADER_KEY = "REQUEST_PAYLOAD_HEADER_KEY";

    @Gateway (requestChannel = "request-message-storage-channel")
    public Message<String> send(Message<String> message);
}

and then running a test such as this one:

package com.l8mdv.sample;

import org.junit.Assert;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.integration.Message;
import org.springframework.integration.support.MessageBuilder;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;

import static com.l8mdv.sample.HeaderManagementGateway.REQUEST_PAYLOAD_HEADER_KEY;

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(locations = {"classpath:META-INF/spring/header-management.xml"})
public class HeaderManagementIntegrationTest {

    @Autowired
    HeaderManagementGateway headerManagementGateway;

    @Test
    public void locatePayloadInHeader() {
        String payload = "Sample test message.";
        Message<String> message = MessageBuilder.withPayload(payload).build();
        Message<String> response = headerManagementGateway.send(message);

        Assert.assertTrue(response.getHeaders().get(REQUEST_PAYLOAD_HEADER_KEY).equals(payload));
    }

    @Test
    public void locateTransformedPayload() {
        String payload = "Sample test message.";
        Message<String> message = MessageBuilder.withPayload(payload).build();
        Message<String> response = headerManagementGateway.send(message);

        Assert.assertTrue(response.getPayload().contains(payload));
    }
}

For full source code and configuration, see the header-management maven module under  https://github.com/mattvickery/l8mdv-si-samples