Sunday, March 21, 2010

Architectural Patterns That Limit Application Scalability & Performance Optimization

Note - Currently working on this blog

Client & Server Pattern


Master & Worker Pattern

Partition For Parallelism 

Common Approach To Improve Throughput

Performance Optimization Concepts





Latency = time delay between starting an activity and when the results are  available /  detectable
Throughput = ratio of number of tasks completed in a unit of time.
Performance (perceived speed / responsiveness) = number of requests made and acknowledged in a unit of time.
Throughput and Performance are often confused!  (sometimes they are the same)
Example:
Average Throughput = 10 tasks / sec
Average Latency = 1000ms (1sec / 10)
Performance = unknown

NOTE - 


To improve performance = reduce latencies (between request and response)
To improve throughput = increase capacity (or reduce total latency)


Only Execute Mandatory Tasks

Increase CPU speed (scale up)

Optimize Algorithm

Exploit Parallelism (Scale Out)

Optimize Large Latencies

Best of all world - Do them all

Reduce Use Of XML


Use Case - Oracle Coherence In Compute Grid

Saturday, March 20, 2010

Flexible Searching Using Compass

            Lucene is a very powerful search engine. Compass is written on top of lucene and provides lucene type search capabilities on top of domain model (jpa entity).In this blog I aim to look into use cases where compass search capabilities can be utilized.
           Compass provides ability to search goggle style on JPA entities. But the question is what are the use cases where somebody would like to have goggle style search on the JPA entities. For example a health insurance claim form contain information like claimant name (first name, last name), address etc. So how to provide search with search information like gaurav malhotra myaddress. The search information can possible comes from the following end points
 UI/Webservice
One can provide UI search screen based on the fixed parameter like
First Name = xxxx
Last Name = yyyyy
-- -- -- -- -- --
and its every easy to construct the SQL/JPQL/JPA2.0 search criteria. But if there is a limitation on the search information and now suppose we want to include the street name also in the search information, this will trigger the change in the UI search screen. Similar challenge exists in case of web services also.

One of nice things about compass is that it fits seamless with JPA (ORM products like EclipseLink). The compass specific search metadata can be directly specified on the JPA entity as shown in the JPA entity Claim.So a typical programming model using compass looks like 




The JPA entity Claim looks like
@Searchable
public class Claim {
    @SearchableId
    private int id;

    @SearchableProperty
    @SearchableMetaDatas({@SearchableMetaData(name = "firstname")})
    private String firstName;

     
    @SearchableProperty
    @SearchableMetaDatas({@SearchableMetaData(name = "lastname")})
    private String lastName;
     
    @SearchableProperty
    @SearchableMetaDatas({@SearchableMetaData(name = "streetname")})
    private String streetName;
     
      -- -- --- ---
  
}

Hence the search code looks like
> Person person = null;
   person = searchService.search(“firstname:gaurav lastname:malhotra streetname=xxx”);
   ** Search Person whose first name is gaurav , last name is malhotra and streetname is xxx
> Person person = null;
   person = searchService.search(“gaurav”);
   ** Search entity which has 'gaurav'

Groovy lovers can also use the search by injecting searchService into the groovy shell and calling search as
Person person = null;
Person = searchService.search(“gaurav malhotra”);  // the search framework automatically finds the Person entity.
Hence some of the import scenarios (EAI) where xml to domain model mapping is provided using groovy logic; adding compass search capabilities to it can be very powerful.

NOTE :- Compass search capability is very vast and support complex querying like inheritance support etc. Refer to Compass Search for more details

Conclusion
          Compass searching provides a very flexible and powerful way of searching capabilities, agnostic of underlying JPA entities, which is sometime referred to google style searching. In my subsequent article I will cover the scabaility of the compass by putting its indexes in the oracle coherence

Predictable Linear Scalability Design Pattern

Aim

            To use a messaging middleware queue to distribute the workload across the jvm is quiet common and helps in linear scalability. This blog explain the design pattern “Polling Consumer” which aids in workload distribution. This blog also explains how the messaging middleware itself become a bottleneck in linear scalability and how to overcome it. Quiet a famous saying I have heard in many of design meetings "Drop a message in a queue, it will be picked up by the consumer to perform the processing. Increase the consumers to increase the performance". Lets see how much truth in it :-)
           

Polling Consumer

In any messaging system (queue), the consumer needs an indication that application is ready so that it can consume the message. The best approach for the consumer is to repeatedly check the channel for message availability. If any message is available, the consumer consumes it (process it). This checking is a continuous process known as polling. The messaging system (queue) can be polled from the multiple jvms. All the consumers/jvms should equally share the workload and should not keep hold of messages on which they are not working.

Solution:
The application should use a polling consumer, one that explicitly makes a call when it wants to receive a message. 

Interactions:
A polling consumer is a message receiver. A polling consumer restricts the number of concurrent messages to be consumed by limiting the number of polling threads. In this way, it prevents the application from being blocked by having to process too many requests, and keeps any extra messages queued up until the receiver can process them.Each consumer/jvm will ask for message whenever it has capacity to perform hence provide perfect load balancing automatically.
Merits
  1. Consumer asks for work (as a message) whenever it has capacity to perform, hence provide efficient workload management.
  2. No message is lost as all the messages are persisted in the messaging system (queue).
  3. Increasing number of consumers (jvms) can increase the scalability/performance to certain level.
Demerits
Predicable scalability/performance cannot be guaranteed, as increasing consumers (jvms) the messaging system can become the bottleneck as shown by the picture below





The messaging middleware (queue) becomes the bottleneck because, as the numbers of consumers are increased to achieve the linear scalability, the messaging middleware fails to handle so many consumers. The messaging middleware  will look like a overloaded horse cart like below



Poor horse chart or should I say messaging middle ware ;-)

The following picture shows the decreasing performance (i.e. no messages processed per seconds) as the numbers of jvms are increased. In below the queue used is a durable queue.
NOTE: - The following tests were performed on windows machine.


Hence with queue in db after the performance reached the optimum level (5 jvms as shown in above picture), adding more jvm results in negative performance.

Predictable Scalability

            One the key feature of Oracle Coherence is to provide “Predictable Scalability” i.e. by increasing a node (a jvm) in a cluster the performance of the system increases. The following picture shows the Queue of the “Polling Consumer” pattern being implemented using Oracle Coherences + AQ that is, AQ being made as the persistent store for the oracle coherence. As a result the consumer always enqueue/dequeue messages from the in-memory queue (hence AQ –DB bottleneck is removed). The enqueue will result a message in the coherence cache and also AQ and similarly dequeue of the message will result in message being removed from the coherence cache and AQ.

In the above design the cache store make sure message are always in sync with persistent storage (db). 
1. enqueue => oracle coherence cache put(message) => AQ enqueue
2. dequeue=> oracle coherence cache remove()   => AQ dequeue
  So the endpoint always perform operation in the cache

Conclusion
      A queue is an efficient way distributing work load. But the linear scalability cannot be always assured.ln functional areas like invoice processing,claim processing etc, using queue for workload distribution may not be always helpful in linear scalability. If the number of the JVMs are less then desired performance can be achieved by fine tunning the consumers.
In my next blog I will share the framework i.e design details to achieve the above. The "Polling Consumer" is an enterprise integration pattern (http://www.eaipatterns.com/PollingConsumer.html). In past, as a part of my job I have developed spring based framework which enable the implementation of enterprise integration design pattern. But spring source team have added a new framework called "Spring Integration", which enable ease of implementing enterprise integration pattern. How to create "Polling Consumer" using spring integration can be found at http://static.springsource.org/spring-integration/reference/htmlsingle/spring-integration-reference.html#endpoint-pollingconsumer

A sneak preview - in order to achieve Polling Consumer backed by queue in a cache (coherence) which also persist messages we need the following
 1) Off course, a queue/channel in oracle coherence backed by persistence storage. Lets call it - CoherenceAwareQueueChannel
2)  A Oracle Coherence cache store which persist the messages
3)  A simple POJO to which poller delivers the message - consumer. In the terminology of enterprise integration its called Service Activator (refer to http://www.eaipatterns.com/MessagingAdapter.html for detail)


Enqueue
Message message = MessageBuilder.withPayload("gaurav in cache").setHeader(JmsHeaders.TYPE, "sometype").build();
coherenceQueue.send(message);

Dequeue

Message msg = (Message) coherenceQueue.receive();

Polling Consumer

<beans:bean id="coherenceQueue" class="com.oracle.aq.cachestore.channel.CoherenceAwareQueueChannel" depends-on="cacheFactoryCreation"/>
<beans:bean id="messageConsumer" class="com.oracle.aq.cachestore.junit.AqCacheStoreTest$ServiceActivator"/>
<service-activator id="coherenceConsumer" input-channel="coherenceQueue"
 ref="messageConsumer" method="onMessage" auto-startup="false">
 <poller>
  <interval-trigger interval="10" />
 </poller>
</service-activator>

As a result of above a message will be passed to the POJO - ServiceActivator method onMessage. The poller will poll the CoherenceAwareQueue every 10 nano seconds. It's also possible to specify the number of polling threads.
The service activator pojo looks like
 public class ServiceActivator {
  
  public void onMessage(Message msg) {
                // Message will be delivered automatically
  }
 }

Simple programming model agnostic of coherence/spring details/jms details so that developer can use it with ease

Wednesday, November 5, 2008


Toplink/Eclipselink Coherence

Integration Using Spring

Introduction

            The aim of this document is to explain how to integrate EclipseLink and Oracle Coherence using Spring Framework.  These white paper target audiences are developers/technical architects/technical managers. This white paper assumes the reader has the knowledge of the following: -

1.       Java EE framework

2.       Spring Framework

Spring framework is collections of smaller frameworks and act as glue, which tie together different layers of enterprise application and provide consistent and simple programming model based on best practices. Some of the key features of the spring framework are: -

·         Provides IOC container (Inversion of control/Dependency Injection)

·         Provides AOP (Aspect Oriented Programming) framework.

·         Provides Data Access framework

·         Provides declarative Transaction management framework.

·         Provides testing framework which enables use to develop code using TDD approach (Test Driven Development)

3.       Eclipse Link

Eclipse Link is a comprehensive open source persistence solution with around 12 years of commercial usage.  Some of the key features of Eclipse Link are: -

·       Fully JPA compliant

·       Enables simplified configuration of the target application server

·       Enable integration with JTA transaction manager

·       Very flexible framework, which allows customizations like SessionCustomizer & DescriptorCustomizer, which allows customizations of eclipse link session.

·       Provide support for concurrency protection – locking

·       EclipseLink cache. It has two types of caches i.e. L1 cache (EntityManager) and L2 cache (EntityManagerFactory)

·       Provides advance query facilitity like object graph loading optimization using Join and Batch Fetch.

 

4.       Oracle Coherence

Oracle Coherence is a JCache-compliant in memory distributed data grid solution for clustered applications and application servers. Some of the key features of Oracle Coherence are: -

·       Container – less clustering of java processing (pojo – pliain old java objects)

·       Real-time event observation (Listener pattern)

·       Parallel queries and aggregation (Purely object based queries)

·       Clustered JMX

·       Pluggable Cache store

 


Overview

Eclipse link and Oracle coherence integration provides highly scalable solution(s) and also provide data grid-computing capabilities. Spring framework allows powerful feature of retrieving objects from the Spring BeanFactory. This feature can be further being extended to retrieve objects configured in a cache scheme from BeanFactory and hence give better control over object instances creation. This is especially true for cache servers configured with CacheStore objects. Typically these CacheStore need to configure with data sources, connection pools, etc. The following document explains how to integrate Eclipse link and Oracle Coherence using Spring Framework and hence provide ability to leverage spring’s framework programming model and ability to provide easy configuration of data sources/pool etc for plain Java object

Eclipse link grid – spring aware cache store/loader

Coherence supports pluggable CacheLoader and CacheStore implementations. As result the entity classes annotated with JPA specifications can directly interact with Coherence API (get, put, etc)

SpringAwareEntityCacheLoader is class which supports reading the JPA entities from the database by implementing load/loadAll methods. SpringAwareEntityCacheStore extendsSpringAwareEntityCacheLoader and also support write, update and delete operations to the database. This class also implements CacheStore and implements the methods store, storeAll, erase, eraseAll.

The figure shows a typical Oracle Coherence and Eclipse Link integration viaSpringAwareEntityCacheStore




CACHE FACTORY SPRING INTEGRATION

            The entry point to access caches or any other services is CacheFactory through the use of its static methods (getCache etc). Coherence provides flexibility to override its default behavior via use of cache configuration file (coherence-cache-config.xml). Hence its possible to provide custom implementations of Coherence interfaces CacheStore  & CacheLoader. These interfaces can be configured via a class-scheme. Class schemes provide a mechanism for instantiating an arbitrary Java object. The class-scheme may be configured to either instantiate objects directly via their class-name (using new operator), or indirectly via a class-factory-name and method-name. The class-scheme must be configured with either a class-name or class-factory-name and method-name.

It is possible to tell Coherence to retrieve objects (cache store) configured in a class-scheme from a Spring BeanFactory instead of creating its own instance.

 

SpringAwareCacheStoreFactory is spring aware class, which extends DefaultConfigurableCacheFactory, and provides the ability to delegate class-scheme bean instantiations responsibility to spring’s BeanFactory class.



Key features of SpringAwareCacheStoreFactory

1.       Application Context Aware class: - Since this class implements spring ApplicationContextAware interface, as result at the time bean post processing, this class will be handed over the current ApplicationContext. Hence it will be useful for the cache servers that require beans from the spring container.

2.       Bean Factory Ware class: - This class also implements spring BeanFactoryAware interface, as result at the time of bean post processing current BeanFactory will be handed over to it. As a result a BeanFactory can be provided at the runtime by an application or provided directly by the spring container. This is useful for Coherence applications running in a container that already has an existing BeanFactory.

Note: - Implementation of spring’s InitializingBean interface will make sure BeanFatory is set before making call to DefaultConfigurableFactory class instantiateAny method is called. This is the key method, which creates object (cache store)-using class-scheme. Typically the object will be configured as spring-bean: myCaheStore in the class-scheme tag in the coherence config file.

3.       Ability to set the path of coherence configuration file and hence clean approach to manage coherence configuration files.

 

Conclusion

            Using Spring as a glue framework to integrate Eclipse Link and Oracle Coherence provides following key benefits: -

 

1.       Ability to inject separate data source to be used by SpringAwareCacheStore to perform the database operations. The data source can be setup in spring configuration file

2.       Ability to provide pool of data sources. Again these data sources pools can be configured in spring configuration file.

3.       Provide seprate EntityManager for the SpringAwareEntityCacheStore as result application’s transaction will never be interfered.

4.       Make it possible to use spring’s JpaDaoSupport and hence spring declarative transaction management and also spring programming model (best practices)


Sunday, September 7, 2008

Eclipse Link and Oracle Coherence integration using Spring

For the last few days I have an intresting journey into Oracle Coherence. Spring being one of the most powerful and widely accepted framework, I thought of playing with it. The belwo article is an outocome of my journey ....

EclipseLink and Coherence can be used together in several combinations. This blog discusses, how to integrate using Spring framework.These options including using Coherence as a Eclipse Link plug-in, using Eclipse Link as a Coherence plug-in via the CacheStore interface and bulk-loading Coherence caches from a Eclipse Link query. Most applications that use Coherence and EclipseLink would like to use a mixture of these approaches. The Eclipse Link API features powerful management of entities and relationships, and the Coherence API delivers maximum performance and scalability.

Aim:

1. To make Oracle Coherence as the L2 shared cache using spring framework.








CacheStore/CachLoader:

Orache Coherence provides above two interfaces based on the Jcache specifications. Jcache specifies API and semantics for temporary, in memory caching of Java objects, including object creation, shared access, spooling, invalidation, and consistency across JVM's.

In order to make Oracle Coherence as the distributed cache, as a first step it is important to make the CacheStore and CacheLoader which will do

· Store/Update/Delete the java object in the oracle coherence

· Retrieve object from the cache.

So lets make two classes EclipseLinkCacheStore which implements CacheStore and EclipseLinkCacheLoder which implements the CacheStore, as shown below:-

public class OhiEclipseLinkEntityCacheLoader extends Base implements CacheLoader {

// Will be injected using spring

private EntityManagerFactory emf;

private EntityManager em;

protected String m_sEntityName = "";

protected Class m_sEntityClass;

public void setEntityName(String entityName) {

this.m_sEntityName = entityName;

}

public void setEntityClass(Class entityClass) {

this.m_sEntityClass = entityClass;

}

public OhiEclipseLinkEntityCacheLoader(EntityManagerFactory emf) {

this.emf = emf;

// Create a new Entity manager to avoid any

// conflicts with the existing Enity manager

// which is using L1 cache

this.em = emf.createEntityManager();

}

- - - - - - - - - - - - - - - - - - - -

- - - - - - - - - - - - - - - - - - - -

// Implement other inteface methods

}

public class OhiEclipseLinkEntityCacheStore extends OhiEclipseLinkEntityCacheLoader implements CacheStore {

private EntityManagerFactory emf;

private String name="";

public OhiEclipseLinkEntityCacheStore(EntityManagerFactory emf) {

super(emf);

- - - - - - - -

- - - - - - -

// Implement other inteface methods

}

Spring can inject the Enity Manager as shown below: -

class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">

class="org.springframework.orm.jpa.vendor.EclipseLinkJpaVendorAdapter">

value="org.eclipse.persistence.platform.database.oracle.OraclePlatform" />

class="org.springframework.instrument.classloading.SimpleLoadTimeWeaver" />

class="org.springframework.jdbc.datasource.DriverManagerDataSource">

value="oracle.jdbc.pool.OracleDataSource" />

value="jdbc:oracle:thin:@xxxx:1521:XE" />

class="org.springframework.orm.jpa.JpaTransactionManager">

ref="entityManagerFactory" />

class="com.oracle.healthinsurance.eclipselink.cachestore.OhiEclipseLinkEntityCacheStore"

scope="prototype">

What’s next?

Lets create spring aware cache store which can pick up the oracle coherence configure files (in a more declrative way rather than passing it using –D options as specified in the oracle coherence documentation)

SpringAwareCacheStore provides a facility to access caches declared in a "cache-config.dtd" compliant configuration file, similar to its super class (DefaultConfigurableCacheFactory). In addition, this factory provides the ability to reference beans in a Spring application context via the use of a class-scheme element. This factor is made ApplicationContextAware so that it will very trivial to get can access to BeanFactory. This can be useful for stand-alone JVMs such as cache servers. It can also be configured at runtime with a pre-configured Spring bean factory. This can be useful for Coherence applications running in an environment that is itself responsible for tarting the Spring bean factory, such as a web container.

The following the code snippet which further extends the spring aware cache factory mentioned in the Oracle Coherence documentations

public class SpringAwareCacheStore extends DefaultConfigurableCacheFactory

implements BeanFactoryAware, ApplicationContextAware, InitializingBean

/**

* This constructor automatically figure out the absolute path to the spring

* configuraion file using DiscoveryUtils. Hence as result cache config file

* can be appeneded at runtime.

*

* If this construtor is used the bean factory is set in the afterProperties

* set method.

*

* @param sPath

*/

public SpringAwareCacheStore(String sPath) {

super(DiscoverUtils.getPathToResouce(sPath));

}

/**

* Appliction context. This application context will be required to get the

* BeanFactory in case this class is instantiated with the constructor which

* take the cache file name.

*/

private ApplicationContext appContext;

public void setApplicationContext(ApplicationContext applicationContext)

throws BeansException {

this.appContext = applicationContext;

}

public void afterPropertiesSet() throws Exception {

if (m_beanFactory == null) {

m_beanFactory = appContext.getAutowireCapableBeanFactory();

// register a shutdown hook so the bean factory cleans up

// upon JVM exit

((AbstractApplicationContext) m_beanFactory).registerShutdownHook();

}

}

// ----- extended methods -----------------------------------------------

// Picked up from the Oracle Coherence documentation

/**

* Create an Object using the "class-scheme" element.

In addition to

* the functionality provided by the super class, this will retreive an

* object from the configured Spring BeanFactory for class names that use

* the following format:

*

*

* <class-name>spring-bean:sampleCacheStore</class-name>

*

*

* Parameters may be passed to these beans via setter injection as well:

*

*

* <init-params>

* <init-param>

* <param-name>setEntityName</param-name>

* <param-value>{cache-name}</param-value>

* </init-param>

* </init-params>

*

*

* Note that Coherence will manage the lifecycle of the instantiated Spring

* bean, therefore any beans that are retreived using this method should be

* scoped as a prototype in the Spring configuration file, for example:

*

*

* <bean id="sampleCacheStore"

* class="com.company.SampleCacheStore"

* scope="prototype"/>

*

*

* @param info

* the cache info

* @param xmlClass

* "class-scheme" element.

* @param context

* BackingMapManagerContext to be used

* @param loader

* the ClassLoader to instantiate necessary classes

*

* @return a newly instantiated Object

*

* @see DefaultConfigurableCacheFactory#instantiateAny( CacheInfo,

* XmlElement, BackingMapManagerContext, ClassLoader)

*/

protected Object instantiateAny(CacheInfo info, XmlElement xmlClass,

BackingMapManagerContext context, ClassLoader loader) {

if (translateSchemeType(xmlClass.getName()) != SCHEME_CLASS) {

throw new IllegalArgumentException("Invalid class definition: "

+ xmlClass);

}

String sClass = xmlClass.getSafeElement("class-name").getString();

if (sClass.startsWith(SPRING_BEAN_PREFIX)) {

String sBeanName = sClass.substring(SPRING_BEAN_PREFIX.length());

azzert(sBeanName != null && sBeanName.length() > 0,

"Bean name required");

XmlElement xmlParams = xmlClass.getElement("init-params");

XmlElement xmlConfig = null;

if (xmlParams != null) {

xmlConfig = new SimpleElement("config");

XmlHelper.transformInitParams(xmlConfig, xmlParams);

}

Object oBean = getBeanFactory().getBean(sBeanName);

if (xmlConfig != null) {

for (Iterator iter = xmlConfig.getElementList().iterator(); iter

.hasNext();) {

XmlElement xmlElement = (XmlElement) iter.next();

String sMethod = xmlElement.getName();

String sParam = xmlElement.getString();

try {

ClassHelper.invoke(oBean, sMethod,

new Object[] { sParam });

} catch (Exception e) {

ensureRuntimeException(e, "Could not invoke " + sMethod

+ "(" + sParam + ") on bean " + oBean);

}

}

}

return oBean;

} else {

return super.instantiateAny(info, xmlClass, context, loader);

}

}

How my Oracle Coherene config file looks like?

Using the above factory the cache config file can be put in the class path and the above factory class will automatically pick it and lauch the Oracle Cohernece.

Here is the sample config file??

*

ohi-cache-distributed

ohi-cache-distributed

spring-bean:entityCacheStore

setEntityName

{cache-name}

5s

true

Below is the spring magic which lanches the above oracle coherence

class="com.oracle.healthinsurance.cache.factory.SpringAwareCacheStore">

value="/META-INF/spring/coherence-cache-config.xml" />

class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">

value="com.tangosol.net.CacheFactory" />

value="setConfigurableCacheFactory" />

This is equivalent to the code

BeanFactory bf = applicationContext.getBeanFactory();

SpringAwareCacheStore scf = new SpringAwareCacheStore /META-INF/spring/",bf);

scf.setBeanFactory(applicationContext.getBeanFactory());

CacheFactory.setConfigurableCacheFactory(scf);

Here my sample Unit test case (based on AbstractJpaTests provided by spring)

NamedCache cache = CacheFactory.getCache("CodFlexCodeSystemsB");

Assert.assertNotNull(cache);

CodFlexCodeSystemsB obj = (CodFlexCodeSystemsB) cache.get(new Long(10001));

System.err.println(">>>" + obj.getSubtype());

Assert.assertNotNull(obj);

obj.setSubtype("SIFC");

cache.put(new Long(10001), obj);

try {

Thread.sleep(8000);

} catch (InterruptedException e) {

// TODO Auto-generated catch block

e.printStackTrace();

}

CodFlexCodeSystemsB obj1 = (CodFlexCodeSystemsB) cache.get(new Long(10001));

System.err.println(obj1.getId());

System.err.println(obj1.getSubtype());

System.err.println("size " + obj1.getCodFlexCodeSystemsTlList().size());

Conclusion

Using spring its very simple to launch oracle coherence as L2 cache and make it enity store. But care should be taken not put all the entity in oracle coherence as it will be negative affect on the performance. After this integration its very simple to exploit the oracle coherence powerful feature like Filters,Query etc.