Object Queue/Pooling

Currently I am in design phase of a system which require high throughput and equally very high level for performance. In one of the design meeting, one of the very experienced member in the team make a very interesting remark, lets make object pool, this will increase the performance. Hmmm. That was really interesting for me and inspired me to write this blog on object pooling.

The myth about the object pooling goes like this “Object pooling works. The idea is that we can reuse objects by pooling them on and off free lists instead of using new and letting the garbage collector pick them up. Once you have more than one thread going off the pool, you need a synchronized free list, which has costs. If the list gets hot and contended, you can get scaling bugs. It gets complicated too fast and is not worth it for small to even moderate sized objects. Use it only for large objects.”

Object Pooling is about objects being pre-created and “pooled” for later use. The concept behind object pooling is that it is far cheaper to access an object from a pool of identical objects rather than create a new instance of the object. Creating a new instance involves the following steps:

1. Loading the class if not already loaded.
2. Obtaining the required memory from the heap.
3. Creating an instance of the class within the obtained memory.

In the case of object pooling all these steps would have been performed for a pre-configured number of objects already. It was argued that one of these pre-configured objects could be utilized when a new instance is required. After use, this object would be returned to the pool. The problems with this paradigm are:

Pooling is not cheap. It requires the following steps as part of its execution:

1. The pool should be locked when the object is being obtained.
2. The pool needs to be potentially scanned for unused objects.
3. The object needs to be marked as _used_.
4. The pool can then be unlocked.

All these steps are not cheap since they involved synchronization locks across multiple threads.

1. Pooling does not automatically support garbage collection. The object needs to be explicitly returned back to the pool to avoid memory leaks. Isn’t garbage collection one of the most compelling reasons to use Java or a similar language in the first place?
2. Objects need to be coded to be “stateless”. Otherwise, the object may have to be re-initialized to ensure that it can be re-used back from the pool. This may be a feature of the pooling implementation but would add overhead during deallocation.

Hence it’s pretty conclusive that object pooling make no sense but resource pooling (like connection pool etc) make lots of sense.

OSGi

OSGi: - Open Services Gateway Initiative

Motivation

JEE world buzzwords have always been scalability, transaction management, security, high availability, manageability, monitoring etc. Trust me coming from my personal experience it is a stereotypical statement, but we do understand these problems and have successfully crafted solutions for each of them to an acceptable degree. But the major challenge lies in JEE based application deployment and things have basically remained unchanged since the very origin of JEE. No denying the implementation of modularization is implemented in code by dividing code base in modules, whether logical, physical or conceptual, at runtime they are seen as one monolithic application in which, making a change (be it large or small), requires a restart.

Modularity in java is achieved by using

· Java class files

· Java Archive (JAR) files

1.Provide form of physical modularity

2. May contain applications, extensions, or services

3. May declare dependencies

4. May contain package version and sealing information

Java provides the mechanisms to do these things, but they are

· Low level

· Error prone

· Ad hoc

Java's shortcoming are particular evident in its support for both modularity and dynamism

Java Modularity Limitations

· Limited scoping mechanisms

§ No module access modifier

· Simplistic version handling

1. Class path is first version found

2. JAR files assume backwards compatibility at best

· Implicit dependencies

1. Dependencies are implicit in class path ordering

2. JAR files add improvements for extensions, but cannot control visibility

· Split packages by default

1. Class path approach searches until it finds, which leads to shadowing or version mixing

2. JAR files can provide sealing

· Unsophisticated consistency model

1. Cuts across previous issues, it is difficult to ensure class space consistency

2. Missing module concept

3. Classes are too fine grained, packages are too simplistic, class loaders are too low level

· No deployment support

Java Dynamism Limitation

· Low-level support for dynamics

- Class loaders are complicated to use and error prone

· Support for dynamics is still purely manual

- Must be completely managed by the programmer

- Leads to many ad hoc, incompatible solutions

· Limited deployment support

- Unable to load modified classes at runtime

OSGi

OSGi framework provides

• Simple component model
• Component life cycle management
• Service registry
• Standard service definitions (separation of specification & implementation)

OSGi Resolves many deficiencies associated with standard Java support for modularity and dynamism

• Defines a module concept

- Explicit sharing of code (i.e., importing and exporting)

• Automatic management of code dependencies

- Enforces sophisticated consistency rules for class loading

• Life-cycle management

- Manages dynamic deployment and configuration

OSGi framework is used as a modularity mechanism for Java and provides logical and physical system structuring. It has benefits for development and deployment. It provides sophisticated dynamic module life-cycle management. It simplifies creation of dynamically extensible systems, where system components can be added, removed, or rebound at run time while the system as a whole continues to function

OSGi framework promotes a service oriented interaction pattern as shown below: -

OSGi Features

1. Simple component and packaging model

Bundles = JARs and contain java classes, resources and meta-data

That is, bundle represents a single component contained in a JAR file. A bundle defines a logical and physical modularity unit with

· Explicit boundaries

- External interface (i.e., exports)

- Internal class path

Java code, resources, and native libraries

· Explicit dependencies

- Package dependencies (i.e., imports)

· Explicit versioning

- Package version, bundle version

· Isolation via class loaders

· Packaging format (bundle JAR file)

• Meta-data explicitly defines boundaries & dependencies as java package imports & exports.
• Dependencies & associated consistency are automatically managed

The framework automatically resolves package dependencies when a bundle is activated

- Matches bundle’s imports to available exports

- Ensures package version consistency

· If a bundle cannot be successfully resolved, then it cannot be activated/used

Hence OSGi is a collection of bundles that interact via service interfaces and these bundles may be independently developed and deployed. Bundles and their associated services may appear or disappear at any time

2. Defines a component life cycle

OSGi framework defines dynamic bundle life cycle

· Possible to install, update, and uninstall code at run time

· Automatic package dependency resolution

· Replaces low-level class loaders

3. Explicitly considers dynamic scenarios
4. Interaction through service interfaces
5. Multi-version support

That is, it’s possible to have more than one version of a shared package in memory at the same time. As result for a given bundle, the service registry is implicitly partitioned according to the package versions visible to it

6. Import version range

That is, exporters still export a precise version, but importers may specify an open or closed version range. It also eliminates existing backwards compatibility assumption.

Import-Package: ClaimService; version=“[1.0.0,1.5.0)”

Note: - Multi-version sharing and importing version ranges make implementation package sharing possible

7. Arbitary export/import attributes

· Exporters may attach arbitrary attributes to their exports, importers can match against these arbitrary attributes

· Exporters may declare attributes as mandatory

· Mandatory attributes provide simple means to limit package visibility

· Importers influence package selection using arbitrary attribute matching

Import-Package: ServiceImpl;

version=“1.0.0”;

myattr=“myvalue”

8. Package Consitency model

· Exporters may declare package “uses” dependencies

· Exported packages express dependencies on imported or other

exported packages, which constrain the resolve process

· The framework must ensure that importers do not violate constraints implied by “uses” dependencies.

Hence its pretty conclusive OSGi will provide in true sense modularization in java and much needed deployment ease

In Memory Computation

I was having a recent chat about caching with someone/group of people and the best part is the magic word used “Caching will improve performance, put it in cache”. It was interesting enough for me/my wife Shikha (she is also a practicing JEE architect) to reflect on and add to my ruminations here in this blog. Just to add this my blog and my ideas and is no way meant to hurt anybody.

The Basic Concept

Yes off course no body can deny, caching is all about performance. The basic rationale behind this is to avoid repeating a complex, input-dependent, stable operation- be it data manipulation, algorithmic computation or expensive calls over the network - by storing the results of the previous computation and returning the results from some kind of a caching repository using some key based on the inputs passed to this operation.

The preceding definition probably has left you more blurry eyed than what you were when you first stumbled on to this page! Like Sherlock Holmes would say, there are certain singular points about caching that are of interest and hence would require further elaboration.

Complex Operation Dependent on Input

Now what is complex operation; I define a complex operation as something that would consume a lot of computing resources. These resources can be CPU resources or network bandwidth or anything else that is sparse and should not be used in a profligate way. This operation should be input dependent, which means that every invocation of this operation must produce the same result given that the inputs are the same. The operation also has to be stable meaning that the results produced by it should not change in a random way. For instance, the results cannot be based on some random number generation or be influenced by the current time of invocation. However, it is acceptable that the results change with time. For instance, it is acceptable to cache data that is editable by the user.

Cache

To re-iterate, the idea in caching is to avoid the computation again by storing the results of the first computation in some place. So where do we store the results? The answer is that we store it somewhere from where we can easily retrieve them again when needed. This place is the caching repository or in short cache. Hence the cache is the place you go to when you want to

• Store results
• Retrieve them in a future call.

Cache Key

We have already said that the computation performed by the “cached operation” produces a result that is only dependent on the input parameters and nothing else. Hence by using the input parameters as the key we should be able to retrieve the results of the previous computation without repeating the entire operation once again. This key that is dependent on the input parameters for the operation is called the cache key. The results are stored in the cache using this key. They are also retrieved using this key.

How is Caching implemented?

This is best illustrated with an example.
Consider the diagram below:

The client makes a call to a complex operation. This call is intercepted typically by a caching interceptor, which computes the key from the input parameters and queries the cache with the key. If a value is found in the cache it is returned back to the client thereby bypassing the call to underlying method. Otherwise, the complex operation is invoked and its results are cached into the cache. The results are then returned back to the client. Thus the cache gets updated for future requests.

Types of Caching

• Reactive
• Pro-active

The above diagram shows reactive caching wherein the cache is only populated on the first request. The cache would never be populated if there had been no requests with the particular input parameters. A second approach is Pro-active caching where some agent running in the background populates the cache actively. The results are returned from the cache.

A combination of both pro-active and reactive caches is also possible. In this approach, the cache is populated pro-actively. When an actual request is made, the cache is first checked to see if it contains the data that needs to be returned. If the data is found it is returned. Else the cache is first populated reactively as before and returned.

Which cache fits where?

Pro-active caching works well in the following cases:

• The total number of items in the cache is limited - in other words the total number of cache keys are finite. This is a corollary of another caching axiom - don’t use caching when the memory requirements are expected to be very high.
• During the life of the application, it is expected that all of the items in the cache are likely to be used at one time or the other.
• Example of pro-active caches can include caches of all the states within a country, cache of the postal codes etc.

Reactive caching is more suitable when

• The cache can potentially have a lot of combination of cache keys.
• The operation is potentially invoked with a vast variation of parameters during the life of the application
• Reactive caches include caches of all transactions for a particular customer, account information etc. This kind of information tends to be too large to be cached pro-actively. The queries against these kinds of information also tend to be arbitrary. For instance, there might be a lot of accounts whose transactions would never be queried during the life of the application and hence it would be wasteful to cache this kind of information.

The Problems involved in using a cache - A Balancing Act

The use of caching, like any software strategy, tends to become a tight ropewalk dictated by a few parameters, which need to be considered carefully. The biggest problem with caching is that of “cache staleness”. What this means is that the caches can potentially contain data that has since been updated at the source. Hence the data retrieved from the cache becomes stale data and unfit for serious applications. Example: Let us say we are caching account balances. The balance has got updated into the database afterwards thereby resulting in stale data in the cache.

This problem tends to get compounded if the data at the source changes frequently. Hence the following observations prevail:

• Caching is easy to implement for “static data”
• Caching for data that changes frequently can only be non-stale if all updates to the data go through the caching layer.

The second point above is interesting. If all the access to the data were accomplished thru the cache, then the cache would automatically be non-stale. Hence isn’t it sensible to keep the cache as close as possible to the source of the data? The answer is a little moot because the whole point of caching is to optimize on the operation. Often the most expensive part of invoking an operation is the network overhead of making this call. If we want to avoid the network overhead, it makes sense to cache the data as close as possible to the consumer of the data and not the producer of the data. Reconciling these two apparently contradictory requirements constitutes an important step in optimizing the cache.

JDK 6 Script Engine

One of the cool feature of jdk1.6 is the ability to call java script from the java code. Have a look at the sample code, which reverse the string using java script

package com.jdk16;

import javax.script.ScriptEngine;
import javax.script.ScriptEngineManager;
import javax.script.ScriptException;

public class ReverseUsingJS {
public static void main(String args[]) {
ScriptEngineManager scriptManager = new ScriptEngineManager();
ScriptEngine engine = scriptManager.getEngineByName("javascript");
try {
engine.put("name", "reverse");
engine.eval("var outputVar = '';for (i = 0; i <= name.length; i++) {"
+ " outputVar = name.charAt(i) + outputVar" + "}");
String name = (String) engine.get("outputVar");
System.out.println(name);
} catch (ScriptException e) {
System.err.println(e);
}
}
}

Inter Process Orchestration

In this white paper I am thinking, infact had starting making the framework, that will enable inter process orchestration, with due respect to BPEL of the world. I have named this framework as "Process Choreographer"
Note :- Process can be any Java Object.

Process Choreographer is a lightweight Java framework for building workflows using java beans to orchestrate events. You can think of Process Choreographer as a simple alternative to BPEL where the workflows are all specified and implemented using Java code rather than declarative XML.

When building highly concurrent or distributed applications (like insurance claim processing) it is very common for there to be many events happening; often asynchronously and in different threads and its very common to need to perform kinds of workflow or orchestration across these events.

Off course one can certainly use things like BPEL to solve these kinds of problems. However often this is a bit heavy weight & complex and you just want to have a bean based workflow using regular Java code to represent the activities involved in the workflow. Further more, BPEL will always comes with the cost of performance. I have heard many times where my client/patners have asked me, we want to run atleast 10 transaction per second, Process Choreographer framework will be answer to these problems.

One of the main goals of Process Choreographer is to reuse what the Java platform is good for in the workflow space; then supplement it with missing abstractions rather than re inventing the wheel. Hence
1) Write a plain old java class (from where every things begins)
2) Use normal java code construct like if - else, while etc
3) Use regular Java fields in your workflow class, then use JDBC/DAO/JPA to deal with
persistence

While working with various client and their concurrent or distributed applications there are many use cases for needing to orchestrate among multiple concurrent events.

1) starting dependent services that is where a parent component is dependent on the child components starting; where the start process may be asynchronous in different threads. e.g. there may be a recovery process on startup which you need to wait for.
2) implementing master-slave type protocols where you need to monitor the state of the master and slave to make decisions on what to do; together with dealing with transitions from Started to Recovering to Running then maybe to FailingOver etc.
3) implementing message orchestration. You may want to implement some simple orchestrations, waiting for either a response to arrive or a timeout to fire etc

Process Choreographer aim to solve to over come above problem by providing the ability to do the following

1. Execute Process In Sequence

2. Execute Process In Parallel Split

3. Process Synchronization : Synchronize two parallel threads of execution.

4. Process Exclusive choice: Choose one execution path from many alternatives

5. Simple Process Merge: Merge two alternative execution paths

Soon I will update you with design of this framework.

Injecting Transactions Into Rules

First of all, some guys may think this slightly strange. With due respect to rules and not making them procedural, there are several business use case where from the THEN part, there is a need to call the web service. I was experimenting this thought on JBoss Rules, Quick Rules etc rule engine. But interestingly , all these rule engine never stops me calling the Web Service but there is no transaction control over it (distributed transaction). Thanks to java open source world and jboss rules guys, I can play with their code.

How can we inject transaction? Thanks to Spring and declarative transactions, this is very much feasible. So here was my game plan in nut shell, introduce a new attribute in the JBoss Rule grammar named "Transaction" and at the time of rule execution run the spring container and inject the declarative transactions.

BPM, BRM in SOA

Introduction

It’s a myth propagated by many consultants and vendors that Business Rule Management System (BRM) and Business Process Management System (BPM) are two alternatives for creating flexible and agile enterprise solution. There is no denying the fact that BPM uses rules for decision making. As always truth lies somewhere in the middle, BPM and BRMS are like two sides of a coin which can’t be separated, especially if used in conjunction with principle of service oriented architecture (SOA)

The underline idea behind this is totally separate process logic from decision logic. So what is process logic? Process logic is the specific logic of the business process such as controlling the sequence of activities, adhering of deadline and handling of exception. It is implemented using Process Engines like jBPM etc. Decision logic represents process independent management policies and principles and are implemented using rule engines like JBoss rules etc as part of BRMS

Principles of SOA

1. Explicit boundaries
   SOA is a design approach for special enterprise solution and special information technology software architecture

=> Consistent result responsibility

2. Shared Contract and Schema, not class that is, service share common contract

SOA is strictly independent of the technology.

=> Unambiguous service level

3. Service orientation is an evolution of component based architectures that is service are reusable

=> Proactive event sharing

4. Vendor Independent

SOA is strictly independent of the technology

5. Policy Driven

6. Services are discoverable

7. Business Driven

The granularity of the process modeling determines the granularity of the business service

8. Loosely coupled
9. Wire format not programming languages APIs
10. Document oriented

Consider a message in string format

2007-11-0642055

and compare it to document format:

2007-1-06

420

55

BPM & BRM in SOA

BPM is a closed loop model consisting of three steps: -

Step 1: Analyze, plan, model, test and simulate business process

Step 2: Execute business process via workflow spanning all applications

(process logic) by mean of process engine on a SOA as the

infrastructure.

Step 3: Plan, monitor and control processes, their performance and the

interplay of all business process.

BRP is a closed loop model consisting of three steps: -

Step 1: Based on business vocabulary it describes rules

Step 2: Describes the lifecycle of rules perform analysis, and design

simulation and test, via execution through rule engine.

Step3: Rule monitoring and controlling including responsibilities.

SOA transforms existing business computing assets into well-defined services. It can work effortlessly with BPM because of the reliance on services. SOA exposes services while BPM consumes them. When properly implemented, SOA opens a vast inventory of services for BPM to piece together into an all-inclusive flow of services.

While BPM defines and orchestrates the flow

The drive to use SOA to create a more agile infrastructure also highlights the importance of externalizing highly volatile business logic that is subject to change.

Highly volatile business logic can be defined as business rules. In the traditional application structure these business rules are buried in the application while in a more modern approach they are separated. Just as process flow can be separated from application code into an external BPM engine, the same can be done with business rules. Separating both process flow and business rules empowers a business analyst to make operational changes more quickly, providing maximum flexibility and adaptability.

An important shared characteristic of BPM, BRM and SOA is that they all deliver tactical cost/time benefits while building a base for competitive growth. Each one contributes to the overall agility of a company's IT infrastructure in the long term.

Business application companies such as Oracle/PeopleSoft/Siebel and SAP are contributing to the new opportunity of SOAs by supplying a new class of business applications called service-oriented business applications (SOBAs). SOBAs provide extended functionality for use on Web services standards and should contribute significantly to a company's repository of business services.

So the evolution toward service orientation as an enterprise elevates process thinking, analytics and performance measurement as core competencies to achieve competitive advantage. Accordingly, companies should establish best practices around these disciplines in parallel with implementing the supporting technologies.

Process and Rules in SOA

Many business areas employ rules. Traditional examples include marketing strategies,
pricing policies, customer relationship management practices, human resources activities,
regulatory constraints, product and service offerings. As rules evaluation matured, areas
such as recommendation technology revolve around rules.
Although the definitions of processes and rules differ the differences are not as clear cut.
Typically both rules and orchestration concepts are intertwined within the definition of a
business process.

Feature of rule engines vs orchestration engine (BPM)

Execution time Rules Engines strive to evaluate business rules as quickly as possible. In
contrast, ORCHESTRATION ENGINEs cater to long-running processes, where services
can take minutes, hours or even days to complete.

Synchronicity Rules evaluation is synchronous. In contrast, processes are intrinsically
asynchronous. Typically the ORCHESTRATION ENGINE invokes services in an
asynchronous manner. The mechanisms required to deal with asynchonicity such as
correlation and compensations are readily available in orchestration environments.

Statefulness Rules Engines are stateless; when a rule fires an engine typically pulls its
inputs from the knowledge base, evaluates it, and then updates the knowledge base.
In contrast, ORCHESTRATION ENGINEs are specifically designed to hold the state
(i.e., execution context) of each active orchestration.

Determinism The Rules Engine fires simultaneously all rules whose conditions are satisfied.
The order in which these rules actually execute is non-deterministic. In contrast,
process implementations strive for close alignment with the business. Business
processes are deterministic, and people go to great lengths to ensure determinism.

Rules are orchestrated by the process engine the same way as other services. This is absolutely consistent and show SOA elegance and power. Decision logic is primarily understood as business logic, as its process independent. This underscores the principle of the resusability of services. Rules are modeled and implemented once and are then available for use in different processes.

The right way of using rule engine by mean of BRM for all process-independent rules in SOA and it bring following merits: -

1. reusability of services and hence higher productivity.

2. rule change control mechanism can be controlled.

3. rule changes are versioned and achieved

4. process independent of rules.

Example, showing rule and process are closely coupled

WHEN status=new
THEN DO
creditRating:=call(creditService,customerData)
status:=ranked
END
WHEN status=ranked AND creditRating=threshold
THEN DO
floodCertification:=call(floodService,propertyAddress)
END
WHEN status=ranked AND creditRating>=threshold
THEN DO
appraisedValue:=call(appraisalService,propertyAddress)
END
WHEN status=ranked AND haveCertification and haveAppraisal
THEN DO
decision:=call(decisionService,application,creditRating,floodCertification,
appraisedValue)
status:=decisioned
END

WHEN status=decisioned
THEN DO
documentation:=call(letterService,application,decision)
status:=documented
END
WHEN status=documented
THEN DO
call(mailingService,documentation)
status:=mailed
END

Challenges in BRM

<<todo>>