Caching Strategy

Why Cache Objects?

Object caching allows applications to share objects across requests and users, and coordinates the objects' life cycles across processes. By storing frequently accessed or expensive-to-create objects in memory, object caching eliminates the need to repeatedly create and load data. It avoids the expensive reacquisition of objects by not releasing the objects immediately after their use. Instead, the objects are stored in memory and reused for any subsequent client request.

Advantages of Caching

One of the main benefits of object caching is the significant improvement in application performance. In a multitier-ed application, data access is an expensive operation compared to other tasks. By keeping frequently accessed data and not releasing it after its first use helps in avoiding the cost and time required for the data's re-acquisition and release.

Following are the benefits of caching:

•        Performance — Fast access of frequently used resources is an explicit benefit of caching. Therefore, when the same resource needs to be accessed again, the resource need not be acquired or fetched from somewhere; it is already available.
•        Scalability — caching by its nature is implemented by keeping hold of frequently used resources and not releasing them. It hence avoids the cost of acquiring (frequently used) resources and their release, which has a positive effect on the scalability.

Disadvantages of Caching

Object caching also includes a few disadvantages. Following are the key disadvantages of caching:-

•       Synchronization Complexity — Depending on the kind of resource, complexity increases because consistency between the state of the cached resource and the original data, which the resource is representing, needs to be ensured.
•        Durability — Changes to the cached resource can be lost when the system crashes. However, if a synchronized cache is used or cache coordination, then this problem can be avoided.
•        Footprint — The run-time footprint of the system is increased as possibly unused resources are cached. However, if an Evictor is used, then the number of such unused cached resources can be minimized.

Caches Classification

 Data-usage predictability influences the caching strategy. Based on the same, cache can be classified as:-

•         Primed Cache: - The primed-cache pattern is applicable when the cache or part of the cache can be predicted in advance. This pattern is very effective in dealing with static. These resources are prefetched and stored in cache during startup of application to give better performance/response time like loading of the web pages (UI).
•        Demand Cache: - The demand cache is suitable when the future resource demand cannot be predicted. The calling module will acquire and store the resource in the cache only when it is needed. This optimizes the cache and achieves a better hit-rate. As soon as the resource is available, it is stored in the demand cache. All subsequent requests for the resource are satisfied by the demand cache

NOTE: The primed cache is populated at the beginning of the application (prefetched/cache warmed), whereas the demand cache is populated during the execution of the application.

The caches can be further broadly classified into the following two types which can in turn, fall either in the category of a primed or demand cache depending on the data-usage predictability:-

1.       ORM cache

2.       In-process cache

ORM/JPA Cache

ORM/JPA framework like Hibernate, EclipseLink,  is a way to bridge the impedance mismatch between objects oriented programming (OOP) and relational database management systems (RDBMS). The ORM’s  (JPA) cache can be layered into two different categories: the read-only shared cache used across processes, applications, or machines and the updateable write-enabled transactional cache for coordinating the unit of work. ORM uses layered architecture to implement two-level caching architecture, the first layer represents the transactional cache and the second layer is the shared cache designed as a process or clustered cache.

Transactional Cache

Entities formed in a valid state and participating in a transaction are stored in the transactional cache. Transactions are characterized by their ACID (Atomicity, Consistency, Isolation, and Durability) properties. Transactional cache demonstrates the same ACID behavior. Transactions are atomic in nature; each transaction will either be committed or rolled back. When a transaction is committed, the associated transactional cache will be updated and synched with the shared cache (explained in the next section). If a transaction is rolled back, all participating objects in the transactional cache will be restored to their pretransaction state

Shared Cache

The shared cache can be implemented as a process cache or clustered cache. A process cache is shared by all concurrently running threads in the same process. A clustered cache is shared by multiple processes on the same machine or by different machines. Distributed-caching solutions implement the clustered cache.

Entities stored in the transactional cache are useful in optimizing the transaction. As soon as the transaction is over, they can be moved into the shared cache. All read-only requests for the same resource can be fulfilled by the shared cache; and, because the shared cache is read-only, all cache coherency problems are easily avoided. For the read-only request there will be no coherency problem as the queries will be executed against the shared cache and transactional cache will not be synchronized with the shared cache.

The shared cache can be implemented using distributed cache. But distributed caches add overheads like serialization/serialization costs along with network traffic to keep the backup copy of data in case of failure. Distributed cache also adds deployment overheads. There could be another strategy to synchronize cache, called Cache Coordination. The cache coordination mechanism works as follows:

1.       Detect a change to an entity

2.       Relay only the name of entity along with its identification i.e. primary key to other nodes. (In case JPA/ORM is used as persistence layer, it’s easy to use JPA callback hooks to relay this information)

3.       Once the notification is received just invalidate shared cache for that entity (using primary key)

Decide What to Cache?

Caching the right data is the most critical aspect of caching. If we fail to get this right, we can end up reducing performance instead of improving it. We might end up consuming more memory and at the same time suffer from cache misses, where the data is not actually getting served from cache but is refetched from the original source.

Cacheable Data Classification In components the data that requires caching can be classified into the following:-

•         Truly Static Data
•          Mostly Static Data
•         In Flight Static Data
•         In Process Data

Truly Static Data

 The data which falls under this category is nonvolatile data i.e. it rarely changes. As a result such data should reside within JVM caches and these caches should never be evicted.

Recommendation – Local Cache (L2 Cache) (like guava caches, never expiring caches)

Mostly Static Data

 The data which falls in this category is also relatively nonvolatile, i. e. data very rarely changes. As a result such data should reside within the JVM caches and these caches will be invalidated for the changed data using ORM/JPA callback hooks.  (refer to cache coordination mechanism above)

Recommendation – Local Cache (L2 cache) + Cache Coordination

In Flight Static Data

The data which falls under this category is volatile by nature but no two threads simultaneously access these. It's worth mentioning here that the object graph of these kinds of data is very huge and take ample amount of time during the ORM’s object construction phase. These objects need to be evicted as soon as they are not in use (dereferenced) hence the cache type SoftWeak is recommended (along with cache coordination)

Recommendations:

1.       LocalCache (L2 cache) with SoftWeak references + Cache Coordination

2.       NO CACHING

In Process Data

The data which falls under this category is also very static but is constructed during the implementation of a functional algorithm like creation of a process derivation rule cache (stored the  entity constructed using complex native SQL). The data like these falls in the "In Process Data" and should live in "In-Process Cache"

Recommendation: Locale Cache (Query Cache) + Cache – Coordination

Chasing the Right Size Cache

There is no definite rule regarding the size of the cache. Cache size depends on the available memory and the underlying hardware and memory settings. There are two possible approaches which can be used to set the size of the caches : -

•            Approach 1: - An effective caching strategy is based on the Pareto principle2 (that is, the 80–20 rule).
•        Approach 2: -Size of the cache = Number of rows for that entity in the database at start of an application.

Cache Taxonomy

Below picture shows cache taxonomy