Ehcache bulk operation APIs

People who have used ehcache know that there is only one bulk operation that is provided as of now which is removeAll(). This operation removes all the entries from the cache. In the next release of ehcache there is a plan to provide these operations as well

Collection <Element> getAll(<Object> keys)
void putAll(Collection<Element> elements)
void removeAll(Collection<Object> keys)

The goal of these new APIs is to provide bulk operations which should be faster than normal operations. Now currently we have these two consistency mode for cache operations

1. Strong: All writes are done under a lock, so once anything is changed (add/remove/update) in the cache, rest all nodes and threads will see the change
2. Eventual: As the name suggests, no clustered lock is taken for each operation and eventually the cache would become coherent. This is used when speed, predictability and uptime is of primary concern in an application.

The challenge is to provide the new bulk APIs to do the operations faster than doing single operation in a loop.

Here is what is planned to achieve this.

1. PutAll (eventual consitency)
We create transaction boundaries by taking a lock and releasing it. There are bunch of optimization which is done like transaction folding etc. Since we already know how many entries we need to put under this call, and since all entries can not be put under one transaction, this can be broken and sent in one batch according to "ehcache.incoherent.putsBatchSize" to the server.
Some thing like
lock.takeLock()
for(int i =0; i < batchSize; i++){
doPut(key, value)
}
lock.releaseLock()
notify listeners for the puts that were done in the for loop

2. PutAll (Strong consitency)
In strong consistency for each operation a lock is taken, operation is performed and then the lock is released. For bulk putAll this can be optimized. To avoid dead lock and be efficient, lock request would be sent asynchronously to the server. Every few milliseconds its checked to see how many lock responses have come from the server. Whatever locks have been granted so far, put will be done for those elements and locks will be released. The putAll call we keep trying to get all the locks and eventually put everything in the cache. Listeners will be immediately notified for all the puts which were done after lock has been granted.

3. removeAll (eventual consitency)
Same as putAll (eventual consistency)

4. removeAll (strong consitency)
Same as putAll (strong consistency)

5. getAll
The implementation can be understood by these steps

1. Collection getAll(Collection) will return a custom collection whose iterator will be overridden
2. The request to server will be based on CDSMDso which will let us split the request in stripes in case of multiple terracotta server stripes.
3. In return we will get a map of keys and object ids of the values, which we will use in our custom iterator
4. When the object ids of the values corresponding to the keys are returned a lookup request will be initiated with a configurable number of object ids batched and the values returned will be added in the local cache
5. When the collection is iterated the value corresponding to the object id associated with the value of a particular key will be returned if present in the local cache. If not then we need to fetch the value from the server in batch
6. Implementation of step 5 is little tricky since some of the values which were added in the local cache while looking up objects might get evicted. The strategy for how the next batch of values will be looked up need to be thought of.

For strong consistency case cluster read lock would be acquired for the whole key set as in the case explained in removeAll(strong consistency) case with the difference of having readlock instead.

Performance Comparison

Right now these new APIs are in development phase. I will be updating the result of this exercise after its development and testing.