The Real Problem: Stateless Applications Eat Developer Brains
The elephant in the room in all stateless architectures is that there is no such thing as "stateless." Pushing state out of the application server doesn't get rid of that state, it just forces you to manage it somewhere else. And, it does so at the expense of your most precious commodity: your poor developer brain. It's hard to write stateless applications. Even if it doesn't seem hard to you, the plain fact is that while you are busy dealing with the effects of a stateless architecture, you could instead be implementing new features and improving existing ones.
The reason externalizing data is hard is because all of the work of managing object state that the JVM used to do for you is now your explicit responsibility. You must now tell the clustering engine that you've made a change so that it may commit your changes to the cluster. And, before reading an object's state, you must remember to check it out of the clustering service.
Managing state data by hand is hard. That's why the JVM goes to so much trouble to do it for you.
Enter Terracotta: Code Like Your Mom Used to Make
What if you didn't have to do any of this funny business to get scalability and reliability? What if the JVM had access to a service that you could plug into to make its heap durable, arbitrarily large, and shared with every other JVM in your application tier? Enter Terracotta, network-attached, durable virtual heap for the JVM. In the spirit of full-disclosure, I'm a co-founder of Terracotta and work there as a software developer.
Terracotta is an infrastructure service that is deployed as a stand-alone server plus a library that plugs into your existing JVMs and transparently clusters your JVM's heap. Terracotta makes some of your JVM heap shared via a network connection to the Terracotta server so that a bunch of JVMs can all access the shared heap as if it were local heap. You can think of it like a network-attached filesystem, but for your object data; see Figure 1.
The Terracotta server handles keeping your clustered object data persistent and coherent and coordinating threads across attached JVMs. The Terracotta client libraries use standard bytecode manipulation techniques to make your existing code cluster aware. All of this is done transparently, driven by a declarative configuration file or through annotations, not through an intrusive API.
Terracotta manages the clustered virtual heap the same way the JVM manages the physical heap. Field updates to clustered objects are automatically and efficiently distributed by Terracotta to the cluster so that all participating JVMs have an up-to-date and stable view of your object data. Terracotta also manages thread interaction between JVMs by making your synchronized(), wait(), and notify() calls cluster aware.
Your code thinks it's talking to threads and objects in the local JVM. Terracotta lets you deploy that same code on as many JVMs as you need to meet your capacity demands without having to write a "stateless" application.
A Terracotta Case Study
To see how this works in the real world, let's take a look at how a large publishing company recently used Terracotta to solve a critical database overload problem in their stateless web application.
The application in question is a test proctoring service that holds examinations for up to 5000 concurrent users. Before using Terracotta, the application was deployed using a stateless architecture backed by a relational database. As users proceeded through the examination, their answers were committed to the database so no in-flight examinations would be in jeopardy should one of the application servers go off-line.
At that level of traffic, the database was already at 70 percent utilization. However, the business was growing and the forecast of double the number of concurrent users in a matter of months presented the application team with a choice between scaling up the overloaded database at a significant cost or investigating some sort of alternative scale-out architecture.
The application team settled on using Terracotta to store the in-flight examination data in the durable virtual heap. They were able to keep state data -- the incomplete examination -- as plain Java objects until the examination was over at which time the results were posted all at once to the database. By eliminating the chatter with the database during the examination, they were able to double the number of concurrent users and simultaneously reduce their database utilization from a hot 70 percent to a much cooler 30 percent.
