Java's new Considered Harmful

Point p = new Point(x, y);</p>

Memory is allocated on the heap for the new object, its instance variables are initialized, the constructor body is executed, and the newly constructed object is returned.

There are several nice things about this mechanism. The syntax is clear and comprehensible. The type of the returned object is apparent. Since clients are forced to call a constructor if they want to create an object, the class writer can enforce correct initialization: From the moment clients first have access to the object, it is in a consistent state. And constructors chain by having a subclass constructor call a superclass constructor, thereby forcing correct initialization of superclasses.

But there are two problems with new:

• It allocates memory from the heap.
• It is not polymorphic.

For each of these, I'll discuss the problem first, then offer solutions.

To be clear, it really is new that I'm railing against, not constructors per se. Despite their name, constructors just initialize objects; they don't allocate them. (It would be interesting to have a language that decoupled initialization from object creation, allowing you to reinitialize existing objects. This feature would be useful in conjunction with object pools.) There are other ways to call constructors that don't have (both) these problems: Calling a constructor via super() does not allocate memory, and calling a constructor with reflection is polymorphic.

Memory Allocation: The Problem

Because memory allocation is new's primary function, it is hard to see how allocating memory from the heap could be a problem. But sometimes you want to allocate memory from somewhere other than the heap, and at other times you don't want to allocate memory at all. In other words, you may want an object, but you may not want to create one, at least not here, not now.

The best example is an instance-controlled class, where the class writer wishes to restrict the number of instances. When the maximum number of instances is one, you have the Singleton pattern (described in Design Patterns by Erich Gamma et al., Addison-Wesley, 1995). Singleton is often used when the object corresponds to a unique item in the real world (such as the keyboard or computer itself), or when the object is the sole manager for some resource. Java's Runtime class is an example of the latter: Its single instance manages interactions between the Java virtual machine and native operating system.

Sometimes only a few instances of a class are desired. Imagine a class representing ASCII characters. There are only 256 of those. If the class is immutable and doesn't carry any other information, only 256 instances of the class need ever exist--more would just clutter memory. For a more dramatic example, consider the Boolean class: Only two instances are ever necessary, one for True and one for False. In addition to saving memory, having only one instance per value also allows testing for equality using the "==" operator, instead of the slower "equals" method.

Both of these are examples of the object-oriented enumerated type idiom, in which a fixed set of values are represented by distinct objects, much as enumerated types in languages such as Pascal and C represent a set of values by distinct integers. This idiom is closely related to the Flyweight pattern (again, see Design Patterns). The class representing the enumerated type should probably be immutable. Otherwise, there is a good chance that data modification by one client will inadvertently affect another because they share pointers to the same object.

So far, I've limited the number of instances because it is logically unnecessary (and perhaps harmful to performance) to have more. Sometimes there is no limit on the number of instances, but the class writer would like the option of handing back an existing instance on occasion.

Swing's border mechanism uses this idea. Each border is an object, but sometimes the same object can be reused for many borders. In particular, the same two instances of BevelBorder are returned whenever users request a raised or lowered bevel.

For a more sophisticated and (currently) hypothetical example, consider the Integer class. There are many possible instances of this class — too many to cache them all — but we might conjecture that some values are more common than others. Probably Integer objects representing the numbers from -2 to 100 are more likely to occur than others. A clever implementation could cache objects for these numbers, saving the time and storage involved in their creation.

This caching scheme absolutely requires immutability, so that the actual identity of an object never matters, only its value. If cached Integers were mutable, someone could change the cached instance representing 1 to have the value 2, which would be mighty confusing.

Object pools are another case in which heap allocation is to be avoided. An object pool is a way to avoid garbage collection overhead. The garbage collector usually does a pretty good job of reclaiming storage, but in rare cases it may be more efficient to allocate/deallocate objects manually. Initially, allocation is done normally with new. But clients manually deallocate objects by calling a method that places the object carcass in a pool, from where it can be reallocated when needed. Of course, it should be transparent to clients whether a requested object is coming from the pool or the Java heap. If the same few objects are allocated/deallocated repeatedly, an object pool can turn a churning garbage collector into a quiet one (see my article "Use Object Pools to Sidestep Garbage Collection," Java Report, September 2000).

While the caches I have been discussing all cut down on memory usage, sometimes the intent is to avoid other kinds of overhead. For instance, object data may be stored in files or a database, and frequently accessed objects may be cached for performance.

The general idea in all these examples is that class writers might wish to cache certain instances to avoid repeated object creation or other overhead. Such caching should be transparent to clients. But it cannot be transparent if clients use new.

In other situations, the idea is not to avoid creating an object, but to create it somewhere other than the heap. For instance, in a distributed system, you may want to instantiate an object on another machine. The real-time specification for Java (see The Real-Time Specification for Java, Greg Bollella et al., Addison-Wesley 2000, and http://www.jcp.org/aboutJava/communityprocess/first/jsr001/rtj.pdf) posits memory regions other than the garbage-collected heap, so that garbage collection does not break real-time guarantees. In such cases, you may not wish to make the origin of the created object transparent to the client, but you still cannot use new, since it always allocates memory on the current machine's garbage-collected heap. (Actually, real-time Java gets around this problem by redefining new to mean "allocate from the current memory region," but redefining new is not a solution available to most of us.)

Of course, you could argue as follows: "You're blaming new for doing what it's supposed to do—allocate memory. Obviously, if you don't want to allocate memory, don't use new. So what's your point?"

My point is that object creation is an implementation detail. Often, I would simply like an Integer object whose value is 3, and I don't much care how I get it. For some classes, to be sure, I do care: I don't want to share someone else's ArrayList in some other address space — I want my own fresh, local one. But often — and always with immutable objects — I don't care. Yet the literature spends a lot of time talking about hiding data and methods, and virtually no time talking about hiding creation. For some classes, hiding object creation can be just as important as hiding object data.

Memory Allocation: The Solutions

The first step in solving any of these memory allocation issues is to declare all constructors private, or at least protected if you wish to support subclasses. Making the constructors private is safer — you can be sure that no clients of your class can use new to instantiate it — but also makes it impossible to declare subclasses. There is a tradeoff between allowing inheritance and encapsulating creation.

To provide a few chosen instances to clients, the simplest thing to do is to put them in public final static variables. For example, Boolean.TRUE and Boolean.FALSE are the only instances of Boolean that need ever exist. (However, the Boolean constructors are public, so clients can create additional instances.)

The problem with using variables is that they expose your policy on object creation just as surely as providing a constructor does, only in the other direction — no new creation, instead of unrestricted creation. This is not an issue if you're sure about what you're doing. If you're implementing an enumerated type and you know you want to limit the possible values to what you've written in your class, then static variables and a private constructor are appropriate. (This could, and perhaps should, have been Boolean's design.) However, if you use a static variable for an object that you intend to be unique (a Singleton), but later decide to allow one instance per thread, you are in trouble: ThreadLocal variables require a method call to retrieve the value, so you'll have to change your static variable to a static method and break clients.

Using a static method in the first place solves the problem with variables, at the cost of some time and a more cumbersome notation. A static method that creates (or may create) objects is known as a static factory method, and is the most common alternative to constructors. Static factory methods are used throughout the Java libraries. For example, Runtime's static getRuntime method returns the virtual machine's Singleton Runtime object. The java.text formatting classes use them to hide the mapping from locales to formatting objects, and in the java.security package they typically map strings naming algorithms to objects that implement them. Most of the wrapper classes have valueOf methods that take a String and return a corresponding instance, though only Boolean's methods avoid object creation by returning one of the constants Boolean.TRUE or Boolean.FALSE. In fact, static factory methods are common enough to have their own naming conventions: Most static factory methods are called getInstance or valueOf.

There are three advantages of static factory methods.

• They hide the details of object creation, just as accessor methods hide the details of data representation. The class writer has complete flexibility about how and when to create objects.
• They enable changes in the creation policy without breaking clients. For example, in Java 1.3, most java.security static factory methods create a new object each time they are called, but because they are methods and not constructors, their implementors can change that policy at any time without breaking any existing code.

• They allow polymorphism.

One problem with static factory methods is that they are inherited. If Point2D defines a static factory method, say valueOf, that returns instances of Point2D, and Point3D inherits from Point2D and neglects to define its own static valueOf method, then calls to Point3D.valueOf will be legal and will invoke Point2D's valueOf method. In all likelihood, this isn't what was intended.

Polymorphism: The Problem

Turning to the second problem: new is famously nonpolymorphic. When I write new Rectangle(), I always get a Rectangle, never anything else, and in particular never a subclass of Rectangle. Much of the time this is what I want, but sometimes it isn't; for example:

• A framework where I request an implementation by name or other attribute, and am given an instance of the appropriate class. Some of the classes in the java.security package, like Signature, work this way. Or imagine a data structure wizard that lets a client specify the desired properties of a data structure (for example, constant average-time access, no duplicates, ordering not required) and returns an implementation satisfying the criteria (HashSet, for instance).
• A drawing program in which users click on a button representing a shape, then drag the mouse across the drawing window. The program draws the shape and creates an instance of the shape class internally. To avoid a clumsy if-then-else in the code, you need a way to associate a button with a mechanism for object creation.

• A platform-independent user-interface framework in which each component has a corresponding native "peer." This is the design of Java's AWT. When you create an AWT Button, the appropriate platform-dependent peer must be created as well.

• A general-purpose multithreaded network server class that creates a connection object for each incoming client. Each connection object runs in a separate thread and handles the interaction with one client. This is a common design for multithreaded servers in Java. To be reusable, the server class must create connections polymorphically.

In all of these cases, the best design uses polymorphism to create objects. The same line of code should be able to return instances of different classes each time it is executed. This rules out using a constructor with new.

Polymorphism: The Solutions

Static factory methods can provide a solution to some — but not all — polymorphism problems. Only the first of the aforementioned examples succumbs to a static factory method. A class that wishes to select from several implementations (typically subclasses) can provide a static method returning the appropriate subclass. For example, Signature and several other classes in java.security have a getInstance method that takes a String naming an algorithm and returns a subclass mapped to the name. Another example is the method Toolkit.getDefaultToolkit in the java.awt package. It takes no arguments but uses a system property to determine which subclass of Toolkit to return.

The problem with static factory methods is that they are static; strictly speaking, a call to one is not polymorphic. That is, when you call Signature.getInstance you are always invoking the same code. Although its strategy for returning an appropriate subclass may be clever and complex, there is no way to use a different strategy.

The obvious antidote — indeed, the only other possibility — is a nonstatic factory method. In other words, you call a method on an object, and you get back an object (possibly a new one). Since instance (that is, nonstatic) method calls can be polymorphic, creation can be as well. There are a variety of ways in which this basic mechanism can be exploited.

In the simplest idea, called the "Factory Method" pattern (see Design Patterns), a factory object is responsible for creation. The factory class is abstract (or is an interface) and declares a method for creating an object. Subclasses implement these methods to create particular classes of objects; in these implementations, new is usually used, although of course any of the memory allocation mechanisms just described can be used as well.

The Java libraries are full of factories. RMI programmers may be familiar with socket factories used to substitute the programmer's sockets for RMI's defaults. RMIClientSocketFactory, in the package java.rmi.server, is an interface with a single method:

Socket createSocket(String host, int port)</p>

You pass an instance implementing the interface to the RMI system, which invokes the createSocket method to obtain a Socket. RMIServerSocketFactory works the same way.

Here, the arguments to the factory method mimic those of the constructor for the class being created, but you could use factories to implement any mapping from arguments to class. The java.net package uses objects called "content handlers" to interpret the content of URLs. The ContentHandlerFactory interface has a single method that takes a String representing a MIME type and returns an instance of ContentHandler. You could implement the factory as a map from MIME types to content handlers if the handlers are immutable.

Although factory methods are usually used to achieve polymorphism, they can also be used to overcome the first problem we discussed, that of memory allocation. Recall that object creation in a distributed system can't use new because it creates objects locally. Enterprise JavaBeans handles this problem by defining a separate "home" interface for each bean. Methods on the home interface are used to create instances of the bean. The home interface acts in part as a factory.

The multithreaded server class could use a factory object that creates an instance of a connection thread each time its method is called. The details resemble those of the RMI socket factories. The server designer would provide an interface:

interface ConnectionFactory {<br>
    Connection createConnection(Socket s);<br>
}</p>

If you wish to use the server code, you would write a class implementing ConnectionFactory and pass it on to the server. The server would then use the factory object to create a new Connection:

Socket s = serverSocket.accept();<br>
Connection c = connectionFactory.createConnection(s);</p>

Although a straightforward factory method technique could be used for the drawing problem, Java's inner classes make it easy to combine the ActionListener interface with object creation. In this piece of code for creating rectangles, the variable drawing is the current drawing, and the Drawing class has an add method that adds a new shape to the drawing.

rectangleButton.addActionListener(new ActionListener() {<br>
       public void actionPerformed(ActionEvent e) {<br>
            drawing.add(new Rectangle());<br>
       }});</p>

The Abstract Factory design pattern takes this a step further: A single class provides methods for creating more than one kind of object. Abstract Factory is ideal for the UI framework situation, and indeed both AWT and Swing use it. The java.awt.Toolkit class is an abstract class with methods createButton, createCheckbox, and so on, that return peer implementations of the various AWT components. The limitation of having a fixed set of methods, and thus components, is overcome in Swing by using a mapping from a generic UI component name (such as "ButtonUI") to the name of an implementing class.

A powerful variant of the factory approach uses an object representing a class to create instances of that class. Java programmers can do this with reflection, using the Class class. Class objects provide a newInstance method for calling a no-argument constructor, as well as Constructor objects that can invoke any constructor. A multithreaded server class could accept a Class instance to use for instantiating connections. In its simplest form, it could use the newInstance method:

Socket s = serverSocket.accept();<br>
Connection c = (Connection) connectionClass.newInstance();</p>

There would have to be some other way to give the Socket to the Connection. Alternatively, a Constructor object for a constructor with a Socket argument could be obtained from the Class object and then invoked.

The required downcast is one of the unattractive things about class-based creations in Java; the other is the variety of exceptions that must be handled, all due to the dynamic nature of this mechanism. For example, an IllegalAccessException is thrown if the class or constructor is not public. With a normal call to the constructor using new, this problem would be detected at compile time.

A variant of the class-based approach appears in real-time Java. Again, the real-time Java spec provides for the existence of memory areas other than the garbage-collected heap. These memory areas are actually objects—instances of class MemoryArea or a subclass — which have two methods, newArray and newInstance — that take a Class object as a parameter and create an instance of that class in the appropriate memory area. You can invoke these methods directly on your MemoryArea of choice, or as mentioned, use new with its altered meaning of creating an object in the currently active MemoryArea.

In Smalltalk, where dynamic typing makes downcasting unnecessary and exceptions are not treated as rigorously as Java, class-based creation is convenient, elegant, and polymorphic (see SmallTalk-80: The Language and Its Implementation, by Adele Goldberg and David Robson, Addison-Wesley, 1983). In fact, it is the usual way to create objects in that language.

Another way to create objects polymorphically is to copy an existing object. This is the Prototype pattern of Design Patterns. In Java, the clone method provides for polymorphic object copying.

You don't see the Prototype pattern much in Java. It is used only once in the J2SE 1.3 libraries, Swing's JEditorPane class (and even there it is employed only as an adjunct to class-based creation with Class.newInstance). A good example of prototypes is a drawing program that provides default properties for each shape, so that when users place a rectangle, it is initially blue with a red border. Users could configure the default properties by manipulating a shape to their liking, then choosing "Make this the default." The easiest way to program that action would be to make a copy of the selected shape, to use as the prototype for future shapes.

Prototypes are also useful when object initialization is expensive, and you anticipate few variations on the initialization parameters. Then you could keep already-initialized objects in a table, and clone an existing object instead of expensively creating a new one from scratch. The objects in the table play the role of prototypes. This is essentially the caching idea described earlier, modified to deal with mutable objects — if the objects were immutable, you could return them directly instead of cloning them.

Cloning is obviously polymorphic since it involves calling a nonstatic method, but it would seem to suffer from the problem that it always allocates memory. Not so. For instance, it is perfectly allowable for an immutable class to have return this as the body of its clone method. The author of Object.clone's contract no doubt had this and similar cases in mind when writing that, while "the general intent" is that x.clone() != x, this is not an "absolute requirement."

Prototype is unique among the object creation techniques in that it doesn't require a class, only another object. Objected-oriented languages such as Self and Omega that do away with classes completely rely on prototypes for creating new objects (see "Self: The Power of Simplicity," by David Ungar and Randall B. Smith, OOPSLA '87 Conference Proceedings, 1987; and Object-Oriented Programming with Prototypes, by Gunther Blaschek, Springer-Verlag, 1994, respectively). If you believe in classes — that is, you believe that it is conceptually clearer to distinguish between classes and objects — then prototypes are a bit unpalatable because they blur that distinction. For instance, consider the server framework previously described, which creates a new connection object for each client. This could be implemented by passing in a connection object as a prototype, to be cloned when a client arrives. But the connection class was written with the intent that each connection object talks to a client. Perhaps the constructor takes a Socket as a parameter, for example. It seems odd to coopt one instance of this class for use purely as a breeder of other connections. The prototype obeys a different set of class invariants than the other instances — it does not require a Socket, for example.

A further problem with using prototypes in Java is the clumsiness of the clone method. The return type is Object, requiring a downcast, and you may need to deal with CloneNotSupportedException. Relief will soon arrive for the first of these problems: Java will have covariant return types, meaning, for example, that Point's clone method can have a return type of Point, even though Object's has a return type of Object. Calls to clone on a Point variable will not require a downcast:

Point p1 = new Point(2, 4);<br>
Point p2 = p1.clone();</p>

Beyond Java

How do other programming languages handle object creation? C++ is just like Java, as you would expect. Eiffel lets class writers designate one or more methods of the class as creation methods; these are called with a special syntax, functionally equivalent to new, and are essentially constructors, except that they do not chain. The problems with new discussed here apply to both C++ and Eiffel.

Smalltalk uses the class-based approach to creation. There is an object corresponding to each class, and instances are created by sending a new message to the class object. This is elegant and polymorphic. Indeed, it is common in Smalltalk programs to pass around a class object when polymorphic creation is required. But Smalltalk's mechanism suffers from a serious drawback: Initialization is not enforced. new creates objects but does not initialize them. The convention is to have an initialize method for each class that is called after new, but you can forget to call initialize, leaving the object in an invalid state. Why not combine creation and initialization by calling initialize from within new? Because new is polymorphic, so it must always take the same number of arguments (namely zero), whereas each class may require a different number of arguments for initialization. Ruby (see Programming In Ruby, by David Thomas and Andy Hunt, DDJ, January 2001) is able to fix this problem because it lets a method be "variadic" — to take any number of arguments. In Ruby, new does call initialize, passing along all its arguments. But both Smalltalk and Ruby are dynamically typed. It doesn't seem possible to have a general mechanism for polymorphic creation and initialization that is statically typed, because polymorphism requires that the creation method have the same argument types, but each class has its own requirements for initialization.

A second drawback with both Smalltalk and Ruby is that initialize, being an ordinary method, does not chain: You must remember to begin your initialize methods with a call to the superclass's initialize method.

For statically typed languages, it may not be possible to do better than the approach taken by Curl (see "The Curl Programming Environment," by Friedger Müffke, DDJ, September 2001). In addition to constructors, whose semantics are close to those of Java, Curl also has factories. A factory is essentially a static factory method, but with a calling syntax identical to that for a constructor. Thus the class writer can choose to provide constructors, factories, or both, and clients cannot tell the difference. This successfully encapsulates object creation.

Conclusion

There is a difference between requesting an object and creating one. The first is an abstraction; it should be designed into a class and should be under the class's control. The second is a low-level implementation detail. Java's new keyword, the standard way to obtain objects, provides only the latter. Thus, most requests for objects end up being nonpolymorphic heap allocations, whether this is a good idea or not. In short, new should be considered harmful for the same reason that goto is considered harmful — although it is an indispensable low-level tool, it must be used with care or hidden behind abstractions.

DDJ