|
|
|
|
Send this page as an email |
|
|
|
Help us improve this content |
|
|
Level: Intermediate Brian Goetz, Chief Consultant, quiotix December 19, 2005
Although a program written in Java does not theoretically have a "memory leak", sometimes objects are not garbage collected after they are no longer part of the logic state of the program. This month, Brian Goetz, an engineer responsible for ensuring application health, discussed common causes of unintentional object retention and showed how to block leaks with weak references.
Objects that are no longer used by garbage collection (GC) recyclersLogicLife Cycle (the time when the application uses it) and the reference to the objectActualThe lifecycle must be the same. In most cases, good Software Engineering Technology guarantees that this is automatically implemented, so we don't have to worry too much about object lifecycle issues. But occasionally we create a reference, which takes a longer time to include objects in the memory than we expected. This situation is calledUnintentional object retention). Memory leakage caused by Global Map The most common cause of unintentional object retention is the useMap Associate metadata with a temporary object. Assume that an object has a medium life cycle, which is longer than the call life cycle of the method assigned to it, but shorter than the application's life cycle, such as the client's socket connection. You need to associate some metadata with this socket, such as generating the user ID for the connection. In the createSocket And cannot add dataSocket Object, because it cannot be controlledSocket Class or its subclass. In this case, the typical method isMap Store the information, suchSocketManager Class: Listing 1. Associate metadata with an object using a Global Map
public class SocketManager { private Map<Socket,User> m = new HashMap<Socket,User>(); public void setUser(Socket s, User u) { m.put(s, u); } public User getUser(Socket s) { return m.get(s); } public void removeUser(Socket s) { m.remove(s); }}SocketManager socketManager;...socketManager.setUser(socket, user);
|
The problem with this method is that the lifecycle of metadata needs to be linked to the lifecycle of the socket, but unless you know exactly when the program no longer needs this socket, rememberMap To delete the corresponding ing. Otherwise,Socket AndUser The object will always stayMap , Far exceeds the response time to the request and to close the socket. This will blockSocket AndUser Objects are garbage collected even if the application no longer uses them. These objects are left uncontrolled, which can easily cause the program to fill up the memory after a long period of operation. Except for the simplest case, find out whenSocket It is a very annoying and error-prone task that is no longer used by programs. You need to manually manage the memory.
Find out memory leakage The first sign of a program memory leak is that it throwsOutOfMemoryError Or the performance is poor due to frequent garbage collection. Fortunately, garbage collection can provide a large amount of information that can be used to diagnose memory leaks. If-verbose:gc Or-Xloggc When JVM is called, a diagnostic information is printed on the console or log file each time the GC is run, including the time it takes, the current heap usage, and the memory size it has recovered. GC usage is not recorded. Therefore, if you need to analyze memory problems or tune the Garbage Collector, it is worthwhile to enable GC logs in the production environment by default. Some tools can output GC logs and display them graphically. jtune is such a tool (see references ). Observe the heap size chart after GC to see the trend of program memory usage. For most programs, memory usage can be divided into two parts:BaselineUse andCurrent Load. For server applications, baseline usage means the memory usage when the application has no load but is ready to accept the request, current load is used in the process of processing the request, but the memory will be released after the request is processed. As long as the load is basically constant, applications usually quickly reach a stable memory usage level. If the memory usage continues to increase when the application Initialization is complete and the load is not increased, the program may retain the generated object when processing the previous request. Listing 2 shows a program with Memory leakage.MapLeaker Process tasks in the thread pool andMap To record the status of each task. Unfortunately, after the task is completed, it will not delete that item, so the state items and task objects (and their internal states) will continue to accumulate. Listing 2. Programs with map-based Memory leakage
public class MapLeaker { public ExecutorService exec = Executors.newFixedThreadPool(5); public Map<Task, TaskStatus> taskStatus = Collections.synchronizedMap(new HashMap<Task, TaskStatus>()); private Random random = new Random(); private enum TaskStatus { NOT_STARTED, STARTED, FINISHED }; private class Task implements Runnable { private int[] numbers = new int[random.nextInt(200)]; public void run() { int[] temp = new int[random.nextInt(10000)]; taskStatus.put(this, TaskStatus.STARTED); doSomeWork(); taskStatus.put(this, TaskStatus.FINISHED); } } public Task newTask() { Task t = new Task(); taskStatus.put(t, TaskStatus.NOT_STARTED); exec.execute(t); return t; }}
|
Figure 1 showMapLeaker The heap size of the application changes over time after GC. The increasing trend is the warning signal of Memory leakage. (In real applications, the slope is not that big, but after collecting GC data for a long enough time, the upward trend is usually very obvious .) Figure 1. Increasing memory usage Trend
After you are sure that there is a memory leak, the next step is to find out which object causes this problem. All memory analyzers can generate heap snapshots decomposed by object classes. There are some good commercial heap analysis tools, but finding out memory leaks doesn't have to spend money on these tools-built-inhprof The tool can also complete this work. To usehprof And keep it track of memory usage.-Xrunhprof:heap=sites Option to call JVM. Listing 3 shows the memory used by the application.hprof Output. (hprof When the application exitskill -3 Or press Ctrl + break in Windows to generate a break .) Note that compared to the two snapshots,Map.Entry ,Task Andint[] The object has increased significantly. See listing 3. Listing 4 showshprof The other part of the output isMap.Entry Call Stack information of the object's distribution point. This output tells us which call chains are generatedMap.Entry Object with some program analysis to find out the memory leak source is generally quite easy. Listing 4. hprof output, showing the allocation point of the map. Entry object
TRACE 300446:java.util.HashMap$Entry.<init>(<Unknown Source>:Unknown line)java.util.HashMap.addEntry(<Unknown Source>:Unknown line)java.util.HashMap.put(<Unknown Source>:Unknown line)java.util.Collections$SynchronizedMap.put(<Unknown Source>:Unknown line)com.quiotix.dummy.MapLeaker.newTask(MapLeaker.java:48)com.quiotix.dummy.MapLeaker.main(MapLeaker.java:64)
|
Weak reference to rescue SocketManager The problem isSocket -User The ing lifecycle should beSocket But the language does not provide any easy way to implement this rule. This makes the program have to use the old technology of manual memory management. Fortunately, since JDK 1.2, the garbage collector provides a way to declare this object's lifecycle dependency, so that the garbage collector can help us prevent this memory leakage-ExploitationWeak reference.
A weak reference is an object calledReferent. When weak references are used, the referent reference can be maintained without being blocked from being collected by garbage collection. When the Garbage Collector traces the heap, if only weak references are referenced to an object, the referent will become a candidate object for garbage collection, just as there is no remaining reference, and all the remaining weak references areClear. (Only weakly referenced objects are calledWeakly reachable).) WeakReference The referent is set during the construction. You can useget() Obtain its value. If the weak reference is cleared (whether the referent has been garbage collected or someone has calledWeakReference.clear() ),get() Will returnnull . Correspondingly, before using the results, always checkget() Whether to return a non-null value, because the referent will always be garbage collected.
When a common (strong) reference is used to copy an object reference, the lifecycle of the referent is limited to at least the same length as that of the copied reference. If you are not careful, it may be the same as the life cycle of the program-if you put an object into a global set. On the other hand, when creating a weak reference to an object, the lifecycle of the referent is not extended at all.When the object is still aliveTo maintain another method to reach it. Weak references are most useful for constructing weak sets, such as those that store metadata about these objects during the rest of the application's use of objects-this isSocketManager Class. This is the most common use of weak references,WeakHashMap It is also added to the JDK 1.2 class library. It uses weak references to the key rather than the value. If a commonHashMap Using an object as the key, the object is mapped fromMap Cannot be recycled before being deleted,WeakHashMap So that you can use an objectMap And does not prevent this object from being garbage collected. Listing 5 showsWeakHashMap Ofget() One possible implementation of the method, which shows the use of weak references: Listing 5. A possible implementation of weakreference. Get ()
public class WeakHashMap<K,V> implements Map<K,V> { private static class Entry<K,V> extends WeakReference<K> implements Map.Entry<K,V> { private V value; private final int hash; private Entry<K,V> next; ... } public V get(Object key) { int hash = getHash(key); Entry<K,V> e = getChain(hash); while (e != null) { K eKey= e.get(); if (e.hash == hash && (key == eKey || key.equals(eKey))) return e.value; e = e.next; } return null; }
|
CallWeakReference.get() It returns a strong reference to the referent (if it is still alive), so there is no need to worry about ing inwhile The loop body disappears because a strong reference prevents it from being collected by garbage collection.WeakHashMap Demonstrate a common usage of weak references-some internal object extensionsWeakReference . The reason is explained in the following section when we discuss the reference queue. ForwardWeakHashMap When adding a ing, remember that the ing may be "detached" in the future, because the key is garbage collected. In this case,get() Returnnull , Which makes the testget() Whether the returned value isnull It becomes more important than usual. Use weakhashmap to block Leakage InSocketManager To prevent leakage.WeakHashMap ReplaceHashMap As shown in Listing 6. (IfSocketManager Thread security is required, so you can useCollections.synchronizedMap() PackagingWeakHashMap ). This method can be used when the ing lifecycle must be associated with the key lifecycle. However, you should be careful not to abuse this technology. In most cases, you should still use commonHashMap AsMap . Listing 6. Use weakhashmap to fix socketmanager
public class SocketManager { private Map<Socket,User> m = new WeakHashMap<Socket,User>(); public void setUser(Socket s, User u) { m.put(s, u); } public User getUser(Socket s) { return m.get(s); }}
|
Reference queue WeakHashMap The ing key is carried with weak references, which makes it possible for applications to collect the key objects out of use,get() Implementation can be based onWeakReference.get() Return?null To distinguish between dead ing and live ing. But this only preventsMap Memory consumption increases by half of the work to be done in the life cycle of the application. You also need to do some work so that after the key object is collected fromMap . Otherwise,Map Will be filled with items corresponding to the dead key. Although this is invisible to the application, it still causes the application to run out of memory because even if the key is collected,Map.Entry And value objects are not collected.
Can be periodically scannedMap , For each weak Reference callget() , And return in get ()null Delete the ing to eliminate the dead ing. However, ifMap There are many active items, so the efficiency of this method is very low. If there is a way to send a notification when the weak referenced referent is collected by garbage collection, this isReference queue. The reference queue is the primary method for the garbage collector to return information about the object lifecycle to the application. Weak references have two constructor functions: one takes only the referent as the parameter, and the other takes the reference queue as the parameter. If weak references are created using the associated reference queue, when the referent becomes the GC candidate object, the referenced object (not the referent) will be cleared after the referenceJoinTo the reference queue. Then, the application extracts the reference from the reference queue and learns that its referent has been collected. Therefore, you can perform corresponding cleanup activities, such as removing the items of objects that are no longer in the weak set. (The reference queue providesBlockingQueue The same out-of-column modes are polled, timed blocking, and untimed blocking .) WeakHashMap There isexpungeStaleEntries() Private method, mostMap It is called in the operation. It removes all invalid references in the reference queue and deletes the associated mappings. Listing 7 showsexpungeStaleEntries() . Used to store key-value ingEntry Type extendedWeakReference Therefore, whenexpungeStaleEntries() When the next invalid weak reference is required, it getsEntry . Use reference queue instead of regular content scanning to clearMap More effective, because the cleanup process does not touch the active items, it only works when there is a reference actually added to the queue.
Listing 7. Possible implementations of weakhashmap. expungestaleentries ()
private void expungeStaleEntries() {Entry<K,V> e; while ( (e = (Entry<K,V>) queue.poll()) != null) { int hash = e.hash; Entry<K,V> prev = getChain(hash); Entry<K,V> cur = prev; while (cur != null) { Entry<K,V> next = cur.next; if (cur == e) { if (prev == e) setChain(hash, next); else prev.next = next; break; } prev = cur; cur = next; } } }
|
Conclusion Weak references and weak collections are powerful tools for managing heap, allowing applications to use more complex accessibility solutions, not just by common (strong) reference provided "either all or no" accessibility. Next month, we will analyzeSoft referenceTo analyze the behavior of the garbage collector when weak references and soft references are used.
References Learning
- For more information, see the original article on the developerworks global site.
- "Focus on performance: optimizing garbage collection": Kirk Pepperdine and Jack Shirazi demonstrate that slow memory leaks ultimately put an unbearable pressure on the garbage collector.
- "Hprof": Sun's article describes how to use the built-in hprof analysis tool.
- Reference objects and garbage collection: This article by Sun was written shortly after the reference object was added to the class library, describing how the Garbage Collector processes the reference object.
- Java Theory and Practice: All series of articles written by Brian Goetz.
- Java Technology Zone: hundreds of articles on various aspects of Java programming.
Obtain products and technologies
- Jtune: a free jtune tool that allows you to use GC logs and graphically display the heap size, GC duration, and other useful memory management data.
Discussion
- Join this forum. (You can also click the forum link at the top or bottom of the article to participate in the discussion .)
- Developerworks blogs: Join the developerworks community.
About the author
|
|
Brian Goetz has been a professional software developer for more than 18 years. He is the chief consultant of quiotix, a software development and consulting firm in Los Altos, California. He participated in several JCP expert groups. Brian'sJava concurrency in practiceThe book will be published by Addison-Wesley by the end of 2005. Please refer to Brian's and upcoming articles on popular publications in the industry. |
|