Sizeof (2) in Java)

After all these preparations, the following is the standard implementation of this graphic traversal:

public static IObjectProfileNode profile (Object obj) { final IdentityHashMap visited = new IdentityHashMap (); final ObjectProfileNode root = createProfileTree (obj, visited, CLASS_METADATA_CACHE); finishProfileTree (root); return root; } private static ObjectProfileNode createProfileTree (Object obj, IdentityHashMap visited, Map metadataMap) { final ObjectProfileNode root = new ObjectProfileNode (null, obj, null); final LinkedList queue = new LinkedList (); queue.addFirst (root); visited.put (obj, root); final ClassAccessPrivilegedAction caAction = new ClassAccessPrivilegedAction (); final FieldAccessPrivilegedAction faAction = new FieldAccessPrivilegedAction (); while (! queue.isEmpty ()) { final ObjectProfileNode node = (ObjectProfileNode) queue.removeFirst (); obj = node.m_obj; final Class objClass = obj.getClass (); if (objClass.isArray ()) { final int arrayLength = Array.getLength (obj); final Class componentType = objClass.getComponentType (); // Add shell pseudo-node: final AbstractShellProfileNode shell = new ArrayShellProfileNode (node, objClass, arrayLength); shell.m_size = sizeofArrayShell (arrayLength, componentType); node.m_shell = shell; node.addFieldRef (shell); if (! componentType.isPrimitive ()) { // Traverse each array slot: for (int i = 0; i < arrayLength; ++ i) { final Object ref = Array.get (obj, i); if (ref != null) { ObjectProfileNode child = (ObjectProfileNode) visited.get (ref); if (child != null) ++ child.m_refcount; else { child = new ObjectProfileNode (node, ref, new ArrayIndexLink (node.m_link, i)); node.addFieldRef (child); queue.addLast (child); visited.put (ref, child); } } } } } else // the object is of a non-array type { final ClassMetadata metadata = getClassMetadata (objClass, metadataMap, caAction, faAction); final Field [] fields = metadata.m_refFields; // Add shell pseudo-node: final AbstractShellProfileNode shell = new ObjectShellProfileNode (node, metadata.m_primitiveFieldCount, metadata.m_refFields.length); shell.m_size = metadata.m_shellSize; node.m_shell = shell; node.addFieldRef (shell); // Traverse all non-null ref fields: for (int f = 0, fLimit = fields.length; f < fLimit; ++ f) { final Field field = fields [f]; final Object ref; try // to get the field value: { ref = field.get (obj); } catch (Exception e) { throw new RuntimeException ("cannot get field [" + field.getName () + "] of class [" + field.getDeclaringClass ().getName () + "]: " + e.toString ()); } if (ref != null) { ObjectProfileNode child = (ObjectProfileNode) visited.get (ref); if (child != null) ++ child.m_refcount; else { child = new ObjectProfileNode (node, ref, new ClassFieldLink (field)); node.addFieldRef (child); queue.addLast (child); visited.put (ref, child); } } } } } return root; } private static void finishProfileTree (ObjectProfileNode node) { final LinkedList queue = new LinkedList (); IObjectProfileNode lastFinished = null; while (node != null) { // Note that an unfinished nonshell node has its child count // in m_size and m_children[0] is its shell node: if ((node.m_size == 1) || (lastFinished == node.m_children [1])) { node.finish (); lastFinished = node; } else { queue.addFirst (node); for (int i = 1; i < node.m_size; ++ i) { final IObjectProfileNode child = node.m_children [i]; queue.addFirst (child); } } if (queue.isEmpty ()) return; else node = (ObjectProfileNode) queue.removeFirst (); } }

This code is the distant parent of the previous Java Q & A, "Attack of the clones." using "clone through reflection. As mentioned above, it caches reflection metadata to improve performance and uses an identity hash ing to mark accessed objects. The profile () method starts from the original object graph of the Spanning Tree With iobjectprofilenode in the width-first traversal, and ends with the rapid and backward traversal of total and allocation of all node sizes. Profile () returns an iobjectprofilenode, that is, the root of the generated tree. Its size is the size of the entire graph.
Of course, output of profile () is useful only when I have a good method to extend it. For this purpose, each iobjectprofilenode must support testing performed by node visitors and node filters:

interface IObjectProfileNode{ interface INodeFilter { boolean accept (IObjectProfileNode node); } // End of nested interface interface INodeVisitor { /** * Pre-order visit. */ void previsit (IObjectProfileNode node); /** * Post-order visit. */ void postvisit (IObjectProfileNode node); } // End of nested interface boolean traverse (INodeFilter filter, INodeVisitor visitor); ... } // End of interface

Node visitors operate on the tree node only when the accompanying filter is null or the filter receives the node. For ease of use, the child node of a node is tested only when the node itself has been tested. Both forward and backward traversal are supported. The dimensions provided by the java. Lang. Object processing program and all the primary data are stored in a pseudo code. This pseudo code is attached to every "real" node of the object instance. This kind of handler node can be passed through iobjectprofilenode. shell () access, or in iobjectprofilenode. the Children () List is displayed to write data filters and visitors so that they can consider the primary data at the same starting point of the instantiated data type.
It's up to you to implement filters and visitors. As a starting point, the objectprofilefilters class (see download in this article) provides several useful stack filters, they help you cut out the large object tree based on the node size, the node size related to the parent node size, and the node size related to the root object.
The objectprofilervisitors class contains the default visitors used by iobjectprofilenode. Dump () and those who can create an XML dump for more advanced object browsing. It is also easy to convert the configuration file to swingtreemodel.
For ease of understanding, we have created a complete dump of the two string arrangement objects mentioned above:

public class Main{ public static void main (String [] args) { Object obj = new String [] {new String ("JavaWorld"), new String ("JavaWorld")}; IObjectProfileNode profile = ObjectProfiler.profile (obj); System.out.println ("obj size = " + profile.size () + " bytes"); System.out.println (profile.dump ()); } } // End of class

The result of this Code is as follows:

obj size = 106 bytes 106 -> <INPUT> : String[] 58 (54.7%) -> <INPUT>[0] : String 34 (32.1%) -> String#value : char[], refcount=2 34 (32.1%) -> <shell: char[], length=9> 24 (22.6%) -> <shell: 3 prim/1 ref fields> 24 (22.6%) -> <shell: String[], length=2> 24 (22.6%) -> <INPUT>[1] : String 24 (22.6%) -> <shell: 3 prim/1 ref fields>

As mentioned above, the internal character arrangement (accessed by Java. Lang. string # value) can be shared by two strings. Even if objectprofiler. profile () points the subordination of the arrangement to the first found string. It still notifies that the arrangement is shared (as shown in its next code refcount = 2 ).

Simple sizeof ()
Objectprofiler. Profile () creates a node image, which is generally several times the size of the original object image. If you only need the root object size, you can use the faster and more effective method objectprofiler. sizeof (), which can be achieved through non-recursive depth-first traversal.

More examples
We apply the profile () and sizeof () functions to a pair of examples.
Javastring is the notorious storage waste collector because it is too common and the efficiency of common strings is quite low. I believe you understand that a common String concatenation operator usually produces compact strings. The following code:
String OBJ = "Java" + new string ("world ");
Generate the following configuration file:

obj size = 80 bytes 80 -> <INPUT> : String 56 (70%) -> String#value : char[] 56 (70%) -> <shell: char[], length=20> 24 (30%) -> <shell: 3 prim/1 ref fields>

The value character arrangement has 20 char characters, although it only needs 9. Compare it with the results of "Java". Concat ("world") or string OBJ = new string ("Java" + new string ("world:

OBJ = new string ("Java" + new string ("world"): OBJ size = 58 Bytes 58-> <input>: String 34 (58.6%) -> string # value: Char [] 34 (58.6%)-> <shell: Char [], length = 9> 24 (41.4%)-> <shell: 3 prim/1 ref fields>

Obviously, if you are allocated through a series operator or stringbuffer. the string attribute constructed by the tostring () function is very relevant to many objects. If you use the Concat () or string replication constructor, you can improve memory consumption.
To further discuss this issue, I provide a slightly esoteric example. The following visitor/Filter checks the object and reports all the Compact strings in it:

class StringInspector implements IObjectProfileNode.INodeFilter, IObjectProfileNode.INodeVisitor { public boolean accept (IObjectProfileNode node) { m_node = null; final Object obj = node.object (); if ((obj != null) && (node.parent () != null)) { final Object parentobj = node.parent ().object (); if ((obj.getClass () == char [].class) && (parentobj.getClass () == String.class)) { int wasted = ((char []) obj).length - ((String) parentobj).length (); if (wasted > 0) { m_node = node.parent (); m_wasted += m_nodeWasted = wasted; } } } return true; } public void previsit (IObjectProfileNode node) { if (m_node != null) System.out.println (ObjectProfiler.pathName (m_node.path ()) + ": " + m_nodeWasted + " bytes wasted"); } public void postvisit (IObjectProfileNode node) { // Do nothing } int wasted () { return 2 * m_wasted; } private IObjectProfileNode m_node; private int m_nodeWasted, m_wasted; }; // End of local class IObjectProfileNode profile = ObjectProfiler.profile (obj); StringInspector si = new StringInspector (); profile.traverse (si, si); System.out.println ("wasted " + si.wasted () + " bytes (out of " + profile.size () + ")");

To use sizeof (), let's take a look at our list () vs arraylist (). This code breeds a list of 1000 null references:

List obj = new LinkedList (); // or ArrayList for (int i = 0; i < 1000; ++ i) obj.add (null); IObjectProfileNode profile = ObjectProfiler.profile (obj); System.out.println ("obj size = " + profile.size () + " bytes");

The size of the generated structure is the total storage implemented by the list. For collections list and arraylist, sizeof () Reports 20,040 bytes and 4,112 bytes respectively. Even if arraylist increases its internal capacity before its size (in this case, it will lose almost 50% of its capacity at any time; this is done to reimburse the cost of inserting constants in installments ), its memory efficiency based on the arrangement design is much higher than the implementation of the two-Link List of sorted list, in this list implementation, a 20-byte node is created to store each value (this does not mean that you should not use the limit list: they guarantee the performance of the outstanding constant insertion, in other things .)

Restrictions
The objectprofiler method is not perfect. In addition to the problem of ignoring storage queues, another serious problem is that Java objects can share non-static data. For example, when the real domain points to global singleton and other shared content, these contents can be shared.
Take decimalformat. getpercentinstance () as an example. Although he returns a new numberformat each time, all these numberformats usually share locale. getdefault () Singleton. Therefore, even if sizeof (decimalformat. getpercentinstance () Reports 1,111 bytes each time, it is estimated that it is too high. This is actually just a manifestation of another conceptual difficulty in defining the Java object's dimensional measurement process. In this case, objectprofiler. sizedelta (Object base, object OBJ) is easy to obtain: This method traverses the object graphics rooted in the base, and then uses the accessed object configuration OBJ during the first traversal. Therefore, the results can be effectively calculated as the total size of data that does not seem to belong to the base obj. In other words, the amount of memory required to instantiate a given obj is equal to the amount of existing memory of the base (the shared object has been effectively deleted ).
Sizedelta (decimalformat. getpercentinstance (), decimalformat. getpercentinstance () Report: Each subsequence format needs 741 bytes to be instantiated. Compared with the sizeof class of Java tip 130, the value of 752 bytes is more accurate, there was a minority-byte deviation, but it was much better than the original sizeof () estimation.
Another type of data that objectprofiler cannot see is local storage allocation. Java. NIO. bytebuffer. the result of allocate (1000) is the structure of the 1050 bytes allocated by the JVM heap, but bytebuffer. allocatedirect (1000) seems to have only 140 bytes; this is because the real storage is allocated in local storage. In this case, you need to discard pure Java and convert it to a Analyzer Based on the JVM analyzer interface (jvmpi.
Another rather vague example of the same problem is measurement throwable. objectprofiler. in the sizeof (New throwable () example, only 20 bytes are reported, which is quite different from the 130 bytes reported by the sizeof class of Java tip 272. The reason is that there is a hidden domain in throwable:
Private transient object backtrace;
JVM uses a special method to process this hidden domain: it is not displayed in the reflection call, even if its definition is seen in the JDK source file. Obviously, JVM uses this property of an object to store some 250 bytes of local data that supports stack backtracking.
Finally, if Java is used in the analysis object process. lang. ref. * The results produced by references are confusing (for example, the results may change between sizeof () copies of the same object ). This is because the weak reference creates redundant parallelism in the application, and the absolute fact of traversing the image may change the reachable state of the weak reference. In addition, the internal structure of Java. Lang. Ref. Reference is openly entered. objectprofiler's behavior is not what Java code should do. Enhance the traversal code to avoid all non-strongly referenced objects (it is not very sure whether the data attribute to the root object is in the first position), which may be the best choice.

Summary
This article discusses how to build a pure Java object analyzer. My experience is that using simple methods such as objectprofiler. Profile () to analyze large data structures can easily save tens or even hundreds of percent of memory consumption. This method is complementary to the commercial analyzer. The commercial analyzer is also designed to demonstrate the very shallow (not graphics-based) view that occurs inside the JVM heap. If there is nothing else, it is also helpful to look at the interior of the object graph.

About the author

Vladimir roubtsov has 14 years of experience programming in a variety of languages since 1995, including Java. Currently, as a senior engineer, he develops enterprise software for Austin's trilogy in Texas.