Automatically detect memory leaks on iOS

The memory of the mobile device is a shared resource. Applications may run out of memory, crash, or experience significant performance degradation.

The Facebook iOS client has many features, and they share the same block of memory space. If any particular feature consumes too much memory, it affects the entire application. This is possible, for example, when this feature causes a memory leak.

When we have allocated a piece of memory and set the object, if we forget to release it after we have finished using it, a memory leak will occur. This means that the system cannot reclaim memory and make it available to others, which in the end means that our memory will gradually run out.

On Facebook, we have many engineers working on different parts of the code base. This inevitably occurs with memory leaks. When memory leaks occur, we need to find and fix them as soon as possible.

Some tools can already find memory leaks, but they require a lot of manual intervention:

1. Open Xcode to compile for performance analysis (profiling).
2. Load instruments.
3. Use the application to try to reproduce as many scenes and behaviors as possible.
4. View memory and leaks.
5. Track the root cause of memory leaks.
6. Fix the problem.

This means that you need to repeat a lot of manual actions every time. For this reason, we may not be able to locate and fix memory leaks as early as possible during our development cycle.

Automation can find memory leaks faster without the need for more developers. To solve this problem, we have made a set of tools to automate the processing and repair of some of the problems in our code base. Today, we are excited to release these tools: Fbretaincycledetector, Fballocationtracker, Fbmemoryprofiler.

Circular references (Retain cycles)

Objective-c uses reference counting to manage memory and free objects that are not in use. Any object in memory can hold ( retain ) other objects, and as long as the previous object needs it, the object remains in memory. One way to see this is for this object to hold other objects.

This works well for most of the time, but when two objects are held with each other, this is an impasse. Directly, or more often, they are connected by indirect objects. This ring of holding references is called Circular reference (Retain cycles).

Circular references cause problems with some columns. In the best case, the object will only occupy a little bit of memory. If the compromised object is actively doing something extraordinary, the rest of the application will have less memory. In the worst case scenario, the application crashes if the leak results in the use of capacity beyond the available memory.

During manual profiling, we found that we tended to have some circular references. We can easily cause memory leaks, but it's hard to find them. Circular reference detectors make it easy to find them.

Detecting circular references at run time

Finding circular references in objective-c is similar to finding loops in a directed acyclic graph, where nodes are objects and edges are references between objects (if object A holds object B, there is a reference between A and B). Our Objective-c object is already in our diagram, and all we have to do is traverse it with a depth-first search.

Http://7i7i81.com1.z0.glb.clouddn.com/blogvideo_memoryleak_1.mp4

It's a little abstract, but it works fine. We have to make sure that we can use objects like nodes, and for each object we can get all the objects it references. These references may weak or may be strong . Only strong references can cause circular references. For each object, we need to know how to find these references.

Fortunately, Objective-c provides a powerful, introspective runtime library. This allows us to have enough data to dig in the graph.

The node in the diagram can be either an object or a block. Let's discuss them separately.

Object

The runtime has many tools that allow us to introspect objects.

The first thing we want to do is get the object's instance variable's layout (Ivar layout).

const char *class_getIvarLayout(Class cls);const char *class_getWeakIvarLayout(Class cls);

For an object, the layout of the instance variable describes where we can find references to other objects. It gives us an index, which means we need to add an offset to the object address to get the address of the object it refers to. The runtime also allows us to get the weak reference instance variable layout (weak Ivar layout).

This also partially supports objective-c++. In objective-c++, we can define objects in structs, but this is not obtained in the instance variable layout. The runtime provides "type encoding" to handle this problem. For each instance variable, the type encoding describes how the variable is structured. If this is a struct, it describes which fields and types it contains. We calculate their offsets, and in the graphs, find the objects they point to.

There are some fringe conditions we won't go into. Most of them are different sets, and we have to enumerate them to get the objects they hold, which can lead to some side effects.

Block

Block is a little different from the object. The runtime doesn't make it easy to see their layout, but we can still guess.

As we work on block, we can use Mike Ash's ideas in his project circle (the first time to inspire Fbretaincycledetector).

We can use the ABI (Application binary interface for blocks-application binary block interface). It describes how the block looks in memory. If we know that the reference we're dealing with is a block, we can throw it in a fake struct to mimic block. After placing the struct in a C language, we can know the object that block holds. Unfortunately, we do not know whether these references are strong or weak references.

To solve this problem, we used a black box technology. We create an object to impersonate the block we want to investigate. Because we know the interface of the block, we know where to find the reference that the block holds. The objects we forge will have "release detectors" in place of these references. Release detectors are small objects that observe the release messages sent to them. When the holder wants to discard its hold, the message is sent to the strong reference object. When we release our bogus objects, we can detect which detectors have received these messages. As long as we know which indexes are in the detectors of the forged objects, we can find the objects actually held in the original block.

Automation

What makes this tool really flash is that it will run continuously and automatically when built within the engineer.

Partial automation of the client is simple. We run a cyclic reference detector on the timer and periodically scan the memory to look for circular references, although this is not entirely a problem. When we first ran the parser, we realized that it wasn't enough to scan the entire memory space very quickly. When it starts to detect, we need to provide it with a set of candidate objects.

In order to solve this problem more effectively, we have developed fballocationtracker. This tool proactively tracks NSObject the creation and deallocation of subclasses. It can fetch any instance of any class with a very small performance overhead.

For the client automation, as long as the NSTimer use of fbretaincycledetector, and then use Fballocationtracker to crawl the instance to match the tracking line.

Now, let's take a closer look at what happens backstage.

A circular reference can contain any number of objects. A bad connection can cause a lot of loops, which is complicated.

In the ring, A→b is a bad connection, creating two rings: a-b-c-d and A-B-C-E.

Here are two questions:

1. We do not want to give a bad connection to the two circular references marked separately.
2. We don't want to tag together two circular references that might represent two questions, even if they share a connection.

So we need to define the cluster group (clusters) for circular references, and given these heuristics, we've written an algorithm to find these problems.

1. Collect all the rings at a given time.
2. For each ring, extract the Facebook-specific class name.
3. For each ring, find the smallest reported ring contained within the ring.
4. The ring is added to the group according to the minimum loop above.
5. Only the minimum ring is reported.

The last part is to find out who has accidentally introduced a circular reference in the first time. We can guess the problems caused by the recent changes through some code of the "GIT/HG responsibility" in the ring. The last person to contact this code will receive the task of repairing the code.

The entire system is as follows:

Manual performance Analysis

While automation helps simplify the process of discovering circular references and reducing the cost of people, manual performance analysis still has its niche. Another tool we created allows anyone to view memory usage, and doesn't even need to plug his phone into a computer.

Fbmemoryprofiler can be easily added to any application, allowing you to manually configure the build file to allow you to run Loop app detection within your application. It uses Fballocationtracker and fbretaincycledetector to implement this functionality.

Http://7i7i81.com1.z0.glb.clouddn.com/blogvideo_memoryleak_2.mp4

Build (Generations)

A great feature of Fbmemoryprofiler is "generate Trace (generation tracking)", similar to Apple's instruments generation tracking. The build is simply a snapshot of all the still alive objects between the two-time markers.

Using the Fbmemoryprofiler interface, we can tag a build, for example, to assign three objects. Then we mark another build, and then we continue to assign the object. The first build contains three objects that we started with. If any of the objects are freed, it will be removed from our second build.

When we have a repetitive task that we think may be memory leaks, it is useful to generate traces, for example, navigating the view controller's entry and exit. Each time we start our task, we mark a build, and then we investigate each subsequent build. If an object should not live for such a long time, we can clearly see it in the Fbmemoryprofiler interface.

Check out

Whether your application is large or small, features are more or less, good engineers should have good memory management. With the help of these tools, we can find and fix these memory leaks more simply, so we can take less time to handle them manually, so we can have more time to write better code. We also hope that you can find that they are useful. Check out on GitHub. Fbretaincycledetector, Fballocationtracker and Fbmemoryprofiler.

Note

You can refer to the usage on Fbmemoryprofiler, or refer to one of my other blogs: fbmemoryprofiler basic tutorial.

Automatically detect memory leaks on iOS