Facebook launches Android image loading library-fresco

Reproduced

• Original link: Introducing fresco:a new Image Library for Android
• Author: Tyrone Nicholas
• Translator: Zhaokaiqiang
• Reviewer: chaossss
• Reviewer: bboyfeiyu
• Reviewer: billionwang
• Status: Complete

It is extremely important to display images quickly and efficiently on Android devices. Over the past few years, we have encountered a lot of problems in how to efficiently store images. The picture is too big, but the memory of the phone is very small. The R, G, B, and alpha channels of each pixel occupy a total of 4byte of space. If the phone's screen is 480*800, then a screen-sized picture will occupy 1.5M of memory. The phone's memory is usually small, especially if the Android device allocates memory for each application. On some devices, only 16MB of memory is allocated to the Facebook app. A picture will occupy One-tenth of its memory.

What happens when your app's memory overflows? Of course it will crash! We have developed a library to solve this problem, we call it fresco. It can manage the images and memory used, and the app no longer crashes.

Memory Area

To understand exactly what Facebook is doing, we need to understand the difference between the heap memory that Android can use. The Java heap memory size of each app in Android is strictly limited. Each object is instantiated in heap memory using Java new, which is a relatively secure area in memory. Memory has a garbage collection mechanism, so when the app is not using memory, the system automatically reclaims the memory.

Unfortunately, the process of memory garbage collection is the problem. When memory is garbage collected, the memory is not just garbage collected, but the Android app is completely terminated. This is one of the most common reasons users use the App when they are stuck or briefly suspended. This makes it frustrating for users who are using the app, and then they may be impatient to swipe the screen or click the button, but the app's only response is to ask the user to wait patiently before the app returns to normal

In contrast, the native heap is assigned by the new C + + program. There is more memory available in the native heap, the app is limited only by the physical available memory of the device, and there is no garbage collection mechanism or other thing to drag. But C + + programmers must reclaim every chunk of memory they allocate, or they will cause a memory leak that eventually causes the program to crash.

Android has another area of memory called Ashmem. It operates more like a native heap, but there are additional system calls. When Android operates the Ashmem heap, it extracts the memory area of the heap that contains the data from the Ashmem heap, rather than releasing it, which is a weak memory release mode, and the part of the memory that is extracted is released only when the system really needs more memory (system memory is not enough). When Android puts this part of the memory that is being extracted back into the Ashmem heap, the previous data is restored to the appropriate location as long as the extracted memory space is not released.

Bitmap that can be eliminated

Ashmem cannot be processed directly by Java applications, but there are some exceptions, and pictures are one of them. When you create a bitmap that is not compressed, the Android API allows you to specify whether it can be cleared.

BitmapFactory.Options = new BitmapFactory.Options();options.inPurgeable = true;Bitmap bitmap = BitmapFactory.decodeByteArray(jpeg, 0, jpeg.length, options);

After the above code is processed, the bitmap that can be purged reside in the Ashmem heap. No matter what happens, the garbage collector does not automatically recycle these Bitmap. When the Android drawing system renders these images, the Android system library extracts the Bitmap from the Ashmem heap, and when the rendering is finished, these Bitmap are returned to their original positions. If a drawn picture needs to be redrawn again, the system just needs to decode it again, which is a very fast operation.

This sounds like a perfect solution, but the problem is that the bitmap decoding operation is running on the UI thread. Bitmap decoding is very CPU intensive, which causes the UI to block when it is too expensive. For this reason, Google does not recommend using this feature. Now they recommend using a different feature,--inbitmap. However, this feature is not supported until after Android3.0. Even so, this feature is not very useful unless all the images in the App are the same size, which is obviously not applicable to Fackbook. Until version 4.4, this restriction was removed. But what we need is a generic solution that can run on Android 2.3-the latest version.

Self - reliance

For the above-mentioned "decoding operation caused the UI to feign death," we found a solution that both UI display and memory management are performing well. If we put the extracted memory area back to the original position before the UI thread renders it, and make sure it is no longer being extracted, then we can put the images in the Ashmem without the problem of the UI suspended animation. Fortunately, there is a function in the NDK of Android that perfectly implements this requirement, named Androidbitmap_lockpixels. This function was originally intended to be executed after calling Unlockpixels to extract the memory area again.

We made a breakthrough when we realized that there was no need for us to do so. If we only call Lockpixels and don't call the corresponding unlockpixels, then we can create a memory-safe image in Java's heap memory without causing the UI thread to loads slow. With just a few lines of C + + code, we solved the problem perfectly.

Write Java code in C + + 's mind

As the Spider-Man said: "The stronger the power, the greater the responsibility." "Bitmap that can be purged are neither reclaimed by the garbage collector nor handled by the ASHMEM built-in cleanup mechanism, which makes it possible to use them to cause memory leaks." So we can only rely on ourselves.

In C + +, the usual solution is to create a smart pointer class that implements a reference count. These need to take advantage of the language features of C + +-copy constructors, assignment operators, and deterministic destructors. This syntax does not exist in Java because the garbage collector can handle all of this. So we have to implement these assurance mechanisms in C + + in some way in Java.

We have created two classes to accomplish this. One of them is called "sharedreference", it has addreference and deletereference two methods, the caller must take the base class object or let it out of scope. Once the reference counter is zeroed, resource processing (bitmap.recycle) occurs.

However, it is clear that it is easy to make mistakes for Java developers to call these methods. The Java language is designed to avoid doing such a thing! So above the sharedreference, we built the Closeablereference class. It not only realizes the Java Closeable interface, but also implements the Cloneable interface. Its constructor and clone () method call AddReference (), and the Close () method calls Deletereference (). So Java developers need to follow the following two simple rules:

1. When assigning closeablereference new objects, call. Clone ().
2. Call. Close () when it is out of scope, usually in the finally block.

These rules can effectively prevent memory leaks and allow us to enjoy native memory management and communication in a large Java program like Fackbook's Android client.

Not just the loader, it's a pipe

It takes a lot of steps to display a picture on a mobile device: several good open source libraries are executed in this order, such as Picasso,universal image Loader,glide and volley, and so on. These open source libraries make a very important contribution to the development of Android. We believe that Fresco will perform better in several important ways.

The difference is that we think of the above steps as pipelines, not just loaders. Each step and other aspect should be as independent as possible, passing the data and parameters in, and then producing an output, as simple as that. It should be possible to do something, either in parallel or serially. Some operations can only be performed under attribute conditions. Some are executed on a thread that has special requirements. In addition, when we consider improving the image, all the images become very complex. Many people use Facebook at low speeds, and we want these people to be able to see pictures as quickly as possible, often even before the pictures are completely downloaded.

Don't fret, embrace the stream

In Java, asynchronous code has historically been executed through the future mechanism. In another thread, the code is committed, and then a future-like object can check that the results of the execution have been completed. However, this is only possible if only one result is assumed. When dealing with progressive images, we want to be able to display the results in a complete and continuous sequence.

Our solution is to define a more generalized version of the future, called DataSource. It provides a subscription method in which the caller must pass in a datasubscriber and executor. Datasubscriber can get the results from datasource to processing and processing, and provides a very simple way to differentiate. Because we need to deal with these objects very often, we must have a definite close call, fortunately, DataSource itself is closeable.

In the background, a new framework called "producer/Consumer" is implemented on each case. The problem is that we are getting inspiration from Reactivex. Our system has a similar interface to Rxjava, but it is more suitable for mobile devices and has built-in support for Closeables.

Keep the interface simple. Producer only a method called Produceresults, this method requires a consumer object. Conversely, consumer has a Onnewresult method.

We use systems like this to connect producer. Suppose we have a producer job of converting Type I to type O, then it should look like this:

public class Outputproducer<i, o> implements producer<o> {private final producer<i> minputproducer;  Public Outputproducer (producer<i> inputproducer) {this.minputproducer = Inputproducer; } public void Produceresults (consumer<o> outputconsumer, Producercontext context) {consumer<i> InputConsu    Mer = new Inputconsumer (Outputconsumer);  Minputproducer.produceresults (Inputconsumer, context);    } private static Class Inputconsumer implements Consumer<i> {private final consumer<o> moutputconsumer;    Public Inputconsumer (consumer<o> outputconsumer) {moutputconsumer = Outputconsumer;      } public void Onnewresult (I newresult, Boolean islast) {O output = Doactualwork (Newresult);          Moutputconsumer.onnewresult (output, islast); }  }}

This allows us to string together very complex steps and to maintain their logical independence.

Full coverage of animations

People who use Facebook like stickers very much, because it can be animated to store GIFs and web formats. If you support these formats, you need to face new challenges. Because each animation is made up of more than one picture, you need to decode each image, store it in memory, and then display it. For larger animations, it is not feasible to put each frame of the picture in memory.

We have built animateddrawable, a powerful drawable that can render animations, while supporting both GIF and WEBP formats. Animateddrawable implements the standard Android Animatable interface, so the caller can start or stop the animation at will. In order to optimize memory usage, if the picture is small enough, we cache the images in memory, but if it is too large, we can quickly decode the images. These behavior callers are fully controllable.

All the backgrounds are implemented in C + + code. We keep a copy of the decoding data and metadata parsing, such as width and height. We cite technical data, which allows multiple Java-side drawables to simultaneously access a WEBP image.

How to Love you? I'll tell you ...

When a picture is downloaded from the Web, we want to display a bitmap. If the download fails, we will display an error flag. When the picture is loaded, we have a gradient animation. By using hardware acceleration, we can scale proportionally, or the matrix is transformed into the size we want and then rendered. We do not always shrink from the center of the image, so we can define our own focal points for shrinking. Sometimes we want to show rounded corners and even rounded pictures. All of these operations should be fast and smooth.

Our previous implementation was to use the Android View Object-The time has come to replace the placeholder view with ImageView. This operation is very slow. Changing the view will force Android to flush the entire layout, which is definitely not what you want to see when the user slides. It is wise to use Android's Drawables, which can be quickly replaced.

So we created the Drawee. This is a picture display framework like the MVC architecture. The model is called Draweehierarchy. It is implemented as a layer of drawables, and for the underlying image, each zengdu has a specific function-imaging, cascading, fading, or scaling.

Draweecontrollers is connected to the image via a pipe-or other image-loading library-and handles picture manipulation in the background. They receive events from pipelines and decide what to do with them. They control the actual operation of the draweehierarchy-whether it is a placeholder picture, an error condition, or a completed picture.

Draweeviews is not much of a function, but it is essential. They monitor Android's view no longer displays system events on the screen. When the picture leaves the screen, Draweeview can tell Draweecontroller to close the image resource that was used. This avoids memory leaks. Also, if it is not already within the screen range, the controller tells the picture pipeline to cancel the network request. Therefore, when scrolling a long list of pictures like Fackbook, there will be no frequent network requests.

Through these efforts, the hard work of displaying pictures is gone. The calling code only needs to instantiate a draweeview and then specify a URI and other optional parameters. The rest of it will be done automatically. Developers do not need to worry about managing image memory or updating the image stream. Fresco did everything for them.

Fresco

After completing this image display and manipulating the complex tool library, we want to share it with the Android developer community. We are pleased to announce that from today onwards, this project has become an open source code!

Murals are painting techniques that have been welcomed by people all over the world for centuries. Many of our great artists use this name, from the Italian Renaissance master Raphael to the mural artist Sri Lanka. We're not pretending to reach this great level, we really want Android developers to enjoy using it just like we did when we first enjoyed creating this open Source library.

More

Fresco Chinese documents

Facebook launches Android image loading library-fresco