Complete parsing of Android DiskLruCache, best solution of hard disk cache, disklrucache

Complete parsing of Android DiskLruCache, best solution of hard disk cache, disklrucache

Reprinted please indicate the source: http://blog.csdn.net/guolin_blog/article/details/28863651

Overview

I remember that I wrote an article long ago.Android efficiently loads large charts and multi-graph solutions, effectively avoiding program OOMThis article is translated from Android Doc. The core solution to prevent multi-graph OOM is to use LruCache technology. However, LruCache only manages the storage and release of images in the memory. If an image is removed from the memory, it needs to be reloaded from the network. This is obviously time-consuming. In this regard, Google provides a hard disk cache solution: DiskLruCache (not officially written by Google, but officially certified ). Unfortunately, the Android Doc does not provide a detailed description of DiskLruCache usage, and there is little information about DiskLruCache on the Internet, so today I want to write a special blog to explain in detail the usage of DiskLruCache and analyze its working principles. This should be the most detailed information about DiskLruCache on the Internet.

Let's take a look at which applications have used the DiskLruCache technology. In my applications, Dropbox, Twitter, Netease news and so on all use DiskLruCache for hard disk caching. Most people in Dropbox and Twitter should have never used it, so let's start with the most familiar Netease news to have an initial understanding of DiskLruCache.

Preliminary Exploration

I believe everyone knows that the data in NetEase news is obtained from the Internet, including a lot of news content and news images, as shown in:

But I don't know if you have found that the content and images are stored in the local cache after they are obtained from the Internet, therefore, even if the mobile phone does not have a network, it can still load news that has been browsed before. The cache technology used is needless to say. Naturally, it is DiskLruCache. So the first question is, where are the data cached on the mobile phone?

In fact, DiskLruCache does not limit the data cache location and can be set freely, however, most applications usually select/sdcard/Android/data/<application package>/cache as the cache path. There are two advantages to choosing this location: first, this is stored on the SD card, so even if more data is cached, it will not affect the mobile phone's internal storage space, as long as the SD card space is sufficient. Second, the path is determined by the Android system as the cache path of the application. When the application is uninstalled, the data here will be cleared together, in this way, there will not be a lot of residual data on the mobile phone after the program is deleted.

Take Netease news as an example. Its client package name is com. netease. newsreader. therefore, the data cache address is/sdcard/Android/data/com. netease. newsreader. activity/cache. Let's go to this directory and check the results, as shown in:

As you can see, there are many folders, because Netease news caches various types of data. For the sake of simplicity, we only need to analyze the image cache, so we can enter the bitmap folder. Then you will see a bunch of files with long file names, which have no naming rules and do not understand what they mean, but if you keep scrolling down, A file named journal is displayed, as shown in:

So what are these files? I believe that some of my friends are confused. The files with a long file name are cached images. Each file corresponds to an image, and the journal file is a log file of DiskLruCache, the operation records of each image are stored in this file. Basically, the journal file indicates that the program uses DiskLruCache technology.

Download

Now that you have an initial understanding of DiskLruCache, let's take a look at the usage of DiskLruCache. Since DiskLruCache is not officially compiled by Google, this class is not included in the Android API. We need to download this class from the Internet and then manually add it to the project. The Source code of DiskLruCache is on Google Source. The address is as follows:

Bytes

If Google Source cannot be openedClick hereDownload the source code of DiskLruCache. After the source code is downloaded, you only need to create a new libcore. io package in the project, and then copy the DiskLruCache. java file to this package.

Open Cache

In this case, we have prepared the preparations. Let's take a look at how to use DiskLruCache. First, you must know that DiskLruCache cannot generate new instances. If we want to create a DiskLruCache instance, we need to call its open () method. The interface is as follows:

public static DiskLruCache open(File directory, int appVersion, int valueCount, long maxSize)

The open () method receives four parameters. The first parameter specifies the data cache address, and the second parameter specifies the version number of the current application, the third parameter specifies the number of cache files that the same key can correspond to. The fourth parameter specifies the maximum number of bytes of data that can be cached.

The cache address has been mentioned earlier. It is usually stored in the/sdcard/Android/data/<application package>/cache path, but at the same time, we need to consider the case that this mobile phone does not have an SD card, or the SD is just removed. Therefore, excellent programs will write a dedicated method to obtain the cache address, as shown below:

public File getDiskCacheDir(Context context, String uniqueName) {String cachePath;if (Environment.MEDIA_MOUNTED.equals(Environment.getExternalStorageState())|| !Environment.isExternalStorageRemovable()) {cachePath = context.getExternalCacheDir().getPath();} else {cachePath = context.getCacheDir().getPath();}return new File(cachePath + File.separator + uniqueName);}

You can see that when the SD card exists or the SD card cannot be removed, call the getExternalCacheDir () method to obtain the cache path. Otherwise, call the getCacheDir () method to obtain the cache path. The obtained path is/sdcard/Android/data/<application package>/cache, the latter obtains the path/data/<application package>/cache.

Then, the obtained path and a uniqueName are spliced and returned as the final cache path. So what is this uniqueName? In fact, this is a unique value set to distinguish different types of data, such as bitmap, object, and other folders seen in the Netease news cache path.

The next step is the application version number. We can use the following code to obtain the version number of the current application:

public int getAppVersion(Context context) {try {PackageInfo info = context.getPackageManager().getPackageInfo(context.getPackageName(), 0);return info.versionCode;} catch (NameNotFoundException e) {e.printStackTrace();}return 1;}

Note that, whenever the version number changes, all data stored in the cache path will be cleared, because DiskLruCache considers that when the application has a version update, all data should be retrieved from the Internet.

There is nothing to explain about the following two parameters. The third parameter is 1, and the fourth parameter is usually 10 MB, which can be adjusted according to your own situation.

Therefore, a very standard open () method can be written as follows:

DiskLruCache mDiskLruCache = null;try {File cacheDir = getDiskCacheDir(context, "bitmap");if (!cacheDir.exists()) {cacheDir.mkdirs();}mDiskLruCache = DiskLruCache.open(cacheDir, getAppVersion(context), 1, 10 * 1024 * 1024);} catch (IOException e) {e.printStackTrace();}

First, call the getDiskCacheDir () method to obtain the cache address path, and then determine whether the path exists. If it does not exist, create it. Call DiskLruCache's open () method to create an instance, and pass in the four parameters.

With the DiskLruCache instance, we can operate on the cached data. The operation types mainly include writing, accessing, and removing. We will learn one by one.

Write Cache

Let's take a look at the writing. For example, there is an image, and the address is:

private boolean downloadUrlToStream(String urlString, OutputStream outputStream) {HttpURLConnection urlConnection = null;BufferedOutputStream out = null;BufferedInputStream in = null;try {final URL url = new URL(urlString);urlConnection = (HttpURLConnection) url.openConnection();in = new BufferedInputStream(urlConnection.getInputStream(), 8 * 1024);out = new BufferedOutputStream(outputStream, 8 * 1024);int b;while ((b = in.read()) != -1) {out.write(b);}return true;} catch (final IOException e) {e.printStackTrace();} finally {if (urlConnection != null) {urlConnection.disconnect();}try {if (out != null) {out.close();}if (in != null) {in.close();}} catch (final IOException e) {e.printStackTrace();}}return false;}

This code is quite basic and I believe everyone can understand it. It is to access the URL passed in urlString and write it to the local through outputStream. With this method, we can use DiskLruCache to write data. The write operation is completed by using the DiskLruCache. Editor class. Similarly, this class cannot be new. You need to call the edit () method of DiskLruCache to obtain the instance. The interface is as follows:

public Editor edit(String key) throws IOException

As you can see, the edit () method receives a parameter key, which becomes the name of the cached file and must correspond to the image URL one by one. So how can we make the key and image URL match one by one? Directly use URL as the key? This is not suitable because the image URL may contain some special characters which may be invalid during file naming. In fact, the simplest way is to perform MD5 encoding on the image URL. The encoded string must be unique and only contain characters such as 0-F, which fully complies with the file naming rules.

Then we will write a method to perform MD5 encoding on the string. The Code is as follows:

public String hashKeyForDisk(String key) {String cacheKey;try {final MessageDigest mDigest = MessageDigest.getInstance("MD5");mDigest.update(key.getBytes());cacheKey = bytesToHexString(mDigest.digest());} catch (NoSuchAlgorithmException e) {cacheKey = String.valueOf(key.hashCode());}return cacheKey;}private String bytesToHexString(byte[] bytes) {StringBuilder sb = new StringBuilder();for (int i = 0; i < bytes.length; i++) {String hex = Integer.toHexString(0xFF & bytes[i]);if (hex.length() == 1) {sb.append('0');}sb.append(hex);}return sb.toString();}

The code is very simple. Now we only need to call the hashKeyForDisk () method and pass the image URL to this method to get the corresponding key.

Therefore, you can now write it like this to get an instance of DiskLruCache. Editor:

String imageUrl = "http://img.my.csdn.net/uploads/201309/01/1378037235_7476.jpg";String key = hashKeyForDisk(imageUrl);DiskLruCache.Editor editor = mDiskLruCache.edit(key);

With DiskLruCache. after the Editor instance, we can call its newOutputStream () method to create an output stream, and then pass it into downloadUrlToStream () to implement the download and write cache functions. Note that the newOutputStream () method receives an index parameter. Because the value of 1 is specified when valueCount is set, you can set index to 0 here. After the write operation is completed, we also need to call the commit () method for submission to make the write take effect. If the abort () method is called, The write is abandoned.

Therefore, the code for a complete write operation is as follows:

new Thread(new Runnable() {@Overridepublic void run() {try {String imageUrl = "http://img.my.csdn.net/uploads/201309/01/1378037235_7476.jpg";String key = hashKeyForDisk(imageUrl);DiskLruCache.Editor editor = mDiskLruCache.edit(key);if (editor != null) {OutputStream outputStream = editor.newOutputStream(0);if (downloadUrlToStream(imageUrl, outputStream)) {editor.commit();} else {editor.abort();}}mDiskLruCache.flush();} catch (IOException e) {e.printStackTrace();}}}).start();

Because the downloadUrlToStream () method is called to download images from the network, make sure that this code is executed in the subthread. Note that at the end of the code, I also called the flush () method. This method is not required for every write operation, but is indispensable here, I will explain its role later.

Now the cache should have been successfully written. Let's go to the cache directory on the SD card and check it, as shown in:

We can see that there is a file with a long file name and a journal file. A file with a long file name is naturally a cached image, because it is named by MD5 encoding.

Read Cache

After the cache is successfully written, we should learn how to read it. The reading method is simpler than writing. It is implemented mainly by using the get () method of DiskLruCache. The interface is as follows:

public synchronized Snapshot get(String key) throws IOException

Obviously, the get () method requires passing in a key to obtain the corresponding cached data, and this key is undoubtedly the MD5 encoded value of the image URL, therefore, the code for reading the cached data can be written as follows:

String imageUrl = "http://img.my.csdn.net/uploads/201309/01/1378037235_7476.jpg";String key = hashKeyForDisk(imageUrl);DiskLruCache.Snapshot snapShot = mDiskLruCache.get(key);

It is strange that the obtained object is a DiskLruCache. Snapshot object. How can we use this object? It's easy to get the input stream of the cached file by calling its getInputStream () method. Similarly, the getInputStream () method also needs to pass an index parameter. It is good to input 0 here. With the input stream of the file, it is easy to display the cached image on the interface. Therefore, the code for reading the complete cache and loading the image to the interface is as follows:

try {String imageUrl = "http://img.my.csdn.net/uploads/201309/01/1378037235_7476.jpg";String key = hashKeyForDisk(imageUrl);DiskLruCache.Snapshot snapShot = mDiskLruCache.get(key);if (snapShot != null) {InputStream is = snapShot.getInputStream(0);Bitmap bitmap = BitmapFactory.decodeStream(is);mImage.setImageBitmap(bitmap);}} catch (IOException e) {e.printStackTrace();}

We used the decodeStream () method of BitmapFactory to parse the file stream into a Bitmap object and set it to ImageView. If you run the program, you will see the following results:

OK. The image is displayed successfully. Note that this is loaded from the local cache rather than from the network. Therefore, this picture can still be displayed even if your mobile phone is not connected to the Internet.

Remove cache

After learning how to write and read the cache, you have mastered the two most difficult operations. Then, you can easily remove the cache. The remove cache is implemented by using the remove () method of DiskLruCache. The interface is as follows:

public synchronized boolean remove(String key) throws IOException

I believe you are quite familiar with it. You need to input a key in the remove () method and then delete the cache image corresponding to this key. The sample code is as follows:

try {String imageUrl = "http://img.my.csdn.net/uploads/201309/01/1378037235_7476.jpg";  String key = hashKeyForDisk(imageUrl);  mDiskLruCache.remove(key);} catch (IOException e) {e.printStackTrace();}

Although the usage is simple, you must know that this method should not be called frequently. Because you do not need to worry about the excessive amount of cached data that occupies too much space on the SD card, DiskLruCache will automatically delete the excess cache based on the Maximum Cache value we set when calling the open () method. You must call the remove () method to remove the cache only when you confirm that the cache content corresponding to a key has expired and need to obtain the latest data from the network.

Other APIs

In addition to writing to the cache, reading from the cache, and removing the cache, DiskLruCache also provides some common APIs. Let's take a look at them briefly.

1. size ()

This method returns the total number of bytes of All cached data in the current cache path, in bytes. If the application needs to display the total size of the current cached data on the interface, you can call this method to calculate it. For example, Netease news has such a function, as shown in:

2. flush ()

This method is used to synchronize the operation records in the memory to the log file (that is, the journal file. This method is very important, because the premise that DiskLruCache can work normally is to rely on the content in the journal File. I previously called this method once when I explained the write cache operation, but not every time I write the cache, the flush () method is called, frequent calls will not bring any benefit, but will only increase the time for synchronizing journal files. The standard method is to call the flush () method once in the onPause () method of the Activity.

3. close ()

This method is used to disable DiskLruCache, which is a method corresponding to the open () method. Once disabled, you cannot call any method in DiskLruCache to operate on cached data. Generally, you should only call the close () method in the onDestroy () method of the Activity.

4. delete ()

This method is used to delete all the cached data. For example, you only need to call the delete () method of DiskLruCache to manually clear the cache in NetEase news.

Interpreting journal

As mentioned above, the premise that DiskLruCache can work normally is to rely on the content in the journal File. Therefore, it is very important for us to understand how DiskLruCache works. So what is the content in the journal file? Let's take a look, as shown in:

Because only one image is cached, there are not several lines of logs in the journal, and we will analyze them in one line. The first line is a fixed string "libcore. io. DiskLruCache", marking the use of DiskLruCache technology. The second row is the version number of DiskLruCache. The value is invariably 1. The third line is the application version number. The version number passed in the open () method will be displayed here. The fourth row is valueCount. This value is also passed in the open () method. Generally, it is 1. The fifth line is a blank line. The first five lines are also called the journal file headers. This part of content is easy to understand, but the next part will be a little bit of a brain.

The sixth line begins with a DIRTY prefix, followed by the key of the cached image. We usually see that DIRTY does not represent any good thing, which means it is a DIRTY data. Yes, every time we call the DiskLruCache's edit () method, a DIRTY record will be written to the journal File, indicating that we are preparing to write a cache data, but we don't know what the result is. Then, the commit () method is called to indicate that the cache is successfully written. a clean record is written to journal, which means that the dirty data is "cleaned" and abort () is called () method indicates that the write to the cache failed. a remove record will be written to journal. That is to say, the key of each DIRTY row should be followed by a CLEAN or REMOVE record. Otherwise, the data is "DIRTY" and will be automatically deleted.

If you are careful enough, you should also notice that the record in the seventh row, in addition to the CLEAN prefix and key, is followed by a 152313 record. What does this mean? In fact, DiskLruCache adds the cached data size at the end of each CLEAN record, in bytes. 152313, that is, the number of bytes of the image we cached, is approximately 148.74 kb. It is just as big as the cached image, as shown in:

The size () method we learned previously can obtain the total number of bytes of All cached data in the current cache path, in fact, its working principle is to add the number of bytes of all CLEAN records in the journal file, and then return the total sum.

In addition to DIRTY, CLEAN, and REMOVE, there is also a record whose prefix is READ, which is very simple. Whenever we call the get () method to READ a cached data, a read record is written to the journal File. Therefore, for apps like Netease news that have large images and data volumes, there may be a large number of READ records in the journal File.

You may be worried. If I keep writing data to the journal File, will the journal File become larger and larger? You don't have to worry about this. DiskLruCache uses a redundantOpCount variable to record the number of user operations. Each time you perform a write, read, or remove cache operation, the value of this variable is increased by 1, when the variable value reaches 2000, it will trigger the reconstruction of the journal event. At this time, it will automatically clear all unnecessary and unnecessary records in the journal, ensure that the journal file size is always within a reasonable range.

Now, we have analyzed DiskLruCache usage and basic working principles. As for the source code of DiskLruCache, it is still relatively simple. Due to space limitations, it will not be available here. Interested friends can explore it on their own. In the next article, I will take you through a project practice to better understand the usage of DiskLruCache.

Hard disk cache

1. Hard disk cache. You can take the hard disk model to the official website and check it.
Your hard disk is cached at 2 MB, with the following specifications: interface type Ultra ATA/capacity 80 GB/2 MB Cache/7200 rpm
Www.segem.com.cn/...1.html
2. You can only adjust the system's virtual memory to achieve repeated read/write operations on the hard disk. This protects the hard disk, and the actual cache of the hard disk cannot be adjusted.

What is cache? What is the 8 M 16 M hard disk cache?

Cache; the English name is Cache, which is also a kind of memory. The data exchange speed is fast and the computing frequency is high.
In the early days, the CPU speed was low and the memory speed could fully meet the system's needs. however, as the CPU speed increases, but the memory speed increases slowly, it becomes a bottleneck of the system. No matter how you increase the CPU frequency, the performance of the entire machine is severely constrained by the memory. at this time, the Cache is born, and its speed is faster than the main memory. As a buffer area of the CPU and memory, it effectively improves the system performance.
Cache is divided into Level 1 cache and level 2 Cache L1 (level 1 cache)
L2 (level 2 cache)
The level-1 cache is mainly used to store CPU commands and Code. Its capacity is relatively fixed. After the required value is reached, increasing its capacity will not improve its performance. The L1Cache varies with CPU; l2Cache is mainly used to store commands of the operating system during computer operation. the capacity and speed of program data and address pointers have a great impact on the system performance. the higher the capacity, the faster the system is ..
The hard disk cache is the place where data is exchanged between the hard disk and the external general route. The read/write process of the hard disk is filled and cleared again and again after the magnetic signal is converted into an electrical signal, then, the data is transmitted step by step according to the PCI bus cycle, so the cache capacity and speed are directly related to the hard disk transmission speed. its capacity is kb. 2 MB and 8 MB and 16 MB
The higher the cache, the higher the hard disk transmission rate.