Reproduced Android Signature mechanism-a detailed description of the signature process

This article was reproduced from: http://www.wjdiankong.cn/android%E7%AD%BE%E5%90%8D%E6%9C%BA%E5%88%B6%E4%B9%8B-%E7%AD%BE%E5%90%8D%E8%BF% 87%e7%a8%8b%e8%af%a6%e8%a7%a3/

First, preface

It's been a long time, not writing the hands of the article is a bit uncomfortable. Today is Christmas, still have to go to work. Because there was a former colleague in the previous days, in the application of the SDK, encountered the issue of the signature, asked me, the results have baffled me. I say that the signature in Android will be familiar to everyone, just for the sake of security, do not let others modify your apk, but we really know how much? So prepare two articles for a good introduction to the Android signature mechanism.

Before talking about Android signatures, we need to know a few knowledge points

1. Data Digest (data fingerprint), signature file, certificate file

2. Jarsign tool signature and signapk tool signature

3. keystore file and Pk8 file, X509.pem file relationship

4, how to manually sign the APK

The four knowledge points described above are the core of today's presentation, and we will look at these issues in one go.

II. Preparation of knowledge

First look at the data summary, signature files, knowledge points of the certificate file

1. Data summary

This knowledge point is very good understanding, Baidu Encyclopedia can, in fact, he is an algorithm, is to a data source after an algorithm to get a digest, also known as data fingerprint, different data sources, data fingerprint affirmation is not the same, and people like.

Message digest algorithm (Messages Digest algorithm) is a kind of algorithm that can produce special output format, its principle is to make some kind of information extraction from raw data according to certain operation rules, the extracted information is called the Message Digest of raw data.
The famous digest algorithm has RSA Company's MD5 algorithm and SHA-1 algorithm and its large number of variants.
The main features of the message digest are:
1) Regardless of how long the message is entered, the length of the computed message digest is always fixed. For example, the message with the MD5 algorithm digest has 128 bits, and the message with the SHA-1 algorithm Digest eventually has a 160 bit output.
2) in general (without considering collisions), as long as the original data entered is different, the summary of the message generated after the digest will also have to be different, even if the original data slightly changed, the output message digest will be completely distinct. However, the same input must produce the same output.
3) has irreversibility, that is, only forward information digest, and can not recover from the digest of any original message.

2. signature files and certificates

Signature files and certificates are in pairs, the two are inseparable, and we can see through the source code, the names of the two files are the same, but the suffix is not the same name.

In fact, the concept of digital signature is very simple. As you know, there are two issues that must be addressed to ensure reliable communication: first of all, to make sure that the source of the message is indeed the person it affirms, and secondly, to ensure that the information is not tampered with by a third party in the process of transmission, even if it is tampered with, it can be discovered.
The so-called digital signature, is to solve these two problems, it is the previously mentioned asymmetric encryption technology and Digital Digest technology, a specific application.
For the sender of the message, a pair of public private key pairs are generated first, and the public key is given to the recipient of the message.
If the sender of the message wants to send a message to the message recipient one day, add another message in the message, in addition to the original message. This message is generated in the following two steps:
1) Extract the message digest for the original message to be sent;
2) The extracted information digest is encrypted with its own private key.
The message obtained through these two steps is the so-called digital signature of the original information.
For the recipient of the information, the information he receives will contain two parts, one is the original message content, and the other is the additional digital signature. He will verify the authenticity of the message by following three steps:
1) Extract the message digest from the original message section, note that the message digest algorithm used here is consistent with the sender;
2) on the additional digital signature, using the pre-obtained public key decryption;
3) Compare the two messages received in the previous two steps. If consistent, the message is indeed sent by the intended sender, and the content has not been tampered with, and conversely, if it is inconsistent, there must be a problem in the delivery process, the message is not trustworthy.
Through this so-called digital signature technology, it is really possible to effectively solve the problem of reliable communication. If the original message was tampered with during the transfer, then the digest of the message that was tampered with is definitely not the same as the original one at the message receiver. Also, because the tamper does not have the private key of the sender of the message, even if he can recalculate the digest of the tampered message, it cannot forge a digital signature.
So, to sum up, the digital signature is only the sender of information can be produced by others can not forge a string of numbers, this string is also the sender of information to send information authenticity of a valid proof.
Do not know that everyone has noticed that the above-mentioned digital signature method, there is a premise that the recipient of the message must be in advance to obtain the correct public key. If a public key is tampered with at first, the villain will be treated as a good person, and the message sent to you by the real sender will be considered invalid. Moreover, many times there is no information channel to communicate the public key beforehand. So how to ensure the security of public key credibility? This depends on the digital certificate to solve.
The so-called digital certificate, generally contains the following content:
Issuing authority for certificates (Issuer)
Validity period of the certificate (validity)
The public key of the message sender
Certificate owner (Subject)
Algorithms used for digital signatures
Digital signatures
It can be seen that digital certificates in fact also used digital signature technology. Only the content to be signed is the public key of the sender of the message, along with some other information. But unlike the ordinary digital signature, the signer in the digital certificate is not an ordinary institution, but a certain public trust institution. It's like your college diploma is usually signed by a respected headmaster. In general, the root certificates of these credible institutions have been pre-installed on your device before they are shipped to the factory. Therefore, the digital certificate can ensure that the public key in the digital certificate is indeed the owner of the certificate, or the certificate can be used to confirm the identity of the other. Digital certificates are mainly used to solve the problem of public key security issue.
In summary, the general flow of digital signature and signature verification is as follows:

3, Jarsign and signapk tools

Once you know the knowledge points for the three files in your signature, continue to look at the two tools signed in Android: Jarsign and signapk

It's easy to confuse the two tools when they start, and what's the difference between them?

In fact, these two tools are well understood, Jarsign is a Java native comes with a tool, he can sign the jar. And signapk is behind the special for Android app apk signature tool, their two signature algorithm is not different, mainly the signature when using the file is not the same, this will lead to a third problem.

4. Difference between KeyStore file and Pk8,x509.pem file

We have learned that Jarsign and signapk two tools can be signed in Android, then they differ in the signature when using a different file

The Jarsign tool is signed with a KeyStore file

The SIGNAPK tool is signed with a PK8,X509.PEM file

When we use the Eclipse tool to write a program, when the debug package, the default is to use the Jarsign tool to sign, and Eclipse has a default signature file:

We can see this default signature of the KeyStore file, of course we can choose our own designated KeyStore file.

Here's another point of knowledge:

We see above have MD5 and SHA1 summary, this is KeyStore file private key Data digest, this information is also we apply for many development platform account when need to fill in the information, such as applying for Baidu map, SDK, etc., will need to fill in the application of MD5 or SHA1 information.

5. Manual Signature APK Package

1 "Signing with Keytool and Jarsigner

Of course, when we release the package at the official signature, we need to create a keystore file of our own:

Here we can name the KeyStore file, and the suffix name is irrelevant. After the file is created, the values for MD5 and SHA1 are also generated, which can be used without logging, and the MD5 and SHA1 values of KeyStore files can be viewed by command.

Keytool-list-keystore Debug.keystore

Of course we all know the importance of this Keytstore file, which is the equivalent of your bank card password. You know.

Here we see automatic signing and generation of an KeyStore file with Eclipse, and we can also use the Keytool tool to generate a KeyStore file. This method is online, there is not too much to introduce. Then we can use the Jarsign to sign the APK package.

We can manually generate a KeyStore file:

Keytool-genkeypair-v-keyalg dsa-keysize 1024-sigalg sha1withdsa-validity 20000-keystore D:\jiangwei.keystore-alias Jiangwei-keypass Jiangwei-storepass Jiangwei

This command is a bit long and there are several important parameters to note:

-alias is the definition alias, here is the debug

-keyalg is the prescribed signature algorithm, here is the DSA, here the algorithm is directly related to the following apk signature file suffix name, will be explained in detail later

Signing with the Jarsigner tool

Jarsigner-verbose-sigalg sha1withdsa-digestalg sha1-keystore D:\jiangwei.keystore-storepass jiangwei D:\123.apk Jian Gwei

So we can successfully sign the APK.

Problems encountered during the signing process:

1 "The certificate chain could not find a problem

This is because the last parameter alias, is the alias of the KeyStore wrong.

2 "generated keystore file when prompted password error this reason is because in the current directory has debug.ketystore, in the generation of a debug.keystore, will be error 3" The problem with the alias is not found because we use the keytool generated KeyStore, the debug alias, this problem has plagued me for a long time, and finally did a lot of examples to find that as long as our KeyStore file alias is debug, Would have reported such a mistake. This should be with the system default signature Debug.keystore in the alias is debug it is related? Did not find the source of Jarsigner, so can only guess, but these three questions to mark here, in case later in the encounter.

Note: Android is allowed to use multiple KeyStore to sign the APK, I am not in the Paste command, I created a few keystore to sign the APK:

Here I extracted the apk after the signature and found that there are three signature files and certificates (. sf/. DSA)

Here I can also notice that we signed with the DSA algorithm, where the file suffix name is DSA

And the filename is the alias of KeyStore.

Hey, here's how we figure out how to use Keytool to generate KeyStore and, with Jarsigner, to sign.

2 "Signing with signapk

Now let's look at the signapk tool to sign:

Java-jar Signapk.jar. Testkey.x509.pem testkey.pk8 debug.apk debug.sig.apk

There are two files needed:. Pk8 and. X509.pem Two files

PK8 is the private key file

X509.PEM is a file that contains a public key

The signature here is not a demonstration, there is nothing wrong here.

However, it is important to note that the name of the three files in the Meta-inf folder in the APK after the signapk signature is like this, because signapk is not in front of the Jarsigner will automatically use aliases to name the file, this is to write dead is cert name, However, the file name does not affect, the later analysis of the APK check in Android will say that only through the suffix name to find files.

3. What is the difference between two types of signatures?

Then the problem comes, Jarsigner signed with the KeyStore file, signapk signed with the PK8 and X509.pem files, and are to the APK signed, then KeyStore files and Pk8, X509.PEM is there any connection between them? The answer is yes, online search, sure enough, they can be transformed, here is not in the analysis of how to convert, online examples seem to be many, there are specialized tools can be converted:

So here we find out the difference and connection between the two signature tools.

Iii. analyzing the signature process mechanism in Android

Let's take a look at the signature mechanism and principle flow in Android from the source point of view

Because the online did not find Jarsigner source code, but found the source of signapk, then we will take a look at the signapk source bar:

SOURCE Location: Com/android/signapk/sign.java

With the above signature we can see that after Android signature apk, there will be a Meta-inf folder, here are three files:

MANIFEST. Mf

CERT. Rsa

CERT. SF

Let's see what these three files are all about.

1, MANIFEST. Mf

Let's take a look at the source code:

public static void main (String [] args) {
    if (args.length! = 4) {
        System.err.println ("Usage: signapk" +
                "publickey.x509 [.pem] privatekey.pk8" +
                "input.jar output.jar");
        System.exit (2);
    }

    JarFile inputJar = null;
    JarOutputStream outputJar = null;

    try {
        X509Certificate publicKey = readPublicKey (new File (args [0]));

        // Assume the certificate is valid for at least an hour.
        long timestamp = publicKey.getNotBefore (). getTime () + 3600L * 1000;

        PrivateKey privateKey = readPrivateKey (new File (args [1]));
        inputJar = new JarFile (new File (args [2]), false); // Do n‘t verify.
        outputJar = new JarOutputStream (new FileOutputStream (args [3]));
        outputJar.setLevel (9);

        JarEntry je;

        // MANIFEST.MF
        Manifest manifest = addDigestsToManifest (inputJar);
        je = new JarEntry (JarFile.MANIFEST_NAME);
        je.setTime (timestamp);
        outputJar.putNextEntry (je);
        manifest.write (outputJar);

        // CERT.SF
        Signature signature = Signature.getInstance ("SHA1withRSA");
        signature.initSign (privateKey);
        je = new JarEntry (CERT_SF_NAME);
        je.setTime (timestamp);
        outputJar.putNextEntry (je);
        writeSignatureFile (manifest,
                new SignatureOutputStream (outputJar, signature));

        // CERT.RSA
        je = new JarEntry (CERT_RSA_NAME);
        je.setTime (timestamp);
        outputJar.putNextEntry (je);
        writeSignatureBlock (signature, publicKey, outputJar);

        // Everything else
        copyFiles (manifest, inputJar, outputJar, timestamp);
    } catch (Exception e) {
        e.printStackTrace ();
        System.exit (1);
    } finally {
        try {
            if (inputJar! = null) inputJar.close ();
            if (outputJar! = null) outputJar.close ();
        } catch (IOException e) {
            e.printStackTrace ();
            System.exit (1);
        }
    }
}
In the main function, we see that we need to enter four parameters, and then we do three things:

Write MANIFEST.MF

//MANIFEST.MF
Manifest manifest = addDigestsToManifest (inputJar);
je = new JarEntry (JarFile.MANIFEST_NAME);
je.setTime (timestamp);
outputJar.putNextEntry (je);
manifest.write (outputJar);
Take a look at the entry method:

/ ** Add the SHA1 of every file to the manifest, creating it if necessary. * /
private static Manifest addDigestsToManifest (JarFile jar)
        throws IOException, GeneralSecurityException {
    Manifest input = jar.getManifest ();
    Manifest output = new Manifest ();
    Attributes main = output.getMainAttributes ();
    if (input! = null) {
        main.putAll (input.getMainAttributes ());
    } else {
        main.putValue ("Manifest-Version", "1.0");
        main.putValue ("Created-By", "1.0 (Android SignApk)");
    }

    BASE64Encoder base64 = new BASE64Encoder ();
    MessageDigest md = MessageDigest.getInstance ("SHA1");
    byte [] buffer = new byte [4096];
    int num;

    // We sort the input entries by name, and add them to the
    // output manifest in sorted order. We expect that the output
    // map will be deterministic.

    TreeMap <String, JarEntry> byName = new TreeMap <String, JarEntry> ();

    for (Enumeration <JarEntry> e = jar.entries (); e.hasMoreElements ();) {
        JarEntry entry = e.nextElement ();
        byName.put (entry.getName (), entry);
    }

    for (JarEntry entry: byName.values ()) {
        String name = entry.getName ();
        if (! entry.isDirectory () &&! name.equals (JarFile.MANIFEST_NAME) &&
            ! name.equals (CERT_SF_NAME) &&! name.equals (CERT_RSA_NAME) &&
            (stripPattern == null ||
             ! stripPattern.matcher (name) .matches ())) {
            InputStream data = jar.getInputStream (entry);
            while ((num = data.read (buffer))> 0) {
                md.update (buffer, 0, num);
            }

            Attributes attr = null;
            if (input! = null) attr = input.getAttributes (name);
            attr = attr! = null? new Attributes (attr): new Attributes ();
            attr.putValue ("SHA1-Digest", base64.encode (md.digest ()));
            output.getEntries (). put (name, attr);
        }
    }

    return output;
}
The code logic is still very simple, mainly depends on the meaning of the loop:

In addition to the three files (MANIFEST.MF, CERT.RSA, CERT.SF), the other files will do a SHA1 algorithm on the file content, which is to calculate the summary information of the file, and then use Base64 to encode, we use the tool below Let's make a case to see if this is the case:

First install the tool: HashTab

: Http://www.baidu.com/s?wd=hashtab&rsv_spt=1&issp=1&f=8&rsv_bp=0&ie=utf-8&tn=baiduhome_pg&bs=hashtable

Then there is another website that calculates Base64 online: http://tomeko.net/online_tools/hex_to_base64.php?lang=en

Then let's start our verification work:

Let's verify the AndroidManifest.xml file, first find this entry in the MANIFEST.MF file, and record the value of SHA1

Then after we install HashTab, find the AndroidManifest.xml file, right-click and select Hashtab:



Copy the value of SHA-1: 9C64812DE7373B201C294101473636A3697FD73C, go to the Base64 conversion website above, and convert it:

nGSBLec3OyAcKUEBRzY2o2l / 1zw =

The content of the entry in MANIFEST.MF is exactly the same

Then we know from the above analysis, in fact, what is stored in MANIFEST.MF is:

Traverse all the entries in it one by one. If it is a directory, skip it. If it is a file, use the SHA1 (or SHA256) message digest algorithm to extract the digest of the file and then BASE64 encode it as the value of the "SHA1-Digest" attribute. Write to a block in the MANIFEST.MF file. This block has a "Name" attribute whose value is the path of the file in the apk package.

2. Let's take a look at the contents of the CERT.SF file
The content here is similar to the content of MANIFEST.MF, let's take a look at the code:

//CERT.SF
Signature signature = Signature.getInstance ("SHA1withRSA");
signature.initSign (privateKey);
je = new JarEntry (CERT_SF_NAME);
je.setTime (timestamp);
outputJar.putNextEntry (je);
writeSignatureFile (manifest, new SignatureOutputStream (outputJar, signature));
Enter the writeSignatureFile method:

/ ** Write a .SF file with a digest the specified manifest. * /
private static void writeSignatureFile (Manifest manifest, OutputStream out)
        throws IOException, GeneralSecurityException {
    Manifest sf = new Manifest ();
    Attributes main = sf.getMainAttributes ();
    main.putValue ("Signature-Version", "1.0");
    main.putValue ("Created-By", "1.0 (Android SignApk)");

    BASE64Encoder base64 = new BASE64Encoder ();
    MessageDigest md = MessageDigest.getInstance ("SHA1");
    PrintStream print = new PrintStream (
            new DigestOutputStream (new ByteArrayOutputStream (), md),
            true, "UTF-8");

    // Digest of the entire manifest
    manifest.write (print);
    print.flush ();
    main.putValue ("SHA1-Digest-Manifest", base64.encode (md.digest ()));

    Map <String, Attributes> entries = manifest.getEntries ();
    for (Map.Entry <String, Attributes> entry: entries.entrySet ()) {
        // Digest of the manifest stanza for this entry.
        print.print ("Name:" + entry.getKey () + "\ r \ n");
        for (Map.Entry <Object, Object> att: entry.getValue (). entrySet ()) {
            print.print (att.getKey () + ":" + att.getValue () + "\ r \ n");
        }
        print.print ("\ r \ n");
        print.flush ();

        Attributes sfAttr = new Attributes ();
        sfAttr.putValue ("SHA1-Digest", base64.encode (md.digest ()));
        sf.getEntries (). put (entry.getKey (), sfAttr);
    }

    sf.write (out);
}
First of all, we can see that we need to make a SHA1 to the entire content of the previous MANIFEST.MF file and put it in the SHA1-Digest-Manifest field:

Let's take a look at the manifest variable that has just been written into the MANIFEST.MF file.


We can verify this:

Then convert it

See, it's the same as the value in the file

Below we continue to look at the code, there is a loop:

Map <String, Attributes> entries = manifest.getEntries ();
for (Map.Entry <String, Attributes> entry: entries.entrySet ()) {
    // Digest of the manifest stanza for this entry.
    print.print ("Name:" + entry.getKey () + "\ r \ n");
    for (Map.Entry <Object, Object> att: entry.getValue (). entrySet ()) {
        print.print (att.getKey () + ":" + att.getValue () + "\ r \ n");
    }
    print.print ("\ r \ n");
    print.flush ();

    Attributes sfAttr = new Attributes ();
    sfAttr.putValue ("SHA1-Digest", base64.encode (md.digest ()));
    sf.getEntries (). put (entry.getKey (), sfAttr);
}

sf.write (out);
Here we still use the mainfest variable we just passed in, traverse the contents of his entry, and then calculate the SHA algorithm in Base64:

In fact, it is to do a SHA for each entry in the MANIFEST.MF file, just save it, and verify it with an example:

Using AndroidManifest.xml as an example, we copy and save the entries in the MANIFEST.MF file to a txt file:


It should be noted here that after we save, we need to add two newlines, we can see the logic in the code:

Then we calculate the SHA value of the txt document:

See, the values calculated here are the same

Here we know what the CERT.SF file does:

1 "Calculate the overall SHA1 value of this MANIFEST.MF file, and after BASE64 encoding, record it in the" SHA1-Digest-Manifest "attribute value of the CERT.SF main attribute block (on the file header)

2》 Calculate the SHA1 of each block in the MANIFEST.MF file one by one, and record it in the block of the same name in CERT.SF after BASE64 encoding. The name of the attribute is “SHA1-Digest

3. Finally, we are looking at the CERT.RSA file
All we see here are binary files, because the RSA file is encrypted, so we need to use the openssl command to view its content

openssl pkcs7 -inform DER -in CERT.RSA -noout -print_certs --text

For this information, you can see the following picture:

Let's take a look at the code:

/ ** Write a .RSA file with a digital signature. * /
private static void writeSignatureBlock (
        Signature signature, X509Certificate publicKey, OutputStream out)
        throws IOException, GeneralSecurityException {
    SignerInfo signerInfo = new SignerInfo (
            new X500Name (publicKey.getIssuerX500Principal (). getName ()),
            publicKey.getSerialNumber (),
            AlgorithmId.get ("SHA1"),
            AlgorithmId.get ("RSA"),
            signature.sign ());

    PKCS7 pkcs7 = new PKCS7 (
            new AlgorithmId [] {AlgorithmId.get ("SHA1")},
            new ContentInfo (ContentInfo.DATA_OID, null),
            new X509Certificate [] {publicKey},
            new SignerInfo [] {signerInfo});

    pkcs7.encodeSignedData (out);
}
We see that here we will use the private key to calculate the signature of the previously generated CERT.SF file, and then write the signature and the digital certificate containing the public key information into CERT.RSA for storage. CERT.RSA is a file that meets the PKCS7 format.

4. Why do you sign like this?
Above we introduced the details of the three files after signing the apk, so let ’s summarize, why is it necessary to encrypt and sign in this way in Android, is this kind of party encryption the most secure? Let's analyze what will happen if the apk file is tampered with.

First of all, if you change any file in the apk package, the summary information of the changed file is different from the inspection information of MANIFEST.MF during the apk installation verification, so the verification fails and the program cannot be successfully installed.
Secondly, if you calculate a new summary value for the changed file, and then change the corresponding attribute value in the MANIFEST.MF file, then it must be different from the summary value calculated in the CERT.SF file, and the verification fails.
Finally, if you still do not give up, continue to calculate the summary value of MANIFEST.MF, and change the value in CERT.SF accordingly, then the digital signature value must be different from the one recorded in the CERT.RSA file, or it will fail.
Can we continue to forge digital signatures? Impossible, because there is no private key corresponding to the digital certificate.
Therefore, if the application to be repackaged can be installed on an Android device, it must be re-signed.

From the above analysis, it can be concluded that as long as any content in Apk is modified, it must be re-signed, otherwise it will prompt installation failure, of course, it will not be analyzed here, and the latter article will focus on analyzing why it will prompt installation failure.

V. Knowledge points sorting out
1. The meaning of data fingerprint, signature file and certificate file

1》 Data fingerprint is to do SHA / MD5 algorithm on a data source, this value is unique

2》 The signature file technology is: data fingerprint + RSA algorithm

3》 The certificate file contains public key information and other information

4》 After Android signature, SF is the signature file and RSA is the certificate file. We can use openssl to view the certificate information and public key information in the RSA file

2. We understand that there are two ways to sign in Android: jarsigner and signapk The difference between these two ways is:

1》 When jarsigner signs, what is needed is the keystore file, while signapk signing is pk8, x509.pem file

2》 The SF and RSA file names after jarsigner signature are the alias of keystore by default, and the file name after signapk signature is fixed: CERT

3 "When we run the Debug program in Eclipse, the default is to use jarsigner to sign, and the default debug.keystore signature file is also used.

4》 Keystore files and pk8, x509.pem files can be converted between each other

Sixth, thinking
After analyzing the signature technology, we accidentally found a problem, namely CERT.SF, MANIFEST.MF, the name field of the content in these two files are the resource name in the apk, then there is a problem, if the resource Very famous Long, and there are many resources in the apk, then these two files will be very large, so can we optimize here? Later in the article on how to reduce the apk size will continue to explain, here first raise this issue.

Resource download: http://download.csdn.net/detail/jiangwei0910410003/9377046

to sum up

Above, we introduced the signing process in Android through the source code. The whole process is still very clear. The article is a bit long. If you have any questions, please remember to leave a message. I will write another sister article later. Article: The signature verification process in Android is explained in detail, looking forward to ~~

[Reprinted] Android signature mechanism-detailed explanation of the signature process