Golang parsing Android Backup files

This is a creation in Article, where the information may have evolved or changed.

I refer to a project on a StackOverflow, Java solution Android backup file. Then translated into the Golang code.
Summarize:
Although the Java language itself is very old, but there are many libraries ah, but people's flow is really useful
Miss Java for the first x days.

Overall backup file: Xx.ab is divided into two parts

• Non-encrypted part
• Encryption section

Analysis

Encryption section

A file information that I have parsed

    magic            string      version          int    compress         int    encryptionAlg    string    userSaltHex      string    ckSaltHex        string    round            int    userIvHex        string    masterKeyBlobHex string

Some debug information, easy to understand the above specific content
···
Agic:android BACKUP
Version:3
Compressed:1
algorithm:aes-256
Iv:1539d61dc22c13259102b5f20b7e9c3d
mk:ec78a761edae9e578a1cae125331bf0c1d9178781c1d997711867ee7717ec30b
MK Checksum:944120ed555a34c6559d1793a322b5b59ced1ab8b19275db7a482984b06ed4a7
Key bytes:ec78a761edae9e578a1cae125331bf0c1d9178781c1d997711867ee7717ec30b
Salt bytes: d12a67d559de5b025e7d0c5d41e27a49c9e7e9502f755e23b3e964969e0aa83dd70235e41c5a72bfd2ea0e2112e36e4827ab54f20fdefc62f353cd557 983378B
Calculated MK checksum (use UTF-8: TRUE): 944120ed555a34c6559d1793a322b5b59ced1ab8b19275db7a482984b06ed4a7
···

By line to explain

1. Magic has the function of title
2. Version only supports 1-4 v2 version above
3. Whether to compress
4. The encryption algorithm used
5. Salt 1 Usersalt
6. Salt 2 Cksalt
7. Rounds
8. Userivhex
9. Masterkeyblob, this is the secret key for encryption.

A little note of 2

• Storing information with UTF-8 in the case of version number >2

A little note of 3

• If it is compressed, it can only be solved with a compressed stream.

A little note of 9

• This section is made up of iv+ keys together.

  The first element of the array represents the length of the IV

Encryption section

After the content, all are encrypted and compressed by the key stream to deal with, but there is a problem of egg, I read the *.ab file, read the first 9 lines, after the contents of the decryption, encountered a decryption bug, the front content is normal, after the GG. Then, my approach is to write this part of the content into a separate file. To decrypt it again.

The difficulty of processing into go

• Whether there is an encrypted stream in go
• Whether there is a compressed stream in go
• Whether key generation (padding, Java BC Proprietary library) in Go is consistent with Java

The result of the final analysis (maybe I'm not familiar with go stream)

• No AES CBC encrypted stream in Go
• No compressed stream in Go (java.util.zip.InflaterInputStream corresponds to what's in go?) )
• Handwriting pkcs#5 padding
• Key generation, go no jce how to do?

Solution Solutions

Pointers of Go no AES CBC encrypted stream

Let's talk about the features of cipher in Go, in CBC mode, the goods will remain in the last encrypted state, that is to say,

var 加密机加密.文本1加密.文本2//与下面的操作等价var 加密机加密.(文本1+文件2)

Before I also have a misunderstanding, that is, stream encryption is not equal to block encryption. I was wrong, in fact, stream encryption, is to read a certain number of bytes at a time, and then read a certain number of bytes, and the previous results in the corresponding mode of operation, such as CBC XOR.
Block encryption, it is directly to encrypt all the content read into memory at once.
So, I use go, can read 16B of content encryption every time, so save memory! Also convenient for the last time to fill!

func (tool *Filetool) temp2() {    println("cbc方式读")    fi, _ := os.Open(tool.temp1file)    defer fi.Close()    aesBlockEncrypter, _ := aes.NewCipher(tool.realkey)    aesEncrypter := cipher.NewCBCDecrypter(aesBlockEncrypter, tool.realIV)    fo, _ := os.Create(tool.temp2file)    defer fo.Close()    //tmp := make([]byte, 16)    read := make([]byte, 16)    write := make([]byte, 16)    frist:=true    var n int    for {        n, _ = fi.Read(read)        if n == 0 {            fo.Write(PKCS5UnPadding(write))            break        } else if !frist {            fo.Write(write)        } else {            frist=false        }        aesEncrypter.CryptBlocks(write, read)        //copy(tmp, write)    }}

No compression stream in Go

This really hurts, only the decrypted content can be written to a file, and then decrypted.

func (tool *Filetool) final() {    inputFile, _ := os.Open(tool.temp2file) //变量指向os.Open打开的文件时生成的文件句柄    zf, err := zlib.NewReader(inputFile)    if err != nil {        return    }    defer zf.Close()    //tarFilePath = filepath.Join(t.parseBackupTask.backupPath, fmt.Sprintf("backup_%d.tar", api.GetEvidenceID()))    //os.MkdirAll(filepath.Dir(path), os.ModePerm)    tarFile:=tool.outfileh    if err != nil {        return    }    defer tarFile.Close()    io.Copy(tarFile, zf)}

Complete code wait, I'll pass it when I'm free.