Build your own yara Database

Build your own yara Database

I have seen some articles about yara before, but it is basically a translation article about how to write rules. However, in actual analysis, the number of rules or the yara rule repository is too small, and many files cannot match the results.

From the actual situation, I may not be able to use yara rules for the time being. Therefore, what I need is to quickly create a yara rule repository-for new malware, it is better to have tools to automatically analyze the sample yara rules and add them to the existing yara rule repository.

Solution

Find the following solutions on the Internet to help solve existing problems:

1. ara official preset rule repository 2. the ClamAV signature is converted to yara rule 3. rules for crawling samples uploaded by others from yara-generator 4. use yara-generator to automatically generate new yara rules

The solution is as follows:

 

Preset rules are obtained from three aspects: First, you can use the official yara preset rules, and then use the tool clamav_to_yara.py to convert the clamav signature To yara rules, finally, yara-generator has many sample rules uploaded by other users. You can use crawlers to crawl these rules into our rule repository.

When analyzing a sample, use the existing yara library for scanning. If no match exists, you can upload the rule to the yara-generator website to automatically analyze the yara rule, and then add the rule to the rule repository. For targeted analysis of malware in some industries, you can manually analyze the keyword string features of some industries and manually modify yara rules.

Get preset rules

1. Official Yara preset rules

The official website of the Yara rule is yarules.com. The rule is stored on github github.com/yara-rules/rules, and the rule ends with yar.

 

2. ClamAV rule conversion to yara

Clam AntiVirus (ClamAV) is a free and open-source AntiVirus software. Updates to the software and virus code are released by the community for free. You can use the clamav_to_yara.py script to directly convert the ClamAV pattern to the yara rule. The main steps are as follows:

1) install clamAV

Installation command apt-get install clamav-freshclam in Linux

2) decompress the signature with the built-in tool

The default rule repository is stored in/var/lib/clamav/main. cvd. You can decompress it using the sigtool that comes with clamav.

Command sigtool-u/var/lib/clamav/main. cvd to decompress the signature. The result is as follows:

 

Main. ndb is the next input file

3) convert a signature to a rule

Conversion command clamav_to_yara.py-f main. ndb-o clamav. yara-s Agent

The converted result clamav. yara is as follows:

 

Script address: https://code.google.com/p/malwarecookbook/source/browse/trunk/3/3/clamav_to_yara.py

3. Crawling rules from yara-generator

Yara-generator (from Joe Sandbox) allows you to view the sample file generation rules uploaded by other users. Its web site is www.yara-generator.net.

 

The yara rule of each sample can be downloaded here. No package download is provided, and a crawler can easily obtain all the rule.

Automatic resolution rules

Without worrying about the sample leakage, you can upload the sample to the www.yara-generator.net for analysis, and then download the corresponding yara rule, which saves the heavy work of writing the yara rule.

 

 

Of course, there are also some omissions in the tool. During the manual analysis process, we can write the discovered features into rules and add them to the file.

Simple scan script

You can use path to quickly customize a multi-to-Multi-scan script. The Code is as follows:

import yaraimport osimport sys def getRules(path):    filepath = {}    for index,file in enumerate(os.listdir(path)):        rupath = os.path.join(path, file)        key = "rule"+str(index)        filepath[key] = rupath    yararule = yara.compile(filepaths=filepath)    return yararule def scan(rule, path):    for file in os.listdir(path):        mapath = os.path.join(path, file)        fp = open(mapath, 'rb')        matches = rule.match(data=fp.read())        if len(matches)>0:            print file,matches if __name__ == '__main__':    rulepath = sys.argv[1]    malpath = sys.argv[2]    yararule = getRules(rulepath)scan(yararule, malpath)

Shows the running result.

The yara rule file is stored under the newrule folder, and a bunch of malicious samples are stored under mal1209. You can rewrite and scan all the folders on the hard disk by yourself. After a variety of rules, a simple complete virus detection software will be completed.

After yara is installed, python can introduce the yara module and import yara

You can use yara. compile () to compile each yara file into a rule object, and then use the match function to match it. For details, see the yara reference document.

The following is a simple example.

rules = yara.compile(filepaths={ 'namespace1':'/my/path/rules1', 'namespace2':'/my/path/rules2' })f = fopen('/foo/bar/my_file', 'rb')matches = rules.match(data=f.read())

More information that can be mined

Yara is a good malware classification software. yara Rules contain a lot of useful string information, such as some URLs and APIs. After manual analysis, you can add the discovered malware features To yara rules.

After accumulating many rules, you can use statistical methods to mine and collect statistics, such as how many malware are targeted at the financial industry and how many malware are targeted at the gaming industry.