A little experience of biological big data processing

In the field of biology, files such as Gff,gtf,bam,sam are often processed. Here I summarize some of my experience in dealing with these documents.

1. Use split split each line, these files are basically tab-separated, so there is split split is very convenient, each line of the last comment information is generally;

2. Find the columns you need and find out all the different elements of this column. Because the files you get are likely to be processed before, with very strong uncertainties. For example, if you get a rice data, you may subconsciously think that there are only 12 possible types of chromosomes, but often this subjective error causes your program to run, so be sure to figure out the elements of each column before processing.

3. Write out classes or functions for your usual functions, which can save you a lot of time.

4. The use of various linguistic analyses, PERL,PYTHON,R is the most commonly used language in biological big data, it is very important to master the advantages of each language and the bioinformatics module of the language. Big data trumps good algorithms, and bioinformatics workers don't necessarily have to be proficient in these languages, but they must learn how to use them efficiently.

A little experience of biological big data processing