Vsearch removal of repetitive sequences and singleton sequences

In 16S data analysis, in order to reduce the clustering time, improve the accuracy, need to remove the repeating sequence, and singleton sequence because there is no other sequence as validation, the credibility is not very high, also need to remove, usually use Usearch to complete these 2 tasks, but Usearch 64-bit is charged, while 32 is usearch on 64-bit Red hat when testing, removing the repeat sequence times wrong, Libgomp:thread creation Failed:resource temporarily unavailable

Baidu after the understanding is because the number of processes reached the upper limit, modified the upper limit or error, abandoned the use of Usearch to remove the repetition sequence, using Vsearch to remove the repeating sequence

Source:

Https://github.com/torognes/vsearch/releases

Installation:

wget https://github.com/torognes/vsearch/archive/v1.11.1.tar.gz

Tar xzvf v1.11.1 cd vsearch-1.11.1/./autogen.sh./configure make make install test: # Remove Repeat Order      Column Vsearch--derep_fulllength raw.reads.fasta--output test.fasta--sizeout # Remove Singleton sequence Vsearch--sortbysize test.fasta--output desingleton.fasta--minsize 2 vsearch and Mothur The result of removing the repeating sequence is exactly the same, actually removing the repeating sequence is the The sequence of identical sequences takes only one bar, and the removal of the singleton sequence is the removal of those sequences that only appear once, in order to deepen understanding, I wrote a perl script to complete these 2 tasks, tested and vsearch consistent, the code is as follows:

#!/usr/bin/perl Usewarnings; UseStrict;my($fasta) =@ARGV;my %unique= ();Local$/ =">";OpenFASTA,$fastaOr die "Can ' t open $fasta \ n"; while(<FASTA>) {    Chomp; Next if/^\s*$/; my($id,$seq) =Split/\n/,$_,2; $seq=~ s/\s+//; Push@{$unique{$seq}},$id;}CloseFASTA;foreach my $x(Keys %unique) {    my $size=scalar@{$unique{$x}}; Unshift@{$unique{$x}},$size;}foreach my $x(Sort{$unique{$b}->[0] <=>$unique{$a}->[0] }Keys %unique) {    my $id=$unique{$x}->[1]; my $size=$unique{$x}->[0]; Next if $size=1; PrintQq{>$id; size=$size; \ n$x\n};}

　

Vsearch removal of repetitive sequences and singleton sequences