6 ways for Linux to delete a large number of files _linux shell

Source: Internet
Author: User
Tags rsync

Create 500,000 files first

Copy Code code as follows:

Test for I in $ (seq 1 500000)
For> do
for> echo Test >> $i. txt
For> Done

1 RM

Copy Code code as follows:

Test Time Rm-f *
Zsh:sure you want to delete all the files in/home/hungerr/test [yn]? Y
Zsh:argument list Too long:rm
Rm-f * 3.63s User 0.29s system 98% CPU 3.985 Total

RM does not work due to excessive number of files.

2 Find

Copy Code code as follows:

Test time find./-type f-exec rm {} \;
Find./-type f-exec rm {} \; 49.86s user 1032.13s system 41% CPU 43:19.17 Total

About 43 minutes.

3 Find with Delete

Copy Code code as follows:

Test time find./-type F-delete
Find./-type f-delete 0.43s User 11.21s system 2% CPU 9:13.38 Total

Spents 9 minutes.

4 rsync
Create an empty folder first Blanktest

Copy Code code as follows:

~ Time Rsync-a--delete blanktest/test/
Rsync-a--delete blanktest/test/0.59s User 7.86s system 51% CPU 16.418 Total

16s, very good very strong.

5 Python

Copy Code code as follows:

Import OS
Import time
Stime=time.time ()
For pathname,dirnames,filenames in Os.walk ('/home/username/test '):
For filename in filenames:
File=os.path.join (Pathname,filename)
Os.remove (file)
Ftime=time.time ()
Print Ftime-stime

Copy Code code as follows:

~ Python test.py
494.272291183

About 8 minutes.

6 Perl

Copy Code code as follows:

Test time Perl-e ' for (<*>) {(stat) [9]< (unlink)}} '
Perl-e ' for (<*>) {(stat) [9]< (unlink)} ' 1.28s user 7.23s system 50% CPU 16.784 Total

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.