Swift distributed storage performance testing and destructive testing

Last Update:2017-04-23 Source: Internet

Author: User

Developer on Alibaba Coud: Build your first app with APIs, SDKs, and tutorials on the Alibaba Cloud. Read more ＞

Continuation of the previous article to do the next test, the environment is a proxy and three storage.

1. Performance test

Test one: Five threads, each of which uploads a file with a size of almost one G

/data/swift/upload> Du-sh Test*.zip

987M Test1.zip

987M Test2.zip

987M Test3.zip

987M Test4.zip

987M Test.zip

Upload

Swift-ahttp://192.168.1.211:8090/auth/v1.0-u system:root-k testpass upload container1 test.zip

Swift-ahttp://192.168.1.211:8090/auth/v1.0-u system:root-k testpass upload container2 test1.zip

Swift-ahttp://192.168.1.211:8090/auth/v1.0-u system:root-k testpass upload container3 test2.zip

Swift-ahttp://192.168.1.211:8090/auth/v1.0-u system:root-k testpass upload container4 test3.zip

Swift-ahttp://192.168.1.211:8090/auth/v1.0-u system:root-k testpass upload container5 test4.zip

Total 8:30:45

Storage node View files

/srv/node/sdb1> Du-sh *

16K accounts

356M async_pending

129M containers

4.9G objects

0 tmp

View the uploaded files

/data/swift/upload> swift-ahttp://192.168.1.211:8090/auth/v1.0-u system:root-k testpass list Container1

Test.zip

/data/swift/upload> swift-ahttp://192.168.1.211:8090/auth/v1.0-u system:root-k testpass list Container2

Test1.zip

/data/swift/upload> swift-ahttp://192.168.1.211:8090/auth/v1.0-u system:root-k testpass list Container3

Test2.zip

/data/swift/upload> swift-ahttp://192.168.1.211:8090/auth/v1.0-u system:root-k testpass list container4

Test3.zip

/data/swift/upload> swift-ahttp://192.168.1.211:8090/auth/v1.0-u system:root-k testpass list Container5

Test4.zip

Test two, 10 threads:

/data/swift/upload/test100m> Du-sh *

505M Test10.zip

505M Test1.zip

505M Test2.zip

505M Test3.zip

505M Test4.zip

505M Test5.zip

505M Test6.zip

505M Test7.zip

505M Test8.zip

505M Test9.zip

Single thread test upload 2.6m/s, download 5.1m/s

Five Thread testing:

Upload

First time 03:20s,2.5m/s

Second Spents 04:30s,1.85m/s

Third Spents 05:50s,1.31m/s

Fourth Spents 06:55s,1.2m/s

Fifth spents 07:00s,1.19m/s

Download

First spents 30.263s, 17.461 MB/s

Second spents 49.415s, 10.691 MB/s

Third spents 54.724s, 9.653 MB/s

Fourth spents 86.473s, 6.108 MB/s

Fifth spents 97.919s, 5.417 MB/s

10 Thread Testing:

/data/swift/upload/test100m> swift-ahttp://192.168.1.211:8090/auth/v1.0-u system:root-k Testpass upload Container1 Test1.zip

Test1.zip

/data/swift/upload/test100m> swift-ahttp://192.168.1.211:8090/auth/v1.0-u system:root-k Testpass upload Container2 Test2.zip

Object PUT failed:http://192.168.1.211:8090:8090/v1/auth_system/container2/test2.zip 404 Not Found [first chars of RE Sponse]

/data/swift/upload/test100m> swift-ahttp://192.168.1.211:8090/auth/v1.0-u system:root-k Testpass upload Container3 Test3.zip

Test3.zip

/data/swift/upload/test100m> swift-ahttp://192.168.1.211:8090/auth/v1.0-u system:root-k Testpass upload Container4 Test4.zip

Object PUT failed:http://192.168.1.211:8090:8090/v1/auth_system/container4/test4.zip 503 Internal Server Error [First Chars of response]

/data/swift/upload/test100m> swift-ahttp://192.168.1.211:8090/auth/v1.0-u system:root-k Testpass upload Container5 Test5.zip

Object PUT failed:http://192.168.1.211:8090:8090/v1/auth_system/container5/test5.zip 404 Not Found [first chars of RE Sponse]

/data/swift/upload/test100m> swift-ahttp://192.168.1.211:8090/auth/v1.0-u system:root-k Testpass upload Container6 Test6.zip

Object PUT failed:http://192.168.1.211:8090:8090/v1/auth_system/container6/test6.zip 404 Not Found [first chars of RE Sponse]

/data/swift/upload/test100m> swift-ahttp://192.168.1.211:8090/auth/v1.0-u system:root-k Testpass upload Container7 Test7.zip

Object PUT failed:http://192.168.1.211:8090:8090/v1/auth_system/container7/test7.zip 404 Not Found [first chars of RE Sponse]

/data/swift/upload/test100m> swift-ahttp://192.168.1.211:8090/auth/v1.0-u system:root-k Testpass upload Container8 Test8.zip

Object PUT failed:http://192.168.1.211:8090:8090/v1/auth_system/container8/test8.zip 404 Not Found [first chars of RE Sponse]

/data/swift/upload/test100m> swift-ahttp://192.168.1.211:8090/auth/v1.0-u system:root-k Testpass upload Container9 Test9.zip

Object PUT failed:http://192.168.1.211:8090:8090/v1/auth_system/container9/test9.zip 404 Not Found [first chars of RE Sponse]

/data/swift/upload/test100m> swift-ahttp://192.168.1.211:8090/auth/v1.0-u system:root-k Testpass upload Container10 Test10.zip

Object PUT failed:http://192.168.1.211:8090:8090/v1/auth_system/container10/test10.zip 404 Not Found [first chars of Response]

But only seven uploads were successful.

/data/swift/upload/test100m> swift-ahttp://192.168.1.211:8090/auth/v1.0-u system:root-k testpass list Container1

Test1.zip

/data/swift/upload/test100m> swift-ahttp://192.168.1.211:8090/auth/v1.0-u system:root-k testpass list Container2

/data/swift/upload/test100m> swift-ahttp://192.168.1.211:8090/auth/v1.0-u system:root-k testpass list Container3

Test3.zip

/data/swift/upload/test100m> swift-ahttp://192.168.1.211:8090/auth/v1.0-u system:root-k testpass list container4

Test4.zip

/data/swift/upload/test100m> swift-ahttp://192.168.1.211:8090/auth/v1.0-u system:root-k testpass list Container5

/data/swift/upload/test100m> swift-ahttp://192.168.1.211:8090/auth/v1.0-u system:root-k testpass list Container6

Test6.zip

/data/swift/upload/test100m> swift-ahttp://192.168.1.211:8090/auth/v1.0-u system:root-k testpass list Container7

Test7.zip

/data/swift/upload/test100m> swift-ahttp://192.168.1.211:8090/auth/v1.0-u system:root-k testpass list Container8

/data/swift/upload/test100m> swift-ahttp://192.168.1.211:8090/auth/v1.0-u system:root-k testpass list Container9

Test9.zip

/data/swift/upload/test100m> swift-ahttp://192.168.1.211:8090/auth/v1.0-u system:root-k testpass List Container10

Test10.zip

Upload

First time 00:39s,12m/s

Second Spents 01:01s,8.19m/s

Third Spents 03:37s,2.3m/s

Fourth Spents 06:16s,1.32m/s

Fifth spents 06:16s,1.32m/s

Sixth spents 07:06s,1.17m/s

Seventh Spents 07:06s,1.17m/s

Download

First spents 73.207s, 7.216 MB/s

Second spents 107.437s, 4.916 MB/s

Third spents 107.581s, 4.910 MB/s

Fourth spents 108.457s, 4.870 MB/s

Fifth spents 117.852s, 4.482 MB/s

Sixth spents 121.565s, 4.345 MB/s

Seventh spents 122.600s, 4.308 MB/s

Eighth spents 122.413s, 4.315 MB/s

Nineth spents 145.383s, 3.633 MB/s

Tenth spents 174.677s, 3.024 MB/s

2. Destructive Testing

Test one: After testing a service that closes a storage node, uploading the file will not synchronize after the storage service is turned on

View a service for a storage node

/srv/node/sdb1> Ps-ef | grep Swift

Swift 5651 10735 5 17:16? 00:01:10/usr/bin/python/usr/bin/swift-object-updater/etc/swift/object-server.conf

Swift 8409 10727 3 17:38? 00:00:01/usr/bin/python/usr/bin/swift-object-auditor/etc/swift/object-server.conf

WAS_WCM 8477 12319 0 17:38 pts/0 00:00:00 grep Swift

Swift 10723 1 0 Mar28? 00:00:50/usr/bin/python/usr/bin/swift-container-updater/etc/swift/container-server.conf

Swift 10724 1 0 Mar28? 00:00:05/usr/bin/python/usr/bin/swift-account-auditor/etc/swift/account-server.conf

Swift 10725 1 0 Mar28? 03:18:56/usr/bin/python/usr/bin/swift-object-replicator/etc/swift/object-server.conf

Swift 10726 1 1 Mar28? 04:23:53/usr/bin/python/usr/bin/swift-container-replicator/etc/swift/container-server.conf

Swift 10727 1 0 Mar28? 00:08:35/usr/bin/python/usr/bin/swift-object-auditor/etc/swift/object-server.conf

Swift 10728 1 0 Mar28? 00:01:47/usr/bin/python/usr/bin/swift-object-expirer/etc/swift/object-expirer.conf

Swift 10729 1 0 Mar28? 00:00:04/usr/bin/python/usr/bin/swift-container-auditor/etc/swift/container-server.conf

Swift 10730 1 0 Mar28? 00:00:00/usr/bin/python/usr/bin/swift-container-server/etc/swift/container-server.conf

Swift 10731 1 0 Mar28? 00:00:00/usr/bin/python/usr/bin/swift-account-server/etc/swift/account-server.conf

Swift 10732 1 0 Mar28? 00:00:02/usr/bin/python/usr/bin/swift-account-reaper/etc/swift/account-server.conf

Swift 10733 1 0 Mar28? 00:00:24/usr/bin/python/usr/bin/swift-container-sync/etc/swift/container-server.conf

Swift 10734 1 0 Mar28? 00:06:13/usr/bin/python/usr/bin/swift-account-replicator/etc/swift/account-server.conf

Swift 10735 1 0 Mar28? 00:00:06/usr/bin/python/usr/bin/swift-object-updater/etc/swift/object-server.conf

Swift 10736 1 0 Mar28? 00:00:00/usr/bin/python/usr/bin/swift-object-server/etc/swift/object-server.conf

Swift 10795 10730 0 Mar28? 00:18:29/usr/bin/python/usr/bin/swift-container-server/etc/swift/container-server.conf

Swift 10796 10730 0 Mar28? 00:17:56/usr/bin/python/usr/bin/swift-container-server/etc/swift/container-server.conf

Swift 10797 10736 0 Mar28? 02:58:11/usr/bin/python/usr/bin/swift-object-server/etc/swift/object-server.conf

Swift 10798 10736 0 Mar28? 03:16:19/usr/bin/python/usr/bin/swift-object-server/etc/swift/object-server.conf

Swift 10799 10731 0 Mar28? 00:04:15/usr/bin/python/usr/bin/swift-account-server/etc/swift/account-server.conf

Swift 10800 10731 0 Mar28? 00:04:30/usr/bin/python/usr/bin/swift-account-server/etc/swift/account-server.conf

Kill Service

# kill-9 ' Ps-ef | Grep-v grep | grep Object-server | awk ' {print $} '

Uploading files

Swift-ahttp://192.168.1.211:8090/auth/v1.0-u system:root-k testpass upload container1 apache_install

Comparison of three storage files before starting the service

[Email protected]:/srv/node/sdb1/objects> ll | Wc-l

21st

[Email protected]:/srv/node/sdb1/objects> ll | Wc-l

Start the service

[Email protected]:/srv/node/sdb1/objects> sudo swift-init all restart

[Email protected]:/srv/node/sdb1/objects> ll | Wc-l

You can see that the files are synchronized

Test Two: Test Delete a storage node the files on the physical server will not synchronize files on other storage nodes afterwards

Delete a file for a storage node

[Email protected]:/srv/node/sdb1/objects> RM-RF *

View sync changes for files