swift分布式儲存效能測試以及破壞性測試

最後更新：2017-04-23 來源：互聯網

上載者：User

創建阿里雲帳戶，並獲得超過 40 款產品的免費試用版；而企業帳戶則可以享有總值 $1200 的免費試用版。立即註冊！

標籤：openstack-swift 測速破壞性測試

延續上一個文章的做接下來的測試，環境是一個proxy和三個storage。

1、效能測試

測試一：五個線程，每個線程上傳一個大小差不多一個G的檔案

/data/swift/upload> du -sh test*.zip

987M test1.zip

987M test2.zip

987M test3.zip

987M test4.zip

987M test.zip

上傳

swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass upload container1 test.zip

swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass upload container2 test1.zip

swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass upload container3 test2.zip

swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass upload container4 test3.zip

swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass upload container5 test4.zip

共用時8:30:45

儲存節點查看檔案

/srv/node/sdb1> du -sh *

16K accounts

356M async_pending

129M containers

4.9G objects

0 tmp

查看上傳的檔案

/data/swift/upload> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass list container1

test.zip

/data/swift/upload> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass list container2

test1.zip

/data/swift/upload> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass list container3

test2.zip

/data/swift/upload> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass list container4

test3.zip

/data/swift/upload> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass list container5

test4.zip

測試二，十個線程：

/data/swift/upload/test100M> du -sh *

505M test10.zip

505M test1.zip

505M test2.zip

505M test3.zip

505M test4.zip

505M test5.zip

505M test6.zip

505M test7.zip

505M test8.zip

505M test9.zip

單個線程測試上傳2.6M/s,下載5.1M/s

五線程測試：

上傳

第一個用時03:20s,2.5M/s

第二個用時04:30s,1.85M/s

第三個用時05:50s,1.31M/s

第四個用時06:55s,1.2M/s

第五個用時07:00s,1.19M/s

下載

第一個用時30.263s, 17.461 MB/s

第二個用時49.415s, 10.691 MB/s

第三個用時54.724s, 9.653 MB/s

第四個用時86.473s, 6.108 MB/s

第五個用時97.919s, 5.417 MB/s

十線程測試：

/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass upload container1 test1.zip

test1.zip

/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass upload container2 test2.zip

Object PUT failed: http://192.168.1.211:8090:8090/v1/AUTH_system/container2/test2.zip 404 Not Found [first 60 chars of response] <html><h1>Not Found</h1><p>The resource could not be found.<

/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass upload container3 test3.zip

test3.zip

/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass upload container4 test4.zip

Object PUT failed: http://192.168.1.211:8090:8090/v1/AUTH_system/container4/test4.zip 503 Internal Server Error [first 60 chars of response] <html><h1>Service Unavailable</h1><p>The server is currently

/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass upload container5 test5.zip

Object PUT failed: http://192.168.1.211:8090:8090/v1/AUTH_system/container5/test5.zip 404 Not Found [first 60 chars of response] <html><h1>Not Found</h1><p>The resource could not be found.<

/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass upload container6 test6.zip

Object PUT failed: http://192.168.1.211:8090:8090/v1/AUTH_system/container6/test6.zip 404 Not Found [first 60 chars of response] <html><h1>Not Found</h1><p>The resource could not be found.<

/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass upload container7 test7.zip

Object PUT failed: http://192.168.1.211:8090:8090/v1/AUTH_system/container7/test7.zip 404 Not Found [first 60 chars of response] <html><h1>Not Found</h1><p>The resource could not be found.<

/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass upload container8 test8.zip

Object PUT failed: http://192.168.1.211:8090:8090/v1/AUTH_system/container8/test8.zip 404 Not Found [first 60 chars of response] <html><h1>Not Found</h1><p>The resource could not be found.<

/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass upload container9 test9.zip

Object PUT failed: http://192.168.1.211:8090:8090/v1/AUTH_system/container9/test9.zip 404 Not Found [first 60 chars of response] <html><h1>Not Found</h1><p>The resource could not be found.<

/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass upload container10 test10.zip

Object PUT failed: http://192.168.1.211:8090:8090/v1/AUTH_system/container10/test10.zip 404 Not Found [first 60 chars of response] <html><h1>Not Found</h1><p>The resource could not be found.<

但是只有七個上傳成功

/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass list container1

test1.zip

/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass list container2

/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass list container3

test3.zip

/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass list container4

test4.zip

/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass list container5

/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass list container6

test6.zip

/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass list container7

test7.zip

/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass list container8

/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass list container9

test9.zip

/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass list container10

test10.zip

上傳

第一個用時00:39s,12M/s

第二個用時01:01s,8.19M/s

第三個用時03:37s,2.3M/s

第四個用時06:16s,1.32M/s

第五個用時06:16s,1.32M/s

第六個用時07:06s,1.17M/s

第七個用時07:06s,1.17M/s

下載

第一個用時73.207s, 7.216 MB/s

第二個用時107.437s, 4.916 MB/s

第三個用時107.581s, 4.910 MB/s

第四個用時108.457s, 4.870 MB/s

第五個用時117.852s, 4.482 MB/s

第六個用時121.565s, 4.345 MB/s

第七個用時122.600s, 4.308 MB/s

第八個用時122.413s, 4.315 MB/s

第九個用時145.383s, 3.633 MB/s

第十個用時174.677s, 3.024 MB/s

2、破壞性測試

測試一：測試關閉一個儲存節點的服務之後上傳檔案會不會在開啟該儲存服務後同步

查看一個儲存節點的服務

/srv/node/sdb1> ps -ef | grep swift

swift 5651 10735 5 17:16 ? 00:01:10 /usr/bin/python /usr/bin/swift-object-updater /etc/swift/object-server.conf

swift 8409 10727 3 17:38 ? 00:00:01 /usr/bin/python /usr/bin/swift-object-auditor /etc/swift/object-server.conf

was_wcm 8477 12319 0 17:38 pts/0 00:00:00 grep swift

swift 10723 1 0 Mar28 ? 00:00:50 /usr/bin/python /usr/bin/swift-container-updater /etc/swift/container-server.conf

swift 10724 1 0 Mar28 ? 00:00:05 /usr/bin/python /usr/bin/swift-account-auditor /etc/swift/account-server.conf

swift 10725 1 0 Mar28 ? 03:18:56 /usr/bin/python /usr/bin/swift-object-replicator /etc/swift/object-server.conf

swift 10726 1 1 Mar28 ? 04:23:53 /usr/bin/python /usr/bin/swift-container-replicator /etc/swift/container-server.conf

swift 10727 1 0 Mar28 ? 00:08:35 /usr/bin/python /usr/bin/swift-object-auditor /etc/swift/object-server.conf

swift 10728 1 0 Mar28 ? 00:01:47 /usr/bin/python /usr/bin/swift-object-expirer /etc/swift/object-expirer.conf

swift 10729 1 0 Mar28 ? 00:00:04 /usr/bin/python /usr/bin/swift-container-auditor /etc/swift/container-server.conf

swift 10730 1 0 Mar28 ? 00:00:00 /usr/bin/python /usr/bin/swift-container-server /etc/swift/container-server.conf

swift 10731 1 0 Mar28 ? 00:00:00 /usr/bin/python /usr/bin/swift-account-server /etc/swift/account-server.conf

swift 10732 1 0 Mar28 ? 00:00:02 /usr/bin/python /usr/bin/swift-account-reaper /etc/swift/account-server.conf

swift 10733 1 0 Mar28 ? 00:00:24 /usr/bin/python /usr/bin/swift-container-sync /etc/swift/container-server.conf

swift 10734 1 0 Mar28 ? 00:06:13 /usr/bin/python /usr/bin/swift-account-replicator /etc/swift/account-server.conf

swift 10735 1 0 Mar28 ? 00:00:06 /usr/bin/python /usr/bin/swift-object-updater /etc/swift/object-server.conf

swift 10736 1 0 Mar28 ? 00:00:00 /usr/bin/python /usr/bin/swift-object-server /etc/swift/object-server.conf

swift 10795 10730 0 Mar28 ? 00:18:29 /usr/bin/python /usr/bin/swift-container-server /etc/swift/container-server.conf

swift 10796 10730 0 Mar28 ? 00:17:56 /usr/bin/python /usr/bin/swift-container-server /etc/swift/container-server.conf

swift 10797 10736 0 Mar28 ? 02:58:11 /usr/bin/python /usr/bin/swift-object-server /etc/swift/object-server.conf

swift 10798 10736 0 Mar28 ? 03:16:19 /usr/bin/python /usr/bin/swift-object-server /etc/swift/object-server.conf

swift 10799 10731 0 Mar28 ? 00:04:15 /usr/bin/python /usr/bin/swift-account-server /etc/swift/account-server.conf

swift 10800 10731 0 Mar28 ? 00:04:30 /usr/bin/python /usr/bin/swift-account-server /etc/swift/account-server.conf

殺死服務

# kill -9 `ps -ef | grep -v grep | grep object-server | awk ‘{print $2}‘`

上傳檔案

swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass upload container1 apache_install

啟動服務之前的三個隱藏檔對比

[email protected]:/srv/node/sdb1/objects> ll | wc -l

啟動服務

[email protected]:/srv/node/sdb1/objects> sudo swift-init all restart

[email protected]:/srv/node/sdb1/objects> ll | wc -l

可以看到檔案已經同步

測試二：測試刪除一個儲存節點物理伺服器上的檔案會不會在之後同步其他儲存節點上的檔案

刪除一個儲存節點的檔案

[email protected]:/srv/node/sdb1/objects> rm -rf *

查看檔案的同步變化