使用Keystone認證多節點安裝Swift

轉載請寫明出處：http://blog.csdn.net/cywosp/article/details/7439440注意：下面被標紅色的地方由於部落格編輯器的原因出現了多餘的資訊 1.  摘要說明

本文所涉及到的所有操作都是在Ubuntu Server 11.10 64位系統上通過驗證。本文參考了Openstack keystone的相關文檔，具體資訊如下：

Linux系統版本：Ubuntu Server 11.10 64-bit oneiricProxy Server IP：   192.168.112.129Storage Server One:192.168.112.130Storage Server Two:192.168.112.131Storage Server Three:192.168.112.132Keystone Server IP：192.168.112.133官方文檔：      www.openstack.org參考文檔：      http://keystone.openstack.org/installing.htmlSwift版本：       1.4.8Keystone版本：    2012.2

2.  為每一台機器建立swift使用者

sudo useradd -mk /home/swift/ -s /bin/bash swiftsudo passwd swift#為swift使用者添加密碼，在此我將其設為了swift編輯/etc/sudoer檔案，在檔案末尾添加如下代碼swift ALL=(ALL) NOPASSWD:ALL

3.  下載源碼（在swift使用者下操作）

1.安裝git工具sudo apt-get install git-core2.在Proxy機器中下載keystone和swift源碼su swift#切換到swift使用者sudo mkdir /home/swift/openstack#建立一個目錄來存放cd /home/swift/openstackgit clone https://github.com/openstack/swift.git#下載swiftcd swiftgit checkout 1.4.8#使用1.4.8版本，在swift目錄下你可以#使用git tag命令查看有多少個版本git clone https://github.com/openstack/keystone.git cd keystonegit checkout 75a8dfe3.在每一台Storage節點的機器中下載swiftsu swift#切換到swift使用者sudo mkdir /home/swift/openstack#建立一個目錄來存放cd /home/swift/openstackgit clone https://github.com/openstack/swift.git#下載swiftcd swiftgit checkout 1.4.8#使用1.4.8版本，在swift目錄下你可以#使用git tag命令查看有多少個版本4.在Auth (keystone)節點的機器中下載keystone和python-keystoneclientsu swift#切換到swift使用者sudo mkdir /home/swift/openstack#建立一個目錄來存放cd /home/swift/openstackgit clone https://github.com/openstack/keystone.git cd keystonegit checkout 75a8dfegit clone https://github.com/openstack/python-keystoneclient.gitcd /home/swift/openstack/python-keystoneclient

4.  安裝swift和keystone以及相關依賴包（在swift使用者下操作）

1.所有Storage節點上的安裝sudo apt-get --option Dpkg::Options::=--force-confold --assume-yes updatesudo apt-get install pep8 pylint python-pip screen unzip wget psmisc git-core lsof vim-nox curl python-mysqldbcd /home/swift/openstack/sudo pip install -r ./swift/tools/ pip-requires#安裝swift的相關依賴，這裡可能需要點時間#安裝swiftcd /home/swift/openstack/swiftsudo python setup.py install --record file.txt#假如要刪除所安裝的東西需要用root使用者來刪除，刪除方法：sudo cat file.txt | xargs rm -rf2.Proxy節點上的安裝sudo apt-get --option Dpkg::Options::=--force-confold --assume-yes updatesudo apt-get install pep8 pylint python-pip screen unzip wget psmisc git-core lsof vim-nox curl python-mysqldbcd /home/swift/openstack/sudo pip install -r ./swift/tools/ pip-requirescd /home/swift/openstack/swiftsudo python setup.py install --record file.txtcd /home/swift/openstack/keystonesudo pip install -r ./tools/pip-requiressudo python setup.py install --record file.txt3.Auth(Keystone)節點的安裝sudo apt-get --option Dpkg::Options::=--force-confold --assume-yes updatesudo apt-get install pep8 pylint python-pip screen unzip wget psmisc git-core lsof vim-nox curl python-mysqldb mysql-server mysql-clientcd /home/swift/openstack/sudo pip install -r ./keystone/tools/pip-requiressudo pip install -r ./ python-keystoneclient/tools/pip-requirescd /home/swift/openstack/python-keystoneclient/sudo python setup.py install --record file.txtcd /home/swift/openstack/keystonesudo python setup.py install --record file.txt

5.  Proxy節點的設定（192.168.112.129）

1.sudo apt-get install memcached#安裝快取服務器修改/etc/ memcached.conf檔案，將-l 127.0.0.1改為-l 192.168.112.129（這裡我是根據我自己的情況設定的，具體原因見第一點中的圖）sudo service memcached restartsudo mkdir  /etc/swiftcd  /etc/swiftsudo chown -R swift:swift /etc/swiftcp  /home/swift/openstack/swift/etc/proxy-server.conf /etc/swift/cp  /home/swift/openstack/swift/etc/swift.conf /etc/swift/2.修改/etc/swift/proxy-server.conf檔案，具體內容如下，原檔案中沒有的項需要自行增加[DEFAULT]bind_port = 8080user = swiftswift_dir = /etc/swiftworkers = 1[pipeline:main]pipeline = healthcheck cache swift3 authtoken keystone proxy-server[app:proxy-server]use = egg:swift#proxyallow_account_management = trueaccount_autocreate = true[filter:keystone]paste.filter_factory = keystone.middleware.swift_auth:filter_factoryoperator_roles = Member,admin[filter:authtoken]paste.filter_factory = keystone.middleware.auth_token:filter_factoryauth_host = 192.168.112.133auth_port = 35357auth_protocol = httpauth_uri = http://192.168.112.133:5000/admin_tenant_name = serviceadmin_user = swiftadmin_password = admin[filter:swift3]use = egg:swift#swift3[filter:healthcheck]use = egg:swift#healthcheck[filter:cache]use = egg:swift#memcache192.168.112.133部分為Auth(Keystone)節點的IP3.修改/etc/swift/swift.conf，‘cynric’部分是隨意更改的，你可以根據自己的需要更改[swift-hash]swift_hash_path_suffix = cynric4.產生相關ring以及builder檔案，使用如下命令產生，加粗部分是根據具體情況而更改的，具體原因見摘要說明裡的圖。每一台機器使用一個域（z1, z2, z3…依次遞增）sudo chown -R swift:swift /etc/swift/*cd  /etc/swiftswift-ring-builder object.builder create 18 3 1swift-ring-builder container.builder create 18 3 1swift-ring-builder account.builder create 18 3 1export HOST_IP=192.168.112.130swift-ring-builder object.builder add z1-${HOST_IP}:6010/sdb1 100swift-ring-builder container.builder add z1-${HOST_IP}:6011/sdb1 100swift-ring-builder account.builder add z1-${HOST_IP}:6012/sdb1 100export HOST_IP=192.168.112.131swift-ring-builder object.builder add z2-${HOST_IP}:6010/sdb1 100swift-ring-builder container.builder add z2-${HOST_IP}:6011/sdb1 100swift-ring-builder account.builder add z2-${HOST_IP}:6012/sdb1 100export HOST_IP=192.168.112.132swift-ring-builder object.builder add z3-${HOST_IP}:6010/sdb1 100swift-ring-builder container.builder add z3-${HOST_IP}:6011/sdb1 100swift-ring-builder account.builder add z3-${HOST_IP}:6012/sdb1 100swift-ring-builder object.builder rebalanceswift-ring-builder container.builder rebalanceswift-ring-builder account.builder rebalance5.啟動proxy服務swift-init  proxy  start

6.  配置Storage節點

因為每個Storage節點的設定基本上是相似的，所以在這裡只拿其中一個節點做樣本（192.168.112.130），其他節點只需要重複一下幾步操作就可以了

1.建立/etc/swift目錄sudo mkdir /etc/swiftsudo chown -R swift:swift /etc/swift/*2.將Proxy節點上/etc/swift/中的account.ring.gz container.ring.gzobject.ring.gz  swift.conf拷貝到當前儲存節點（192.168.112.130） /etc/swift目錄中，可使用如下命令scp  swift@192.168.112.129:/etc/swift/*.ring.gz  /etc/swift/scp  swift@192.168.112.129:/etc/swift/swift.conf  /etc/swift/sudo chown -R swift:swift /etc/swift/*3.更改/etc/rsyncd.conf檔案，如果該檔案不存在則需要自行建立，內容如下uid = swiftgid = swiftlog file = /var/log/rsyncd.logpid file = /var/run/rsyncd.pidaddress = 127.0.0.1#這裡也可以改為192.168.112.130[account]max connections = 2path = /srv/node/read only = falselock file = /var/lock/account.lock[conainer]max connections = 2path = /srv/node/read only = falselock file = /var/lock/container.lock[object]max connections = 2path = /srv/node/read only = falselock file = /var/lock/object.lock編輯/etc/default/rsync:將RSYNC_ENABLE設定為true更改好之後，重啟該服務sudo service rsync restart4.儲存點的設定這裡有分兩種情況來設定儲存點a.假設你的系統裡有一個單獨分區，使用此分區來做儲存點，在這裡假設系統中有/dev/sdb1（註：這雷根據你自己系統的情況而定）這個分區未被使用，我們用它來做儲存點。sudo mkdir -p /srv/node/sdb1sudo mkfs.xfs -i size=1024 /dev/sdb1 #以xfs方式格式化分區sudo chmod a+w /etc/fstabsudo echo “/dev/sdb1 /srv/node/sdb1 xfs noatime,nodiratimenobarrier,logbufs=8 0 0” >> /etc/fstab  #系統啟動時自動掛載，這裡的sdb1是一定不能改的，因為在做Proxy節點產生相應的ring檔案時使用了sdb1 (swift-ring-builder object.builder add z1-${HOST_IP}:6010/sdb1 100)的時候，加入需要更改則兩個地方都需要改sudo mount /srv/node/sdb1sudo chown -R swift:swift /srv/node/sdb1sudo chmod a+w -R /srv/node/sdb1b.如果系統裡沒有單獨的分區來做儲存點，則需要建立一個臨時分區來做儲存點sudo mkdir -p /srv/node/sdb1sudo dd if=/dev/zero of=/srv/swift-disk bs=1024 count=0 seek=1000000 #這個命令是在/srv/下建立一個名為swift-disk的儲存區，你可以改變seek的大小來改變swift-disk的大小sudo mkfs.xfs -i size=1024 /srv/swift-disksudo chmod a+w /etc/fstabsudo echo “/srv/swift-disk /srv/node/sdb1 xfs loop,noatime,nodiratime,nobarrier,logbufs=8 0 0” >> /etc/fstab    #系統啟動時自動掛載sudo mount /srv/node/sdb1sudo chown -R swift:swift /srv/node/sdb1sudo chmod a+w -R /srv/node/sdb1sudo chmod a+w /srv/swift-diskc.建立相關的目錄sudo mkdir /var/run/swiftsudo chown swift:swift /var/run/swiftsudo chmod a+w /var/run/swiftd.在/etc/rc.local的exit 0之前加入下列三行mkdir /var/run/swiftchown swift:swift /var/run/swiftchmod a+w /var/run/swift5.Swift檔案配置建立/etc/swift/account-server.conf檔案，並加入如下配置[DEFAULT]devices = /srv/nodemount_check = falsebind_port = 6012user = swiftbind_ip = 0.0.0.0workers = 2[pipeline:main]pipeline = account-server[app:account-server]use = egg:swift#account[account-replicator][account-auditor][account-reaper]建立/etc/swift/object-server.conf檔案[DEFAULT]devices = /srv/nodemount_check = falsebind_port = 6010user = swiftbind_ip = 0.0.0.0workers = 2[pipeline:main]pipeline = object-server[app:object-server]use = egg:swift#object[object-replicator][object-updater][object-auditor]建立/etc/swift/ container-server.conf檔案[DEFAULT]devices = /srv/nodemount_check = falsebind_port = 6011user = swiftbind_ip = 0.0.0.0workers = 2[pipeline:main]pipeline = container-server[app:container-server]use = egg:swift#container[container-replicator][container-updater][container-auditor][container-sync]6.啟動swift服務sudo chown -R swift:swift /etc/swift/*swift-init all start#當啟動的時候可能會報WARNING: Unable to increase file descriptor limit.  Running as non-root?  這是正常情況

7.  Auth(Keystone)節點的配置

sudo mkdir /etc/keystonesudo chown -R swift:swift /etc/keystonecp -r /home/swift/openstack/keystone/etc/* /etc/keystone1.修改/etc/keystone/keystone.conf檔案將connection = sqlite:///keystone.db更改為connection = mysql://keystone:keystone@127.0.0.1/keystone將[identity]下的driver設定成如下driver = keystone.identity.backends.sql.Identity將[catalog]下的driver設定成如下driver = keystone.catalog.backends.sql.Catalog其他的保持不變即可2.Mysql的設定mysql -u root -p#以root身份登入mysql資料庫在資料庫中做如下操作CREATE DATABASE keystone;GRANT ALL ON keystone.* TO 'keystone'@'%' IDENTIFIED BY 'keystone';commit;修改/etc/mysql/my.conf檔案將bind-address = 127.0.0.1改為bind-address = 0.0.0.0重啟mysql服務sudo service mysql restart3.同步資料庫建立相應的資料庫表keystone-manage db_sync#執行成功之後，在mysql的keystone資料庫中將會建立一下表，你可以登陸資料庫查看+------------------------+| Tables_in_keystone     |+------------------------+| ec2_credential         || endpoint               || metadata               || migrate_version        || role                   || service                || tenant                 || token                  || user                   || user_tenant_membership |+------------------------+4.建立相應的keystone使用者以及keystone服務端點#!/usr/bin/env bashADMIN_PASSWORD=adminENABLE_SWIFT=1ENABLE_ENDPOINTS=1KEYSTONE_CONF=${KEYSTONE_CONF:-/etc/keystone/keystone.conf}SERVICE_PASSWORD=${SERVICE_PASSWORD:-$ADMIN_PASSWORD}# Extract some info from Keystone's configuration fileif [[ -r "$KEYSTONE_CONF" ]]; then    CONFIG_SERVICE_TOKEN=$(sed 's/[[:space:]]//g' $KEYSTONE_CONF | grep ^admin_token= | cut -d'=' -f2)    CONFIG_ADMIN_PORT=$(sed 's/[[:space:]]//g' $KEYSTONE_CONF | grep ^admin_port= | cut -d'=' -f2)fiexport SERVICE_TOKEN=${SERVICE_TOKEN:-$CONFIG_SERVICE_TOKEN}if [[ -z "$SERVICE_TOKEN" ]]; then    echo "No service token found."    echo "Set SERVICE_TOKEN manually from keystone.conf admin_token."    exit 1fiexport SERVICE_ENDPOINT=${SERVICE_ENDPOINT:-http://127.0.0.1:${CONFIG_ADMIN_PORT:-35357}/v2.0}function get_id () {    echo `"$@" | grep ' id ' | awk '{print $4}'`}# TenantsADMIN_TENANT=$(get_id keystone tenant-create --name=admin)SERVICE_TENANT=$(get_id keystone tenant-create --name=service)DEMO_TENANT=$(get_id keystone tenant-create --name=demo)# UsersADMIN_USER=$(get_id keystone user-create --name=admin \                                         --pass="$ADMIN_PASSWORD" \                                         --email=admin@example.com)DEMO_USER=$(get_id keystone user-create --name=demo \                                        --pass="$ADMIN_PASSWORD" \                                        --email=admin@example.com)# RolesADMIN_ROLE=$(get_id keystone role-create --name=admin)MEMBER_ROLE=$(get_id keystone role-create --name=Member)KEYSTONEADMIN_ROLE=$(get_id keystone role-create --name=KeystoneAdmin)KEYSTONESERVICE_ROLE=$(get_id keystone role-create --name=KeystoneServiceAdmin)SYSADMIN_ROLE=$(get_id keystone role-create --name=sysadmin)# Add Roles to Users in Tenantskeystone user-role-add --user $ADMIN_USER --role $ADMIN_ROLE --tenant_id $ADMIN_TENANTkeystone user-role-add --user $DEMO_USER --role $MEMBER_ROLE --tenant_id $DEMO_TENANTkeystone user-role-add --user $DEMO_USER --role $SYSADMIN_ROLE --tenant_id $DEMO_TENANTkeystone user-role-add --user $ADMIN_USER --role $ADMIN_ROLE --tenant_id $DEMO_TENANT# TODO(termie): these two might be dubiouskeystone user-role-add --user $ADMIN_USER --role $KEYSTONEADMIN_ROLE --tenant_id $ADMIN_TENANTkeystone user-role-add --user $ADMIN_USER --role $KEYSTONESERVICE_ROLE --tenant_id $ADMIN_TENANT# ServicesKEYSTONE_SERVICE=$(get_id \keystone service-create --name=keystone \                        --type=identity \                        --description="Keystone Identity Service")if [[ -n "$ENABLE_ENDPOINTS" ]]; then    keystone endpoint-create --region RegionOne --service_id $KEYSTONE_SERVICE \        --publicurl 'http://localhost:$(public_port)s/v2.0' \        --adminurl 'http://localhost:$(admin_port)s/v2.0' \        --internalurl 'http://localhost:$(admin_port)s/v2.0'fiif [[ -n "$ENABLE_SWIFT" ]]; then    SWIFT_SERVICE=$(get_id keystone service-create --name=swift \                            --type="object-store" \                            --description="Swift Service")    SWIFT_USER=$(get_id keystone user-create --name=swift \                                             --pass="$SERVICE_PASSWORD" \                                             --tenant_id $SERVICE_TENANT \                                             --email=swift@example.com)    keystone user-role-add --tenant_id $SERVICE_TENANT \                           --user $SWIFT_USER \                           --role $ADMIN_ROLE    keystone endpoint-create --region RegionOne --service_id $SWIFT_SERVICE \        --publicurl 'http://192.168.112.129:8080/v1/AUTH_$(tenant_id)s' \        --adminurl 'http://192.168.112.129:8080/' \        --internalurl 'http://192.168.112.129:8080/v1/AUTH_$(tenant_id)s'fi                                               將以上shell代碼拷貝到一個檔案中，然後執行（在Auth(Keystone)主機中）。其建立了以下主要關係的資料：Tenant               User      Roles             password-----------------------------------------------------------admin                admin     admin             adminservice              swift     admin             admin       demo                 admin     admin             admindemo                 demo      Member,sysadmin   admin

注意：在建立swift的endpoint時，各個url所指向的必須是Proxy節點，   例如上面IP地址（192.168.112.129）。如果有多個Proxy節點則需要加入多個endpoint。

8.  開啟各個節點的服務（swift使用者下操作）

Proxy節點：swift-init  proxy  start

各個Storage節點：swift-init  all  start

Auth(Keystone)節點：

    sudo  screen  -S  keystone  #建立一個名為keystone的臨時終端，這樣 可以隱藏多餘的列印資訊

    su  swift   #切換到swift使用者

    keystone-all   #這裡會輸出很多資訊，調試的時候可以用到

    迅速按下Ctrl+a  Ctrl+d鍵，此時會返回類似於這樣的資訊[detached from
4334.key]，記住紅色部分的編號，要想恢複原來keystone臨時終   端時可以使用命令：sudo screen -r 4334

9.  驗證與使用

a.  驗證整個儲存架構是否成功（在Proxy節點上或者安裝了swift的節點上操作）

swift  -A  http://192.168.112.133:5000/v2.0 -U admin -K admin stat -V 2

執行成功會返回類似如下的資訊：

Account:AUTH_308722b8cc8747a5afdd9b7b1f6155e8

Containers:0

Objects:0

Bytes:0

Accept-Ranges:bytes

b.  用curl測試

curl -d '{"auth": {"tenantName": "admin", "passwordCredentials":{"username": "admin", "password": "admin"}}}' -H "Content-type: application/json" http://192.168.112.133:35357/v2.0/tokens | python -mjson.tool