CentOS上安裝MooseFSDistributed File System2種方式

方法一

概述：
MooseFS是一種Distributed File System，MooseFS檔案系統結構包括以下四種角色：
　　1 管理伺服器managing server (master)
　　2 中繼資料Log Service器Metalogger server（Metalogger）
　　3 資料存放區伺服器data servers (chunkservers)
　　4 客戶機掛載使用client computers

 

各種角色作用:
　　 1 管理伺服器:負責各個資料存放區伺服器的管理,檔案讀寫調度,檔案空間回收以及恢複.多節點拷貝
　　 2 中繼資料Log Service器: 負責備份master伺服器的變化記錄檔，檔案類型為changelog_ml.*.mfs，以便於在master server出問題的時候接替其進行工作
　　 3 資料存放區伺服器:負責串連管理伺服器,聽從管理伺服器調度,提供儲存空間，並為客戶提供資料轉送.
　　 4 用戶端: 通過fuse核心介面掛接遠端管理伺服器上所管理的資料存放區伺服器,.看起來共用的檔案系統和本地unix檔案系統使用一樣的效果.

安裝環境
Master Server
master               10.0.0.141    Centos5.5(64X)

Metalogger Server
metalogger           10.0.0.142    Centos5.5(64X)

Chunk Servers
chunk01              10.0.0.143    Centos5.5(64X)
chunk02              10.0.0.144    Centos5.5(64X)

Client Server
client01              10.0.0.150    Centos5.5(64X)
client02              10.0.0.155    Centos5.5(64X)

—————————————————-
master 安裝(10.0.0.141)

 

mkdir -p /root/tools/moosefs/
cd /root/tools/moosefs/

/usr/sbin/groupadd -g 660 mfs
/usr/sbin/useradd -g mfs mfs -u 660 -s /sbin/nologin

wget http://www.moosefs.org/tl_files/mfscode/mfs-1.6.20-2.tar.gz
tar zxvf mfs-1.6.20-2.tar.gz
cd mfs-1.6.20-2
./configure
–prefix=/elain/apps/mfs
–sysconfdir=/elain/apps/mfs/etc
–localstatedir=/elain/apps/mfs/lib
–with-default-user=mfs
–with-default-group=mfs
–disable-mfschunkserver
–disable-mfsmount
make && make install
cd ..

cd /elain/apps/mfs/etc/
cp mfsmaster.cfg.dist mfsmaster.cfg
cp mfsmetalogger.cfg.dist mfsmetalogger.cfg
cp mfsexports.cfg.dist mfsexports.cfg

vi mfsexports.cfg

10.0.0.0/24             /       rw,alldirs,maproot=0

cd /elain/apps/mfs/lib/mfs
cp metadata.mfs.empty metadata.mfs

vi /etc/hosts

10.0.0.141         mfsmaster

運行mfs服務
/elain/apps/mfs/sbin/mfsmaster start

master metaloggers module: listen on *:9419
master chunkservers module: listen on *:9420
main master server module: listen on *:9421

運行監控平台
/elain/apps/mfs/sbin/mfscgiserv

starting simple cgi server (host: any , port: 9425 , rootpath: /elain/apps/mfs/share/mfscgi)

這時，可通過瀏覽器訪問：http://10.0.0.141:9425/  查看 master 的運行情況

—————————————————–
metalogger   安裝(10.0.0.142)

cd /root/tools/
/usr/sbin/groupadd -g 660 mfs
/usr/sbin/useradd -g mfs mfs -u 660 -s /sbin/nologin

wget http://www.moosefs.org/tl_files/mfscode/mfs-1.6.20-2.tar.gz
tar zxvf mfs-1.6.20-2.tar.gz
cd mfs-1.6.20-2
./configure
–prefix=/elain/apps/mfs
–sysconfdir=/elain/apps/mfs/etc
–localstatedir=/elain/apps/mfs/lib
–with-default-user=mfs
–with-default-group=mfs
–disable-mfschunkserver
–disable-mfsmount
make && make install
cd ..

cd /elain/apps/mfs/etc/
cp mfsmetalogger.cfg.dist mfsmetalogger.cfg

vi /etc/hosts

10.0.0.141         mfsmaster

/elain/apps/mfs/sbin/mfsmetalogger start

—————————————————-
Chunk01  安裝(10.0.0.143)

cd /root/tools/

/usr/sbin/groupadd -g 660 mfs
/usr/sbin/useradd -g mfs mfs -u 660 -s /sbin/nologin

wget http://www.moosefs.org/tl_files/mfscode/mfs-1.6.20-2.tar.gz
tar zxvf mfs-1.6.20-2.tar.gz
cd mfs-1.6.20-2
./configure
–prefix=/elain/apps/mfs
–sysconfdir=/elain/apps/mfs/etc
–localstatedir=/elain/apps/mfs/lib
–with-default-user=mfs
–with-default-group=mfs
–disable-mfsmaster
make && make install
cd ..

cd /elain/apps/mfs/etc/
cp mfschunkserver.cfg.dist mfschunkserver.cfg
cp mfshdd.cfg.dist mfshdd.cfg

vi /etc/hosts

10.0.0.141         mfsmaster

mkdir /data/mfschunks{1,2}
chown -R mfs:mfs /data/mfschunks{1,2}

vi /elain/apps/mfs/etc/mfshdd.cfg

/data/mfschunks1
/data/mfschunks2

/elain/apps/mfs/sbin/mfschunkserver start

————————————————-
Chunk02  安裝(10.0.0.144)
cd  /root/tools/
/usr/sbin/groupadd -g 660 mfs
/usr/sbin/useradd -g mfs mfs -u 660 -s /sbin/nologin

wget http://www.moosefs.org/tl_files/mfscode/mfs-1.6.20-2.tar.gz
tar zxvf mfs-1.6.20-2.tar.gz
cd mfs-1.6.20-2
./configure
–prefix=/elain/apps/mfs
–sysconfdir=/elain/apps/mfs/etc
–localstatedir=/elain/apps/mfs/lib
–with-default-user=mfs
–with-default-group=mfs
–disable-mfsmaster
make && make install
cd ..

cd /elain/apps/mfs/etc/
cp mfschunkserver.cfg.dist mfschunkserver.cfg
cp mfshdd.cfg.dist mfshdd.cfg

vi /etc/hosts

10.0.0.141         mfsmaster

mkdir /data/mfschunks{1,2}
chown -R mfs:mfs /data/mfschunks{1,2}

vi /elain/apps/mfs/etc/mfshdd.cfg

/data/mfschunks1
/data/mfschunks2

/elain/apps/mfs/sbin/mfschunkserver start

———————————————————
Client01 Server安裝 (10.0.0.150)

cd /root/tools/
/usr/sbin/groupadd -g 660 mfs
/usr/sbin/useradd -g mfs mfs -u 660 -s /sbin/nologin

wget http://cdnetworks-kr-2.dl.sourceforge.net/project/fuse/fuse-2.X/2.8.5/fuse-2.8.5.tar.gz
tar -zxvf fuse-2.8.5.tar.gz
cd fuse-2.8.5
./configure
make
make install
cd ..

運行以下export命令，否則掛載moosefs系統會失敗

vi /etc/profile
export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:$PKG_CONFIG_PATH
source /etc/profile

cd /root/tools/moosefs/
wget http://www.moosefs.org/tl_files/mfscode/mfs-1.6.20-2.tar.gz
tar -zxvf mfs-1.6.20-2.tar.gz
cd mfs-1.6.20-2
./configure
–prefix=/elain/apps/mfs
–sysconfdir=/elain/apps/mfs/etc
–localstatedir=/www/lib
–with-default-user=mfs
–with-default-group=mfs
–enable-mfsmount
make && make install
cd ..

vi /etc/hosts

10.0.0.141         mfsmaster

掛接操作
mkdir -p /elain/data/htdocs/elain/upload
/elain/apps/mfs/bin/mfsmount /elain/data/htdocs/elain/upload/ -H mfsmaster
umount -l /elain/data/htdocs/elain/upload

——————————————————-
Client02 Server安裝 (10.0.0.155)
cd /root/tools/
/usr/sbin/groupadd -g 660 mfs
/usr/sbin/useradd -g mfs mfs -u 660 -s /sbin/nologin

wget http://cdnetworks-kr-2.dl.sourceforge.net/project/fuse/fuse-2.X/2.8.5/fuse-2.8.5.tar.gz
tar zxvf fuse-2.8.5.tar.gz
cd fuse-2.8.5
./configure
make
make install
cd ..

運行以下export命令，否則掛載moosefs系統會失敗

vi /etc/profile
export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:$PKG_CONFIG_PATH
source /etc/profile

#載入fuse
modprobe fuse

[root@web1 ~#]lsmod |grep fuse                                                                                           
fuse                   83057  8

wget http://www.moosefs.org/tl_files/mfscode/mfs-1.6.20-2.tar.gz
tar -zxvf mfs-1.6.20-2.tar.gz
cd mfs-1.6.20-2
./configure
–prefix=/elain/apps/mfs
–sysconfdir=/elain/apps/mfs/etc
–localstatedir=/elain/apps/mfs/lib
–with-default-user=mfs
–with-default-group=mfs
–enable-mfsmount
make && make install
cd ..

vi /etc/hosts

10.0.0.141         mfsmaster

掛接操作
mkdir -p /mnt/mfs
/elain/apps/mfs/bin/mfsmount /mnt/mfs -H mfsmaster

安裝完畢。

停止 MooseFS
為了安全停止 MooseFS 叢集,建議執行如下的步驟:

umount -l /mnt/mfs                                                  #用戶端卸載MooseFS 檔案系統
/elain/apps/mfs/sbin/mfschunkserver stop                            #停止 chunk server 進程
/elain/apps/mfs/sbin/mfsmetalogger stop                             #停止 metalogger 進程
/elain/apps/mfs/sbin/mfsmaster stop                                 #停止主控 master server 進程

安全的啟動MooseFS叢集

/elain/apps/mfs/sbin/mfsmaster start                                #啟動 master 進程
/elain/apps/mfs/sbin/mfschunkserver start                           #啟動 chunkserver 進程
/elain/apps/mfs/sbin/mfsmetalogger start                            #啟動 metalogger 進程
/elain/apps/mfs/bin/mfsmount /mnt/mfs -H  master                    #用戶端掛載 MooseFS 檔案系統
 

MooseFS 命令的使用方法及參考文檔：
http://www.moosefs.org/reference-guide.html#using-moosefs

http://www.moosefs.org/moosefs-faq.html

FAQ:
/elain/apps/mfs/bin/mfsmount /elain/data/htdocs/elain/upload -H master
mfsmaster accepted connection with parameters: read-write,restricted_ip ; root mapped to root:root
fuse: device not found, try 'modprobe fuse' first
error in fuse_mount
答：modprobe fuse
[root@client01 ~]# lsmod |grep fuse
fuse                   83057  6

 

方法二

環境介紹：
OS: CentOS 6.3 x86_64
server1 - mfsmaster & mfschunkserver
server2 - mfsmetalogger & mfschunkserver & mfsclient
server3 - mfschunkserver

1. 什麼是MooseFS
MooseFS是一個具有容錯能力的Distributed File System，支援掛載(通過FUSE)和與本地檔案系統相同的操作。

MooseFS具有以下幾個角色：
主伺服器：Managing Server(master server)
資料節點：Data server(chunk servers)
備份伺服器：Metadata backup servers(metalogger servers)
用戶端：Client computers that access(mount) the files in MooseFS

2. 安裝配置主伺服器(僅在server1上)
sudo rpm --import http://apt.sw.be/RPM-GPG-KEY.dag.txt
sudo rpm -ivh http://pkgs.repoforge.org/rpmforge-release/rpmforge-release-0.5.3-1.el6.rf.x86_64.rpm
sudo yum install mfs

cd /etc/mfs
sudo cp mfsmaster.cfg.dist mfsmaster.cfg
sudo cp mfsmetalogger.cfg.dist mfsmetalogger.cfg
sudo cp mfsexports.cfg.dist mfsexports.cfg

修改主伺服器的地址，比如10.197.29.251
sudo vim /etc/mfsmetalogger.cfg

1 MASTER_HOST = 10.197.29.251
初始化資料檔案
cd /var/mfs
sudo cp metadata.mfs.empty metadata.mfs

安裝WEB介面支援
sudo yum install mfs-cgi httpd
sudo vim /var/www/html/mfs/index.html

1 document.location.href="/cgi-bin/mfs/mfs.cgi"
sudo vim /etc/httpd/conf/httpd.conf

1 Listen 9480

sudo /etc/init.d/mfsmaster restart
sudo /etc/init.d/httpd restart

訪問WEB介面

http://server1:9480/mfs/

如下圖所示

3. 安裝配置備份伺服器(僅在server2上)
sudo rpm --import http://apt.sw.be/RPM-GPG-KEY.dag.txt
sudo rpm -ivh http://packages.sw.be/rpmforge-release/rpmforge-release-0.5.2-2.el5.rf.x86_64.rpm

sudo yum install mfs

cd /etc/mfs
sudo cp mfsmetalogger.cfg.dist mfsmetalogger.cfg

sudo vim /etc/mfsmetalogger.cfg

1 MASTER_HOST = 10.197.29.251
sudo /etc/init.d/mfsmetalogger restart

4. 安裝配置資料節點(在server1-3上)
sudo rpm --import http://apt.sw.be/RPM-GPG-KEY.dag.txt
sudo rpm -ivh http://packages.sw.be/rpmforge-release/rpmforge-release-0.5.2-2.el5.rf.x86_64.rpm

sudo yum install mfs

cd /etc/mfs
sudo cp mfschunkserver.cfg.dist mfschunkserver.cfg
sudo cp mfshdd.cfg.dist mfshdd.cfg

sudo vim mfschunkserver.cfg

1 MASTER_HOST = 10.197.29.251
sudo vim mfshdd.cfg

1 /mnt/mfschunks1 

2 /mnt/mfschunks2
sudo mkdir /mnt/mfschunks1 /mnt/mfschunks2
sudo chown -R daemon:daemon /mnt/mfschunks*

sudo /etc/init.d/mfschunkserver restart

訪問WEB介面，檢查新增的chunkserver

http://server1:9480/cgi-bin/mfs/mfs.cgi?sections=CS

5. 安裝配置用戶端(在server2-3上)
sudo rpm --import http://apt.sw.be/RPM-GPG-KEY.dag.txt
sudo rpm -ivh http://packages.sw.be/rpmforge-release/rpmforge-release-0.5.2-2.el5.rf.x86_64.rpm

sudo yum install mfs-client

sudo mkdir -p /mnt/mfsmount
sudo chown -R daemon:daemon /mnt/mfsmount

sudo mfsmount /mnt/mfsmount -H 10.197.29.251

1 mfsmaster accepted connection with parameters: read-write,restricted_ip ; root mapped to root:root
df -h | grep mfs

1 10.197.29.251:9421         25G     0   25G   0% /mnt/mfs
6. 測試MooseFS
在server2上建立檔案server2.txt：
cd /mnt/mfsmount
echo "I am from server2" > server2.txt

在server3上檢查檔案是否存在：
cd /mnt/mfsmount

ls

1 server2.txt