After Ceph deployment RBD block device read and write only about 10M, slow more turtle speed, like, how to solve! , the Bo friends to guide ..., thank you .....

1 Describe my deployment environment first:

2 osd,1 monitor,1 Console Management Server, 1 client, are 24 cores, 64G memory, 1.6T SSD flash card, gigabit network card; The Ceph version currently installed is 0.94.7.

2 Current status

I use the DD command to write 5G data, using Iostat to observe%util immediately 100% Ah, while the await indicators are more than 4,000, and at this time the network bandwidth is only used around 10M.

650) this.width=650; "Src=" Http://s4.51cto.com/wyfs02/M01/83/A2/wKioL1d5IcmzM6gWAABCGTKBIMI488.png-wh_500x0-wm_3 -wmp_4-s_3460037820.png "title=" dd test "alt=" Wkiol1d5icmzm6gwaabcgtkbimi488.png-wh_50 "/>

Using the Fio tool to write 1G data in sequence, the BW test is only 7M and ioPS is more than 800,

Fio--name=seqwrite--filename=/mnt/test--numjobs=2--bs=4k-size=1g--ioengine=libaio--iodepth=32--direct=1--rw= Write--group_reporting--randrepeat=0

Current configuration file:

[Global]
Fsid = 87c84fff-c652-4873-953b-85549fa400b9
Mon_initial_members = Cmon
Mon_host = 172.16.1.144
auth_cluster_required = Cephx
auth_service_required = Cephx
auth_client_required = Cephx
Filestore_xattr_use_omap = True
Osd_pool_default_size = 2
Osd_pool_default_min_size = 1
Osd_pool_default_pg_num = 128
Osd_pool_default_pgp_num = 128


Filestore_fd_cache_size = 204800
Filestore_omap_header_cache_size = 204800
Filestore_fiemap = True
Filestore_wbthrottle_xfs_bytes_start_flusher = 500000000
Filestore_wbthrottle_xfs_indoes_start_flusher = 500
Filestore_wbthrottle_xfs_indoes_hard_limit = 500000
Filestore_wbthrottle_xfs_ios_start_flusher = 50000
Filestore_wbthrottle_xfs_bytes_hard_limit = 500000000
Filestore_wbthrottle_xfs_ios_hard_limit = 500000
Filestore_queue_max_ops = 5000
Filestore_queue_max_bytes = 1024000000
Filestore_queue_committing_max_ops = 50000
Journal_queue_max_ops = 500000
Journal_queue_max_bytes = 10240000000
Osd_journal_size = 10000
Ms_dispatch_throttle_bytes = 104857600000
OSD OP threads = 4
OSD Disk threads = 2
Filestore OP threads = 6
OSD_MOUNT_OPTIONS_XFS = "Rw,noexec,nodev,noatime,nodiratime,nobarrier"

This article is from the "Little Brother" blog, please be sure to keep this source http://waq1688.blog.51cto.com/2621504/1795425

After Ceph deployment RBD block device read and write only about 10M, slow more turtle speed, like, how to solve! , the Bo friends to guide ..., thank you .....