reliable, but also save the investment of new programs. The disaster-tolerant scheme based on storage virtualization technology emerges. The new virtual disaster recovery solution, using the network's existing transmission capabilities, will be transferred to the remote connected storage systems, all of the above mentioned problems will be solved.
By moving data replication functionality from disk groups to network-based, Universal Central storage Service programs, data replication is no longe
technologies, including linear linear link, similar to JBOD technology, can be used to organize multiple disks into a disk, and mirror mirroring technology, The data can be backed up; snapshot is the snapshot volume technology, you can save the system a moment ago, and so on if the system error, need to recover, you can use the snapshot to recover; Multipath Multipath allows the server to have multiple lines to transmit data, and when a problem occur
First, we will introduce a script that yahoo once used to collect hardware information. You can view and download hwconfig here. It is executed on one of my systems as follows:View Code BASH
12345678910111213141516171819
[root@jay-linux test-programs]# ./hwconfig Summary: QCI QSSC-S4R, 1 x 000 2.27GHz, 47.1GB / 48GB 1067MHz DDR3System: QCI QSSC-S4RProcessors: 1(of 4) x 000 2.27GHz 133MHz FSB (HT enabled, 10 cores, 80 threads)Memory: 47.1GB / 48GB 1067MHz DDR3 == 12 x 4
"TechTarget Chinese Original"Microsoft introduced storage Spaces Direct in Windows Server Technical Preview 2. This feature expands local storage to a highly available (HA) storage System.For example, Storage Spaces Direct supports insertion into the PCI Express bus using low-cost, low-performance, high-capacity SATA-like drives and solid-state devices such as SSD and serial storage architectures. This approach is known as nonvolatile memory Express (NVMe) or nonvolatile Storage Control Interfac
What is Windows Server 2012 R2 Storage Spaces? What functions does Storage Spaces provide for data center and Storage management?
Storage Spaces Storage space) is Microsoft's answer to Storage virtualization and software-defined Storage, allowing enterprise data centers running Windows Server 2012 and Windows Server 2012 R2 to virtualize shared Storage for configuration and management.
Storage space depends on Inexpensive Disk Arrays, rather than proprietary and expensive architectures such as
First, we will introduce a script that yahoo once used to collect hardware information. You can view and download hwconfig here. It runs on one of my systems as follows: ViewCodeBASH12345678910111213141516171819 [root @ jay-linuxtest-programs] #./hwconfig
First, we will introduce a script that yahoo once used to collect hardware information. You can view and download hwconfig here. It is executed on one of my systems as follows:View Code BASH
12345678910111213141516171819
[root@jay-l
and Redundancy when possible
10. Select a disk and Add to Array-> Accept DG-> next; (Note that if the disk is JBOD, change it to Make Unconf Good !!!! This step is required; otherwise, the disk cannot be selected later !!!)
650) this. width = 650; "src =" http://www.bkjia.com/uploads/allimg/131228/005912O51-5.jpg "title =" IMG_20130718_120459.jpg "/>
11. Add to SPAN-> next
650) this. width = 650; "src =" http://www.bkjia.com/uploads/allimg/13122
"alt =" "/>
Excellent: minimum resource overhead (depending on the RAID level );
Poor: If a RAID fails, the entire RAID group will be rebuilt. RAID group reconstruction takes a long period of time, resulting in performance degradation;
3. 15 disks are in three RAID groups, and one LUN is output for each RAID group:
650) this. width = 650; "width =" 237 "height =" 158 "border =" 0 "src =" http://www.bkjia.com/uploads/allimg/131228/0154434938-5.png "alt =" "/>
Excellent: Three I/O queues are ava
. click start 7. configuration wizard 8.new configuration has three options: clear configuration, new configuration, and add configuration. we select new configuration to clear the old one first, create a new one. 9. select Manual configuration and Redundancy when possible 10. select a disk and Add to Array-> Accept DG-> next; (note that if the disk is JBOD before, change it to Make Unconf Good !!!! This step is required; otherwise, the disk cannot be
)
Determine how YARN handles resource allocations
Identify the workflow of MapReduce job running on YARN
Determine which files you must the change and the How on order to migrate a cluster from the MapReduce version 1 (MRv1) to mapredu CE version 2 (MRV2) running on YARN
3. Hadoop Cluster Planning (16%)
Principal points to consider in choosing the hardware and operating systems to host a Apache Hadoop cluster
Analyze the choices in selecting an OS
Understand kernel tuning and disk
space of multiple disks is combined with a large continuous space;Free space: sum (s1,s2,...) ;Common levels: RAID-0, RAID-1, RAID-5, raid-10,raid-50, JBOD;
Soft RAID implementations:
MDADM: Provides management interface for soft raid;Add redundancy to the spare disk;Combined with the MD (Multi devices) in the kernel;RAID devices can be named/dev/md0,/dev/md1,/DEV/MD2,/dev/md3, and so on;Implementation of software RAID:Mdadm: a modular to
, independent of the San and NAS with a lot of hard disk to do large scale scaling.hadoop is also developed on the basis of the same horizontal expansion concept. So is Hadoop using today's popular blade servers, Sans, and virtualization technologies to improve performance like traditional services? Not really First, the features of Hadoop operations. Hadoop is very sensitive to IO performance and there will be a lot of IO operations in the process. Hadoop does not require RAID, instead, using
) is also known as RAID 0 + 1, a solution that combines storage performance and data security. It provides the same data security as RAID 1, and the storage performance is similar to RAID 0, but the storage cost is high. Take four hard disks as an example, that is, each hard disk is a group, each group stores the same data, and the two groups of data are split by a group of data. jbod, also known as spanning, is composed of hard disks of different cap
HDFS File System Optimization
1. System Perspective
Storage Method. namenode adopts raid1 + 0, and datanode adopts the disk group jbod mode.
In the scenario of sequential file reading, such as mapreduce, you can adjust the size of the file system pre-read cache.
Set noaptime and nodiratime for File Mounting to improve File System Performance
2. HDFS Parameter Optimization
DFS. namenode. handler. Count (the default value is 10 and the value is 64)DFS.
settings of the raid control card. When we install the raid control card and connect the two hard disks, the following figure will appear when the system starts up.Select "1. Set RAID configuration" in the menu and press the Enter key to go to the "set RAID configuration" page. The RAID Controller will use a period of time to identify the hard disk. Then, move the cursor to the hard disk, press the Space key to select the disk, and press the Enter key to confirm the selection, A new window is d
1. What is raid0?
Raid0, which divides data into several halves and writes data to multiple disks A: A1 A2 ...... Performance improvement. At least two disks are required.
Merge multiple disks into a large disk with no redundancy and parallel I/O, with the fastest speed. RAID 0 is also called a zone set. It associates multiple disks to form a large disk. When storing data, the data is segmented by the number of disks and written into these disks at the same time. Therefore, RAID 0 is the fastes
disk and Add to Array-> Accept DG-> next; (Note that if the disk is JBOD before, change it to Make Unconf Good !!!! This step is required; otherwise, the disk cannot be selected later !!!) 11. Add to SPAN-> next12. enter the raid0 option. Here, you only need to enter a select size (set the available space of raid0 in upper case). According to the green font prompt on the right, there will be available space for the current disk, therefore, when the s
:
Linear: similar to Jbod. The storage space of multiple disks can be spliced together and used as one disk.
Mirror: similar to an image.
Snapshot: Like a VM, a Snapshot can retain the status of data when a Snapshot is created. A snapshot is another path for accessing the same data. (Mainly used for data backup)
Multipath: multi-path. The multi-path must be configured by the DM module, so that the path of the data storage device can be searched throug
extend volume.
650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M02/30/88/wKioL1OoMISCyQZlAARjSwpOfLk228.jpg "Title =" image44.png "alt =" wkiol1oomiscyqzlaarjswpoflk228.jpg "/>
47. Enter a value (for example, 27.9 GB) in the maximum value and click OK.
650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M00/30/88/wKioL1OoMJXBrlZmAARinaKRpMU675.jpg "Title =" image45.png "alt =" wkiol1oomjxbrlzmaarinakrpmu675.jpg "/>
48. confirm that the disk has additional available space. (I
RaidPerformance of disks 1.定位 2.旋转 3.命令队列 4.数据库传输HBA: Host-based adapterRaid:redundent array of inexpensive Disks inexpensive disk arrayIndependent Standalone disk arrayLevel: Only used to identify the different ways of disk composition, can not replace data backup raid0:提高读写性能,没有容错能力。至少需要两块磁盘 100% raid1:至少需要两块硬盘,镜像,硬件容错能力,读性能提升,写性能下降 50% raid4:至少需要三块硬盘,允许一块盘出错,读写性能提升,(n-1)/n raid5: 至少需要三块硬盘,允许一块盘出错,读写性能提升,(n-1)/n raid6: 至少需要四块磁盘,允许两块盘出错。读写性能提升,(n-2)/n raid10:至少需要四块磁盘。允许不同组内个各坏一块盘,读写性能提升
The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion;
products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the
content of the page makes you feel confusing, please write us an email, we will handle the problem
within 5 days after receiving your email.
If you find any instances of plagiarism from the community, please send an email to:
info-contact@alibabacloud.com
and provide relevant evidence. A staff member will contact you within 5 working days.