hadoop distributed file system architecture and design

Foreman Mode Contractor-Worker mode: I have read a book titled large-scale distributed system architecture and design practices. The "contractor-worker" mode is used in the user module of anycmd. AC is an implicit project. A project has only one contractor. A contractor can hire many workers. A worker is only employed

OverviewDistributed filesystem: Distributed File System, DFS, also known as the network file system. A file system that allows files to be shared across multiple hosts on a network, all

1, two sets of equipment, respectively, two groups of services deployed.2, the main server one master, the main standby hot switch. Business Server, one master, one standby, load balancing. The primary primary replication of the database.3, the main server is eager to switch, using the heartbeat server monitoring, while hanging, the other side automatically start.4, for the business Server, the primary service is designated by the master standby, monitoring the main business Server, the main bus

name. After successful matching, the offset of the file in SD is obtained. The 9-bit offset (512 bytes, one sector) is the position of the file. Fopen returns a handle that corresponds to a data structure. The structure generally includes the start position, length, and current read pointer position of the file. The record start position and length are used to p

latest consistent fsimage and editlog. Namenode exists at a single point in HDFS. If the machine where namenode is located is incorrect, manual intervention is required.5) deleting a file does not immediately remove the namespace from the namenode, but is stored in the/trash directory and can be recovered at any time until the specified time is exceeded.Besides the fault tolerance of hadoop,

the Zookeeper directory　　　　　　　　　　　　Copy this path, and then go to config file to modify this, and the rest do not need to be modified　　　　　　　　　　　　After the configuration is complete, start zookeeper, and in the Zookeeper directory, execute the command: bin/zkserver.sh start　　　　　　　　　　　　View zookeeper status can be seen as a stand-alone node　　　　　　command to enter the client: bin/zkcli.sh　　　　　　To create a command for a node:Create/test "Test-data"　　　　　　V

; Application Layer: website application cluster; Vertical Split by business, such as product applications and member centers; Service layer: provides public services, such as user services, order services, and payment services; Data Layer: supports relational database clusters (supports read/write splitting), NOSQL clusters, Distributed File System clusters, and

Basic information of hadoop technology Insider: in-depth analysis of mapreduce architecture design and implementation principles by: Dong Xicheng series name: Big Data Technology series Publishing House: Machinery Industry Press ISBN: 9787111422266 Release Date: 318-5-8 published on: July 6,: 16 webpage:: Computer> Software and program

the underlying details of the distribution. Make full use of the power of the cluster for high-speed operation and storage. Hadoop implements a Distributed File system (Hadoop distributedfile System), referred to as HDFs. HDFs is

, and end users extract the statement result file via the Hadoop client ( Hadoop itself is also a distributed file system with the usual file access capabilities.3. Clearing: Import the

In 2 b Enterprise Services, cloud computing, mobile Internet, professional cloud platform services, distributed technology for the support platform for the normal operation of key technologies. From the perspective of commercial profits and operational costs, every effort to squeeze dry the server's performance to a large extent affect the business value of the site, so the pursuit of performance, distributed