Design and implementation of cloud security storage system based on Hadoop

Design and implementation of cloud security storage system based on Hadoop

Guangdong University of Technology Shangxiangyang

Cloud computing is the development of distributed computing, parallel computing and grid computing, which brings about fundamental changes in human life, production mode and business model, and is one of the focuses of the whole society. Cloud storage is a new concept that extends from the concept of cloud computing, it refers to the use of cluster applications, grid technology or Distributed file system functions, the network of a large number of different types of storage devices through the application software together to work together to provide data storage and business access functions of a system. With the wide application of cloud computing technology and cloud storage technology, the security problems have been highlighted, and security problems have become the key factors that hinder the development of cloud computing and cloud storage. Hadoop, an open source distributed computing platform under the Apache Software Foundation, provides developers with a distributed system infrastructure that allows users to develop distributed applications without understanding the underlying details of a distributed system. As a core component of Hadoop, the Distributed File System (HDFS) has a powerful storage capability and is suitable for cloud storage system, but there are some defects in its design, which is still developing, and there is room for improvement and improvement. In this paper, the HDFS multi-replica storage technology and identity authentication technology are studied, and an improved method is proposed for the existing problems. This paper studies the security Problem of data storage in the cloud computing environment based on Hadoop, designs the cloud security storage system, and realizes the system prototype. The main tasks are as follows: (1) This paper analyzes the problem that the storage efficiency of the HDFS multi-copy storage scheme is not high and the load imbalance is easy to appear, proposes a distributed storage scheme based on information dispersing method, designs an improved file Block storage strategy, and verifies the effectiveness of the scheme. (2) It is pointed out that the identity authentication of the HDFS system has some problems, such as the user is apt to impersonate, and a safe and effective one-time password authentication scheme is designed, and the authentication scheme is combined with HDFS authentication to authenticate the user and improve the security of HDFs. (3) Using JSP technology combined with HADOOPAPI developed a cloud security storage system based on Hadoop, visualized the HDFs operation interface, realized Encrypting file transmission, permission control, security sharing and so on, overcame the security hidden danger in the data transmission and storage of HDFS system. Improve the security of user data. The innovation of the thesis includes: (1) A distributed storage scheme based on information scatter algorithm is proposed, an improved file Block storage strategy is designed, and the storage load balancing ability of HDFS is improved; (2) a secure and effective one-time password authentication scheme is designed, This password authentication scheme is combined with HDFS authentication to improve the security of HDFs; (3) A cloud security storage system based on Hadoop is designed, and the system prototype is realized.

Design and implementation of cloud security storage system based on Hadoop