Cloudeep
Introduction to OSS
-1
---- Adam
Cloudeep
In the past, the team has been committed to developing a similar
Amazon S3
(
Http://aws.amazon.com/
) And
Google Storage
(
Http://code.google.com/apis/storage/docs/overview.html
) To provide users
Rest
Oriented
Object
Cloud storage system;
Cloudeep
Early narration based on
Hadoop
Object Storage System implementation method (
Http://blog.csdn.net/Cloudeep/archive/2009/08/05/4412958.aspx
);
While
Cloudeep
The OSS adopts a new architecture.
Keywords:
Object Storage System
,
Amazon S3
,
Google Storage
,
Hadoop
Cloudeep
Introduction to OSS Functions
-
Provides large-scale storage space services for Internet users and enterprise users (
PB
)
- Supports writing, reading, and deleting objects, but does not support object modification (update and append)
- Support Bucket
-
Any smaller
5G
Object (file)
-
Supports the storage of user-defined metadata attributes
-
Support for Bucket objects
ID
Lexicographic operations
-
Object support
ACL
Access Control
-
High scalability. The system supports seamless online resizing.
-
High Availability: supports as many online service capabilities as possible, and supports multi-copy control and repair
-
Final data consistency
-
Supported
Rest
Access interface
-
Supports restoring previous versions (new)
2
,
Cloudeep
Introduction to the architecture of the Object Storage System
Figure
1 cloudeep
Simple Object Storage System Architecture
-
Object meta data service layer
: Stores metadata information of objects, stores location information of objects after chunking, stores location scheduling of data after chunking, repairs data copies, and monitors data health;
-
Object block service layer
: Storage and maintenance of object chunk data; copy and migration of data;
-
Chubby-like Cluster
: Provides network environment information services (topology services, Event Notification Services) and bottom-granularity lock services for the system;
-
Cloud storage Web Service
: Provide object
Internet
Access interface
3
, Based on
Hadoop
How to Implement Object Storage Service
(
Http://blog.csdn.net/Cloudeep/archive/2009/08/05/4412958.aspx
)
-
Low system availability:
HDFS
Layer exists
Namenode
Single point of failure;
Bigtable
The metadata layer of is prone to local
Range
If the service is unavailable (
Range
Of
Takeover
Time required );
-
System complexity is too high: in this solution, there are too many subsystems
HDFS
,
Hbase
,
Mapreduce
Etc. The system is too complex to facilitate deployment and maintenance;
-
System message overhead is too high: System
HDFS
Designed for large files, the message process is not suitable for small object storage, with high overhead and low efficiency;
-
Too many system layers: accessing an object involves
Hbase
Metadata access,
HDFS
File System
Namenode
Access,
Datanode
Low access efficiency;
- ...
