MongoDB-based file server

 

MongoDB-based file server

 

Recently, I complained about the company's file server. The existing file server saves the files to the server hard disk through the interface provided by the server and divides the files by ID. After regular backup to other servers, an accident occurred so long that the hard disk was formatted by a colleague, leading to the loss of some data. Therefore, if we re-design the file server, we will consider using MongoDB for Distributed synchronization.

 

 

Mongodb benefits:

1. Simple backup and Maintenance

2. You can use the distributed synchronization data of MongoDB. Even if a server fails, the fileserver does not

3. The speed will be faster (I guess)

 

File server structure:

1. Use WebService to provide interfaces for adding, deleting, and obtaining files. MongoDB only saves the file content.

2. Save the file to the MongoDB Database

3. The storage file is composed of three MongoDB database servers. One is the arbiter server, and the other two are the master and slave servers.

 

Mongodb server environment setup:

 

1. download the latest version of MongoDB for Windows http://www.mongodb.org/downloads

2. decompress the file and rename the folder MongoDB. Create a new data and log folder under the MongoDB folder and copy the two files as D: \ mongodb2, D: \ mongodb3.

 

3. Configure MongoDB 1

Locate the MongoDB/bin folder by running the CMD command, and run the following command to start MongoDB:

D: \ MongoDB \ bin> mongod -- rest -- replset myset -- Port 20720 -- dbpath D: \ MongoDB \ data -- logpath D: \ MongoDB \ log

-- Replset myset specifies that MongoDB runs in replication mode and the Set Name Is myset.

-- Rest is used to start the admin UI. You can view the server status through http: // localhost: 28017/_ replset.

-- Dbpath specifies the database file path

-- Logpath specifies the log file path

 

4. Configure MongoDB 2

The configuration method is the same as MongoDB 1, but the port is changed to 20721

D: \ mongodb2 \ bin> mongod -- rest -- replSet myset -- port 20721 -- dbpath d: \ mongodb2 \ data -- logpath d: \ mongodb \ log

 

5. Configure Mongodb 3

The configuration command is as follows:

D: \ mongodb2 \ bin> mongod -- rest -- replSet myset -- oplogSize 8 -- port 20722 -- dbpath d: \ mongodb3 \ data

 

View http://www.mongodb.org/download/attachments/9830402/mongodb+replica+sets+intro.pdf details

 

Now all three mongodb servers are started. Oh, you need to tell Mongodb 1 his companion Mongodb 2. The specific command is as follows:

D: \ mongodb \ bin> mongo -- host ChrisCheung_PC // here is my computer name, or 127.0.0.1

Run

> Rs. initiate ()

> Rs. add ("ChrisCheung_PC: 20721") // you can use rs. conf () to view the computer name.

{"OK": 1}

 

In this case, add mongodb of port 20721 to myset and execute rs. conf () to view the corresponding information.

 

Next, add Mongdb 3 to the group and agree to execute it in mongodb 1:

> Rs. add ({_ id: 2, host: "ChrisCheung-PC: 20722", arbiterOnly: true })

{"OK": 1}

If the host is incorrect, the following error is reported:

 

{
"Assertion": "need most members up to reconfigure, not OK: chrischeung
-PC: 20721 ",
"Assertioncode": 13144,
"Errmsg": "DB assertion failure ",
"OK": 0
}

 

 

 

Up to now, we have three mongodb servers running normally and configured them as the Replica Sets synchronization mode.

 

Program Implementation. Below is a simple class I wrote

 

using System;using System.Collections.Generic;using System.Linq;using System.Text;using MongoDB.Driver;using MongoDB.Driver.Builders;using MongoDB.Bson;namespace FileUtility{    public class DocHelper    {        private string connStr = "";        public string ConnStr        {            get             {                if (string.IsNullOrEmpty(connStr))                {                    throw new ArgumentNullException("Connection string did not specify!");                }                return connStr;            }        }        public DocHelper()        {            connStr = System.Configuration.ConfigurationManager.AppSettings["FileDb"];        }        public DocHelper(string connectionString)        {            this.connStr = connectionString;        }        /// <summary>        /// save document conent        /// </summary>        /// <param name="content">file content</param>        /// <returns>file id in mongodb database</returns>        public string AddDoc(byte[] content)        {            MongoServer server = MongoServer.Create(this.ConnStr);            try            {                Doc doc = new Doc() { ID = Guid.NewGuid().ToString(), data = content };                server.Connect();                MongoDatabase db = server.GetDatabase("ecDocs");                MongoCollection<Doc> docs = db.GetCollection<Doc>("ecFiles",SafeMode.True);                docs.Insert(doc);                return doc.ID.ToString();            }            catch (Exception ex)            {                throw ex;            }            finally             {                if (server != null)                    server.Disconnect();            }        }        /// <summary>        /// delete doc from the mongodb server        /// </summary>        public void DeleteDoc(string docID)        {            MongoServer server = MongoServer.Create(this.ConnStr);            try            {                server.Connect();                MongoDatabase db = server.GetDatabase("ecDocs");                MongoCollection<Doc> docs = db.GetCollection<Doc>("ecFiles", SafeMode.True);                var queryDoc = Query.EQ("_id", docID);                docs.Remove(queryDoc);            }            catch (Exception ex)            {                throw ex;            }            finally            {                if (server != null)                    server.Disconnect();            }        }        /// <summary>        /// get the doc content from mongodb        /// </summary>        /// <param name="docID">doc id in mongodb</param>        /// <returns>document content</returns>        public Doc GetDoc(string docID)        {            MongoServer server = MongoServer.Create(this.ConnStr);            try            {                server.Connect();                MongoDatabase db = server.GetDatabase("ecDocs");                MongoCollection<Doc> docs = db.GetCollection<Doc>("ecFiles", SafeMode.True);                Doc doc = docs.FindOneById(docID);                return doc;            }            catch (Exception ex)            {                throw ex;            }            finally            {                if (server != null)                    server.Disconnect();            }        }    }}

 

Entity class:

 

using System;using System.Collections.Generic;using System.Text;using MongoDB.Bson.DefaultSerializer;using MongoDB.Bson;namespace FileUtility{    public class Doc    {        [BsonId]        public string ID { get; set; }        public byte[] data { get; set; }    }}

 

Test code:

 

Code

Fileutility. dochelper helper = new fileutility. chelper ("MongoDB: // localhost: 20720 ");
Byte [] bytes = file. readallbytes ("E: \ photo \ 20110101-ski \ dscn1621.jpg ");
String id = helper. adddoc (bytes );

FileUtility. Doc doc = helper. GetDoc (id );
File. WriteAllBytes ("d: \ image.jpg", doc. data );

 

 

After inserting an image, go to http: // localhost: 28017/_ replSet optime to view the database logs. You can see that the two servers have inserted the same records, for example:

 

 

 

Well, a simple file server is built. Simply inserting some data, I feel that mongodb is currently using a very large hard disk space. What are your suggestions for file server? Thank you.

 

References:

Http://www.cnblogs.com/daizhj/archive/2010/09/08/1821481.html

Http://www.cnblogs.com/daizhj/archive/2010/09/07/1820528.html

Http://www.mongodb.org/