. Net platform, implementation of Distributed File storage,. net platform file storage

. Net platform, implementation of Distributed File storage,. net platform file storage
Problems encountered

When a Web program uses a server, files uploaded by the client are generally stored on this server. However, it won't work in the cluster environment. If each server stores the files it receives, it will be messed up. The database clearly contains records of this attachment, but this file cannot be found. Therefore, files need to be centrally managed and provide a uniform path to the servers in the cluster.

NFS-based Distributed File storage implementation

Network File System (NFS) is a shared folder that can be used to store distributed files. You only need to share the folder on the file server, specify the permissions of the corresponding account, and set the account and password for the Web server to access the shared folder, the web server can operate files on the file server like local files. The file access path under NFS is in a fixed format, known as UNC (Universal Naming Convention) and starts.

To access files under NFS in UNC mode, two APIs provided by windows are required: WNetAddConnection2 and WNetCancelConnection2. WNetAddConnection2 can use the specified account and password to create a UNC connection, and then the program can directly access the files under the UNC.

First, create a FileServerConnection class to manage the connection. The specific code is as follows:

Public class FileServerConnection {private string uncName; private string username; private string password; /// <summary> /// constructor /// </summary> /// <param name = "uncName"> complete UNC path </param> /// <param name = "username"> User name for accessing the shared connection </param> // <param name = "password"> password for accessing the shared connection </param> public FileServerConnection (string uncName, string username, string password) {this. uncName = uncName; this. username = u Sername; this. password = password;} // <summary> // Connect to the file server // </summary> public void Connect () {var netResource = new NetResource {Scope = resourcquota. globalNetwork, ResourceType = ResourceType. disk, DisplayType = ResourceDisplayType. share, RemoteName = this. uncName. trimEnd ('\')}; var result = WNetAddConnection2 (netResource, password, username, 0); if (result! = 0) throw new Win32Exception (result);} // <summary> // release the connection /// </summary> public void Disconnect () {WNetCancelConnection2 (this. uncName, 0, true);} [DllImport ("flat. dll ")] private static extern int WNetAddConnection2 (NetResource netResource, string password, string username, int flags); private static extern int WNetCancelConnection2 (string name, int flags, bool force );}

Several struct codes used in FileServerConnection are as follows:

[StructLayout(LayoutKind.Sequential)]public class NetResource{    public ResourceScope Scope;    public ResourceType ResourceType;    public ResourceDisplayType DisplayType;    public int Usage;    public string LocalName;    public string RemoteName;    public string Comment;    public string Provider;}public enum ResourceScope{    Connected = 1,    GlobalNetwork,    Remembered,    Recent,    Context} ;public enum ResourceType{    Any = 0,    Disk = 1,    Print = 2,    Reserved = 8,}public enum ResourceDisplayType{    Generic = 0x0,    Domain = 0x01,    Server = 0x02,    Share = 0x03,    File = 0x04,    Group = 0x05,    Network = 0x06,    Root = 0x07,    Shareadmin = 0x08,    Directory = 0x09,    Tree = 0x0a,    Ndscontainer = 0x0b}

Then, when the Web program starts, you only need to create a FileServerConnection instance and then call its Connect method. To prevent exceptions caused by repeated connection creation, you can DisConnect before Connect. The specific call code is as follows:

fsConnection= new FileServerConnection (storeRootPath, username, password);fsConnection.Disconnect();fsConnection.Connect();

A file storage server and a large disk based on the DFS Distributed storage solution have been able to cope with most situations. If you want a larger storage capacity, a larger throughput, and a more secure and reliable file Deployment Solution, you can use DFS on windows server. DFS is essentially based on NFS, but it can be used to organize too many file servers to provide a unified access path, so as to support large capacity and large throughput. In addition, you can configure to store the same file on different servers. After one server fails, the file will not be lost; DFS can also be integrated with the Active Directory, even if the root server fails, it still does not affect server usage, thus ensuring the reliability of file storage. For how to install and use DFS in Windows Server, refer to "using FastDFS.
In the net architecture, how does one understand distributed file storage?

The system architecture is okay, but --- but the system crashes, why? The system does not take into account the massive number of users, but also the massive number of files. users' photo albums and images are all stored in one partition of the WEB server, with each user having a directory, however, when the performance monitor is turned on, the disk's IO is surprisingly high and basically has no response time. As we all know, the file system is also a database, and it doesn't matter if you have a single large file. The key is that the entire file is more than 300 GB of fragmented files, massive read/write operations, system crashes, and data loss, A chain in the file system is broken, and all user data is lost !!! This is a very heavy problem. The system has stopped for a whole month to recover data (it is easy to separate files, but there is currently no software to organize the software architecture for massive files ). Solution: modify the program architecture and perform distributed file storage (it took 8 days for the program to be modified, but the file transfer took nearly a month), and 0.2 million users were lost.

In the net architecture, how does one understand distributed file storage?

What is distributed storage:
Unlike the centralized storage (such as file and data servers and EMC storage products) of a large number of applications, distributed storage does not store data on one or more specific nodes of an enterprise, instead, you can use the disk storage space on each machine (either a desktop PC or a server) in the enterprise through the network, and make these distributed storage resources into a virtual storage device, data is stored in all corners of an enterprise.