Question: Background of grid data transmission and management :)

(Copyright. Do not reprint it !)

In the ever-changing development of computers and communication equipment, the emerging infrastructure of grid has brought about a new network boom. The grid tries to connect resources on the Wide Area Network and hopes to establish a higher level of sharing in existing technologies to provide sharing of computing, storage, and other resources. Data transmission and management not only occupy an unshakable position in the data grid for processing massive data, but also play a crucial role in the fields of computing grids and service grids, efficient, reliable, and convenient data migration and replication are related to the efficiency of the entire system.

The gridftp protocol provides an effective solution for data transmission and management between file systems with heterogeneous storage in the grid. It is based on the FTP protocol and has been extended, supports GSI and Kerberos authentication, dynamic adjustment of TCP buffer/window size, parallel and band transmission, resumable and partial transmission, and third-party transmission. In this way, data transmission ensures reliable performance and security.

Due to the diversity of grids and the rapid development of wireless devices, we have to consider optimizing the wireless network environment in future data transmission protocols. Wireless Internet access statistics first appeared in CNNIC's China Internet Network Development Statistics Report (January 2005). The report shows that 1% of Internet users (nearly one million) use wireless Internet access, 3.5 million of Internet users use mobile terminals, Information appliances, and other devices to access the Internet. The characteristics of a wireless network are different from those of the TCP protocol. Due to its high bit error rate, long latency, and variable bandwidth and latency, the transmission performance is greatly affected.

In addition, TCP traffic control parameters and algorithms cannot adapt to the increasing bandwidth of the Wide Area Network. Similarly, the dynamic window adjustment algorithm of gridftp may also have some problems. Gridftp supports only distributed file systems, and the transmission rate of different servers may not be taken into account. It should be improved to support duplicate transmission and dynamic load balancing. The authentication system of the gridftp server involves security certificates. It takes too much time to log on when transferring large volumes of files. Such application problems impose higher requirements on the Performance of gridftp.

At the same time, there are still various data transmission protocols in the grid, such as the most common FTP and HTTP, European DataGrid and CERN Alpha Magnetic Spectrometer (AMS) the bbftp and sftp used by the project to transmit large files bring confusions and inconvenience to common users. Users usually need to wander between different software and use the local system as a transit for data transmission. Cross-OS and cross-protocol software is urgently needed to solve this problem. More importantly, the complementarity between various protocols enables data transmission in the grid environment to meet different needs: ultra-large file transmission, small sensitive file transmission, cross-firewall transmission, and so on. On this basis, it is encapsulated into a grid service based on WSRF for third-party transmission and file management between multiple data servers. It implements adaptive and reliable transmission of protocols, and provides automatic transmission recovery, which makes up for the shortcomings of transmission software that requires manual participation. It can also monitor and query transmission statuses.

A portal is the next-generation desktop. It transmits e-commerce applications to various client devices through the Web, providing site users with single-point access to various types of information and applications. Portlet is the heart of a portal and a reusable component that provides access to applications, Web-based content, and other resources. The data transmission software and grid service are released to the website in the form of a Portlet, providing users with intuitive and convenient interfaces is an important way to display the grid.