The algorithm of file backup and distribution dynamic adjustment in Telecom data Warehouse based on cloud platform
Yang Miaomiao Li Fai Liu Jinghu
Hadoop leverages a unified backup mechanism to ensure data availability and fault tolerance. However, for important blocks of files, you can only set the number of replicas by modifying the configuration file. By analyzing the access situation of data Warehouse files in telecom industry, this paper proposes a dynamic adjustment algorithm (BPDA) for file backup and storage to improve the data locality, so as to reduce the I/O and network consumption caused by data transmission, and to improve query efficiency. The optimal backup number and backup location of each file are obtained by using the multiple replica consistency hashing algorithm, probability sampling and competitive weakening algorithm. Experimental results show that the proposed algorithm can improve query correlation efficiency.
The algorithm of file backup and distribution dynamic adjustment in Telecom data Warehouse based on cloud platform
The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion;
products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the
content of the page makes you feel confusing, please write us an email, we will handle the problem
within 5 days after receiving your email.
If you find any instances of plagiarism from the community, please send an email to:
info-contact@alibabacloud.com
and provide relevant evidence. A staff member will contact you within 5 working days.