Website Operations

File compression has two main benefits, one is to reduce the space occupied by storage files, the other is to speed up the data transmission. In the context of Hadoop's big data, these two points are particularly important, so let me now look at the file compression in Hadoop. Hadoop supports a wide variety of compression formats, we look at a table: Deflate is also using the LZ77 algorithm and Huffman coding (Huffman http://www.aliyun.com/zixun/aggregation/1552.html ...

Linux is a child of love, OS x and Windows systems are always there, but open source Linux is silently supporting something unusual in the corner. The executive director of the Linux Foundation told us, "You use Linux every day, but you don't know it's always in your life." "Mobile phones, securities institutions, high-speed rail, traffic control, banks, nuclear submarines, automobiles ..." This almost forgotten system is far more important than what you see. 1. Http://www.aliyun.com/zix ...

Visualization is one of the hottest areas of cloud application. Gathered countless Daniel, small and innovative enterprise representatives. 　　This paper, from the Stanford visualization group led by Pat Hanrahan and Jeffrey Heer two-bit visualization, focuses on sharing the image's theme-Color extraction algorithm. 　　The visualization and Visual analysis Group of CAD&CG State Key Laboratory of Zhejiang University in particular, the paper was collated, the following is a summary of the article. Stanford Visualization Group is very necessary to introduce, the leader of the two Daniel is Pat Hanra ...

&http://www.aliyun.com/zixun/aggregation/37954.html ">nbsp; For Telecom Enterprises, database security is very important. Just imagine the problem with the recharge system? What happens to the system when a mobile phone user checks the bill at the end of the month? The following is a telecom enterprise database operators in the database security aspects of some of the experience, as well as the star-Chen database security experts to give suggestions, hope to have some inspiration and reference. Database version ...

Hortonworks has released a preview release of the next generation of Apache Hadoop. 　　The Apache Hadoop commitment expands the range of types that can be applied to analysis on a data-processing platform. The new Apache Yarn Scheduler replaces the founder of MapReduce Hortonworks, one of the core engineers who developed Hadoop, by providing a more general resource management framework Arun Murthy said: "Hadoop 2.0 is a fundamental architectural change, ...

It's been a long time since the deletion volume error makes volume in http://www.aliyun.com/zixun/aggregation/16539.html ">error_deleting state, There was this problem when I deleted a volume yesterday, and here's a way to record the solution. Note that I am here for the back-end with iSCSI mode, specifically to me here is the TGT+LVM way. Cause I am currently experiencing the deletion of V ...

For Cos and DSCP, just the criteria for categorization, you can set the trust yourself. and Cos and DSCP there is a mapping between the priority of the packet, just identify the different priorities, according to the priority of the package to choose a different queue, different out of the queue of bandwidth resources, congestion when the proportion of discarded. To achieve the goal of quality of service. The implementation of QoS is based on the DiffServ system of the IETF. DiffServ system stipulates that each transmission message will be classified into different categories in the network, the classification information is contained in the IP message header, diffs ...

Today, I attended 3 keynotes,42 sessions of 8, and a lot of vendors to discuss technology, is really a big bang day. Hadoop has been around for 7 years since its inception, and this year has seen many new changes: 1, Hadoop is recognized as a set of industry data standard open source software, in a distributed environment to provide a large number of data processing capacity (Gartner). Almost all major manufacturers revolve around Hadoop development tools, Open-source software, commercial tools, and technical services. This year, large IT companies, such as ...

1 Hadoop Cluster Planning 1.1 Total A, B, C three machines; 　　1.2 A as master,b as slave1,c as Slave2; 1.3 IP &http://www.aliyun.com/zixun/aggregation/37954.html ">nbsp; a:192.168.1.103; b:192.168.1.104; c:192.168.1 ...

In recent years, the development of the NoSQL database has become more and more hot, but this does not mean that developers will abandon the original can be directly from the huge amount of data to obtain the knowledge of the SQL query. Please visit the original view video (need to flip wall) Sqlstream CEO Damian Black in http://www.aliyun.com/zixun/aggregation/13821.html ">gigaom The Businessesflat-out meeting brought their ...

Refer to Hadoop_hdfs system dual-machine hot standby scheme. PDF, after the test has been added to the two-machine hot backup scheme for Hadoopnamenode 1, foreword currently hadoop-0.20.2 does not provide a backup of name node, just provides a secondary node, although it is somewhat able to guarantee a backup of name node, when the machine where name node resides ...

GitHub is an open source software collaboration platform that provides a platform for developers to communicate. Git was originally a distributed software versioning program written by Linus Torvald to manage the development of Linux kernel code. GitHub implements Git's code hosting, http://www.aliyun.com/zixun/aggregation/9591.html "> Versioning features, and you can join a project development team through fork projects.

Take hadoop0.21 as an example. Namenode.java: The main maintenance file system namespace and file metadata, the following is a description of the code.  /**********************************************************&http://www.aliyun.com/zixun/aggregation/37954.html ">nbsp; ...

The disadvantage of using HDFS to save a large number of small files with use using the: 1.Hadoop Namenode saves "meta information" data for all files in memory. According to statistics, each file needs to consume NameNode600 bytes of memory. If you need to save a large number of small files will cause great pressure on the namenode. 2. If the use of Hadoop MapReduce small file processing, then the number of Mapper will be the number of small files into a linear correlation (Note: Filei ...

1, namenode Start load metadata scenario analysis Namenode function call Fsnamesystemm read dfs.http://www.aliyun.com/zixun/aggregation/11696.html "> Namenode.name.dir and Dfs.namenode.edits.dir build Fsdirectory. Fsimage class Recovertransitionread and ...

The following is my hive installation process: Hive is the most commonly used tool in Hadoop, can be said to be a required tool. According to the official Apache documents, recommended to use SVN download compiled, document address: Https://cwiki.apache.org/confluence/display/Hive/AdminManual+Installation but build , because of the dependence, the whole long time, under a lot of packages also did not succeed. Recommended use of tar.gz bag, direct ann ...

1, metadata (Metadata): Maintenance of HDFs file System file and directory information, divided into memory metadata and metadata file two kinds. Namenode maintains the entire metadata. HDFs implementation, the method of periodically exporting metadata is not adopted, but the backup mechanism of metadata mirroring file (fsimage) + day file (edits) is adopted. 2, Block: The contents of the file. Search path Flow: &http://www.aliyun.com/zixun/aggregation/37 ...

SDN (software-defined network software tabbed receptacle) is the hottest technology in a new generation of data centers. All major network equipment manufacturers have issued their own SDN strategy, imagine in this new field of competition. However, according to IDC's latest research results: In 2013, the entire enterprise network industry market value of 42 billion U.S. dollars, of which nearly half from the 2~3 layer Network Switch Market, SDN market only 168 million U.S. dollars, and by 2016 SDN market can reach 2 billion U.S. dollars. From the entire Network line ...

hadoophttp://www.aliyun.com/zixun/aggregation/17253.html > FAQ:------------------------------------------ -----------------WARN mapred. localjobrunner:job_local910166057_0001 org ...

The NSA's "massive surveillance" data center in the Utah State Desert can screen and analyze most of the world's network traffic, from geographical locations, audio and video files, e-mail, instant messaging, social networks and other digital documents. Of course, it's not just the NSA that can track our digital footprint, but in this time of personal privacy, all kinds of government and business companies can master our words and every move. 　　As a common netizen, is there a reliable way to protect themselves? Earlier, the Washington Post had given five individuals to avoid NSA surveillance ...