Big Data

There is a concept of an abstract file system in Hadoop that has several different subclass implementations, one of which is the HDFS represented by the Distributedfilesystem class. In the 1.x version of Hadoop, HDFS has a namenode single point of failure, and it is designed for streaming data access to large files and is not suitable for random reads and writes to a large number of small files. This article explores the use of other storage systems, such as OpenStack Swift object storage, as ...

Today, open source firewalls are numerous. 　　This article will cover 10 of the most practical open source firewalls that fit your business needs. 1. Iptables Iptables/netfilter is the most popular command line based on firewalls. It is the safe line of defense for Linux servers. Many system administrators use it to fine-tune the server. The function is to filter packets from the network stack in the kernel, including: listing the contents of the packet filtering rule set, and executing fast because it only checks the header of the packet, and the administrator can ...

With the continuous expansion of Hadoop application, many people fall into the blind worship of it, think it can solve all problems. While Hadoop is a great distributed large http://www.aliyun.com/zixun/aggregation/14206.html "> Data Computing Framework, Hadoop is not omnipotent." For example, the following scenarios are not suitable for using hadoop:1, low latency data access to Hadoop and not for ...

The 2013 will soon be over, summarizing the major changes that have taken place in the year hbase. The most influential event is the release of HBase 0.96, which has been released in a modular format and provides many of the most compelling features. 　　These characteristics are mostly in yahoo!/facebook/Taobao/millet and other companies within the cluster run a long time, can be considered more stable available. 1. Compaction Optimization HBase compaction is a long-standing inquiry ...

The end of 2013, we based on the past year's user access, exchange and sharing and the project itself update frequency and other aspects of the open source China's nearly 30,000 open source software statistics, so that the top 10 most popular open source software, for reference only. The list is mainly for domestic open source software, the list of 10 open source software is not the same type, although put together is not very scientific. 　　We only select from a few angles, including user access, software updates, and user discussion of the software. 1. Goagent ...

The performance of the MapReduce paradigm is not always ideal in the face of large-scale computational-intensive algorithms. 　　To address its bottlenecks, a small start-up team built a product called PARALLELX, which will bring significant improvements to the Hadoop task by leveraging the computing power of the GPU. Parallelx's co-founder, Tony Diepenbrock, says this is a "GPU compiler that translates user code written in Java into OpenCL and is shipped on Amazon's AWS GPU Cloud ...

Naresh Kumar is a software engineer and enthusiastic blogger, passionate and interested in programming and new things. 　　Recently, Naresh wrote a blog, the open source world's two most common database MySQL and PostgreSQL characteristics of the detailed analysis and comparison. If you're going to choose a free, open source database for your project, you may be hesitant between MySQL and PostgreSQL. MySQL and PostgreSQL are free, open source, powerful, and feature-rich databases ...

As a software developer or DBA, one of the essential tasks is to deal with databases, such as MS SQL Server, MySQL, Oracle, PostgreSQL, MongoDB, and so on. As we all know, MySQL is currently the most widely used and the best free open source database, in addition, there are some you do not know or useless but excellent open source database, such as PostgreSQL, MongoDB, HBase, Cassandra, Couchba ...

Over the past three years, the Hadoop ecosystem has expanded to a large extent, with many major IT vendors introducing Hadoop connectors to enhance the top tier of Hadoop or the Hadoop release that the vendor uses. 　　Given the exponential growth in the deployment rate of Hadoop and the growing depth and breadth of its ecosystems, we wonder whether the rise of Hadoop will lead to the end of traditional data warehousing solutions. We can also put this issue in a larger context to discuss: to what extent, large data will change ...

Hadoop is a large data distributed system infrastructure developed by the Apache Foundation, the earliest version of which was the 2003 original Yahoo! Doug cutting is based on Google's published academic paper. Users can easily develop and run applications that process massive amounts of data in Hadoop without knowing the underlying details of the distribution. The features of low cost, high reliability, high scalability, high efficiency and high fault tolerance make Hadoop the most popular large data analysis system, yet its HDFs and mapred ...

What exactly is hive? Hive was originally created and developed in response to the need for management and machine learning from the massive emerging social network data generated by Facebook every day. So what exactly is the definition of Hive,hive's official website wiki? The Apache hive Data Warehouse software provides query and management of large datasets stored in distributed, which itself is built on Apache Hadoop and provides the following features: it provides a range of tools Can be used to extract/Transform Data/...

Twill, formerly known as Weave, has now become one of the new members of the http://www.aliyun.com/zixun/aggregation/14417.html ">apache incubator Project, 　　It is designed to simplify the operation of applications in Yarn/hadoop. The fact that Hadoop is now a compelling technology solution is almost no doubt. The success of this project has been achieved with the release of its version 2.0.

To use Hadoop, data consolidation is critical and hbase is widely used. In general, you need to transfer data from existing types of databases or data files to HBase for different scenario patterns. The common approach is to use the Put method in the HBase API, to use the HBase Bulk Load tool, and to use a custom mapreduce job. The book "HBase Administration Cookbook" has a detailed description of these three ways, by Imp ...

Http://www.aliyun.com/zixun/aggregation/14223.html "> Application system, the log is an indispensable important part of all the application of error information should be able to find in the log file, 　　Some application system log may be very small, some large application system log is quite large, while the log file must be user-friendly and search, to have a high performance, otherwise it will affect the performance of the application system. Because the log usually involves I.

Within a few years, the NoSQL database has focused attention on performance, scalability, flexible patterns, and analytical capabilities. 　　Although relational databases are still a good choice for some use cases, like structural data and applications that require acid transactions, NoSQL is more advantageous in the following use cases: The data stored is essentially semi-structured or loosely-structured. 　　Requires a certain level of performance and scalability. 　　The application to access the data is consistent with the final consistency. Non-relational databases typically support the following features: Flexible ...

In the work life, some problems are very simple, but often search for half a day can not find the required answers, in the learning and use of Hadoop is the same. 　　Here are some common problems with the Hadoop cluster settings: 3 models that 1.Hadoop clusters can run?  Single-machine (local) mode pseudo-distributed mode 2. 　　Attention points in stand-alone (local) mode? In stand-alone mode (standalone) ...

Facebook is the world's biggest social networking site, and its growth is driven by open source power. 　　James Pearce, the head of Open-source project, said that Facebook began with the first line of writing its own PHP code, starting with the MySQL INSERT statement, and that open source has been incorporated into the company's engineering culture. Facebook is not only open source, but also open source its internal projects, internal results feedback to the open source community, it can be said that this is a great company should be the attitude. By constantly open source yourself ...

"Now is the best time for companies to apply Hadoop. "Jeff Markham, Hortonworks's chief technology officer, said in a speech at the 2013 China Hadoop Technology Summit, which was held at the end of November. At this summit, Hadoop entered the 2.0 era as the focus of people's talk. 　　Jeff Markham says Hadoop 2.0 has a stronger, broader range of new features that meet the needs of enterprise users, making up for the lack of Hadoop 1.0 and more in line with the needs of business users. Ha ...

A new Hadoop SQL database query engine http://www.aliyun.com/zixun/aggregation/14417.html ">apache Tajo recently won the South Korean telecom operators SK Telecom's favor. Geun-tae Park, senior manager of SK Telecom Data Technology Laboratory, said: After extensive research on the current available data analysis technology, we found that the Apache hatching project Tajo can be implemented in had ...

Netflix recently launched an Open-source tool called Suro that collects event data from multiple http://www.aliyun.com/zixun/aggregation/15818.html "> Application Servers" and real-time directed delivery to target data platforms such as Hadoop and Elasticsearch. 　　Netfix's innovation is expected to be a major data-mainstream technology. Netflix uses Suro for real-time orientation of data sources to target hosts, Su ...