Test the infobright engine of MySQL

I have changed from Oracle practitioners to MySQL practitioners. Of course, the reason for giving up oracle is mainly because I can write as little as possible in the new unit.Code.

Now I am facing a data warehouse. Just like the previous company, the data warehouse makes our technical staff most vulnerable to the large data volume, storage, I/O, efficiency is disappointing. Every time there are some statistical analysis requirements, the query in the List table is simply a thank you to me. So what is the best solution? I think many people will tell me how to use hadoop. Yes, hadoop is now the most popular big data solution, but there is a problem with hadoop, that is, no one in our team can do it now. It takes a long time to learn and build it, in addition, I am not willing to give up the traditional relational database easily. To be honest, I can even resist mango.

I think someone previously recommended greenplum to us in the unit. According to my search, greenplum is fast and postgresql-based. It was originally a relational database, there is no pressure to learn, but commercial Authorization requires money. Now we can't apply for funds.

I naturally thought of the columnar database sybaseiq. Of course, it also requires money, but the infobright engine of MySQL does not need money, at leastCommunityThe version is free of money. This is the benefit of plug-in databases. There is always an engine suitable for you. The Community version of infobright does not support DML, but it is nothing for me to test, and it will be nothing in the future. Because the data is imported once a day, loaddata can be fully held.

At http://www.infobright.org/, I'm comfortable with seeing organd I'm willing to pay for it. For installation, install this item directly after it is downloaded. MySQL does not need to be installed because it comes with one. It seems that the built-in infobright only supports the MyISAM and memory engines, but this is not a problem. InnoDB can download another MySQL. The installation of the Linux version may be a little complicated. What I can find on the Internet is based on a copy written by a person named Zhang banquet. I have no right to speak without testing. I am using the Windows version, one way to the next can be, after installation only need to modify its configuration file my-ib.ini can be, it is written in very clear.

Let's talk about the test result. I tested it on an enterprise machine. According to the confidentiality principle of the Organization, I cannot bring it back. But through my three-day simple test, on my PC (I5 processor, 2 GB memory, Windows XP SP2, 7200 GB hard drive (RPM )). It takes about 2 minutes to use loaddata to import 2 GB of data. A 0.1 billion-piece data table is associated with its own Query Count (*), that is, 2 minutes. Performance is quite tough. If you take the database test to the PC server, it is estimated that the effect will shine.

For more scientific tests, see http://www.actionsky.com/infobright/styles/performance/default.html. This seems to be the proxy of infobright in China.

let's talk about it. When I was in a company, the most important thing I did was to optimize SQL. I also did a lot of things under the Oracle framework before, but with the increase in data volume, how can I optimize it, i/O is also a bottleneck, and indexing is useless. I feel helpless. At that time, I was wondering if it was true that the relational database would be over. Will Mongo be the world of nosql in the future? After tests over the past few days, I found it still takes some time to replace nosql with relational databases. In systems that require real-time processing such as Weibo, nosql may be very popular, but in the data warehouse field, I think the relational data represented by columnar databases will shine brightly, now we can take a look at the mainstream: sybaseiq, which may be the earliest columnar database; Nanda universal GBASE, national goods boutique; infobright, open-source, if capable, you can change it yourself. By the way, greenplum is fast, but greenplum is built on PostgreSQL. PG is open-source. If the company is powerful, it can even build a database with no less performance than GP, I hope to use it in the future.