Search and BI tools combine enterprise-wide search in the ascendant

Intermediary transaction SEO diagnosis Taobao guest Cloud host technology Hall

"Research has shown that complete corporate search has repeatedly become the most prioritized consideration in the minds of the subject," said Sue Feldman, an analyst at the Internet Data Center (IDC). "Over the past two years, companies have become even more important when they find that their decision-making system has been ignoring the content level," she said. "

Most of the data you need is hidden------but where? They are hidden in data that is not organized, and deep in the network.

Random search techniques are useful for retrieving basic data from a network, or for determining the location of a desired file on your hard drive. But only business intelligence (bi:business FDI) tools can truthfully information stored in a large corporate database. As a result, the demand for search engines and BI tools is growing for the knowledge worker. For them, the data that needs to be discovered and analyzed is seen.

It may exist in a structured database of companies, or it may exist in unstructured files and e-mails, on a regular network, or on numerous web sites that cannot be traced to a common web search engine. It exists and is hidden in a deep network.

Combination of search engine and BI tools

Combining search engines with bi tools makes things easier. data, whether organized or organized, on a regular network or on a deep network, can be accessed via a generic web search engine interface. Let's assume that if you want to understand why a particular product is falling. You can start with a customer Relationship Management database (CRM) Search request to determine who stopped ordering. But you may also want to find out if those customers have emailed or phoned to talk about their products.

These may also exist in CRM (Customer relationship Management) files, or it may be recorded in your notes by representatives of the call center that you specialize in dealing with unhappy customers.

But the point is, you can't get a full picture by looking at a data resource. The good news is that you have a lot of data sources, lots of tools, and many ways to improve your ability to get the data you need.

This article explores the combination of search engine and BI tools, an emerging market in which some manufacturers and industry analysts call it other names: Federated search, Corporate Search, custom search, knowledge management and information access. No matter what the terminology, it is stated that the combination is a growing trend.

The more data, the more data you need to mine

For example, Endeca, fast search, and other businesses are touting their ability to combine search engines and bi tools. Other businesses, including bright Planet (Bright planet) and deep Web Technologies (Deep Network technology), provide a common interface for searching multiple databases.

"The first step is to recognize the status of analysts, who are traditional consumers of BI, who have to go beyond their existing tools," said Ramon Barquín, head of the Barquín International and the first president of the Data Warehousing Society at the Washington consulting firm. "

Barquín insists that bi analysts, knowledge workers and random web searchers have missed out on "90% or more of the data they should have found for them------or because they can't search for corporate resources, such as files and e-mails, that are not organized, Or because they don't have tools that go beyond the normal web interface.

Deep networks, sometimes called hidden networks, or invisible network), including some Internet sites, if you know the Web site, you can visit it, and search in it, but if you do not know the site, the content of these sites will not be found by the commercial web search engine.

Some of the usual deep Web sites include those that specialize in weather data, airline travel schedules and price sites, stock sites, patent sites, phone number sites, and more. Researchers and web developers have a clearer picture of how much resources are hidden in a deep network than random users. They also know more about tools that enhance the breadth and depth of web search, such as sitemap Kyoto, mod oai, and OAIster.

"In 2000, we did some analysis and found that," said Jerry Tardiff, co-founder of the Bright Planet (Bright planet), a deep web search company The amount of files coming from those deep Web databases is much larger than the internet that everyone says is-------200 to 500 times times larger than a known internet network. "

So we need to consider what these efforts will bring to us, and what can be done today to make use of it? And what can we expect in the near future?

A common dream across two areas

It seems logical that anyone would want to get as much data from each search as possible, and that the truth is not that simple. Personal things, business strategies, and technology make this scenario complicated. For example, BI and search are traditionally divided into two different types of users, each of them operating tools.

"For a random user who is conducting an ad search, he wants to combine the data with the organized data in the search." The application of an organized bi interface is completely wrong. "For these users, search engines offer better apps," says Boris Evelson, an analyst at Forrester Research. "

In this case, it is best to apply a search engine that can comb both organized and non organized data at the back end and display the results in a single window. Such products are available from Endeca and fast. At the same time, some web search engines, such as ALACRA, Closerlooksearch and northern Light, are also working to develop this technology for enterprises.

On the other hand, Evelson continues, the sample example of a simple search is "no need to work for a heavy burden analyst, who needs to slice, organize, and integrate data (as outlined in a field or kind of data), from a summary to a detail, from one dimension to another." "

Evelson adds that these users need a traditional BI interface, like business objects or Cognos products, especially if they want to "analyze a pattern across time, then jump a step, analyze a pattern across geographies, and then combine the two, See how one of them affects the other. "

The BI tool produces the depth of understanding that text searches cannot achieve, but because of the hard structure of the underlying database, their insights are limited. In contrast, text search is not enforced by the bi "rules" and is not limited by the data structures that BI runs. The goal of the combination is: To enable people to conduct information mining and BI analysis of organized data, and to allow people to do random searches of organised information.

The progress made in the combination of BI and search

Some companies gain new insights by letting the average employee, not just the analyst, use the BI tool. For example, Labarge, a manufacturer of electronic components, placed the BI front-end of IBM's WebSphere and data-building tools in the main framework of the company's Enterprise resource Planning (ERP) system, enabling hundreds of employees to use the tool directly. Previously, employees had to rely on IT staff to report to them.

According to the information operations chief George Hayward, since the staff themselves are directly teaching the data, their understanding of the data relationship will be better. and learned to look at the data as a whole, rather than as a series of reports.

"They were able to use methods that had never been thought of before to make multi-level search requests for multiple data sources," he said. "They can ask better questions and get answers for themselves," he said. "

The difference between organized and non organized data is obvious. Business organizations are taking a variety of approaches to finding information and federated information from structured and unstructured data. One approach is to give the user a single search interface, and then detach the search request from the database request and another search request to invoke other resources. These resources to invoke may include their own hard drives, corporate storage networks, or the Internet. Then, after the search results are consolidated, the user will be shown a single display.

Another method is to add to the data storage repository a text file that is not organized, or related data. Then, only a single search query is made to the storage warehouse. The third approach is useful for businesses with hundreds of reports. This approach is to describe the reports that can be found through corporate search engines so that users can have access to the full report when necessary. Last fall, for example, Hyperion (now Oracle) announced a plug-in for its system search tool, allowing a comprehensive search of Hyperion company data stored by Google OneBox.

Although the consolidation of BI and search is underway, some of the benefits of the search function cannot be ignored. These are the advantages that anyone using the weakest search engine can tell you. There is a big difference between finding all the "black" or "white" words from the database, and finding only black or white widgets, or finding people whose surname is black or white. Company analysts, researchers, and knowledge work to find everything that is "about them", not "everything".

Enterprise-wide Search

According to Sue Feldman, an Internet resource Centre analyst, "there is a survey showing that full corporate search has repeatedly become a priority in the minds of the subjects." "It has become more important over the past two years, when companies have discovered that their decision-making systems have been ignoring content levels," she said. "

"Content," Feldman refers to data stored in content management systems, rather than data management systems, such as text data, HTML files, and other materials. "Very often, the most important business data is unstructured or semi-structured. "She continued.

One of the methods of mining unstructured or semi-structured data is the application of federated search engines. Federated search engines can run search requests for several databases at the same time. The user enters a single search term and chooses which database to search. The search engine contains the necessary APIs (application interfaces) to facilitate running these search requests on the appropriate database.

For example, the Bright Planet (DQM) search engine can run search requests on 70,000 public databases, as well as within a commercial organization's internal database. David Fuess, a computer scientist at the Livermore National Laboratory in Lawrence, said the group used the DQM search engine to find non-US users of dual-use export restrictions for American manufacturers. But he said: "Trying to build a joint search on your own might be daunting."

"You must first understand the type of search request that you are willing to run, and understand how to guide them on every resource you want to use," he said. "The biggest problem is discovering the depth of network resources available and understanding each interface." "

Bright Planet help fuess save trouble. Because it has assembled many of the available sources, and as part of DQM, the necessary links have been established. In addition to making searches on different databases easier, DQM also allows users to make the same search queries at set intervals, while search engines present only the latest or most recently updated data------so that the user does not have to read the same material every time.

"Google makes the search look simple," Fuess said, "but in fact when it comes to completeness, the search is not that simple." Using services like DQM helps to recognize this. If the relevant data is available somewhere, we can create the greatest opportunity to discover it. "

Commercial applications

Mixing bi and search engine business applications is still a sensitive topic, according to Forrester's Evelson. Few businesses are willing to discuss exactly what they are doing in this field. However, after all, some companies are willing to talk.

For example, a Saint Louis chemical company, Sigma-aldrich, uses Endeca's search technology to provide customers with the latest information about their products. Users search or click on a list of chemicals to get a description of it, formula, chart and price, whether there is inventory and storage in which warehouse, its material Safety Data description (MSDS) and other related data.

"If someone searches for a pharmaceutical product from Sigma Aldrich, they will also need to know the authenticity certificate and the certificate of origin." "If they're going to separate search for these two items, then we're wasting the user's time," says Carl Turza, director of information.

In addition to being released and operated at the site, he plans to expand the use of Endeca across the enterprise, including customer service and business intelligence.

"Because I don't need to predict these search requests, I just have to be able to point out the tools available for the existing content, get my in-house product manager to search it, get to know it, and expose the bugs." "he said." When people turn to search applications that weaken the web's base, the real value becomes apparent. "

At the same time, there is an example. The National Association for Education (NEA) has four Oracle databases. Its database contains membership records, financial and legal information for 14,000 affiliate sites and 320,000 participants. It uses fast radar "search BI" products, mining information in the Data Warehouse.

"We have a lot of data and we really want to find the right data and show it in an understandable way," said Bill Thompson, NEA's financial and Member Services Manager. It could be a scorecard, a graph, a speedometer. In order for the data to be presented to the user in the best possible way, no matter what form it is. "

Such applications are developing and revealing, and later this year, the "Search bi" function may be more complete.

Mining data that is not organized

These tools are still very new, but it is possible, just an intermediate step in the evolutionary process of "data acquisition and analysis". Fast, for example, has just purchased radar products last year and has established partnerships with information-forming companies Cognos and other companies. Bi manufacturer Cognos, Hyperion, information builders, and SAS support the Onebox services that Google offers to businesses. With Onebox, the database can be searched.

The end result may be that these bi/search solutions are ultimately merged with the database itself. It is rumored that Microsoft (NASDAQ: MSFT) wants to buy Yahoo. Forrester's Evelson predicts that database vendors will end up moving away from their current relational structure and moving closer to a search-based relational structure in order to better address the problem of unstructured data.

"The latest version of the relational database can handle XML (Extensible Markup Language), but that's just another structured data," he says. "They can't handle random text and e-mail messages that really do not have organized data, such as word processing files." However, database search engines have a perfect performance for similar things and embed structured data search capabilities into these search engines. "(Zebian: Admin01)