The sharp weapon of Telecom enterprises ' participation in competition-data warehouse and data mining

Data Warehouse and data mining--a sharp weapon to participate in the competition of digital telecommunication enterprises
The solution of Guangdong Telecom Data Warehouse based on Sybase

Guangdong Institute of Telecommunication Science and technology

1 overview

With the opening of the telecom market, the competition will be more and more fierce. The reduction of profit makes it necessary to change from extensive operation to intensive operation, while the management decision requires as much quantitative basis as possible and as fast as possible. All of this requires technical support----data warehousing and data mining Guangdong Telecom Science and Technology Research Institute is the highest technical department of Communication Technology Support and scientific research development in South China, which is to support the system maintenance, network management system development, multimedia research, Network Technology and market research, billing system research and development, personnel training and measurement testing seven aspects of work. Research Institute currently has a more than 900 people in the prime of the energetic, strong research and development team.

The THINKER-BC2000 Multimedia network integrated business management system developed by the Institute is a unified integrated business management system. It provides a stable and flexible business support platform for telecom operators and ISPs at all levels. The platform can provide all the basic business of multimedia data communication and a variety of value-added services, the platform has a strong scalability, with rapid new business generation, promotion capabilities. The system can flexibly define the tariff policy of each service, and the bundled sales preferential strategy of each business, and provide the accurate and real-time billing function for each service. The Data Warehouse decision support system is developed based on Sybase Data Warehouse solution. Business decision makers can use this system to quickly and accurately understand the development of each business and provide a solid foundation for further decision support work.

2 main features of demand

Large amount of data
At present, the total number of dial-up users in Guangdong Public multimedia communication network has reached 700,000. According to business needs analysis, Guangdong province 163/169 net to 2002 end of the total number of users will reach more than 8 million, of which dial-up registered users up to 4 million, the main call users 3 million, card users 1 million, line users will also reach more than 10,000 households. There is a lot of important information hidden behind these vast numbers of data.
Complex business Structure
The end user of the system is the various departments within the telecommunications, so the end-user needs are different.
Constantly changing
Business policies are constantly changing. Performance is the constant change of tariff policy.
3 Program Introduction

The Data Warehouse system model of THINKER-BC2000 Multimedia Network integrated service management system is shown in the following figure:

　　
3.1 System software Platform

The implementation of data warehouse is a rather complicated process, which mainly includes five parts: Data Warehouse design modeling, data transformation and integration, data storage and management, data analysis and presentation, and Data Warehouse maintenance and management.

Sybase provides a complete set of packages covering the entire Data Warehouse lifecycle: Warehouse Studio, which includes data warehousing modeling, data integration and transformation, data storage and management, metadata management, and data visualization analysis. The following explains how we combine Sybase's products to do data warehouse design and development.
Design and modeling of Data Warehouse
The design work of data Warehouse plays an important role in decision support system, it needs to decide the subject according to the decision demand, from data source to data submission, to design the data organization of data warehouse logically, and to organize and provide information according to the way that business users can understand.

At this stage, we use the PowerDesigner warehousearchitect. Warehousearchitect is a highly optimized database tool, which is widely used in reverse engineering, modeling and Data warehouse design of data sources to meet the needs of each business. By integrating logic design, physical design and application modeling, Warehousearchitect facilitates the development and realization of data Warehouse.
Data Transformation and Integration
One of the biggest challenges in building a data warehouse is how to transform the original business data into a consistent format to better serve decision support. This includes testing and purifying the accuracy and consistency of existing data, transforming, extracting, transforming, loading to a data mart or data Warehouse, and periodically updating and managing the data. Powermart as a data extraction tool, from a variety of heterogeneous data sources to extract data, in the data extraction process, users can according to different extraction stages, flexible customization of various data extraction process, and regularly load data into the Data warehouse.

Powermart is an integrated suite of software products for building and managing data marts and analytics applications. Powermart delivers an open, scalable solution that focuses on the full lifecycle of data marts and analysis of application development and product management, enabling fast-changing large amounts of data on multiple platforms as a source of data, complex transformation processing, and support for high-speed data loading. Its metadata repository can coordinate and drive a series of core functions, including extraction, transformation, loading and management.

Powermart's graphical user interface helps data warehouse managers easily design complex source-to-target mappings that can then be automatically executed by Powermart powerful servers.
Data storage and Management
The data warehouse can be stored in a multidimensional database or a relational database or other special storage mode. Data storage to ensure data security, integrity, consistency, but also have a complex analysis of the efficiency of the query.

We chose Sybase's Data Warehouse products Adaptive Server IQ. Adaptive Server IQ is a relational database optimized for high performance decision support and Data Warehouse establishment. The key technology in IQ is vertical data storage (through columns rather than rows), bit-wise query indexing, and data compression.
Data analysis and presentation
Online analytical Processing (OLAP) is an analytical processing technology that collects information from an enterprise's data collection and uses mathematical operations and data processing techniques to provide a flexible, interactive approach to statistical, trending, and predictive reporting. Through a variety of OLAP tools to data warehouse data in Multidimensional Analysis, summary, form a chart or report form, so that decision-makers can clearly and intuitively see the results of the analysis, which is the data Warehouse system to achieve the goal.

The development and application of data Warehouse mainly includes structure design, data centralized organization and management, fast and efficient data access and so on. The access to data is generally done by more mature business intelligence tools, therefore, unlike OLTP system, data Warehouse system front-end development programming volume is relatively small, but its maintenance work time span is large, because the decision support application is more arbitrary, it is impossible to like the business system that fixed a unified operation mode.

BusinessObjects, as an early business intelligence provider to enter the Chinese market, has a streamlined, functional, and intuitive front-end presentation metadata management, which complements the IQ's high speed query efficiency.
Maintenance and management of data Warehouse
Metadata is data about data, the data that can represent and define the meaning of the data and the relationship between the components of the system, including keywords, attributes, data description, physical data structure, source data structure, mapping and transformation rules, synthesis algorithm, code, default value, security requirements and data time limit, etc. Managing metadata is the key to managing a data warehouse.

Sybase's Warehouse control Center provides a guaranteed technology for data warehousing solutions through centralized management of the metadata warehouse. From design and development to implementation to end-user access, the intensive integration and management of metadata generated by tools and databases guarantees the establishment of a truly enterprise-class data warehouse. Warehouse Control Center is a data Warehouse metadata management tool for data Warehouse developers based on Intellidex technology that can collect, capture, store, manage, and distribute logical, physical, and contextual information in a data warehouse environment. Instead of having to take care of its physical storage location is on the federated Data Warehouse, the distributed data Warehouse or both. Business users can browse metadata objects that are generated based on their requirements, and can even use publishing and enlistment performance requests or select additional performance.

3.2 System hardware Platform
Data Warehouse Server:
SUNE5500/8 cpu/4g ram/18g Hard Drive
Run Sybase IQ. Data Migration server:
IBM Netfinity7600 Intel PIII550/4 cpu/1g ram/36.4 G HD NT4
Run Powermart Server Management Web server two:
IBM Netfinity7600 Intel PIII550/4 cpu/1g ram/36.4 G HD NT4
Run the Bo Web Intelligence and warehouse control Center Server respectively. Storage network:
San Systems with Veritas
3.3 Main functions of the system
The data Warehouse and decision support system can effectively isolate the application of online transaction processing and online statistical analysis. It ensures the safe, stable, reliable and efficient operation of the business management system online, and also ensures that the decision support system can obtain the statistic data quickly and timely. The provincial and municipal business managers are able to produce predefined standard statistical reports on a monthly basis. Through a very simple and easy-to-use graphical interface, the business analyst can quickly and accurately query the semantic layer and quickly display the required business data, information and analysis results in a rich form, providing accurate basis for leading decision-making. To provide data mining capabilities, to explore the potential impact of business development factors.
Provide service to customer management system, provide customers with quick billing and various service list inquiries. And to provide the means to excavate big customers.

4 concluding remarks

We have successfully implemented the Data Warehouse system and developed a statistical analysis reporting system based on Bo (Business Object), using Sybase's Data Warehouse solution in the Guangdong visual and Fujian 163 online.

At present, Guangdong Province is building a new generation of multimedia network integrated business management system, including the latest version of the Data Warehouse system. And this data warehouse system is also using Sybase's Data Warehouse solution.