IBM software Architecture deals with astronomical data

IBM recently released a prototype of a software architecture capable of handling massive data streams. IBM's software is built for the ska (KM array) telescope project, which automatically classifies objects. Melanie Johnston-hollitt, a radio astronomer from Victoria University in Wellington, New Zealand, developed the system in collaboration with IBM.

The main purpose of the SKA project is to make unprecedented and accurate observations of the power supply through radar and antenna networks across Australia, New Zealand or across South Africa. One of the major design challenges is how to deal with the raw data of 1EB per day (1EB=1024PB). This is the amount of data that needs to be processed when the world's largest and most sensitive radio telescope is ready, and the project is expected to start in 2016. IBM says the amount of data already exceeds the total daily traffic on the Internet. The equivalent of more than 15 million 64G of ipod to store this data.

IBM announced the prototype system completed on November 10. This new automated data management software architecture is likely to make it easier for researchers to gather useful information from large data-gathering projects such as this square kilometer array of global telescopes, with which we can explore the mysteries of the universe.

With the help of Dr. Melanie Johnston-hollitt, IBM created the information-intensive framework (IIF). IBM mentioned that the software uses the International Virtual Astronomical Association ontology method to classify the collected data into concepts that astronomers can understand and provide intelligent "guided search" functions. The technical basis of ontology is the Network Ontology language (OWL). Astronomers hope to improve efficiency and creativity by automatically classifying.

Although the IIF was originally developed for the SKA project, it can also be applied in other areas. Douglas WATT,IBM, New Zealand's chief technology officer, explained that while the project was initially developed for SKA projects, the final project results could also be applied to other business organizations facing the "Data torrent". We have identified several local scenarios that benefit from automated performance data analysis and monitoring of trend changes, anomaly recognition, and increased decision making capabilities. Potential customers range from independent manufacturers and telecommunications companies to the entire transport network and healthcare systems.

In addition to the above topics, the IIF's future work will also include improving performance by leveraging parallel processing.

Readers interested in the SKA project can view pictures on Flickr, which explains some of the most impressive ska details.

(Responsible editor: The good of the Legacy)