Interview:yarcdata on BIG DATA and SEMANTIC WEB

August by Ian Jacobs | Posted in: interviews, Semantic Web

I recently spoke with Shoaib Mufti, Yarcdata Vice President of the R & R, about Big Data and Semantic Web technology. Yarcdata is Cray subsidiary, accustomed to crunching lots of data. Yarcdata recently joined.

Ian: Why did Yarcdata join the consortium?

SM: Yarcdata have products, including the Urika data appliance, for manipulating graphs. Instead of reinventing the wheel, we found benefits in existing standards. We also decided to contribute and the significant semantic effort at the website.

Ian: How does your see Big Data relating to the Semantic Web?

SM: We think that for Big Data need standards, and that linked Data standards fit the bill. What's exciting is the opportunity-find in data something of value, the was non-obvious. For the need tools to join terms, reason and inference, and query. These is the fundamentals for getting value from big data. There is open standards for these capabilities and so we had moved away from proprietary solutions.

Ian: Let's start with a story.

SM : One of our customers are in the financial sector. They is very interested in understanding how changes in one part of their portfolio affect the rest of their portfolio. For example, if a company goes bankrupt, what's the downstream effect on other assets in a mutual fund? What is happens to the companies that depended on the now bankrupt company? How should the financial institution reorganize it portfolio based on these events?

SM: The first challenge the financial institution faces is data integration. They want to integrate public and private data, in large amounts. Because the market moves quickly and the financial stakes is enormous, they need fast integration. They cannot afford to wait months for the results of a analysis.

SM: The second challenge relates to query. They has multiple questions to ask through billion triples, and they need to reduce the cost of running those queries. This particular company identified some "forbidden queries" they would leads to a huge performance hits on their servers.

Ian: How does Semantic Web technology help?

SM: It makes them less dependent on database optimization. RDF and SPARQL is schema-less, which makes it much faster and easier to ask ad-hoc questions without of the performance hit. The flexibility to do AD-HOC queries efficiently have given this company a big competitive advantage.

SM: The story doesn ' t end there. Although their initial interest concerned portfolio optimization, the company found another use for the technology. There is legal penalties and public relations nightmares around insider trading. Detecting insider trading is challenging and can happen in many ways (such as someone providing a friend with insider info rmation). This financial institution realized they could use Semantic Web technology to detect insider trading effectively and Impro VE compliance.

Ian: The second time in a recent months people has told me about Semantic Web technology and compliance; See my interview with Paul Groth and Luc Moreau.

SM: that example of serendipity are not unique. In the We held a contest-the yarcdata graph Analytics challenge-for people to solve some Big Data Graph problems. The winners, from the Institute for Systems Biology (ISB), studied drug repurposing.

Ian: What's drug repurposing?

SM: I ' ll explain with an example:viagra. Viagra was originally developed for managing heart problems. The trials revealed an interesting side effect. The drug was repurposed.

Ian: Yay, science!

SM: Drug companies realize that for a number of "failed" projects, there is great opportunities to repurpose the Drugs. Our contest winners studied some data sets and found that a particular HIV drug could is repurposed to treat breast cancer . By querying diverse data sets from the literature and clinical trials, they were able to find a common pathway. The whole project took about six weeks, which was astonishing compared to the usual time it takes to develop a drug. What's more, the FDA approval time for repurposed drugs are much shorter than for new drugs.

Ian: How do you have these technologies benefited yarcdata?

SM: First, in cost savings. We can use software available in the ecosystem instead of writing proprietary utilities. For instance, we used-to-convert relational data to graphs with a proprietary tool; Now we can use the something like D2R. There is many such tools, related to inference and other capabilities.

SM: The second benefit is value. RDF is simpler than we former custom format, and this have made data integration both simpler and faster for us. One of our engineers ran a data integration project using the other techniques; The integration required several months. With RDF it took a week. And, we can more easily reuse existing data sets.

SM: I think many organizations face similar data integration challenges. In the enterprise, people use a bunch of heterogeneous systems:email, plain text, unstructured data, and structured data. Managing all of the data are a big challenge for any organization.

Ian: Does any of your customers choose to migrate to RDF after you ' ve worked with them?

SM: Absolutely. People use we appliance on premises since much of their data are sensitive for them. We work with them to convert their data to RDF, which we feeds to our appliance. They see firsthand our efficient conversion process and what fast we can do integration. One large clinic with a lot of unstructured data in data warehouses is so impressed they issued a directive the any new Data they create is available in RDF. Once people understand the simplicity, they say "Let's make this the the-the-do things going forward." Only a few of our customers is doing this now and we see it increasing.

Ian: Which industries do you see adopting RDF?

SM: I think the farthest along is life sciences, then financial, and then US government.

Ian: Shoaib, thank very much for sharing those stories!

Post Navigation

← News from the Automotive and Web Platform business Group | Blog Home | interview:alcatel-lucent on WebRTC with Anne Lee →

Interview:yarcdata on BIG DATA and SEMANTIC WEB