Metamodel: Provides unified data access across a variety of data stores

Recently, Human inference and the Apache Software Foundation (ASF) announced the donation and acceptance of the Metamodel project separately, Metamodel will become an Apache incubator project in the future. Previously, Metamodel followed the LGPL license, managed by the human inference product development team, but has now shifted to ASF and will follow new licenses, new communities, and new management. So what exactly is this project? What is it used for?

Metamodel is a Java class library designed to provide a single interface that can interact with any data store, whether it is a relational database, a NoSQL database, or a spreadsheet or other format file. We mean, by interacting, searching for metadata, querying, writing, or changing the data stored in the data store. It is clear that high-level abstractions can lose some detail and risk excessive generalization and loss of important features. We do not want to reduce the functionality of a relational SQL database to a full table scan such as (SELECT * from [table]). On the other hand, we don't want to expose features that can only be used on a specific brand-specific version of the SQL Server, because these features cannot be used on any other data store. Ultimately, we want to build data interaction based on existing conventional skills, such as standard SQL.

Meta data processing

So what is the abstraction of the Metamodel project for the method of data storage? The project publishes a very SQL-like query model through the Java interface, and sometimes the model can be parsed from the character. By defining a query as a regular Java object, you can easily parse the query (depending on the underlying technology) and choose the best execution strategy based on the underlying technical details. This means that Metamodel not only provides an interface, but also provides a complete query engine that can handle some or all of the tasks while querying. If you are using a relational JDBC database, a 99% query is executed on the local engine of the database. However, if you use Metamodel, you can use its query engine to slice and dice the data to perform the same query on a CSV file or Excel spreadsheet. At the same time, you do not have to modify the query statement at all.

Of course, this assumes that the metadata for your data store is compatible with the data structure. Different data stores expose or infer meta data in different ways. The JDBC database typically exposes metadata through the JDBC metadata API. File formats, such as CSV and Excel tables, are not explicitly defined, and you can usually read the first line of the file to get their metadata. There is a relatively extreme situation, there are some NoSQL database even no metadata. Metamodel lets you choose how to expose metadata, you can specify whether to expose it programmatically, or to infer metadata by checking the first n records of the data store.

The most central design in Metamodel is the DataContext interface, which represents the data store, which can be used to browse and query the data store. In addition, its sub-interface Updateabledatacontext represents a writable data store that can be used to update data. Once you have a DataContext instance, you can learn more or less about using the class library code completion function. Here are a few typical examples of DataContext implementations, let's see how they are instantiated:

A DataContext for a CSV file Updateabledatacontextcsv = new Csvdatacontext (new file ("Data.csv")); A DataContext for a Excel spreadsheet Updateabledatacontext Excel = new Exceldatacontext (New File ("Spreadsheet.xlsx"  
 
       ));  
A DataContext for a JDBC database (can use either DataSource or Connection) Java.sql.DataSourcedataSource = ...  
 
       UPDATEABLEDATACONTEXTJDBC = new Jdbcdatacontext (DataSource); A DataContext for this XML file (where metadata is automatically inferred) DataContext XML = new Xmldomdatacontext (new 
 
       File ("Data.xml")); A DataContext for connecting to Salesforce.com ' s data Web services Updateabledatacontextsalesforce = newsalesforced  
 
       Atacontext (username, pw, securitytoken);
          A in-memory DataContext for POJOs (useful for testing and mocking) person Record1 = ...
Person Record2 = ... Tabledataprovidertabledataprovider = new Objecttabledataprovider ("persons", PERSON.CLASS, Arrays.aslist (Record1, record2));
 
 Updateabledatacontextpojos = new Pojodatacontext ("schema", Tabledataprovider);

Metadata is extremely important for metamodel, not only for managing data structures, but also for defining queries. If your query is to ensure that the appropriate metadata is used, you need to devote a great deal of effort to ensuring the security of the query. Therefore, as a developer, you must hold the metadata object before querying. For example, if you know that there is a table, the table name is Order_line, it has one column is price, and the other is order_id, then you can use the usual hard-coded way to query for the required metadata (obviously, only if you are familiar with the data store to use this way):

Datacontextdatacontext = ...//The DataContext 

object represents the ' connection '  
          Table orderlines = DataContext . Gettablebyqualifiedlabel ("Order_lines");  
          Column Price = Orderlines.getcolumnbyname (' price ');
          Column orderId = Orderlines.getcolumnbyname ("order_id");

In addition, you can use the API to dynamically acquire metadata based on exploratory methods. When you want to provide users with customized queries, this way can play a huge role in this way, you can provide users with the available tables and columns, let the user select the desired query.

schema[] schema = Datacontext.getschemas ();  
table[] tables = Schemas[0].gettables ();  
column[] columns = Tables[0].getcolumns ();

Metamodel also has a very important idea that it treats metadata, queries, and other data interactions as objects. A query is an ordinary Java object in Metamodel, so you can tamper with it before you execute it, or you can distribute it. This allows applications to create composite workflows that are done in collaboration with different blocks of code, and no longer have to face lengthy SQL string operations when optimizing the query plan. It is also useful for type safety, for example, a query model that describes a model based on a type-safe structure such as columns and tables, rather than using ambiguous string constants.