Apache Ingite of Memory Database

In the previous article, we did the technology selection of the memory database:

Memory Database Technology Selection

In this article, we continue to delve into Apache Ignite while sharing some of us. NET coding practices.

First, Apache Ignite is an in-memory data organization that is a high-performance, integrated, and distributed memory platform that performs transactions and calculations in real-time in large data sets, with an order of magnitude improvement over traditional disk-or flash-based technologies.

which

Data Grid: The Ignite memory data grid is an in-memory key-value store that can cache data in the memory of a distributed cluster.
It uses strong semantic data location and relational data routing to reduce the noise of redundant data, so that it can increase the number of nodes linearly, up to hundreds of nodes.
The Ignite data grid is fast enough to undergo official testing and is currently one of the fastest implementations in a distributed cluster that supports transactional or atomic data.

SQL grid: memory SQL Grid provides the functionality of a distributed memory database for Apache Ignite, which is horizontally extensible, fault tolerant, and compatible with the ANSI-99 standard of SQL. The SQL grid supports full DML commands, including SELECT, UPDATE, INSERT, merge, and delete. Supports distributed SQL join associations at the same time

RDBMS Integration: Ignite supports integration with a variety of persistent storage, it can connect databases, import schemas, configure index types, and automatically generate all the necessary XML or mapping configurations and Java domain Model Pojo, which can be easily downloaded and copied into their own projects.
Ignite can be integrated with any JDBC-enabled relational database, including Oracle, PostgreSQL, MS SQL Server, and MySQL

The features of Apache Ignite are:

• Distributed key-value storage: The Ignite data grid is an in-memory key-value store, a distributed partitioned hash, in which each node in the cluster holds part of all data, which means that the more nodes within the cluster, the more data can be cached. Ignite determines the location of the data through a pluggable HA algorithm, each client can determine which node a key belongs to by inserting a custom hash function, and does not require any special mapping services or named nodes.
• Memory optimization: Ignite supports 2 modes of data caching in memory, both inside and outside the heap. When the cache data occupies a large heap and exceeds the Java Master heap space, the out-of-heap storage can overcome the long pauses caused by the JVM garbage collection (GC), but the data is still in memory.
• SQL query: Ignite supports querying the cache using standard SQL syntax (ANSI 99) and can use any of the SQL functions, including aggregation and grouping.
• Distributed affinity: Ignite supports distributed SQL associations and cross-cache associations.
• Acid transaction: Ignite provides a fully acid-compliant distributed transaction to ensure consistency. Supports optimistic and pessimistic concurrency models and isolation levels for read commits, replicable reads, and serialization. The ignite transaction uses the two-phase commit protocol, and the optimizations for many of the phases of the commit are also properly performed.
• Same-write and same-read: Write mode allows data to be updated in the database, and read-through mode allows reading from the database.
• Database Asynchronous Update: Ignite provides an option to perform database updates asynchronously through a write-behind cache
• Automatic persistence: Automatically connects to the underlying database and generates XML object-relational mapping configuration and Java domain model Pojo
• Database support: Ignite can be automatically integrated with external databases, including RDBMS, NoSQL, and HDFs.

Apache Ignite has a very advanced clustering capability and is very flexible to deploy.

• Node equality: Ignite no master node or server node, no worker node or client node, all nodes are equal according to Ignite's point of view. But developers can configure nodes as Master,worker or client and data nodes.
• Autodiscover mechanism: The Ignite node is automatically aware of the scalability of the cluster, do not need to restart the cluster, simply start the newly joined nodes and then they will automatically join the cluster. This is done through a
• Discovery mechanism, which he can find each other, ignite by default the TCP/IP protocol is used as the implementation of node discovery, or it can be configured as multicast-based or static IP-based, and these methods are suitable for different scenarios.
• Deployment mode: The ignite can be run independently or in a cluster, or it can embed several jar packages inside the application to run in embedded mode, run in a Docker container and in environments such as Mesos and yarn, run in a physical machine, or run in a virtual machine. This broad adaptability is one of his great advantages.
• Configuration: Most of the configuration options for Ignite are supported by the spring-based approach to XML configuration and programmatically through Java code.
• Client and server: Each node in the ignite is equal, but the node can be configured as a client or server, the server node participates in caching, computation, streaming and so on, while the Native Client node provides the ability to connect the server remotely. The Ignite Native Client can use the full ignite API, including near caches, transactions, calculations, streams, services, and so on.
• All ignite nodes are started by default in service-side mode, and client mode needs to be explicitly enabled.

The above outlines the architecture and functionality of Apache Ignite, and now let's share the technical prototyping validation done in the code sample:

1. Launch Apache Ignite

The code starts a ignite node by calling Ignition.start ().

Execute Apache directly. Ignite.exe can also start a Ignite node, which internally references the Apache.Ignite.Core.dll, calling the Ignition.start () method

• Ignite can host in console and WinForm
• Ignite relies on Oracle JDK 7 and later
• Ignite can be run independently
• There are cross-process access scenarios

2. Create the specified cache area

3. Data Write Cache

4. Data query

5. Data Correlation Query

6. Querying a specified field

7. Full-Text Search

8. Apache Ignite Cluster deployment

Node equality

Ignite no master node or server node, no worker node or client node, all nodes are equal according to Ignite's point of view. But developers can configure nodes as Master,worker or client and data nodes.

Automatic discovery mechanism

The ignite nodes are automatically aware of the scalability of the cluster, do not need to restart the cluster, simply start the newly joined nodes and then they will automatically join the cluster. This is achieved through a discovery mechanism, he nodes can find each other, ignite by default using the TCP/IP protocol as the implementation of node discovery, can also be configured to multicast-based or static IP-based, these methods for different scenarios.

Deployment mode

The ignite can be run on its own or in a cluster, or it can embed several jar packages in an embedded application, run in a Docker container and in environments such as mesos and yarn, can run in a physical machine, or run in a virtual machine. This broad adaptability is one of his great advantages.
How to configure
Most of the configuration options in ignite are supported by the spring-based approach to XML configuration and the programmatic configuration of Java code, which is an important advantage.

9. Client and server

Each node in the ignite is equal, but the node can be configured as a client or server, the server node participates in caching, computation, streaming, and so on, while the Native Client node provides the ability to connect the server remotely. The Ignite Native Client can use the full ignite API, including near caches, transactions, calculations, streams, services, and so on.

The code starts in client mode ignite

Client/server architecture, brings a lot of problems!!

• Each host that uses ignite installs the JDK and initiates a ignite client node
• High cost of development, operation and management

Apache Ignite REST API

Apache Ignite provides restful APIs that support reading, writing, performing tasks, obtaining metrics, and more on the cache.
Enable HTTP connection: Copy Libs\optional\ignite-rest-http to Libs\ignite-rest-http.
Http://localhost:8080/ignite?cmd=version
Get or create cache
Http://localhost:8080/ignite?cmd=getorcreate&cacheName=partionedCache
Add
Http://localhost:8080/ignite?cmd=add&key=newKey&val=newValue&cacheName=partionedCache
Get
Http://localhost:8080/ignite?cmd=get&key=newKey&cacheName=partionedCache
SQL Query
http://localhost:8080/ignite?cmd=qryexe&type=Person&pageSize=10&cacheName=Person&arg1=1000 &arg2=2000qry=salary+%3e+%3f+and+salary+%3c%3d+%3f

The above is the technical sharing of the entire Apache ignite.

Zhou

2017/8/26

Apache Ingite of Memory Database