Event stream processing framework nesper for. net

Complex Event Processing introduces the open-source Esper. nesper is an event stream processing (ESP) and complex event processing (CEP) system, which can monitor the event stream and trigger certain actions when a specific event occurs-it can be seen that the database is in turn, the statements are fixed, and the data stream is in and out. Event processing is a development trend in the software industry. It has been added to this market by several large manufacturers and many startups. Examples of common applications include automatic system transactions, bam, RFID, advanced monitoring systems, fraud detection, and even direct integration into SOA.

Microsoft has a streaminsight integrated with SQL Server, but it is not open-source. Open-source nesper is another option other than streaminsight. Although streaminsight is integrated in SQL Server, it does not need SQL server. It works a bit like a database inversion and Esper. The nesper engine allows applications to store query and run data in place of data storage and query and store data. Nesper provides two mechanisms to handle events:
1. nesper provides an Event Mode language to specify expression-based event mode matching. This pattern matching engine is implemented through a state machine. The event processing method matches the expected queue or nonexistent event or event combination. It includes the relationship between events based on time.
2. nesper also provides event stream query. This can enable the event stream to analyze the requirements of the CEP application. Event stream query provides window, aggregation, connection, and analysis functions to process event streams. These queries are implemented through the EPL statement (the Custom Event processing language (event processing Lauguage -- EPL) can express rich event conditions, relevance, and Time Window spans as much as possible, therefore, the required development effort is minimized during system development ). EPL is used for views. The view indicates that the constructed data needs to be put into an event stream to drive the data flow. Process the data in the process of data flow to obtain the final result we need.

Like SQL, EPL uses select clauses and where clauses. However, the EPL language replaces the tables in the database with the concepts of event stream and views. Similar to tables in SQL statements, views in EPL are defined as data that can be queried and filtered. Views sorts events, obtains analysis data from event attributes, groups events, and independently processes event attribute values.
The following is a simple EPL statement that calculates the average price of the stock event stream in the last 30 seconds.

Select AVG (price) from stocktickevent. Win: Time (30 sec)

The following EPL statement returns the average price of each feature category within 100ticks.

Select symbol, AVG (price) as averageprice from stocktickevent. Win: length (100) group by symbol

The following example connects two event streams. The first event stream contains events that trigger warnings within 30 minutes. The second event is a refund within the last 30 seconds. The event stream is connected through the accout number.

Select fraud. accountnumber as accntnum, fraud. warning as warn, withdraw. amount as amount, max (fraud. timestamp, withdraw. timestamp) as timestamp, 'withdrawraud 'As DESC from fraudwarningevent. win: Time (30 min) as fraud, withdrawalevent. win: Time (30 sec) as withdraw where fraud. accountnumber = withdraw. accountnumber

3. event pattern matching
Pattern matching is implemented through the state machine. (NFA, DFA, and so on. I learned the compilation principles ). The expression of a pattern can contain a filtering expression combined with a pattern operation. Mode expressions can be nested. Below are five types of pattern operators ):

(1) Create and terminate the control pattern finder OPERATOR: Every

(2) logical operators: And, or, not

(3) Temporal operators (controlling event sequence):->

(4) Where-condition (used to terminate the Event Filter and pattern finder): for example, Timer:

(5) event time observed by the observer: for example, Timer: interval, Timer:

Below is a simple model, which gives a warning to IBM stocks whose price is greater than 80 within 60 seconds:

Every stocktickevent (symbol = "IBM", price> 80) Where Timer: Within (60 seconds)

Send a warning every 5 minutes:

Every Timer: At (5 ,*,*,*,*)

After event a occurs, event B or event C occurs:

A-> (B or C)

In one mode, a property of the next event must match the attribute of the first event:

Every A = eventx-> every B = eventy (objectid = A. objectid)

4. Combination of pattern matching and event stream analysis
When a series of event streams are matched by the pattern, the results of the pattern match can be applied for analysis and processing. The following pattern describes the difference between the ID of an event within 10 seconds and that of the subsequent event. This statement further counts all Id groups in this case.

Select a. ID, count (*) from pattern [every A = status-> (Timer: interval (10 sec) and not status (ID = A. ID)] group by ID
5. named Windows)
The naming window is a global data window. It can participate in many query statements and be queried, inserted, or deleted. The naming window is similar to the table concept in the database system.
Next, we will create a naming window:
Create window alertnamedwindow as (origin string, priority string, alarmnumber long)
The naming window can also trigger query, modification, and deletion when an event is reached. Here is an example of statistics:

On triggerevent select count (*) from alertnamedwindow

 

Nesper for. net

Esper status quo: event stream processing framework

Https://bitbucket.org/emertechie/syslognet

Http://www.cppblog.com/mysileng/archive/2012/10/11/193182.html

Get started with CEP