Start the Esper tour in 5 minutes and start the esper tour

Start the Esper tour in 5 minutes and start the esper tour

Original Author: CoffeeOneSugar

Translation: Liu binhua

 

In my previous article, I mentioned my recent passion for Complex Event Processing (CEP) (Complex Event Processing ). In short, CEP uses data streams as input and redirects data (or part of data) to listeners based on a series of predefined rules, or Pattern in data discovery). CEP is particularly useful when a large amount of data is generated and needs to be analyzed in real time.

 

There is a very good software project that allows you to do this, called ESPER. You can find the website of this project here. Esper provides programmers with a language called EPL, which is similar to the SQL language. It can be used to configure rules and patterns that will be applied to data streams.

 

The documents attached to Esper are quite complete, but there is no actual example, which makes it difficult to use. So here is a 5-minute Esper guide. Although the following example is written in Java, Esper supports both Java and C. I suppose you have downloaded Esper. If not, click this link. After extracting the downloaded file, you should find a folder named esper-3.1.0 somewhere on your disk. (TRANSLATOR: This article is earlier. The latest esper version is 5.2.0)

 

Before you start, you need to add several library files to your project, of course, esper-3.1.0.jar is one of them, you also need four other third-party library files, they can be found in the esper-3.1.0.jar/esper/lib folder.

 

Let's start with 5 minutes. Before you drop the data to be analyzed into the CEP Engine Pipeline, You need to structure the data into objects. Let's use a simple example to write a class (or its bean) to describe the price of a stock at a given time:

 

import java.util.Date;    public static class Tick {        String symbol;        Double price;        Date timeStamp;         public Tick(String s, double p, long t) {            symbol = s;            price = p;            timeStamp = new Date(t);        }        public double getPrice() {return price;}        public String getSymbol() {return symbol;}        public Date getTimeStamp() {return timeStamp;}         @Override        public String toString() {            return "Price: " + price.toString() + " time: " + timeStamp.toString();        }    }

 

It has three attributes: stock code, price, and timestamp. Before we start to generate hundreds of millions of data, we need to notify the engine of what objects it needs to process. This is achieved by using a Configuration object when instantiating the CEP engine:

 

import com.espertech.esper.client.*; public class main {    public static void main(String [] args){     //The Configuration is meant only as an initialization-time object.    Configuration cepConfig = new Configuration();    // We register Ticks as objects the engine will have to handle    cepConfig.addEventType("StockTick",Tick.class.getName());    // We setup the engine    EPServiceProvider cep = EPServiceProviderManager.getProvider("myCEPEngine",cepConfig);   }}

 

For testing purpose, we now create a function to generate random data and drop them into the CEP engine. We call this function "GenerateRandomTick", which uses the EPRuntime object as the parameter, this object is used to pass events to the CEP engine:

 

import java.util.Random;import com.espertech.esper.client.*; public class exampleMain {   private static Random generator=new Random();    public static void GenerateRandomTick(EPRuntime cepRT){    double price = (double) generator.nextInt(10);    long timeStamp = System.currentTimeMillis();    String symbol = "AAPL";    Tick tick= new Tick(symbol,price,timeStamp);    System.out.println("Sending tick:" + tick);    cepRT.sendEvent(tick);}     public static void main(String[] args) {     //The Configuration is meant only as an initialization-time object.      Configuration cepConfig = new Configuration();     cepConfig.addEventType("StockTick",Tick.class.getName());       EPServiceProvider cep=EPServiceProviderManager.getProvider("myCEPEngine",cepConfig);       EPRuntime cepRT = cep.getEPRuntime();    }}

 

Now we have a workable CEP engine and continuously input false data. It is time to create our first rule. In the Esper statement, our first EPL statement. To do this, we need to ask the engine administrator to record our statements. The CEP engine then filters the data it receives based on the definition of the EPL statement. An event is triggered when the data meets the selection conditions or modes in the statement.

public static void main(String[] args) {    //The Configuration is meant only as an initialization-time object.    Configuration cepConfig = new Configuration();    cepConfig.addEventType("StockTick",Tick.class.getName());    EPServiceProvider cep = EPServiceProviderManager.getProvider("myCEPEngine",cepConfig);    EPRuntime cepRT = cep.getEPRuntime();     // We register an EPL statement    EPAdministrator cepAdm = cep.getEPAdministrator();    EPStatement cepStatement = cepAdm.createEPL("select * from " +                                "StockTick(symbol='AAPL').win:length(2) " +                                "having avg(price) > 6.0");  }

 

Here, we set the rule to trigger an event whenever the average value of the last two data attempts is greater than 6.0.

 

The next step is to create a listener and associate it with the events generated by our selection rules. You can do this:

 

cepStatement.addListener(new CEPListener());

 

There are different ways to implement listeners. The following is the simplest one. Here, the listener simply prints the object it receives from the engine:

 

public static class CEPListener implements UpdateListener { public void update(EventBean[] newData, EventBean[] oldData) {         System.out.println("Event received: "                            + newData[0].getUnderlying());    }}

 

So far, it looks pretty good. Now is the time to test our code. Let's generate some data to see if everything works normally. You can add the following lines to the main function:

 

for(int i = 0; i< 5; i++)    GenerateRandomTick(cepRT);

 

Now all the code looks as follows (I put the Tick class and the entry function together so that you can copy and paste them into a file and run them)

 

import com.espertech.esper.client.*;import java.util.Random;import java.util.Date; public class exampleMain {     public static class Tick {        String symbol;        Double price;        Date timeStamp;         public Tick(String s, double p, long t) {            symbol = s;            price = p;            timeStamp = new Date(t);        }        public double getPrice() {return price;}        public String getSymbol() {return symbol;}        public Date getTimeStamp() {return timeStamp;}         @Override        public String toString() {            return "Price: " + price.toString() + " time: " + timeStamp.toString();        }    }     private static Random generator = new Random();     public static void GenerateRandomTick(EPRuntime cepRT) {         double price = (double) generator.nextInt(10);        long timeStamp = System.currentTimeMillis();        String symbol = "AAPL";        Tick tick = new Tick(symbol, price, timeStamp);        System.out.println("Sending tick:" + tick);        cepRT.sendEvent(tick);     }     public static class CEPListener implements UpdateListener {         public void update(EventBean[] newData, EventBean[] oldData) {            System.out.println("Event received: " + newData[0].getUnderlying());        }    }     public static void main(String[] args) { //The Configuration is meant only as an initialization-time object.        Configuration cepConfig = new Configuration();        cepConfig.addEventType("StockTick", Tick.class.getName());        EPServiceProvider cep = EPServiceProviderManager.getProvider("myCEPEngine", cepConfig);        EPRuntime cepRT = cep.getEPRuntime();         EPAdministrator cepAdm = cep.getEPAdministrator();        EPStatement cepStatement = cepAdm.createEPL("select * from " +                "StockTick(symbol='AAPL').win:length(2) " +                "having avg(price) > 6.0");         cepStatement.addListener(new CEPListener());        // We generate a few ticks...        for (int i = 0; i < 5; i++) {            GenerateRandomTick(cepRT);        }    }}

Output:

 

log4j:WARN No appenders could be found for logger (com.espertech.esper.epl.metric.MetricReportingPath).log4j:WARN Please initialize the log4j system properly.Sending tick:Price: 6.0 time: Tue Jul 21 01:11:15 CEST 2009Sending tick:Price: 0.0 time: Tue Jul 21 01:11:15 CEST 2009Sending tick:Price: 7.0 time: Tue Jul 21 01:11:15 CEST 2009Sending tick:Price: 4.0 time: Tue Jul 21 01:11:15 CEST 2009Sending tick:Price: 9.0 time: Tue Jul 21 01:11:15 CEST 2009Event received: Price: 9.0 time: Tue Jul 21 01:11:15 CEST 2009

 

As you can see, only the average data of the last two rows is greater than 6, so only one event is finally triggered by the engine. Pretty good!

 

Oh, you may be worried about the first line in the output. Yes, there is a small problem here. In fact, the log generation package log4j used by Esper causes this warning. It can be configured through a file named log4j. xml, and you can find it under the/etc directory in an example in esper-3.1.0/examples. I don't think it is a good idea to create an xml configuration file for all our programs. So in the following code, we use some tips to configure our logger, add some import and code at the beginning of your file:

 

import org.apache.log4j.ConsoleAppender;import org.apache.log4j.SimpleLayout;import org.apache.log4j.Level;import org.apache.log4j.Logger;//and this in the main function before the rest of your code:public static void main(String [] args){       SimpleLayout layout = new SimpleLayout();      ConsoleAppender appender = new ConsoleAppender(new SimpleLayout());      Logger.getRootLogger().addAppender(appender);      Logger.getRootLogger().setLevel((Level) Level.WARN);(...)

 

5 minutes later.

 

In the next article, I will go deeper into EPL statements and provide code to connect two engines to implement the so-called event refinement: it seems that the author has never updated since, so don't count on the future :))

 

Original article address: Workshop /. Liu binhua yuanchuang translation. For more information, see the source.