Drools Stream Integration

This passage discusses what to integrate a provided drools package into DataStream application.

Packaging:
If a MAVEN project is provided by customer. In this case, you need to ensure the Pom file contains the following:

<dependencyManagement> <dependencies> <dependency> <groupid>org.drools</groupid&        Gt <artifactId>drools-bom</artifactId> <type>pom</type> <version>xxx</version&        Gt <scope>import</scope> </dependency> </dependencies> </dependencyManagement> <d ependencies> <dependency> <groupId>org.kie</groupId> <artifactid>kie-api</artif actid> </dependency> <dependency> <groupId>org.drools</groupId> <artifactid& gt;drools-compiler</artifactid> <scope>runtime</scope> </dependency> <dependency>        Other dependencies</dependency> </dependencies> <build> <plugins> <plugin> <groupId>org.kie</groupId> <artifactId>kie-maven-plugin</artifactId> <version&gt ; xxx</version> <extensions>true</extensions> </plugin> </plugins> </build& Gt

In addition, a file kmodule.xml must is added to src\main\resources\META-INF folder. A minimum kmodule.xml likes like the following.

<kmodule xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"        xmlns="http://www.drools.org/xsd/kmodule">      <kbase name="defaultKBase" default="true" eventProcessingMode="cloud"     equalsBehavior="equality" declarativeAgenda="enabled" >        <ksession name="ksession1" type="stateless" default="true"/>    </kbase></kmodule>  

The default stateless ksession is mandatory.

Rule files can be put in main/resources as normal

The command to create JAR file is still as mvn package normal. However, the jar created is a bit different. Here is a screenshot

Note that there are Knowledge base cache file and Kmodule file in Meta-inf. The rule files in Main/resources is shifted out into the root folder.

What if the customer does not provide a MAVEN project? I guess the best strategy are to create a Maven project by ourselves. If source code is provided, we just import source code to the MAVEN project, otherwise, use customer provided jar as a M Aven dependency?

Note that Kie module is introduced only after Drools 6. So I don ' t think this would work for Drools 5 and below. Also, for Drools integration in Streamtau, we is using the latest version 7.2.1. So whether earlier version like 6.x is fully compatible still remains a question.

Invocation:
Load Rules:
First create a kieservices singleton instance.
private final KieServices kieServices = KieServices.Factory.get();

Load the Drools package into system:

 protected DroolsDataHolder doLoadDroolsModule(DroolsLoadParam droolsLoadParam) {        DroolsParameters origParams = droolsLoadParam.getDroolsParam();        String moduleName = origParams.getModuleName();        try {            InputStream is = droolsDataLoader.getDroolsModuleAsStream(droolsLoadParam);            KieContainer curContainer = DroolsUtils.buildContainer(kieServices, is);            return new DroolsDataHolder(curContainer);        }        catch (Exception ex) {            logger.error("Error loading drools " + moduleName, ex);        }        return null;    }

Droolsdataloader is a interface that's designed to loads drools package as stream (via either file system or restful int Erface)
Droolsutils is the utility class, builds a kiecontainer from stream.

public static KieContainer buildContainer(KieServices kieServices, InputStream stream) throws Exception {        Resource wrapped = kieServices.getResources().newInputStreamResource(stream);        KieModule curModule = kieServices.getRepository().addKieModule(wrapped);        ReleaseId releaseId = curModule.getReleaseId();        logger.info("Release id generated for module: {}", releaseId);        KieContainer kContainer = kieServices.newKieContainer(releaseId, DroolsUtils.class.getClassLoader());        return kContainer;    }

The returned Droolsdataholder is merely a wrapper of Kiecontainer

public class DroolsDataHolder {    private final KieContainer kieContainer;    public DroolsDataHolder(KieContainer kieContainer) {        this.kieContainer = kieContainer;    }    public KieContainer getKieContainer() {        return kieContainer;    }    public void destroy() {        kieContainer.dispose();    }}

The loaded droolsdataholder would be cached unless rule is changed, which triggers a reload operation

  public Droolsdataholder getorloaddroolsmodule (Droolsloadparam droolsloadparam) {droolsparameters origPa        Rams = Droolsloadparam.getdroolsparam ();        String modulename = Origparams.getmodulename ();        Datalock.readlock (). Lock ();            try {droolsdataholder Curholder = Containers.get (modulename);            if (Curholder! = null) {return curholder;            } datalock.readlock (). Unlock ();            Datalock.writelock (). Lock ();            try {return doupdatedroolsmodule (Droolsloadparam);                } finally {Datalock.readlock (). Lock ();            Datalock.writelock (). Unlock ();        }} finally {Datalock.readlock (). Unlock (); }    }

Invoke the Drools module:
in stream environment, only stateless drools knowledge session is Suppor Ted for now. The main reason is, and the stream is executed in a distributed environment. The session'll be created on multiple JVMS, so it's virtually hard-to-share all the facts globally. Evaluating the rule is quite simple, it's composed of 3 steps:

1. Convert stream data to rule input Pojo

     public class<?> getrulepojoclass (Droolsloadparam DROOLSL    Oadparam, String inputpojoclassname) {droolsparameters origparams = Droolsloadparam.getdroolsparam ();    String modulename = Origparams.getmodulename ();    Droolsdataholder Curdataholder = This.getorloaddroolsmodule (Droolsloadparam);    if (Curdataholder = = null) {throw new IllegalArgumentException ("No drools module found by name:" + modulename);        try {ClassLoader cl = Curdataholder.getkiecontainer (). getClassLoader ();        class<?> Inputpojoclass = Cl.loadclass (inputpojoclassname);    return inputpojoclass;    } catch (Exception e) {throw rtexception.from (e); }}

The good thing about Drools module is, it provides a self contained class loading environment. So third party jar dependencies is unlikely to cause conflict with the outside runtime environment. However, when we build a input event to Drools engine, we need to use the Kiecontainer's class loader to find the input E Vent class referenced in rule.

2. Build a stateless Kie session and invoke the rule

     public list<object> Evaluate (Droolsloadparam Droolsloadparam, list<object> facts) {if (logger.isdebugenabled ()) {Logger.debug ("Start evaluating drools, input is: {}, module name is: {}", Array    S.aslist (Facts), Droolsloadparam.getdroolsparam (). Getmodulename ());    } droolsparameters Origparams = Droolsloadparam.getdroolsparam ();    String modulename = Origparams.getmodulename ();    Droolsdataholder Curdataholder = This.getorloaddroolsmodule (Droolsloadparam);    if (Curdataholder = = null) {throw new IllegalArgumentException ("No drools module found by name:" + modulename);    } statelesskiesession cursession = Curdataholder.getkiecontainer (). Newstatelesskiesession ();    Cursession.execute (facts);
    return facts;}  

3. Convert rule evaluation result back to stream data

under the hood:
Drools class relations

Things to note:
Drools package can is large and the current approach caches all loaded Drools package in memory. The loading time and memory consumption might be a bottleneck of scalability. A better approach'll be building a standalone rule server, where it manages rules and exposes a rest API to stream appli cation.

Find out input metadata for rule:it are possible to find out the Java class of each rule variable. This is useful as a hint-to-map stream data to rule input.

  public static map<string, class<?>> Getruleinputmeta (kiebase kiebase, String rulepkgname        , String rulename) {Ruleimpl r = (Ruleimpl) kiebase.getrule (Rulepkgname, RuleName);        list<ruleconditionelement> elements = R.GETLHS (). GetChildren ();        Pattern curpattern = null;        String curid = null;        ObjectType curobjtype = null;        map<string, class<?>> result = new hashmap<string, class<?>> ();  for (Ruleconditionelement nextelem:elements) {if (Nextelem instanceof Pattern) {Curpattern =                (Pattern) Nextelem;                Curobjtype = Curpattern.getobjecttype ();                Curid = Curpattern.getdeclaration (). Getidentifier ();            Result.put (Curid, Curobjtype.getvaluetype (). Getclasstype ());    }} return result; }

Maven Shade plugin and drools jar:
The Drools Java API, multiple jars need to be included as Maven dependency.

However, the special thing about drools jars are the each one contains a file kie.conf (Eg Drools-core.jar, Kie-internal.jar ). The default behavior of Maven shade plugin is that kie.conf would overwrite each other and causes a runtime error when DEPL Oying the shaded jar to flink. Mitigation to this problem are to configure Maven shadow plugin parameters properly so that the content of each kie.conf WI ll be appended to the combined file instead of overwritten.

<build> <plugins> <plugin> <groupid>org.apache.maven.                        Plugins</groupid> <artifactId>maven-shade-plugin</artifactId> <executions> <execution> &LT;PHASE&G                                                T;package</phase> <goals>                                        <goal>shade</goal> </goals>                                                        <configuration> <transformers combine.children= "Append" > <transformer implementation= "Org.apache.maven.plugins.shade.resource.Appen Dingtransformer "> <resource>meta-inf/kie.conf</            Resource>                                            </transformer> </trans Formers> </configuration> </execution&gt                        ; </executions> </plugin> </plugins></build>

Drools Stream Integration