Apache Camel and Spring Boot integration, through FTP timing acquisition, processing files

1. Summary:

This project is mainly through the spring platform configuration camel, FTP, the timing from the FTP server to download files to the local, parse files, stored in the database and other functions.

2. Construction of Empty Project:

Spring boot has several mechanisms for automatically generating empty projects: CLI, Spring tool suite, Web site Spring initializr, we choose the third one.

2. Visit site http://start.spring.io/, such as
   
4. When dependencies adds a dependency package, enter Camle, JDBC, and MySQL in the box to automatically eject the prompt, confirming that it is selected, such as:
   
5. Click the Generate Project button to build the project and import it into Ecipse, add camel-ftp dependency in Pom.xml, and note that the version number is the same as Camel-spring-boot-stater
   

   <Dependency>    <groupId>Org.apache.camel</groupId>    <Artifactid>Camel-ftp</Artifactid>    <version>2.18.0</version></Dependency>

6. The full version of the Pom.xml file is as follows:
   

   <Dependencies>    <Dependency>        <groupId>Org.apache.camel</groupId>        <Artifactid>Camel-spring-boot-starter</Artifactid>        <version>2.18.0</version>    </Dependency>    <Dependency>        <groupId>Org.apache.camel</groupId>        <Artifactid>Camel-ftp</Artifactid>        <version>2.18.0</version>    </Dependency>    <Dependency>        <groupId>Org.springframework.boot</groupId>        <Artifactid>Spring-boot-starter-jdbc</Artifactid>    </Dependency>    <Dependency>        <groupId>Mysql</groupId>        <Artifactid>Mysql-connector-java</Artifactid>        <Scope>Runtime</Scope>    </Dependency>    <Dependency>        <groupId>Org.springframework.boot</groupId>        <Artifactid>Spring-boot-starter-test</Artifactid>        <Scope>Test</Scope>    </Dependency></Dependencies>

8. When all dependent jars are downloaded locally, the base project is completed.

3. Configure camel to complete downloading files from FTP server to local

2. Configure the remote FTP server address, port, user name, and password information in application.properties
   

   ftp.server.info=sftp://172.16.20.133:22/. /home/temp/data?username=root&password=root&delay=5s&move=done& Readlock=renameftp.local.dir=file:c:/ftp/test

   Note: The SFTP server's file location is relative to the root log-in relative address (is here to pit to), delay=5s is every 5 seconds to scan the FTP server for new file generation, if you have to download to the local, and transfer files on the server to the Done folder (/home/ Temp/data/done), Readlock=rename can prevent camel from reading files being written

3. Configure routing, complete file download
   

   @Component Public classDownloadroutedemoextendsRoutebuilder {Private StaticLogger Logger = Loggerfactory.getlogger (Downloadroutedemo.class ); @Value ("${ftp.server.info}")    PrivateString Sftpserver; @Value ("${ftp.local.dir}")    PrivateString downloadlocation; @Override Public voidConfigure ()throwsException {from (Sftpserver). to (downloadlocation). log (Logginglevel.info, logger,"Downloaded file ${file:name} complete."); }}

   Note: to inherit camel Routebulider, override the Configure method to download the file locally from the FTP server and output the filename (the necessary information required at runtime is configured in the Application.properties file)

5. In order for the Java process to run in the background, you need to add the following configuration in the Application.properties file
   

   Camel.springboot.main-run-controller=true

7. All work to download files from the FTP server is complete, run Camelftpspringapplication.java, if you have files in the appropriate location on your FTP server, they will be downloaded to a locally configured folder

4. Resolve local files and save to database through camel timing

2. Add the following configuration in the Application.properties
   

   Route.parserfile.info = {{ftp.local.dir}}?delay=10s&move=done&readlock= Renameroute.parserfile.dir = {{Ftp.local.dir}}/done

   Note that two curly braces are a configuration that references other variables

3. Write parsing files, inbound programs and other processors
   

   @Component Public classLocationfileprocessorImplementsProcessor {Private StaticLogger Logger = Loggerfactory.getlogger (locationfileprocessor.class ); @Value ("${ftp.local.dir}")    PrivateString Filedir; @Autowired OrderService OrderService;//business logic processing Components@Override Public voidProcess (Exchange Exchange)throwsException {genericfilemessage<RandomAccessFile> infilemessage = (genericfilemessage<randomaccessfile>) Exchange.getin (); String FileName= Infilemessage.getgenericfile (). GetFileName ();//file nameString Splittag = File.separator;//System file SeparatorsLogger.info (Filedir + Splittag + fileName);//absolute path to fileOrderservice.process (Filedir + Splittag + fileName);//Resolve inbound and other operations            }}

   
   
   
4. Configure routing to complete the concatenation of business logic
   

   @Component Public classLocaltransformrouteextendsRoutebuilder {Private StaticLogger Logger = Loggerfactory.getlogger (localtransformroute.class ); @Value ("${route.parserfile.info}")    PrivateString location; @Value ("${route.parserfile.dir}")    PrivateString Locationdir;        @Autowired Locationfileprocessor Locationfileprocessor; @Override Public voidConfigure ()throwsException {from location] process (locationfileprocessor). to (Locationdir). log (Logginglevel.info, logger, "Tirans file ${file:name} complete."); }}

   Note that the process configuration is more than the route above, that is, the business logic processing configuration

6. At this point, all the work has been completed, re-execute Camelftpspringapplication.java to achieve the ftp file timed download, business processing, etc. (which saves a lot of, such as warehousing operations, etc.)

Note: Just CAMLE Spring FTP demo demo, to be used in production, there are many needs to improve the place

Apache Camel and Spring Boot integration, through FTP timing acquisition, processing files