Kettle Series-My open source Kettle management project [Kettle-manager] Introduction

Kettle Management Tools

• A web-side management tool developed specifically for kettle, an excellent ETL tool.

Project Introduction

• Kettle as a very good open source ETL tool has been very widely used, the general use of the use of client operations management, but the problem is that when put into production environment, often hundreds or hundreds of of jobs need to manage, when the use of client management is difficult to complete. I know the solution useful command line calls, Operation Trouble, in short or difficult to use, there is the development of web-based management tools, kettle comes with a web-based management tool, the interface is very simple not to say, it is difficult to use, basic can not be put into production environment, Others have not noticed that there is a better tool for managing many jobs.
• Based on the above situation, and my recent two years of Kettle custom development and use experience. I have designed and developed this kettle web-side management tool.

Project Features

1. Job run parameter settings, you can set the operation parameters on the page, and each time the job runs, the default values are overwritten with the parameters set, which is not used by many people.
2. Timing settings, the interface is consistent with the start control interface in Kettle, which is probably one of the best features.
3. The operation and stop of the operation, the core function is not said.
4. Real-time view of the job run log, the maximum number of lines can be configured, this function looks inconspicuous, it should be very useful.
5. In the background there is a job specifically to complete the job log to be run to log files, files according to the talent folder, log file size can be configured, if you feel occupied space, you can design a job to clean up the previous log files, and then provide the job. You can also configure the log file to not be written.
6. The operational status of the job is periodically reflected in the management interface, and we can filter the jobs we need to run or stop in batches with a variety of criteria.
7. The system is based on database resource library design, temporarily do not consider supporting file resource library.
8. The system also comes with a kettle plugin, similar to the custom class control that kettle already has (you can write Java code in the conversion). The control is designed to avoid every need to develop a kettle control, from the background metadata management to the interface design. The control takes a flexible json pass parameter, and then inherits a base class to complete a task, that is, based on the control, you only need to develop a specific operation class. Interested to have a look.

Interface

Instructions for use

1. Download the packaged software from the Appendix address.
2. For the first time, user km between Oracle Center, and then the DMP in Doc\sql into the KM user, then execute the script in the Doc\sql\mynote.sql, modify the Kettle repository table structure.
3. Set environment variables according to the actual situation: Org.osjava.sj.root=e:\kettle\simple-jndi. This variable specifies the Jndi file location and then copies the file: Doc\ sundries \jdbc.properties to the directory, modifying the content according to the actual situation.
4. Jobs that will log and run state modifications: doc\ sundries \ Job log records. KJB Import your repository.
5. Modify the Log profile yourself: Log4j.xml, you can adjust the log directory and so on.
6. The default is the configuration file, which needs to be modified according to the actual situation.
7. You can modify the contents of the Start.bat file to modify the ports used.
8. Double-click the Run Start.bat script to start the project.
9. Normal operation can be accessed, the default address is: http://127.0.0.1:85, if the runtime modifies the port, the corresponding changes here.

Development Notes

1. Generally do not need to develop their own, direct use of the line, if there is a bug or new features need, you can send me an email.
2. This project is based on open source project Eova development: Http://git.oschina.net/eova/eova.
3. If you want to further expand the functionality, it is recommended to first build the kettle development environment, you can refer to my blog: http://www.cnblogs.com/majinju/p/4455107.html
4. My blog also has a number of articles about kettle source code, etc., can be used as a reference.

Appendix

1. Project Address: Https://github.com/ma459006574/kettle-manager
2. My e-mail: [email protected].
3. Software downloads that have been packaged

Kettle Series-My open source Kettle management project [Kettle-manager] Introduction