pentaho kettle

Write your own script that automates the deployment of kettle in Linux, including some of the problems encountered in scripting.Kettle is the official website version Pdi-ce-6.1.0.1-196.zipScript:#!/bin/Bash#record The current directory!Mulu=`pwd' #The output of java_home number of Bytesc=`Echo$JAVA _home|WC-C 'Echo "Tips:install JDK rather than jre! Configuration Java_home"#PleaseInstallJdkifJava_home Bytes is equal to1if[$c-eq1]; ThenEcho "Please in

Tags: kettle H2 database table input placeholder parameter variableimplementing dynamic SQL queries in KettleIn ETL projects, it is common to execute some SQL statements, such as querying data, based on runtime input parameters. This article describes dynamic queries and parameter queries through the table input ("table input") step in kettle. The sample code uses the in-memory database (H2), which can be r

. Database resource pool configuration When you open kettle, the Library selection dialog box pops up, you can click the plus sign to create a new library, and if you have already opened kettle, you can connect to the repository after you have disconnected the repository in the Tools-> repository, or you can open the Create Library dialog box. Clicking the plus sign to create a new library will bring up th

Javascript step error handling if you are familiar with the incorrect features of kettle conversion, you may want to know how to use it in javascript steps. The steps and user interfaces that support the error mechanism are the same. Right-click the javascript step and select "define error handling" to enable and configure error handling. Javascript code can identify error lines and forward them to the configuration Javascript step error handling if y

The role of the resource pool: The repository is used to save the operation steps and related logs, conversions, JOB and other information. Conversion tasks created by the user through the graphical interface can be saved in the repository. The repository can be a variety of common databases where users access resources in the resource pool via username/password, which is the default user name/password admin/admin. The resource pool is not required, and if there is no repository, the user can al

Check whether the empty line ETL processing in kettle data streams sometimes requires data generation but no data input. This may cause some problems. Therefore, the ETL data stream is usually required to generate a blank line of data; sometimes some clustering functions are required for processing, which means that when no data is input, the generated value is 0. This article describes how to detect and process empty row data streams. Check whether t

Delegate is an application type that encapsulates a named or anonymous method. A delegate is used to pass a method as a parameter to another method. Coding specification for delegates in the. NET Framework: 1 The name of the delegate type should end with EventHandler. 2 The prototype definition of the delegate type: there is a void return value that accepts two parameters: an object, a second EventArgs (or inherits from it) 3 The name of the event is delegated to remove EventHandler remaind

Kettle Connection Database Kettle The way to connect data There are many ways to connect to a database, which is more convenient than using Jndi, and will not modify too many files because of problems such as database migrations Configured in the Jdbc.properties file ${kettle_home}/simple-jndi inside the Kettle home directory Oracle's configuration is (where

Download Kettile Java must be installed before installing kettle, if the system memory is greater than 4G, it is best to install 64-bitDownload link after decompression Locate the Spoon.bat file in the root directory, open it with an editor, and locate the following code pentaho_di_java_options= "-xms1024m" "-xmx2048m" "-xx:maxpermsize=256m" -XMS: Setting JVM Initial Memory-XMX: Set JVM maximum available memory-xx:maxpermsize: Set JVM maximum allo

Tags: core needs installation Steps Reference statement exception set image procedure1. Database Link DriverIf the corresponding database link driver is not installed, in the process of database linking, it may be reported that a database connection can not find an exception, so you need to download the corresponding driver (installation step can see "How to download Java connection MySQL driver jar package on the official website?") "), put the Lib folder in

kettle+mongodb data sync to MySQL 1. Preface:The date type in MongoDB is stored in UTC (coordinated Universal time), and the isodate type is equal to GMT (GMT). And Beijing is in the area of +8, so MONGO Shell will gmt+0800 the current time minus 8, stored in GMT time.2. Overview of Extraction Operations650) this.width=650; "Src=" Http://s3.51cto.com/wyfs02/M01/89/C8/wKiom1gcMc_CAetyAAAa3XTzyqU778.png-wh_500x0-wm_3 -wmp_4-s_2776987160.png "title=" 52[

Tags: mapping demo database connection open log Width style technology share displayCase Purpose: When there are tens of thousands of records or more data in Excel, use Kettle to import into the database. Speed can feel much faster than copy-paste. Here I will demonstrate that there are more than 150,000 records of the data in the "data.xlsx" table, using kettle to import to the testuser user of

1 Create a good conversion under Windows and place it under one of the Linux folders. such as: Put the 0816_em.ktr file under the/home/cfj/kettle/testdemo.Note that the database user name and password may need to be changed if it is a job file such as: JOB0816_01.KJB. Remember to change the path in which it is stored, such as:2 Edit the script file, put it in the corresponding folder, and give permissionThe permissions are given in the following ways:

Label: Use SP data to work BS Database SQL Oracle TT These days, the implementers collaborate on Bi projects. The development of the project is coming to an end. The work in the future is mainly to define indicators and multi-dimensional analysis models and ETL processes. The following is a summary of the work: Environment: System: ERP, his and other subsystems ------- mainly business data and some data dictionaries (sqlserver2005) Bi SYSTEM -------------------------- metadata and basic da

Kettle incremental update design skills First, you need to determine whether you are processing a dimension table. If it is a dimension table, it may beSCDSituation, you can useKettleOfDimension LookupTo solve this problem, if you are dealing with fact tables, the methods may be different. The main difference between them is that the primary key judgment method is different. Fact tables generally have a large amount of data. You must first determin

Environment Description:Now a project has a lot of homework, need to know which ktr run successfully after each run, which failedProblem solving:The following is a specific operating procedureFirst create a database tableCREATE TABLE test_1 (id int,name VARCHAR), INSERT into Test_1 VALUES (1, ' HSJ '); CREATE TABLE test_2 (id INT PRIMARY KEY not null,name VARCHAR (10));After the database table is established, the new KTR,KTR is as follows:In the interface right--"conversion settings", you can s

Today encountered a problem: different databases need to use full connectivity, so do not hesitate to merge join Plug-in, but in the process of using a lot of problems encountered. After you connect to get the field, a repeating field appears. Workaround: Change the field you want to associate to a different name.2. The data obtained is not the data we want:For example:Table A:1 A2 bTable B:1 of3 placesFull connection expected effect:1 A's2 b NULL3 null PlacesThe result is:1 a 1 of2 b NU

First, the purposeMerge tables on different servers onto another server. For example, merge table B on server 1 on table A and server 2 to table C on server 3Requirements: Table A needs to be cropped (removing unnecessary fields), table B needs to add some fieldsIi. Methods of Use(1) Create a new Table C (field that conforms to the actual system design) in the database on server 3(2) Create a new table input, connect to server 1, select the table you want to use by getting the SQL statement, or

For any two buildings, the minimum spanning tree is obtained by the shortest path between them.The answer to the query (x, y) is the maximum value of the top right of the X-to-y path on the minimum spanning tree.The BFS finds the closest building to each point and the distance to it, and can find only the edges of the junction to use, using these edges to seek MST.#include 　　BZOJ4242: Kettle

Use the app Send mail icon: The information needed in this icon is not manually filled, but is taken from the previous stream.This generates data using the Generate records chart, which is given to the Send mail control using:In fact, the overall process is equivalent to kettle know your mailbox user name and password, using the standard interface to call the form of operation of your mailbox.Copyright NOTICE: This article for Bo Master original artic