pentaho kettle

Label:??Kettle Version 4.1.0 Open the Spoon software and double-click Spoon.bat under Windows. Creates a transform. Create two data sources, one for Oracle and one for MySQL. CREATE TABLE input. Create a field selection Create the table output to map the value field of the input table to the STR field of the output table. Note: If the target table does not have a new one, you sh

In daily development, such a situation often arises. There are a number of functions that are generic. And every JavaScript is written once, which is very disturbing for maintenance. Therefore, the public functions need to be written in an external JS file. In this case, we need to introduce external public files. The following is the introduction of external public JS files in the JavaScript component in the conversion.//Load JS fileLoadscriptfile (getvariable ("Internal.Transformation.Filename

SQL statements manually.Kettle: Data quality features in the GUI, you can manually write SQL statements, Java scripts, regular expressions to complete the data cleansing.Informatica: A product dedicated to Informatica data quality to ensure qualityInaplex Inaport: Data cleansing is easier because only specific data is processed. Monitoring:Talend: There are monitoring and logging toolsKettle: There are monitoring and logging toolsInformatica: Very detailed monitoring and logging toolsInaplex I

1, the MySQL stored procedure, can realize the dynamic table reads, satisfies the more complex business logicDROP PROCEDURE if existsP_get_car_trace;delimiter//CREATE PROCEDUREp_get_car_trace ()beginDeclareVar_sqlvarchar( -);DeclareVar_tabe_namevarchar( -);SetVar_tabe_name= 'geom1';SetVar_sql=Concat_ws (' ','Select Id,astext (GEO) Geo from', var_tabe_name);Set @sql =Var_sql;PREPARES1 from @sql; EXECUTES1;deallocate PrepareS1;End //delimiter;2, SQL Server stored procedures, using the steps of the

Label:Kettle linking strings to each databaseSybase:To_db_url = jdbc:sybase:tds:192.168.168.163:5000/testdb?charset=eucgbjconnect_version=6 TO_DB_DRIVER = Com.sybase.jdbc3.jdbc.SybDataSource #com. Sybase.jdbc3.jdbc.SybDriver Driver The following table input components do not support aliases. #net. Sourceforge.jtds.jdbc.Driver drivers The following table input components support aliases. to_db_user = Test TO_DB_PWD = Test Oracle:DEST_DB_URL=JDBC:ORACLE:THIN:@192.168.168.162:1521:ORCL Dest_db_driv

website Link: http://wiki.pentaho.com/display/EAI/Call+DB+Procedure DescriptionCalling the database stored procedure step allows the user to execute a database stored procedure and obtain the results. Stored procedures or methods can only return

1. It can only have one primary query result set. 2. Place the $ {parameter name} parameter in the SQL statement at the bottom of the DATA page in the upper-right corner. The parameter must have a default value. Otherwise, no DATA is displayed in

I. Extracting data from HDFS to an RDBMS1. Download the sample file from the address below.Http://wiki.pentaho.com/download/attachments/23530622/weblogs_aggregate.txt.zip?version=1&modificationDate =13270678580002. Use the following command to place

The layout of the dashboard is very troublesome. This CSS layout is a virtue. This mainly involves layout and compontens layout size settings. Layout layout includes row objects and column objects, as well as images and HTML objects. A row object

Http://www.aboutyun.com/thread-7450-1-1.html There is a very large table: Trlog The table is about 2T.Trlog:CREATE TABLE Trlog(PLATFORM string,user_id int,Click_time String,Click_url string)Row format delimitedFields terminated by ' t ';

Reprint Source: http://www.cnblogs.com/wxjnew/p/3417942.html Since the server is a Linux system, but the feeling of Linux graphics is not strong, so from contact with the kettle has been operating in the Windows system ETL design and processing. Now you need to look in Linux to see if the kettle repository is connected properly, and to schedule the kettle job o

Scenario: Assume that there is a large amount of data in a table, you need to cycle the incremental extraction by a single timeMethod: The loop call is implemented mainly through the job itself, similar to the loop of function self-invocation.1.JOB

1. Design an incremental configuration table, Etl_incremental, to configure data such as delta time for the table2. The full image of the incremental job is as follows:2.1 Get the increment time variable and set the increment variable2.2 Increment

Cost:Software costs include software products, pre-sales training, after-sales consulting, and technical support.Open-source products are free of charge and the cost is mainly training and consulting, so the cost will remain at a low

When data is extracted from the production environment to the warehouse, Chinese characters in the target database are garbled. My environment is from MySQL to MySQL. There are no heterogeneous databases at the moment, and the architecture is

The data in the database may be transferred in different databases for replacement. Because different databases may use different character sets, the resulting data may be garbled. This time, we ran data in a job. After the data was run, the

Intermediate database ----> organization code table local database ------> organization tableThe structure of the two tables is as follows:Organization Code Table IDOrganization table ID FlagThis is the requirement. If the ID in my local

Http://wiki.pentaho.com/display/BAD/Extracting+Data+from+HBase+to+Load+an+RDBMS1) New conversion--Big Data--Hbase Input double hit Open2) Create a table in hbase 3) Click Get table Names4) Create Mapping Click the Get table names button in the drop-

Set Variable: Add a variableTable input: Reads a record from the database and finally saves a SELECT statementTable output: Writes data from a stream to a database in a specified tableEmpty operation: Usually used to judge a branch of a branching,

Scene:Determine if the extracted data already exists in the target library (if the same patient already has a treatment plan number):1. If it does not exist, then grouprowno=1 and insert into Target library (judging if the outer associated field is