First step: Enter Clientshell[Email protected]:~$sqoop.sh ClientSqoop Home directory:/home/fulong/sqoop/sqoop-1.99.3-bin-hadoop200sqoop shell:type ' help ' or ' \h ' for help.
Sqoop:000>Set server--host FBI003--port 12000--webapp sqoopServer is set successfullysqoop:000>Show version--allClient Version:sqoop 1.99.3 revision 2404393160301df16a94716a3034e31b03e27b0b Compiled by Mengweid on Fri Oct 18 14:15: 2013server version:sqoop 1.99.3 revision 2404393160301df16a94716a3034e31b03e27b0b Compiled by Mengweid on Fri Oc T 14:15:53 EDT 2013Protocol version: [1]
Step Two: Create a connection
Sqoop:000>
Create connection--cid 001Creating connection for connector with ID1Please fill following values to create new connection objectName:147patents
Connection Configuration
JDBC Driver Class:Oracle.jdbc.driver.OracleDriverJDBC Connection String:JDBC:ORACLE:THIN:@192.168.0.147:1521:ORCLGBKUsername:ZhaobiaoPassword:******JDBC Connection Properties:there is currently 0 values in the map:entry#
Security Related configuration options
Max Connections:
-New connection is successfully created with validation status FINE and persistent ID 1
Step Three: Create a job
I tried the update command here by the way. So the first time you create the job, you enter the wrong tablename:
sqoop:000> Create jobrequired argument--xid is missing.sqoop:000>Create job--xid 1--type ImportCreating job for connection with ID 1Please fill following the values to create new job ObjectName:importpatents
Database Configuration
Schema Name:
ZhaobiaoTable Name:
ZhaobiaoTable SQL statement:table Column names:partition column name:nulls in Partition column:boundary query:
Output Configuration
Storage type:0: Hdfschoose:0Output format:0: text_file 1:sequence_filechoose:0Compression format:0: NONE 1:default 2:deflate 3:gzip 4:bzip2 5:lzo 6:lz4 7:snappychoose:0Output directory:/data/zhaobiao
Throttling Resources
Extractors:Loaders:New
Job was successfully created with validation status FINE and persistent
ID 1
sqoop:000> Update jobrequired argument--jid is missing.sqoop:000>
Update Job --jid 1 Updating job with ID 1Please update Job metadata:Name:ImportPatents
Database Configuration
Schema name:zhaobiaotable Name:cms_news_0625Table SQL statement:table Column names:partition column name:nulls in Partition column:boundary query:
Output Configuration
Storage type:0: hdfschoose:0output format:0: Text_file 1:sequence_filechoose:0compression format:0: NONE 1 : DEFAULT 2:deflate 3:gzip 4:bzip2 5:lzo 6:lz4 7:snappychoose:0output directory:/data/zhaobiao
Throttling Resources
Extractors:Loaders:Job was successfully updated with status FINE
Summary:
1.create Job must specify a connection ID
2.create After the job is complete. Assigned a job ID on its own initiative
3.update job. Need to specify job ID
4. Create the specified in the job procedure
Output directory Specifies the path in HDFs
Fourth step: Run the job
Sqoop:000>Start Job--jid 12014-06-27 12:44:17 CST:FAILURE_ON_SUBMITException:org.apache.hadoop.mapred.FileAlreadyExistsException:Output Directory Hdfs://fulonghadoop/data/zhaobiao already existssqoop:000> start Job--jid 1Submission detailsjob ID: 1Server url:http://fbi003:12000/sqoop/created by:fulongcreation date:2014-06-27 12:44:53 CSTLastly updated By:fulonge Xternal id:job_1403248336137_0005http://FBI003:8088/proxy/application_1403248336137_0005/2014-06-27 12:44:53 cst:booting-progress is not availablesqoop:000>
status Job--jid 1Submission Detailsjob id:1server url:http://fbi003:12000/sqoop/created by:fulongcreation date:2014-06-27 12:44:53 CST Lastly updated by:fulongexternal id:job_1403248336137_0005 http://FBI003:8088/proxy/application_1403248336137_000 5/2014-06-27 12:50:11 CST:RUNNING-0.00%
sqoop:000> status Job--jid 1Submission detailsjob id:1server url:http://fbi003:12000/sqoop/created By:fulongcreati On date:2014-06-27 12:44:53 cstlastly updated by:fulongexternal id:job_1403248336137_0005 Http://FBI003:8088/pro Xy/application_1403248336137_0005/2014-06-27 13:36:49 CST:RUNNING-25.00%sqoop:000> status Job--jid 1Submission detailsjob id:1server url:http://fbi003:12000/sqoop/created By:fulongcreati On date:2014-06-27 12:44:53 cstlastly updated by:fulongexternal id:job_1403248336137_0005 Http://FBI003:8088/pro Xy/application_1403248336137_0005/2014-06-27 13:39:24 CST:RUNNING-35.00%sqoop:000> status Job--jid 1Submission detailsjob id:1server url:http://fbi003:12000/sqoop/created By:fulongcreati On date:2014-06-27 12:44:53 cstlastly updated by:fulongexternal id:job_1403248336137_0005 Http://FBI003:8088/pro Xy/application_1403248336137_0005/2014-06-27 13:47:19 CST:RUNNING-40.00%Sqoop:000>
status Job--jid 1Submission Detailsjob id:1server url:http://fbi003:12000/sqoop/created by:fulongcreation date:2014-06-27 12:44:53 CST Lastly updated by:fulongexternal id:job_1403248336137_0005 http://FBI003:8088/proxy/ application_1403248336137_0005/2014-06-27 14:08:36 cst:succeededcounters: org.apache.hadoop.mapreduce.jobcounter Slots_millis_maps: 16499226 total_launched_maps:10 slots_millis_reduces:0 OTHER_LOCAL_MAPS:15&N Bsp org.apache.hadoop.mapreduce.lib.output.fileoutputformatcounter bytes_written:421226094 org.apache.hadoop.mapreduce.lib.input.fileinputformatcounter &NBsp bytes_read:0 org.apache.hadoop.mapreduce.taskcounter map_input_records:0 MERGED_MAP_OUTPUTS:0&NBS P physical_memory_bytes:1185689600 &NBSP ; spilled_records:0 COMMITTED_HEAP_BYTES:627179520&NB Sp cpu_milliseconds:307570 failed_shuffle:0 virtual_memory_bytes:6542311424 &NB Sp split_raw_bytes:1437 MAP _output_records:644083 gc_time_millis:5279 &N Bsp Org.apache.hadoop.mapreduce.filesystemcounter file_write_ops:0 & nbsp file_read_ops:0 File_lar ge_read_ops:0 file_bytes_read:0 hdfs_bytes_read:1437 file_bytes_written:931 780 hdfs_large_read_ops:0 hdfs_write_ops:20 hdfs_read_ops:40 &N Bsp hdfs_bytes_written:431907904 org.apache.sqoop.submission.counter.sqoopcounters Rows_read: 644083Job executed successfully
the next question: How do I import data into hbase?
"Gandalf" Sqoop1.99.3 basic operations-Import Oracle data into HDFs
