Hadoop on Mac with intellij idea-4 jar package

This article describes how to use intellij idea to package a project, that is, to package a jar package.

Environment: Mac OS X 10.9.5, intellij idea 13.1.4, hadoop 1.2.1

Hadoop is stored in a virtual machine. The host machine is connected through SSH, And the IDE and data files are stored in the host machine. Idea runs on JDK 1.8 and uses JDK 1.6 for idea engineering and hadoop.

 

In the first article of this series of blog posts, I had to directly debug the code because I didn't use idea to create a jar package. The input path does not exist. Today, I typed a few jar packages and the results were not satisfactory. By default, idea packages all dependent libraries and exported packages are too large. Although available, I have an obsessive-compulsive disorder, I want to create a minimum jar package. Because the database files required for hadoop running are already on the server, I just need to package my own classes.

By referring to Vic's blog, we finally got a jar package that only contains our own project.

Step 1, Command +; or file-> project structure... open the project structure.

Step 2, Click artifacts, and then click Add.

Step 3Select jar> empty in the pop-up menu, as shown in.

Step 4, Fill in the jar package name and output path, as shown in.

Step 5Add the module that outputs to the jar package in output layout, as shown in.

Step 6In the displayed window, specify a module, as shown in.

After confirmation, if the module contains only one main function, it is automatically specified. Otherwise, you must manually specify it.

Step 7, [Optional] manually specify the main function.

Everything is ready, as shown in.

Step 8, Packaging.

Build-> Build artifacts

In the pop-up menu, select build or rebuild.

Finally, because the main class is specified, the hadoop execution script should ignore the main class parameter, that is

Hadoop jar [jar package] [input path] [Output path]

If the hadoop jar hadoop-example-1.2.1.jar wordcount input ouput is written as the example jar, wordcount is treated as an input parameter as required.

Finally, as a beginner, I have encountered many problems.

Hadoop on Mac with intellij idea-4 jar package