Spark1.6.2 Java Implementation read JSON data file into MySQL database

PublicClassMainImplementsSerializable {/*** */PrivateStaticFinalLong Serialversionuid =-8513279306224995844L;PrivateStaticFinal String Mysql_username ="Demo";PrivateStaticFinal String mysql_pwd ="Demo";PrivateStaticFinal String Mysql_connection_url ="Jdbc:mysql://192.168.1.91:3306/demo";PrivateStaticFinal Javasparkcontext sc =New Javasparkcontext (New Sparkconf (). Setappname ("Sparksavetodb"). Setmaster ("local[*]"));PrivateStaticFinal SqlContext SqlContext =New SqlContext (SC);public static void main (string[] args) { //Sample data-frame loaded from a json filedataframe USERSDF = Sqlcontext.read (). JSON ( "Users.json"); //Save data-frame to MySQL (or any other JDBC supported databases) Properties Connectionproper Ties = new Properties (); Connectionproperties.put ( "user", Mysql_username) connectionproperties.put (" users ", connectionproperties);}           

In order to write the data for easy testing, we need a JSON file, similar to the below:

{"id":994,"Name":"Betty","Email":"Bsmithrl@simplemachines. org ","City":"Eláteia","Country":"Greece","IP":"9.19.204.44"},{"id":995,"Name":"Anna,""Email":"ALEWISRM@canalblog. com ","City":"Shangjing","Country":"China","IP":"14.207.119.126"},{"id":996,"Name":"David","Email":"Dgarrettrn@japanpost. JP ","City":"Tsarychanka","Country":"Ukraine","IP":"111.252.63.159"},{"id":997,"Name":"Heather","Email":"Hgilbertro@skype. com ","City":"Koilás","Country":"Greece","IP":"29.57.181.250"},{"id":998,"Name":"Diane","Email":"DDANIELSRP@statcounter. com ","City":"Transmitted","Country":"Colombia","IP":"19.205.181.99"},{"id":999,"Name": "Philip",  "email":  "Pfullerrq @reuters. com ", " City ": " El Cairo ",  "country":  "Colombia",  "IP": " 210.248.121.194 "},{" id ": 1000,  "name":  "Maria",  "email":  "Mfordrr @shop-pro.jp",  "City":  "Karabash",  "country":  "Russia",  "IP":  "224.21.41.52"}        

When reading a file, the Users.json needs to be in the same directory as the jar package, and the test takes the local run mode:

DataFrame usersDf = sqlContext.read().json("users.json");

The code in this line of mode (Savemode.append) to pay special attention, this makes each write data is added to the data table. Otherwise you will always be prompted: Exception in Thread "main" java.lang.RuntimeException:Table users already exists.

Usersdf. Write (). Mode (Savemode. Append). JDBC (Mysql_connection_url, "Users ", connectionPrope rties)                

Spark1.6.2 Java Implementation reads JSON data file into MySQL database