Simple conversion of case1:list () to Dataframe ()
Step1: We first create a case class
Case Class ResultSet (Masterhotel:int,
Quantity:double,
Date:string,
Rank:int,
Frcst_cii:double,
Hotelid:int)
Step2
Initialize the ResultSet class, there are many ways to get the data definition ResultSet class from the relational database,
Direct definition of a resultset list, etc.
Val x1=list (ResultSet (1001,12, "2016-10-01", 1, 13.44,1001),
ResultSet (1002,12, "2016-10-01", 3, 13.44,1002),
ResultSet (1004,15, "2016-10-02", 10, 18.44,1004),
ResultSet (1005,5, "2016-10-02", 40, 5.67,1005)
)
Val Dataset1=sqlcontext.createdataframe (x1)
Scala> Dataset1.show ()
+-----------+--------+----------+----+---------+-------+
|masterhotel|quantity| date|rank|frcst_cii|hotelid|
+-----------+--------+----------+----+---------+-------+
| 1001| 12.0|2016-10-01| 1| 13.44| 1001|
| 1002| 12.0|2016-10-01| 3| 13.44| 1002|
| 1004| 15.0|2016-10-02| 10| 18.44| 1004|
| 1005| 5.0|2016-10-02| 40| 5.67| 1005|
+-----------+--------+----------+----+---------+-------+
CASE2: How an array of tuple elements is converted to a list, then the class is converted to Dataframe ()
Val X2=array (1001,12, "2016-10-01", 1, 13.44,1001),
(1002,12, "2016-10-01", 3, 13.44,1002),
(1004,15, "2016-10-02", 10, 18.44,1004),
(1005,5, "2016-10-02", 40, 5.67,1005)
)
Val x3= (0 until X2.length). Map (i = resultset (x2 (i). _1,X2 (i) _2,x2 (i). _3,X2 (i). _4,X2 (i). _5,X2 (i). _6)//How to access the tuple
Val X4=x3.tolist
Val Dataset2=sqlcontext.createdataframe (x4)
Scala> Dataset2.show ()
+-----------+--------+----------+----+---------+-------+
|masterhotel|quantity| date|rank|frcst_cii|hotelid|
+-----------+--------+----------+----+---------+-------+
| 1001| 12.0|2016-10-01| 1| 13.44| 1001|
| 1002| 12.0|2016-10-01| 3| 13.44| 1002|
| 1004| 15.0|2016-10-02| 10| 18.44| 1004|
| 1005| 5.0|2016-10-02| 40| 5.67| 1005|
+-----------+--------+----------+----+---------+-------+
CASE3: Arrays and lists can be converted to each other
Scala> Val A=array. toList
A:list[int] = List (1, 2)
Scala> A.toarray
Res69:array[int] = Array (1, 2)
Case4:dataframe () can be converted directly into an array of arrays in CAE2
