[Spark] [Python] RDD FlatMap Operation Example

Example of the RDD FlatMap operation:

FlatMap, performs a function operation on each element (line) of the original Rdd, and then "beats" each line

[Email protected] ~]$ HDFs dfs-put cats.txt
[Email protected] ~]$ HDFs dfa-cat cats.txt
Error:could not find or load main class DFA
[Email protected] ~]$ HDFs dfs-cat cats.txt
The Cat on the mat
The aardvark sat on the sofa

Mydata=sc.textfile ("Cats.txt")

Mydata.count ()
OUT[14]: 2

Mydata.take (2)
OUT[15]: [u ' the Cat on the mat ', U ' the aardvark sat on the sofa ']

Myflatdata=mydata.flatmap (Lambda line:line.split ("))
Myflatdta.count ()
OUT[19]: 11

Myflatdata.take (2)
OUT[20]: [u ' the ', U ' Cat ']

Myflatdata.take (11)
OUT[21]:
[u ' the ',
U ' Cat ',
U ' on ',
U ' the ',
U ' Mat ',
U ' the ',
U ' Aardvark ',
U ' sat ',
U ' on ',
U ' the ',
U ' sofa ')

[Spark] [Python] RDD FlatMap Operation Example