Spark growth Path (one)-ngram

Ngram Introduction

N-gram Code

Object Ngramexample extends Sparkobject {

  def main (args:array[string]): Unit = {
    val worddataframe = spark.create DataFrame (Seq (
      (0, Array ("Hi", "I", "heard", "about", "Spark")),
      (1, Array ("I", "Wish", "Java", "could", "use", " Case "," classes ")),
      (2, Array (" Logistic "," Regression "," models "," is "," neat ")))
    . TODF (" id "," words "

    ) Val Ngram = new Ngram (). SETN (2). Setinputcol ("words"). Setoutputcol ("Ngrams")

    val ngramdataframe = Ngram.transform ( Worddataframe)
    ngramdataframe.select ("Ngrams"). Show (False)
  }

}

Execution Results

+------------------------------------------------------------------+ |ngrams
| +------------------------------------------------------------------+
| [Hi I, I heard, heard about, about Spark] | | [I wish, Wish Java, Java could, could use, use case, Case classes]| |
[Logistic regression, regression models, models is, is neat] | +------------------------------------------------------------------+