Sparksql External DataSource Easy to use Avro

Download Source & Compile:

git clone https://github.com/databricks/spark-avro.gitSBT/SBT Package

Maven GAV:

groupid:com.databricks.sparkartifactid:spark-avro_2.10version:0.1

$SPARK _home/conf/spark-env.sh

Export spark_classpath=/home/spark/software/source/spark_package/spark-avro/target/scala-2.10/ Spark-avro_2. Ten-0.1. Jar: $SPARK _classpath

Test data Download:

wget https://

Scala API:

== Sqlcontext.avrofile ("file:///home/spark/software/data/episodes.avro"  ) Import Sqlcontext._episodes. Select (' title '). Collect ()

Sql:

CREATE Temporary TABLE episodesusing Com.databricks.spark.avroOPTIONS (path "file:///home/Spark/ software/data/Episodes.avro "); Select *  from episodes;

Sparksql External DataSource Easy to use Avro