[Interactive Q & A sharing] Stage 1 wins the public welfare lecture hall of spark Asia Pacific Research Institute in the cloud computing Big Data age

Spark Asia Pacific Research Institute Stage 1 Public Welfare lecture hall [Stage 1 interactive Q & A sharing]

 Q1: sparkHow can I support ad hoc queries? Isn't it spark SQL? Is it hive on Spark?

The technology that spark1.0 used to support ad hoc queries is shark;

The ad hoc query technology supported by Spark 1.0 and spark 1.0.1 is spark SQL;

Spark SQL is the core of ad hoc queries for unreleased spark 1.1. We expect hive on spark to support ad hoc queries;

 Q2:Does spark 1.0.0 support hive on spark, does it support CLI interactive access?

Spark1.0.0 does not support hive on spark;

Hive on spark is under development and is expected to be released at spark 1.1;

Currently, spark 1.0.0 does not support CLI access;

 Q3: spark SQLHow can hbase be combined?

Spark SQL and hbase use the RDD function of spark core;

When using hbase, You need to import hbase packages on spark.

Currently, spark 1.0.0 does not support CLI access;

 Q4: sparksqlIs SQL supported? Can the instructor directly convert the current PLSQL to sparksql?

Spark SQL now supports the basic functionality of the SQL-92 and will not be enhanced in subsequent versions;

PLSQL cannot be directly converted into spark SQL;

For better SQL support, you can consider the hive in Spark SQL function in spark1.0.0 and spark1.0.1;

 Q5:If hive on spark is supported, when will spark SQL be used and hive on spark be used?

Hive on spark is expected to be released in Spark 1.1. The core function is to use all the existing hive data and functions on spark;

Spark SQL can be used if it is not related to hive;

Theoretically, with the continuous enhancement of spark SQL in future versions, everything in hive can be done;

 Q6: spark SQLCan I use group?

Spark SQL can use the group by function;

Support for group by in schemardd;

Groupby(Groupingexprs: expression *) (aggresponexprs: expression *): schemardd

L performs a grouping followed by an aggregation.

 

Q7: spark SQLWhich of the following SQL-supported operations have not been found on the official website?

Official Document URL of spark SQL API

Http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark. SQL .SchemaRDD;

Official Website for spark SQL

Http://spark.apache.org/docs/latest/sql-programming-guide.html