Course Outline:
Section 1th introduction of the project and what can be learned in this course, how to apply it to the actual project 00:09:43 min .
2nd. Installation and use of Scala and IDE and installation of MAVEN plugin 00:07:04 minutes
3rd CentOS Environment Preparation (Java environment, hosts configuration, firewall off) 00:06:24 min
4th Scala Basics-1 00:08:51 min
5th Scala Basics Tutorial-functions and Closures-2 00:30:07 min
6th Scala Basics Tutorial-Arrays and collections-3.1 00:48:33 min
7th Scala Basics Tutorial-Arrays and collections-3.2 00:14:16 min
8th Scala Basics Tutorial-Classes and objects -400:23:06 minutes
9th Scala Basics-Feature and pattern matching -500:13:46 minutes
10th Scala Basics Tutorial-Regular expressions and exception handling -600:12:41 minutes
11th Scala Basics-Knowledge review 00:15:58 min
12th NoSQL Database MongoDB installation 00:04:57 min
section 13th Spring data for mongodb-simple connection mongodb00:07:52 min
section 14th Spring data for mongodb-spring configuration +crud operation (repo not implemented, default action) 00:36:20 min
section 15th Spring Data for mongodb-implementation Repo Interface +mongotemplate+crud operation 00:36:17 min
16th Spring data for mongodb-paged query 00:13:32 min
17th Section Zookeeper cluster installation 00:13:41 min
18th Section Zookeeper Basic introduction -100:22:36 minutes
19th Section Zookeeper working principle-election process (Basic Paxos algorithm) -200:24:27 min
20th Section Zookeeper working principle-election process (Fast Paxos algorithm) -300:31:16 min
21st kafka-Background and architecture introduction 00:12:28 min
22nd Section Kafka cluster installation and testing 00:14:29 minutes
23rd Kafka Data Sending and receiving implementation-java00:31:28 minutes
24th HDFs Stand-alone installation deployment 00:18:51 min
25th section connecting HDFs query store-java00:35:45 minutes
section 26th machine learning Basic Linear algebra introduction 00:05:08 min
27th Section Ikanalyzer Chinese word breaker 00:17:54 min
28th Section Ikanalyzer Chinese Word breaker tool with Java application 00:16:29 minutes
29th Spark and eco-circle introduction 00:11:45 min
Section 30th Introduction and principle of spark running architecture job,stage,task00:26:19 minutes
31st Spark programming model RDD design and operation principle 00:15:48 min
32nd Pure Handwriting First Spark application: wordcount00:23:57 minutes
section 33rd Rdd common function Introduction 00:29:22 minutes
section 34th Spark SQL Introduction, DataFrame creation and use, RDD DataFrame DataSet Mutual conversions 00:12:54 min
section 35th Spark Streaming introduction 00:12:56 min
36th Spark Streaming+kafka Integration Operation 00:18:44 min
37th section Avro combined with Maven for serialization and deserialization 00:21:07 min
Section 38th Spark ML (machine learning) Introduction (supervised learning, semi-supervised learning, unsupervised learning) 00:13:59 min
section 39th feature extraction: Introduction to TF-IDF principle 00:17:49 min
40th section feature extraction: TF-IDF code implementation 00:26:37 minutes
41st Section Clustering algorithm: Introduction to Kmeans principle 00:20:55 min
42nd Section Clustering algorithm: Kmeans code implementation 00:20:03 minutes
43rd section Other spark ml algorithms simple introduction 00:03:48 min
section 44th Spark Connection MongoDB code implementation 00:13:08 minutes
45th Section Mesos Overview of the overall architecture 00:08:25 min
46th Section Mesos installation deployment 00:12:04 minutes
47th Spark on Mesos installation deployment 00:11:12 min
48th. System Architecture Re-introduction + Technology Tandem Introduction (all the learning techniques are integrated into the project) 00:03:57 min
section 49th Project code: Parent project, managing versions of each jar 00:03:47 min
Section 50th Project code: AVRO serialization jar for client and machine learning to implement serialization and deserialization 00:04:46 min
section 51st Project code: Kafka send data jar, call to app and implement word cut and send data to kafka00:06:23 minutes
section 52nd Project Code: Tool class jar for operation of HDFs, word-cutting and operation mongodb00:03:28 minutes
section 53rd Project code: Manipulating the class jar, invoking the tool class to specifically cut words and data for cleaning and storing to hdfs00:05:34 minutes
section 54th Project code: Machine Learning Collection jar, mainly used for storing record00:02:56 minutes
The 55th section of the Project code: Machine learning algorithm jar, mainly for TF-IDF and Kmeans calculation, mainly to achieve upstream and downstream enterprises, supply and demand upstream and downstream model calculation 00:07:11 min
section 56th Project code: Streaming compute jar, mainly accepts the data load model that the client sends to Kafka to calculate 00:04:35 minutes
Section 57th Project code: Test simulation jar, main simulation implementation user load Avro serialized jar write data to kafka00:01:51 minutes
58th Spark on MESOS Deployment submission parameter description 00:08:17 min
section 59th Spark code submitted to Mesos run (spark-submit) 00:07:13 min
60th. Overall flow of the project running through, results show 00:06:54 minutes
61st Spark Tuning introduction 00:08:01 minutes
62nd Spark-based machine learning project-Intelligent Customer System Combat Course summary 00:04:12 min
63rd. Practical work and interview attention 00:03:45 minutes
: Baidu Network disk download
Original address: http://linyunbbs.com/thread-2142-1-1.html
Big Data-spark-based machine learning-smart Customer Systems Project Combat
