Working mechanism of Spark streaming

1. Working mechanism of Spark streaming

Spark Streaming is an extension of the Spark core API that enables the processing of high-throughput, fault-tolerant real-time streaming data.

Support for data acquisition from a variety of data sources, including KAFK,Flume,Twitter,ZeroMQ,Kinesis, and TCP sockets,

After fetching data from a data source, you can use advanced functions such as map,reduce,join, and window to handle complex algorithms.

Finally, the processing results can be stored in the file system, database and field dashboards.

2. Basic steps for writing the spark streaming program:

Working mechanism of Spark streaming