collections between nodes, or store collections into Tachyon. We can StorageLevel set these storage levels by passing an object to persist() the method. cache()method uses the default storage level- StorageLevel.MEMORY_ONLY . The complete storage level is described below:
Storage level
meaning
memory_only The
stores the RDD as a non-serialized Java object in the JVM. If the RDD does not fit in m
space will be occupied. Therefore, you need to generate a spark deployment package in the next step;
Spark deployment package Generation Command make-distribution.sh-- Hadoop version: Version Number of hadoop. If this parameter is not added, the hadoop version is 1.0.4.-- With-yarn: whether to support hadoop yarn. If no parameter is added, yarn is not supported.-- With-Hive: whether to support hive in Spark SQL. If this parameter is not added, hive is not supported.-- Skip-Java-test: Indicates
to disk. Subsequent read memory_and_disk_2 _2 on demand from memory and disk, with each partition backed up to 2 cluster nodes, and the other ibid memory_and_disk_ser similar to Memory_only_ser, Memory-less RDD partition data is persisted to disk, avoids, avoids again when need to recalculate memory_and_disk_ser_2 _2, will backup each partition to 2 cluster nodes, other ibid off_heap off_heap parameter is persisted to the distributed memory file system by default. The memory_only_ser,off_heap r
55)E2de b A4 E1 5d 5e Ed (6c) FB Bayi e F9 bd d4 f4 8b de 1dE2f7 BA D (3 4b 3e DC d0 1c D1 A6 2b 4e 8d 8f d4E310 2 E1 6b 1a (EA) C0 CF A0 ab f3 m C6 B4 C8 8d D7-E7e329 C 5b A4 Ba-F5 4a B7 EC 67 Bayi 5c DF 5a 8fe342 C4 6d F7 A3 1f E5 3 A5 0 8 m 7c 9c B0e35b AB 5d DF E0 B9 FC 5 AC D9 (AB A2) CE 5c F0e374 D9 A8 3b D3 E6 4c 8c B4 4e F2 5e 6d 7a 9b D3e38c C4 F8 3 6f f4 F8 9b 62 F3 6 to D0 0 km/DC F9 77E3A5 3b D9 EC F3 B8 D9 db 8c F1 e5 5e f2 e5 2e B0 b 6e E2 81 25E3be 8e b5 DD 5b F9 af Ed 6 cf C9 1d
configuration skips theDC Environment we have configured the relevant policies and user account information;We will synchronize the specified OU later650) this.width=650; "title=" image "style=" border-top:0px;border-right:0px;background-image:none;border-bottom:0 px;padding-top:0px;padding-left:0px;border-left:0px;margin:0px;padding-right:0px; "border=" 0 "alt=" image "src=" Http://s3.51cto.com/wyfs02/M01/06/1E/wKiom1myATnTSF3TAAFKW1O0v5U698.png "he
The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion;
products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the
content of the page makes you feel confusing, please write us an email, we will handle the problem
within 5 days after receiving your email.
If you find any instances of plagiarism from the community, please send an email to:
info-contact@alibabacloud.com
and provide relevant evidence. A staff member will contact you within 5 working days.