ProtocolBuffer 結合 LZO在 Hadoop中的使用（一）

ProtocolBuffer結合LZO在 Hadoop中的使用（一）1.ProtocolBuffer

首先介紹一下ProtocolBuffer吧，可以參考：Protocol Buffer官網

Protocol buffers are a flexible, efficient, automated mechanism for serializing structured data – think XML, but smaller, faster, and simpler. You define how you want your data to be structured once, then you can use special generated source code to easily
write and read your structured data to and from a variety of data streams and using a variety of languages. You can even update your data structure without breaking deployed programs that are compiled against the "old" format.

簡而言之，就是說Protocol buffers能靈活有效地序列化結構化的資料。

接下來是Java中使用它的教程：Java Protocol Buffers

(1)定義訊息格式在.proto檔案

package tutorial;option java_package = "com.example.tutorial";option java_outer_classname = "AddressBookProtos";message Person {  required string name = 1;  required int32 id = 2;  optional string email = 3;  enum PhoneType {    MOBILE = 0;    HOME = 1;    WORK = 2;  }  message PhoneNumber {    required string number = 1;    optional PhoneType type = 2 [default = HOME];  }  repeated PhoneNumber phone = 4;}message AddressBook {  repeated Person person = 1;}

這是一個.proto檔案的例子，

這裡挺好理解這個檔案的，需要說明一下的是，如果不定義java_outer_classname，那麼就會只用檔案名稱作為classname，欄位分為required，optional和repeated，其中repeated指的是欄位可能重複出現，

(2)編譯你的Protocol Buffers

1.首先你需要下載安裝環境：下載安裝

2. 運行如下代碼：

protoc -I=$SRC_DIR --java_out=$DST_DIR $SRC_DIR/addressbook.proto

這時，你就會得到com/example/tutorial/AddressBookProtos.java類

(3)使用 Java protocol buffer API 去讀寫訊息

// required string name = 1;public boolean hasName();public String getName();// required int32 id = 2;public boolean hasId();public int getId();// optional string email = 3;public boolean hasEmail();public String getEmail();// repeated .tutorial.Person.PhoneNumber phone = 4;public List<PhoneNumber> getPhoneList();public int getPhoneCount();public PhoneNumber getPhone(int index);

這是相關的欄位，分別解析成了java不同的類型，

同時Person.Builder

// required string name = 1;public boolean hasName();public java.lang.String getName();public Builder setName(String value);public Builder clearName();// required int32 id = 2;public boolean hasId();public int getId();public Builder setId(int value);public Builder clearId();// optional string email = 3;public boolean hasEmail();public String getEmail();public Builder setEmail(String value);public Builder clearEmail();// repeated .tutorial.Person.PhoneNumber phone = 4;public List<PhoneNumber> getPhoneList();public int getPhoneCount();public PhoneNumber getPhone(int index);public Builder setPhone(int index, PhoneNumber value);public Builder addPhone(PhoneNumber value);public Builder addAllPhone(Iterable<PhoneNumber> value);public Builder clearPhone();