How to implement serialization and deserialization in memory with Avro

Source: Internet
Author: User
Tags serialization

In recent projects to serialize data sent to Kafka using Avro, there are two ways to serialize with Avro: one is to serialize in memory and the other is a data file format.

Change how to choose it.

If you want to integrate Avro into an existing system, it is better to serialize memory.

In other cases, consider using the Avro data file format.

Avro official on-line serialization of the data file format is very clear, this time not to repeat, just to explain how to serialize in memory.

Let's take a simple Avro model as an example


"Type": "Record",

"Name": "Pair",

"Doc": "A pair of strings.",

"Fields": [

{"Name": "Left", "type": "String"},

{"Name": "Right", "type": "String"}



This mode is stored in a road strength (usually under the resources Road), and named PAIR.AVSC (AVSC is the common extension of Avro mode files).

Schema schema= Schema.parse (GetClass () getresourceasstream ("PAIR.AVSC");//declaring the mode to be loaded

Creates an instance of the Avro record, builds a Avro Utf8 instance for the recorded string field

Genericrecord datum=new Genericdata.record (schema);

Datum.put ("left", New Utf8 ("L"));

Datum.put ("Right", New Utf8 ("R"));

Serializing records to an output stream

Bytearrayoutputstream out=new Bytearrayoutputstream ();

Datumwriter<genericrecord> write=new genericdatumwriter<genericrecord> (schema);//DatumWriter Translate data objects into types that the encoder object can understand,

Encoder encoder= New Binaryencoder (out);//The data stream is then written by Encoder.

Write.write (Datum,encoder);

Encoder.flush ();

Out.close ();


Datumreader<genericrecord> reder=new genericdatumreader<genericrecord> (schema);

Decoder decoder=decoderfactory.defaultfactory (). Createbinarydecoder (Out.tobytearray (), NULL);

Genericrecord (Null,decoder);

Assertthat (Result.get ("left"). ToString (), is ("L"));

Assertthat (Result.get ("right"). Tostring,is ("R"));

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.