How to implement serialization and deserialization in memory with Avro

In recent projects to serialize data sent to Kafka using Avro, there are two ways to serialize with Avro: one is to serialize in memory and the other is a data file format.

Change how to choose it.

If you want to integrate Avro into an existing system, it is better to serialize memory.

In other cases, consider using the Avro data file format.

Avro official on-line serialization of the data file format is very clear, this time not to repeat, just to explain how to serialize in memory.

Let's take a simple Avro model as an example

{

"Type": "Record",

"Name": "Pair",

"Doc": "A pair of strings.",

"Fields": [

{"Name": "Left", "type": "String"},

{"Name": "Right", "type": "String"}

]

}

This mode is stored in a road strength (usually under the resources Road), and named PAIR.AVSC (AVSC is the common extension of Avro mode files).

Schema schema= Schema.parse (GetClass () getresourceasstream ("PAIR.AVSC");//declaring the mode to be loaded

Creates an instance of the Avro record, builds a Avro Utf8 instance for the recorded string field

Genericrecord datum=new Genericdata.record (schema);

Datum.put ("left", New Utf8 ("L"));

Datum.put ("Right", New Utf8 ("R"));

Serializing records to an output stream

Bytearrayoutputstream out=new Bytearrayoutputstream ();

Datumwriter<genericrecord> write=new genericdatumwriter<genericrecord> (schema);//DatumWriter Translate data objects into types that the encoder object can understand,

Encoder encoder= New Binaryencoder (out);//The data stream is then written by Encoder.

Write.write (Datum,encoder);

Encoder.flush ();

Out.close ();

Deserialization

Datumreader<genericrecord> reder=new genericdatumreader<genericrecord> (schema);

Decoder decoder=decoderfactory.defaultfactory (). Createbinarydecoder (Out.tobytearray (), NULL);

Genericrecord Result=reader.read (Null,decoder);

Assertthat (Result.get ("left"). ToString (), is ("L"));

Assertthat (Result.get ("right"). Tostring,is ("R"));