How to add protobuf support for asp.net core? asp. netprotobuf

How to add protobuf support for asp.net core? asp. netprotobuf

Preface

In some applications with high performance requirements, protocol buffer serialization is used, which is better than Json. Protocol buffer is more backward compatible.

Asp.net core adopts the new MiddleWare method. Therefore, to use protobuf serialization, you only need to use Protobuf-net to modify the object to be serialized, add the corresponding Formatter during MVC initialization.

No time to explain. Get on the bus.

Obtain Zaabee. AspNetCoreProtobuf through NuGet

Install-Package Zaabee.AspNetCoreProtobuf

Modify the ConfigureServices method in the Startup. cs file.

public void ConfigureServices(IServiceCollection services){  services.AddMvc(options => { options.AddProtobufSupport(); });}

Get started ...... In this case, you can use the content-type of application/x-protobuf to enable asp.net core to use protobuf for serialization/deserialization.

Test code

Add the following DTO to the asp.net core project:

[ProtoContract]public class TestDto{  [ProtoMember(1)] public Guid Id { get; set; }  [ProtoMember(2)] public string Name { get; set; }  [ProtoMember(3)] public DateTime CreateTime { get; set; }  [ProtoMember(4)] public List<TestDto> Kids { get; set; }  [ProtoMember(5)] public long Tag { get; set; }  [ProtoMember(6)] public TestEnum Enum { get; set; }}public enum TestEnum{  Apple,  Banana,  Pear}

Create an XUnit project and use Nuget to reference Microsoft. AspNetCore. TestHost to create a test class.

public class AspNetCoreProtobufTest{  private readonly TestServer _server;  private readonly HttpClient _client;  public AspNetCoreProtobufTest()  {    _server = new TestServer(      new WebHostBuilder()        .UseKestrel()        .UseStartup<Startup>());    _client = _server.CreateClient();  }  [Fact]  public void Test()  {    // HTTP Post with Protobuf Response Body    _client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/x-protobuf"));    var dtos = GetDtos();    var stream = new MemoryStream();    ProtoBuf.Serializer.Serialize(stream, dtos);    HttpContent httpContent = new StreamContent(stream);    // HTTP POST with Protobuf Request Body    var responseForPost = _client.PostAsync("api/Values", httpContent);    var result = ProtoBuf.Serializer.Deserialize<List<TestDto>>(      responseForPost.Result.Content.ReadAsStreamAsync().Result);    Assert.True(CompareDtos(dtos,result));  }  private static bool CompareDtos(List<TestDto> lstOne, List<TestDto> lstTwo)  {    lstOne = lstOne ?? new List<TestDto>();    lstTwo = lstTwo ?? new List<TestDto>();    if (lstOne.Count != lstTwo.Count) return false;    for (var i = 0; i < lstOne.Count; i++)    {      var dtoOne = lstOne[i];      var dtoTwo = lstTwo[i];      if (dtoOne.Id != dtoTwo.Id || dtoOne.CreateTime != dtoTwo.CreateTime || dtoOne.Enum != dtoTwo.Enum ||        dtoOne.Name != dtoTwo.Name || dtoOne.Tag != dtoTwo.Tag || !CompareDtos(dtoOne.Kids, dtoTwo.Kids))        return false;    }    return true;  }  private static List<TestDto> GetDtos()  {    return new List<TestDto>    {      new TestDto      {        Id = Guid.NewGuid(),        Tag = long.MaxValue,        CreateTime = DateTime.Now,        Name = "0",        Enum = TestEnum.Apple,        Kids = new List<TestDto>        {          new TestDto          {            Id = Guid.NewGuid(),            Tag = long.MaxValue - 1,            CreateTime = DateTime.Now,            Name = "00",            Enum = TestEnum.Banana          },          new TestDto          {            Id = Guid.NewGuid(),            Tag = long.MaxValue - 2,            CreateTime = DateTime.Now,            Name = "01",            Enum = TestEnum.Pear          }        }      },      new TestDto      {        Id = Guid.NewGuid(),        Tag = long.MaxValue - 3,        CreateTime = DateTime.Now,        Name = "1",        Enum = TestEnum.Apple,        Kids = new List<TestDto>        {          new TestDto          {            Id = Guid.NewGuid(),            Tag = long.MaxValue - 4,            CreateTime = DateTime.Now,            Name = "10",            Enum = TestEnum.Banana          },          new TestDto          {            Id = Guid.NewGuid(),            Tag = long.MaxValue - 5,            CreateTime = DateTime.Now,            Name = "11",            Enum = TestEnum.Pear          }        }      }    };  }}

Why protobuf?

Because fast ...... In our testing of using business data, protobuf's serialization/deserialization performance is about three times that of Json.net, And the serialized volume is about 1/2 of the Json size, this can improve the webapi throughput performance to a considerable extent.

In addition, there is a loss of precision in the processing of floating point numbers in Json, because the secure integer of the number type in JS is 53 bits. When we use the snowflake algorithm to provide a global incremental ID, duplicate primary keys will be generated due to loss of precision. In addition, if the DateTime type is passed for the same reason, the time matching error is also caused by inconsistent milliseconds. The general solution is to use string transmission. However, this is because the remedies do not solve the problem from the root cause. Therefore, we should directly use protobuf for processing.

Disadvantages of protobuf

The DTO layer must reference protobuf-net to add features, which causes code intrusion to some extent. Basically, DTO belongs to POCO. If you rely on a third-party package, you will always feel a little inactive ...... In addition, the serialized data of protobuf is not visualized. Therefore, if you are using message queue or request monitoring, you must consider whether protobuf is suitable for use cases.

Principle

Asp.net core is implemented based on middleware. Its default JsonFormater (based on Json.net) is provided. asp.net core selects the corresponding Formater based on content type to process Object serialization, including InputFormatter (deserialization) and OutputFormatter (serialization ). Therefore, in addition to protobuf, we can also add or replace other serialization methods. For example, we can use Jil to replace Json.net to improve Json performance.

The source code for the above implementation and Demo and test has been put on GitHub.

Summary

The above is all the content of this article. I hope the content of this article has some reference and learning value for everyone's learning or work. If you have any questions, please leave a message to us, thank you for your support.