A passthrough solution for data result sets

I. background

Many people have experienced such an application environment: the server obtains the result set by querying the database for client data requests, and then returns the result set to the client. Here, to make it easier for the server to expand or contract data requests from the client, the server must not know the format of the data result set returned by the client each time.

So how can we make the server expand without knowing the format of the client request dataset, that is, the transparent binary format? Think about it: serialize the query result dataset into a common format, and then the client deserializes the data format as needed.

 

Ii. Solution

After learning about Windows ado, we know that its recordset supports serializing data to a memory buffer. It can also be deserialized from the memory buffer into A recordset object. Well, there are goals and methods. Of course, we can support passthrough of data result sets.

The problem now is how to serialize the result set to a memory segment and deserialize it into a data result set. To overcome complexity, humans rely on abstract layering-isolation complexity. So how to abstract a data result set? I thought about it for a while!

Isn't a data result set a two-dimensional table, and a two-dimensional table having rows, columns, and cells? There are cells in the row and column names in the column (convenient for fast indexing ).

The following is the abstract code about rows, columns, and cells.

 

Struct cell {size_t col_index _; size_t row_index _; typedef STD: Allocator <char> allocator_type; typedef STD: basic_string <char, STD: char_traits <char>, allocator_type> data_type; data_type data _; cell (): col_index _ (0), row_index _ (0) {} cell (size_t col_ I, size_t row_ I, const void * data, size_t Len): col_index _ (col_ I), row_index _ (row_ I) {data _. resize (LEN); const char * TMP = static_cast <const Cha R *> (data); STD: Copy (TMP, TMP + Len, data _. begin ();} cell (Cell & RHs): col_index _ (RHS. col_index _), row_index _ (RHS. row_index _), data _ (STD: Move (RHS. data _) {} cell & operator = (Cell & RHs) {If (& RHS! = This) {col_index _ = RHS. col_index _; row_index _ = RHS. row_index _; Data _ = STD: Move (RHS. data _);} return * This;} PRIVATE: Cell (const cell &); cell & operator = (const cell &); Public: size_t size () const {// data attribute size size_t size = sizeof (row_index _) + sizeof (col_index _) + sizeof (Data _. size () + Data _. size (); Return size;} void set (size_t col_ I, size_t row_ I, const void * data, size_t Len) {col_index _ = col_ I; row_index _ = row_ I; Data _. resize (LEN); const char * TMP = static_cast <const char *> (data); STD: Copy (TMP, TMP + Len, data _. begin ();} void clear () {col_index _ = 0; row_index _ = 0; Data _. clear ();}};

The cell contains row numbers, columns, and data.

class col{    friend ostream& operator << (ostream & , const col &);    friend istream& operator >> (istream & , col &);    friend bool operator==(const col &lhs, const col &rhs);public:    typedef std::string name_type;    typedef size_t id_type;private:    name_type name_;public:    col()    {}    col(const std::string &name)        : name_(name)    {}    col(col &&r)        : name_(std::move(r.name_))    {    }    col &operator=(col &&r)    {        if( &r != this )        {            name_ = std::move(r.name_);        }        return *this;    }private:    col(const col &);    col &operator=(const col &);public:    void name(const name_type &name)    { name_ = name; }    const name_type &name() const    { return name_; }    size_t size() const    {        return sizeof(name_.length()) + name_.length() * sizeof(name_type::value_type);    }    void clear()    {        name_.clear();    }};

Col only contains one name attribute.

 

Class row {typedef STD: Allocator <cell> allocator_type; typedef STD: vector <cell, allocator_type> cells_type; cells_type cells _; friend ostream & operator <(ostream &, const row &); friend istream & operator> (istream &, row &); friend bool operator = (const row & LHS, const row & RHs); public: row () {} explicit row (size_t CNT) {cells _. resize (CNT);} row (row & RHs): cells _ (STD: Move (RHS. cells _)) {} Row & operator = (row & RHs) {If (& RHS! = This) {cells _ = STD: Move (RHS. cells _);} return * This;} private: Row (const row &); row & operator = (const row &); Public: Cell & operator [] (size_t index) {assert (cells _. size ()> index); Return cells _ [Index];} const cell & operator [] (size_t index) const {assert (cells _. size ()> index); Return cells _ [Index];} size_t size () const {// get all data sizes size_t size = sizeof (cells _. size (); size + = STD: Accumulate (cells _. begin (), cells _. end (), 0, [] (size_t CNT, const cell & Val)-> int {CNT + = Val. size (); Return CNT;}); Return size;} void reserve (size_t CNT) {cells _. reserve (CNT);} void clear () {STD: for_each (cells _. begin (), cells _. end (), [] (Cell & Val) {Val. clear () ;}); cells _. clear ();}};

The row contains the set of all cells.

 

A hot cell, Col, and row can be assembled into a table.

class table{    struct impl;    impl *impl_;    friend ostream& operator << (ostream & , const table &);    friend istream& operator >> (istream & , table &);    friend bool operator==(const table &lhs, const table &rhs);    friend class table_iterator;public:    table();    ~table();private:    table(const table &);    table &operator=(const table &);public:    void reserve_col(size_t col_cnt);    void reserve_row(size_t row_cnt);    size_t insert_col(const std::string &name);    size_t insert_row();    void set_cell(size_t col_i, size_t row_i, const void *data, size_t size);    size_t size() const;    void clear();    size_t col_cnt() const;    size_t row_cnt() const;    const std::string &col_name(size_t index) const;    std::pair<const char *, size_t> get_data(size_t row_i, size_t col_i) const;    std::pair<const char *, size_t> get_data(size_t row_i, const std::string &name) const;};

Table_iterator is a forward iterator used to traverse tables.

Ostream and istream are used to serialize the member attributes of a table. Let's look at the use case.

Int _ tmain (INT argc, _ tchar * argv []) {// column name STD: String cols_name [] = {"test1", "Test2 ", "test3"}; // cell data STD: String cell_data [] = {"1", "1.01", "test_data" ,}; Database: Table T; // insert columns and rows for (size_t I = 0; I! = _ Countof (cols_name); ++ I) size_t col_ I = T. insert_col (cols_name [I]); const size_t row_num = 5; For (size_t I = 0; I! = Row_num; ++ I) T. insert_row (); // write the cell data for (size_t I = 0; I! = _ Countof (cols_name); ++ I) {for (size_t J = 0; J! = Row_num; ++ J) {T. set_cell (I, j, cell_data [I]. c_str (), cell_data [I]. size () ;}// serialization buffer STD: vector <char> buffer; buffer. resize (T. size (); // serialized to the buffer Database: ostream OS (& buffer [0], buffer. size (); OS <t; // deserialization Database: Table tt; Database: istream in (buffer. data (), buffer. size (); In> tt; // compare and print assert (t = TT); STD: Copy (Database: table_iterator (TT), Database :: table_iterator (), STD: ostream_iterator <STD: String> (STD: cout, ""); Return 0 ;}

Iii. Summary

There are many implementation methods for the passthrough of data result sets. You can also choose mature solutions, such as Google protobuf, which can be serialized from the database to the memory and then transmitted over the network, then let's get started with xxoo!

I think the implementation of this kind of version is not brilliant, but this abstract process is worth recording. You are welcome to comment on it.

Here is the address on git, https://github.com/chenyu2202863/smart_cpp_lib/tree/master/test/database_test