Implement iterative access in WCF

In general, in the following scenarios, we need to use the iterator to access the sequence instead of returning all sequences at once:

1. There are many sequences of content, and a large amount of memory is required to return all results at a time.
2. It takes a long time to return all sequences and requires rapid response.
3. No need to read all the sequences. You only need to find the key data and then terminate the traversal.

This rule is also applicable to network programming. In the above scenarios, we often need to apply the iterator mode to obtain data. A typical example is the traversal in SNMP. The SNMP protocol even defines the get-next Standard Operation to implement iterative access. Another familiar example is that the select Operation in the database reads all data through the cursor.

A simple Attempt

So how can we implement iterative access in WCF? If we want to provide an iterative access interface for data traversal, a more intuitive idea is to achieve this:

[ServiceContract]
Public
Interface
IService1
{
[OperationContract]
IEnumerable <int> GetValue ();
}

However, when we generate the client code, we will find that the interface generated by the client is:

Public
Int [] GetValue ()

The returned value is int [] instead of IEnumerable <int>. That is to say, the server waits until the result is completed. The client reads the result at a time, which is equivalent to the implicit call of a ToArray function on the server, iterative access is not possible.

Therefore, we must implement iterative access by ourselves. There are two common methods: 1. index-based access, 2. for cursor-based access, the following describes the two methods.

Index-based iterative access

A typical example of indexed access is the get-next operation of snmp. The basic principle is as follows:

1. Obtain the index of the first and first values through an empty index.
   
2. Obtain the index of the second value and the second value based on the index of the first value.
   
3. Repeat Step 1 until all values are obtained.
   

The method is relatively simple and can be implemented by a function.

Server:

[ServiceContract]
Public
Interface
IService1
{
[OperationContract]
Bool GetNextValue (ref
Int? Index, out
Int value );
}

Public
Class
Service1: IService1
{
Int [] valueList = new
Int [] {2, 3, 5, 8, 4 };

Public
Bool GetNextValue (ref
Int? Index, out
Int value)
{
Index = (index ?? -1) + 1;

If (index = valueList. Length) // The Last vertex, cannot getnext
{
Value = 0;
Return
False;
}

Value = valueList [(int) index];
Return
True;
}
}

Client:

Int? Index = null;
Int value = 0;
While (client. GetNextValue (ref index, out value ))
{
Console. WriteLine (value );
}

Although this method seems relatively simple, friends who have implemented mib know that this access method has a large constraint on the data structure:

• The data source must maintain a key that can be iterated so that next-key can be found based on the key.
  
• The data source must be queryable so that next-value can be found based on next-key.
  
• Each time you obtain data, you need to go through a complete query. The performance is poor and the data source must be an efficient data structure.
  

Due to these constraints, it is often necessary to store it into a tree-like data structure (the array structure usually cannot meet the demand for adding/deleting performance), but this structure may not be applicable to its own business. Therefore, two sets of data structures are stored in most cases. One is used to traverse mib and the other is used to implement services. When business data is added or deleted, the tree needs to be updated synchronously, which is very troublesome.

An index-based iteration is very different from the traditional yield generation iterator: index-based iteration does not save the iteration state, so it is necessary to deduce the iteration by index, each iteration is a brand new query with low efficiency and high restrictions.

Cursor-based iterative access

In view of this, we can save the iteration status on the server side in a way similar to the SQL-select cursor. When performing iterative access, we only need to move the cursor down. In C #, we can easily generate a cursor iteration on the server side using the yield syntax sugar. A major problem is how to pass the cursor-IEnumerator to the client.

As we have tried before, IEnumerator and IEnumerable cannot be directly returned to the client (unable to serialize), so we need to map and query the iterator by returning a client Key.

The basic implementation is as follows:

Server:

[ServiceContract]
Public
Interface
IService1
{
[OperationContract]
Guid GetIterator ();

[OperationContract]
Bool GetNextValue (Guid iteratorKey, out
Int value );
}

Public
Class
Service1: IService1
{
Int [] valueList = new
Int [] {2, 3, 5, 8, 4 };
Static
Dictionary <Guid, IEnumerator <int> iterators = new
Dictionary <Guid, IEnumerator <int> ();

Public
Guid GetIterator ()
{
Var iterator = valueList. Select (I => I). GetEnumerator ();
Var key = Guid. NewGuid ();

Iterators [key] = iterator;
Return key;
}

Public
Bool GetNextValue (Guid iteratorKey, out
Int value)
{
Var iterator = iterators [iteratorKey];

If (! Iterator. MoveNext ())
{
Iterator. Dispose ();
Iterators. Remove (iteratorKey );

Value = 0;
Return
False;
}
Else
{
Value = iterator. Current;
Return
True;
}
}
}

Client:

Var iteratorKey = client. GetIterator ();
Int value = 0;
While (client. GetNextValue (iteratorKey, out value ))
{
Console. WriteLine (value );
}

Although it seems that this method is more complex than the previous index, this process basically achieves the Key ing between the iterator and Key. This part of code is common, if you convert it into a generic Helper class, you can directly reuse it to any place where you need to use this mode. The implementation process of iteration only uses valueList. select (I => I ). getEnumerator () is implemented, which is much simpler than the previous method (index iteration in actual projects is much more complicated than the previous example ).

A hidden BUG:

After reading the previous Code, experienced users will find that the implementation has a Bug: when the client only accesses part of the data, it will exit the time, and the iterator saved on the server will not be released. There are usually two solutions to this problem:

1. The server provides the Dispose interface to notify the client to release the iterator after traversal.
   
2. The server performs timeout processing on the iterator according to a certain mechanism. For example, if there is no iteration access within one minute, the iterator is automatically deleted.
   

The two methods have their own advantages and disadvantages and can be used in combination. However, it is not reliable for the client to actively call the Dispose interface (the client process may be accidentally killed after several Data Accesses), and the server still needs to maintain the Timeout Policy, or set the service to session mode. when the end of the session is detected (the client exits) and all the iterators of the session are released, I will not implement the code.