Custom iterators (this is a good introduction to functional programming)

Custom iterators

Copyright 2007 by Bill Wagner (http://www.srtsolutions.com/public/blog/20574)

In this article, I'll discuss custom iterators, predicates, and generic methods. you can combine these three techniques to create small building blocks that plug together in a variety of ways to solve everyday development problems. once you can recognize when to apply this pattern, you will be able to create a large number of reusable building blocks.

When we build software, we rarely work with one instance of a data type. Instead, most of our work centers on collections of data:Lists,Arrays,Dictionaries, Or other collections. As a result, so much of our code appears in a series of loops:

foreach ( Thing a in someContainer)
    doWorkWith(a);

That's simple enough, and there's little point in trying to get more reuse from that simple construct. but most of our daily work isn' t really that simple. sometimes the iteration creates a new collection, or a new sequence. sometimes the iteration shocould only affect some of the elements in a collection, not all them. sometimes the iteration will remove some elements from the collection. with all the different variations, we're back to copying code and modifying it. that lowers our productiinstances. when you create M iterators and predicates instead of copying and modifying loops, you decouple the common actions from the specific code. and, often, you will use the output from one iteration as the input to another. by using custom iterators to perform the iteration, you can save memory and increase performance as well.

Let's start with a simple sample, and modify that code to create a set of reusable custom iterators and predicates. suppose you 've been asked to print out a list of people you know from the New York City phone book. it is a contrived problem, but it exhibits the characteristics of dynamic real-world problems. you need to process a large number of objects (the list of all entries in the New York City phonebook ). you need to extract a subset of the properties of each object (the first and last names ). finally, you need to filter the input set to generate the output set (only the people you know ). your first attempt might look like this:

// First attempt:
public List<string> PeopleIKnowInNewYork()
{
    IEnumerable<PhoneBookEntrys> newYorkNumbers =
       PhoneBook.FindListFor(“New York”);
    List<string> peopleIKnow = new List<string>();
    foreach ( PhoneBookEntry ph in newYorkNumbers)
    {
        string name = string.Format(“{0} {1}”, ph.FirstName, ph.LastName);
       if ( RecognizePerson( name ) )
            peopleIKnow.Add(name);
    }
    return peopleIKnow;
}

This Code does produce the proper output, but there's a lot to criticize. you 've loaded every name in the New York phone book into one list. that's very wasteful. chances are you know a very small percentage of people in New York. it is a big place, after all. however, by creating a localListOf all the people in New York, you prevent the garbage collector from freeing any of those entries until the entire list is processed. at best, that's very wasteful. if you don't require a very large memory configuration, your application probably fails. also, from a design standpoint, it falls short of expectations. chances are that as this imaginary sample application grows, you will get other requests for similar, but not identical, features. for example, you may be asked to find everyone you called in the last six months, and print out the phone number you dialed. right now, that produces another method that has almost the exact same contents, with one or two changes.

As a first step to creating more usable code, you can create a custom iterator that returns a sequence of names from a sequence of phonebookentries. That wocould look like this:

IEnumerable<string> ConvertToNames(IEnumerable<PhoneBookEntry> list)
{
    foreach ( PhoneBookEntry entry in list)
        yield return string.Format(“{0} {1}”, entry.FirstName, entry.LastName);
}

This additional method changes your peopleiknow method to this:

// Second attempt:
public List<string> PeopleIKnowInNewYork()
{
    IEnumerable<PhoneBookEntry> newYorkNumbers =
        PhoneBook.FindListFor(“New York”);
    List<string> peopleIKnow = new List<string>();
    foreach ( string name in CovertToNames(newYorkNumbers))
    {
        if ( RecognizePerson( name ) )
            peopleIKnow.Add(name);
    }
    return peopleIKnow;
}

It's a little better. now, anytime you get a new request that requires you to convert a set of phone entries to a set of strings, you 've already got the method to do it. one more quick modification to the method saves you a lot of memory. if you examine the application, you'll almost certainly find that you never need the full list of names. you really only need to enumerate the list of names. so, you can try this modification and see if everything still compiles:

public IEnumerable<string> PeopleIKnowInNewYork()

That works, so you can changePeopleiknowinnewyorkMethod to a custom enumerator method:

// Third attempt:
public IEnumerable<string> PeopleIKnowInNewYork()
{
    IEnumerable<PhoneBookEntry> newYorkNumbers =
        PhoneBook.FindListFor(“New York”);
    foreach ( string name in ConvertToNames(newYorkNumbers))
    {
        if ( RecognizePerson( name ) )
           yield return name;
    }
}

Let's stop a minute and consider what you 've got, and how you can use these techniques in other use cases and other applications. for this discussion, let's assume you 've changedPhonebook. findlistfor ()To be an enumerator method as well.

You started with a pipeline that looked like this:

1. Create a list of every phone book entry in the New York phonebook
2. Examine every entry in that list
3. Create name from the phone book entry
4. If the name is recognized, add it to the output list

By changing methods to enumerators, you 've created a pipeline that looks like this:

1. Read an entry from the phone book.
2. Create a name for that entry
3. If the name is recognized, return the name
4. Repeat

That's a good start. This set of changes took away much of the memory pressure that this method placed on the system. There may only be onePhonebookentryAnd one string representation of a name in memory at one time. AnyPhonebookentryObjects already processed are eligible for garbage collection.

You can do better by introducing predicates, in the form. net delegates. the code you created is a little bit more reusable, but still suffers from being very specific to the operation at hand: Finding recognized names from the New York phone book. in order to make this code more reusable, You Need To parameterize that specific portion of the algorithm. the problem, though, is that the specific portion of the algorithm is actually code. luckily,. net Framework and C # have a way to introduce code as a parameter to a method: A delegate.

Start with yourConverttonamesMethod. With that as a base, you can build what we want: a method that transforms an input sequence into an output sequence of a different type. Here isConverttonamesMethod:

IEnumerable<string> ConvertToNames(IEnumerable<PhoneBookEntry> list)
{
    foreach ( PhoneBookEntry entry in list)
        yield return string.Format(“{0} {1}”, entry.FirstName, entry.LastName);
}

What you want is to pass the code 'string. Format (...)" As a parameter to the method. So you change the signatureConverttonamesLike this:

// A generic method to transform one sequence into another:
delegate Tout Action<Tin, Tout>(Tin element);
IEnumerable<Tout> Transform<Tin, Tout>(IEnumerable<Tin> list, Action<Tin, Tout> method)
{
    foreach( Tin entry in list)
        yield return method(entry);
}

This has some new syntax, but it's really fairly simple. the delegate definition defines the signature of any method that takes one input parameter and returns a single object of another type. transform simple defines a pipeline that returns the output of the Delegate for every object in the input sequence. it's the same thing as converttoname, but it can be used for any input type, any output type, and any algorithm that transforms one type into another.

You 'd call that method like this:

// Fourth attempt:
public IEnumerable<string> PeopleIKnowInNewYork()
{
    IEnumerable<PhoneBookEntry> newYorkNumbers =
        PhoneBook.FindListFor(“New York”);
    foreach ( string name in Transform<PhoneEntry, string>(newYorkNumbers,
        delegate(PhoneEntry entry)
        {
            return string.Format(“{0} {1}”, entry.FirstName, entry.LastName);
        }))
    {
        if ( RecognizePerson( name ) )
            yield return name;
    }
}

There are a few new concepts here, so let's go over it in detail. this code is a little easier to understand if you change the structure just a bit. so here's the second version:

// Fifth attempt:
public IEnumerable<string> PeopleIKnowInNewYork()
{
    IEnumerable<PhoneBookEntry> newYorkNumbers =
        PhoneBook.FindListFor(“New York”);
    IEnumerable<string> names = Transform<PhoneEntry, string>(newYorkNumbers,
        delegate(PhoneEntry entry)
        {
            return string.Format(“{0} {1}”, entry.FirstName, entry.LastName);
        });
    foreach (string name in names)
    {
        if ( RecognizePerson( name ) )
            yield return name;
    }
}

The expression defining names defines a new enumeration over all the names harvested from the phone book. it defines the delegate method inline, making use of anonymous delegates. note that I'm only creating a single line delegate here. I 'd recommend against creating complicated methods as anonymous delegates. but, when you need to wrap an existing function call in a delegate, this syntax is much simpler for other developers to follow.

There's one last loop to convert to something more generic: the loop that checks for names you know. This one can be refactored into a well-known generic pattern:

delegate bool Predicate<T>(T inputValue);
IEnumerable<T> Filter<T>(IEnumerable<T> list, Predicate<T> condition)
{
    foreach (T item in list)
        if (condition(item))
            yield return item;
}

You shoshould recognize this pattern by now. the filter is just a pipeline that returns all members of a list that matches the condition specified by the predicate. one last look at that peopleiknowinnewyork method shows how you can use it:

// Fifth attempt:
public IEnumerable<string> PeopleIKnowInNewYork()
{
    IEnumerable<PhoneBookEntry> newYorkNumbers =
        PhoneBook.FindListFor(“New York”);
    IEnumerable<string> names = Transform<PhoneEntry, string>(newYorkNumbers,
        delegate(PhoneEntry entry)
        {
            return string.Format(“{0} {1}”, entry.FirstName, entry.LastName);
        });
    return Filter(names, delegate(string name)
        { return RecognizePerson(name);} );
}

The final version is quite a bit different than what you started with, and some may argue that it's not that different. but it depends on how you apply it. the final version is structured around much more powerful building blocks. you built a generic method that filters a list of any type, as long as you supply the specific condition for a single item. you built a generic method that transforms a sequence of one type into a sequence of another type. you can leverage this same technique for other operations. this method samples an input sequence, returning every nth item:

IEnumerable<T> NthItem<T>(IEnumerable<T> input, int sampleRate)
{
    int sample = 0;
    foreach (T aSample in input)
    {
        ++sample;
        if (sample % sampleRate == 0)
            yield return aSample;
    }
}

This method generates a sequence, based on some factory method you define:

delegate T Generator<T>();
IEnumerable<T> Generate<T>(int number, Generator<T> factory)
{
    for (int i = 0; i < number; i++)
        yield return factory();
}

And this method merges two sequences of the same type into a new type combining them:

delegate Tout MergeOne<Tin, Tout>(Tin a, Tin b);
IEnumerable<Tout> Merge<Tin, Tout>(IEnumerable<Tin> first,
    IEnumerable<Tin> second, MergeOne<Tin, Tout> factory)
{
    IEnumerator<Tin> firstIter = first.GetEnumerator();
    IEnumerator<Tin> secondIter = second.GetEnumerator();
    while (firstIter.MoveNext() && secondIter.MoveNext())
        yield return factory(firstIter.Current, secondIter.Current);
}

By separating the actions on a single object from the actions on a sequence of types, you created a set of more powerful building blocks to work with very large sets of sophisticated data types.

Summary

In this article, I showed you how custom iterators, delegates, and generic methods can be combined to create more powerful Reusable Building Blocks for your applications. by applying these techniques yourself, you'll find your uses for these techniques, and you'll be more ready for C #3.0, where your of these techniques enjoy more embedded language support.