Fake Fibers using Async CTP


This is another PoC, building recursive code with continuations using the Async CTP.

The code creates a fake fiber, which can be suspended and resumed, thus allowing us to “step” through its actions.
This technique could be useful when building an interpreting language where you might want to step through the expressions.

    using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Threading;
using System.Runtime.CompilerServices;


namespace ConsoleApplication7
{
    class Program
    {
        static void Main(string[] args)
        {
            FakeFiber f = new MyFiber();

            f.Run();
            while (true)
            {
                //tell fiber to continue
                f.Continue();
                Console.ReadLine();
            }
        }
    }

    public class MyFiber : FakeFiber
    {

        //recursive loop that never throws stack overflow
        async void DoLoop(int count)
        {
            await Yield(); //clear callstack

            Console.WriteLine("{0} {1}", count, System.Threading.Thread.CurrentThread.ManagedThreadId); 
            
            //we can fetch values from other functions too
            //w/o blowing the call stack
            var i = await IntFunc();
            
            Console.WriteLine("got func result {0}",i);
                     
            if (count == 0)
                return;

            DoLoop(count - 1);
        }

        private async Task<int> IntFunc()
        {
            await Yield(); //clear callstack
            return 1;
        }

        public override void Run()
        {
            DoLoop(100000);
        }
    }

    public abstract class FakeFiber
    {
        public abstract void Run();
        private Task currentTask;
        public bool IsCompleted = false;
        public void Continue()
        {
            var task = currentTask;
            if (task != null)
            {
                task.Start();
                task.Wait();
            }
        }

        protected Task Yield()
        {
            currentTask = new Task(() => { this.currentTask = null; });
            return currentTask;
        }
    }
}

F# style Mailbox / Agent using C# async CTP


More info on TPL DataFlow can be found here : http://www.microsoft.com/downloads/en/details.aspx?FamilyID=d5b3e1f8-c672-48e8-baf8-94f05b431f5c

Here is a (naive) F# style Mailbox / Agent using the C# async CTP :

    class Program
    {
        static void Main(string[] args)
        {
            var agents = new List<MyAgent>();
            //create 100 000 agents
            for (int i = 0; i < 100000; i++)
                agents.Add(new MyAgent());

            //buffer some messages
            agents.AsParallel().ForAll(a => a.Send(DateTime.Now.ToString()));

            //start agents
            agents.AsParallel().ForAll(a => a.Run());

            //send some more messages
            agents.AsParallel().ForAll(a => a.Send(DateTime.Now.ToString()));

            Console.ReadLine();
        }
    }

    public class MyAgent : Mailbox<string>
    {
        
        public override async void Run()
        {
            while (true)
            {
                var message = await Receive();                
                Console.WriteLine("Agent {0} got {1}",this.Id, message);
            }
        }
    }

    public abstract class Mailbox<T>
    {
        private static int id;

        protected Mailbox()
        {
            this.Id = id++;
        }

        public int Id { get;protected set; }

        BufferBlock<T> buffer = new BufferBlock<T>();

        
        public abstract void Run();
        
        
        public void Send(T message)
        {
            buffer.SendAsync(message);
        }

        protected async Task<T> Receive()
        {
            return await buffer.ReceiveAsync();
        }
    }

This way, we can spawn thousands of agents w/o allocating threads for each of them.
The message loop is executed on the threadpool for each iteration and then suspended untill a new message arrives.

Consuming WCF services in Silverlight using Async CTP


Here is a small sample of how you can consume WCF services using the new Async CTP features.

Example, filling a listbox with categories of some sort.

private async void FillCategories()
{
    var client = new MyServiceReference.MyServiceClient();
    //yield untill all categories have been fetched.
    var categories = await client.GetCategoriesTaskAsync();
    categoriesListBox.DataContext = categories;
}

..elsewhere..

//this extension makes it possible to get a Task of T back from our service client
public static class MyServiceClientExtensions
{
    public static Task<IList<Category>> 
                GetCategoriesTaskAsync(this MyServiceClient client)
    {
        var taskCompletion = new TaskCompletionSource<IList<Category>>();
        client.GetCategoriesCompleted += (s, e) =>
                {
                    if (e.Error != null)
                        taskCompletion.TrySetException(e.Error);
                    else
                        taskCompletion.TrySetResult(e.Result);
                };
        client.GetCategoriesAsync();

        return taskCompletion.Task;
    }  
}

Async CTP first impressions


This is just my first observations. nothing fancy..

I’ve just installed the new Async CTP (for C# 5 async features)
The SP1 Refresh can be found here: http://msdn.microsoft.com/sv-se/vstudio/async/

The new async and await features makes async programming so much sweeter, it lets you write async code in a sequential manner.
e.g.

static async void ShowGoogleHtmlCode()
{
  WebClient client = new WebClient();
   var result = await client.DownloadStringTaskAsync("http://www.google.com");
   Console.WriteLine(result);
}

This code looks sequential, but the code will return/yield back to the caller when it hits the “await” keyword.
Once the expression after “await” completes, the code will continue to run.

So far so good.
However, there are some design considerations.

Consider this:

public async void SetupUi()
{
  var blogService = new BlogServiceClient();
     var categories = await blogService.GetCategories();
  var latestPosts = away blogService.GetLatestPosts();

     this.Categories.DataSource = categories;
     this.Posts.DataSource = latestPosts;
}

This code will call GetCategories and GetLatestPosts at the same time and then wait for _both_ of them to complete before continuing to fill the GUI elements.
If you are doing a Silverlight app, then you probably want to display each GUI element as soon as possible, and thus, the above code could be rewritten to:

public async void SetupUi()
{
  SetupCategories();
  SetupPosts();
}

public async void SetupCategories()
{
  var blogService = new BlogServiceClient();
     var categories = await blogService.GetCategories();
     this.Categories.DataSource = categories;
}

public async void SetupPosts()
{
  var blogService = new BlogServiceClient();
  var latestPosts = away blogService.GetLatestPosts();
     this.Posts.DataSource = latestPosts;
}

This way, the async calls will be independent of each other and fill the corresponding GUI element once the data it needs has been fetched.

Also, if you were to write the first code in this way:

public async void SetupUi()
{
  var blogService = new BlogServiceClient();
     var categories = await blogService.GetCategories();
     this.Categories.DataSource = categories;

  var latestPosts = away blogService.GetLatestPosts();
     this.Posts.DataSource = latestPosts;
}

In this case, the call to GetCategories would be invoked, and the code waits for it to complete.
Once the categories have been fetched, the GUI element is filled and _then_ the call to GetLatestPost() would be invoked.
Thus, the async calls would not execute at the same time and the time span for the SetupUI to complete would be the same as if all of it were sync code.

Thats all for now..

O/R Mapping and domain query optimizations


One of the cons of O/R mapping is that the abstraction is a bit too high.
You write object-oriented code and often forget about eventual performance problems.

Take this (somewhat naive) example:

class Customer
{
   ...
  public double GetOrderTotal()
   {
       var total = ( from order in this.Orders
                        from detail in order.Details
                        select detail.Quantity * detail.ItemPrice)
                       .Sum();

       return total;
   }
}

For a given customer, we iterate over all the orders and all the details in those orders and calculate the sum of quantity multiplied with itemprice.
So far so good.

This will work fine as long as you have all the data in memory and the dataset is not too large, so chances are that you will not notice any problems with this code in your unit tests.

But what happens if the data resides in the database and we have 1000 orders with 1000 details each?
Now we are in deep s##t, for this code to work, we need to materialize at least 1 (cust) + 1000 (orders) * 1000 (details) entities.
The DB needs to find those 1 000 001 rows , the network needs to push them from the DB server to the App server and the App server needs to materialize all of it.
Even worse, what if you have lazy load enabled and aren’t loading this data using eager load?
Then you will hit the DB 1 000 001 times… GL with that! :-)

So clearly, we can not do this in memory, neither with lazy load nor eager load.

But what are the alternatives?
Make an ad hoc sql query?
In that case, what happens to your unit tests?

Maybe we want to keep this code, but we want to execute it in the database instead.

This is possible if we stop beeing anal about “pure POCO” or “no infrastructure in your entities”

Using an unit of work container such as https://github.com/rogeralsing/Precio.Infrastructure

We can then rewrite the above code slightly:

class Customer
{
   ...
  public double GetOrderTotal()
   {
  var total = ( from customer in UoW.Query<Customer>() //query the current UoW
                        where customer.Id == this.Id //find the persistent record of "this"
                        from order in customer.Orders
                        from detail in order.Details
                        select detail.Quantity * detail.ItemPrice)
                       .Sum();

       return total;
   }
}

This code will run the query inside the DB if the current UoW is a persistent UoW.
If we use the same code in our unit tests and use an in mem UoW instance, this code will still work, if our customer is present in the in mem UoW that is..

So the above modification will reduce the number materialized entities from 1 000 001 to 1 (we materialize a double in this case)

I don’t know about you , but I’d rather clutter my domain logic slightly and get a million times better performance than stay true to POCO and suffer from a broken app.