Genetic Math

Gold Parser Sample

The code creates a fake fiber, which can be suspended and resumed, thus allowing us to “step” through its actions.
This technique could be useful when building an interpreting language where you might want to step through the expressions.

    using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Threading;
using System.Runtime.CompilerServices;


namespace ConsoleApplication7
{
    class Program
    {
        static void Main(string[] args)
        {
            FakeFiber f = new MyFiber();

            f.Run();
            while (true)
            {
                //tell fiber to continue
                f.Continue();
                Console.ReadLine();
            }
        }
    }

    public class MyFiber : FakeFiber
    {

        //recursive loop that never throws stack overflow
        async void DoLoop(int count)
        {
            await Yield(); //clear callstack

            Console.WriteLine("{0} {1}", count, System.Threading.Thread.CurrentThread.ManagedThreadId); 
            
            //we can fetch values from other functions too
            //w/o blowing the call stack
            var i = await IntFunc();
            
            Console.WriteLine("got func result {0}",i);
                     
            if (count == 0)
                return;

            DoLoop(count - 1);
        }

        private async Task<int> IntFunc()
        {
            await Yield(); //clear callstack
            return 1;
        }

        public override void Run()
        {
            DoLoop(100000);
        }
    }

    public abstract class FakeFiber
    {
        public abstract void Run();
        private Task currentTask;
        public bool IsCompleted = false;
        public void Continue()
        {
            var task = currentTask;
            if (task != null)
            {
                task.Start();
                task.Wait();
            }
        }

        protected Task Yield()
        {
            currentTask = new Task(() => { this.currentTask = null; });
            return currentTask;
        }
    }
}

Here is a (naive) F# style Mailbox / Agent using the C# async CTP :

    class Program
    {
        static void Main(string[] args)
        {
            var agents = new List<MyAgent>();
            //create 100 000 agents
            for (int i = 0; i < 100000; i++)
                agents.Add(new MyAgent());

            //buffer some messages
            agents.AsParallel().ForAll(a => a.Send(DateTime.Now.ToString()));

            //start agents
            agents.AsParallel().ForAll(a => a.Run());

            //send some more messages
            agents.AsParallel().ForAll(a => a.Send(DateTime.Now.ToString()));

            Console.ReadLine();
        }
    }

    public class MyAgent : Mailbox<string>
    {
        
        public override async void Run()
        {
            while (true)
            {
                var message = await Receive();                
                Console.WriteLine("Agent {0} got {1}",this.Id, message);
            }
        }
    }

    public abstract class Mailbox<T>
    {
        private static int id;

        protected Mailbox()
        {
            this.Id = id++;
        }

        public int Id { get;protected set; }

        BufferBlock<T> buffer = new BufferBlock<T>();

        
        public abstract void Run();
        
        
        public void Send(T message)
        {
            buffer.SendAsync(message);
        }

        protected async Task<T> Receive()
        {
            return await buffer.ReceiveAsync();
        }
    }

This way, we can spawn thousands of agents w/o allocating threads for each of them.
The message loop is executed on the threadpool for each iteration and then suspended untill a new message arrives.

Example, filling a listbox with categories of some sort.

private async void FillCategories()
{
    var client = new MyServiceReference.MyServiceClient();
    //yield untill all categories have been fetched.
    var categories = await client.GetCategoriesTaskAsync();
    categoriesListBox.DataContext = categories;
}

..elsewhere..

//this extension makes it possible to get a Task of T back from our service client
public static class MyServiceClientExtensions
{
    public static Task<IList<Category>> 
                GetCategoriesTaskAsync(this MyServiceClient client)
    {
        var taskCompletion = new TaskCompletionSource<IList<Category>>();
        client.GetCategoriesCompleted += (s, e) =>
                {
                    if (e.Error != null)
                        taskCompletion.TrySetException(e.Error);
                    else
                        taskCompletion.TrySetResult(e.Result);
                };
        client.GetCategoriesAsync();

        return taskCompletion.Task;
    }  
}

I’ve just installed the new Async CTP (for C# 5 async features)
The SP1 Refresh can be found here: http://msdn.microsoft.com/sv-se/vstudio/async/

The new async and await features makes async programming so much sweeter, it lets you write async code in a sequential manner.
e.g.

static async void ShowGoogleHtmlCode()
{
  WebClient client = new WebClient();
   var result = await client.DownloadStringTaskAsync("http://www.google.com");
   Console.WriteLine(result);
}

This code looks sequential, but the code will return/yield back to the caller when it hits the “await” keyword.
Once the expression after “await” completes, the code will continue to run.

So far so good.
However, there are some design considerations.

Consider this:

public async void SetupUi()
{
  var blogService = new BlogServiceClient();
     var categories = await blogService.GetCategories();
  var latestPosts = away blogService.GetLatestPosts();

     this.Categories.DataSource = categories;
     this.Posts.DataSource = latestPosts;
}

This code will call GetCategories and GetLatestPosts at the same time and then wait for _both_ of them to complete before continuing to fill the GUI elements.
If you are doing a Silverlight app, then you probably want to display each GUI element as soon as possible, and thus, the above code could be rewritten to:

public async void SetupUi()
{
  SetupCategories();
  SetupPosts();
}

public async void SetupCategories()
{
  var blogService = new BlogServiceClient();
     var categories = await blogService.GetCategories();
     this.Categories.DataSource = categories;
}

public async void SetupPosts()
{
  var blogService = new BlogServiceClient();
  var latestPosts = away blogService.GetLatestPosts();
     this.Posts.DataSource = latestPosts;
}

This way, the async calls will be independent of each other and fill the corresponding GUI element once the data it needs has been fetched.

Also, if you were to write the first code in this way:

public async void SetupUi()
{
  var blogService = new BlogServiceClient();
     var categories = await blogService.GetCategories();
     this.Categories.DataSource = categories;

  var latestPosts = away blogService.GetLatestPosts();
     this.Posts.DataSource = latestPosts;
}

In this case, the call to GetCategories would be invoked, and the code waits for it to complete.
Once the categories have been fetched, the GUI element is filled and _then_ the call to GetLatestPost() would be invoked.
Thus, the async calls would not execute at the same time and the time span for the SetupUI to complete would be the same as if all of it were sync code.

Thats all for now..

http://www.amazon.com/Essentials-Metaheuristics-Sean-Luke/dp/0557148596/

(Featuring EvoLisa inspired images ;-) )

Take this (somewhat naive) example:

class Customer
{
   ...
  public double GetOrderTotal()
   {
       var total = ( from order in this.Orders
                        from detail in order.Details
                        select detail.Quantity * detail.ItemPrice)
                       .Sum();

       return total;
   }
}

For a given customer, we iterate over all the orders and all the details in those orders and calculate the sum of quantity multiplied with itemprice.
So far so good.

This will work fine as long as you have all the data in memory and the dataset is not too large, so chances are that you will not notice any problems with this code in your unit tests.

But what happens if the data resides in the database and we have 1000 orders with 1000 details each?
Now we are in deep s##t, for this code to work, we need to materialize at least 1 (cust) + 1000 (orders) * 1000 (details) entities.
The DB needs to find those 1 000 001 rows , the network needs to push them from the DB server to the App server and the App server needs to materialize all of it.
Even worse, what if you have lazy load enabled and aren’t loading this data using eager load?
Then you will hit the DB 1 000 001 times… GL with that! :-)

So clearly, we can not do this in memory, neither with lazy load nor eager load.

But what are the alternatives?
Make an ad hoc sql query?
In that case, what happens to your unit tests?

Maybe we want to keep this code, but we want to execute it in the database instead.

This is possible if we stop beeing anal about “pure POCO” or “no infrastructure in your entities”

Using an unit of work container such as https://github.com/rogeralsing/Precio.Infrastructure

We can then rewrite the above code slightly:

class Customer
{
   ...
  public double GetOrderTotal()
   {
  var total = ( from customer in UoW.Query<Customer>() //query the current UoW
                        where customer.Id == this.Id //find the persistent record of "this"
                        from order in customer.Orders
                        from detail in order.Details
                        select detail.Quantity * detail.ItemPrice)
                       .Sum();

       return total;
   }
}

This code will run the query inside the DB if the current UoW is a persistent UoW.
If we use the same code in our unit tests and use an in mem UoW instance, this code will still work, if our customer is present in the in mem UoW that is..

So the above modification will reduce the number materialized entities from 1 000 001 to 1 (we materialize a double in this case)

I don’t know about you , but I’d rather clutter my domain logic slightly and get a million times better performance than stay true to POCO and suffer from a broken app.

This is a small framework for UoW/Workspace support in .NET with Linq support.

The framework contains a Unit of Work implementation and providers for Entity Framework 4, NHibernate and MongoDB(using NoRM).
There is also a small incomplete Blog sample project included.

There is no documentation nor setup scripts so you’re on your own if you try it.
If you get it to run, be sure to add all indexes to the xml field – primary , secondary – path , secondary – value , secondary – property

Executing this Linq query:

var query = (from order in ctx.GetCollection().AsQueryable()
                where order.OrderTotal > 100000000
                where order.ShippingDate == null
                where order.OrderDetails.Sum(d => d.Quantity * d.ItemPrice) > 10
                select new
                {
                    OrderTotal = order.OrderDetails.Sum(d => d.ItemPrice * d.Quantity),
                    CustomerId = order.CustomerId ,
                    Details = order.OrderDetails
                })
                .Take(5);

Will yeild this Sql + XQuery:

select top 5 Id,DocumentData.query(
'<object type="dynamic">
 <state>
  <OrderTotal type="decimal">
   {fn:sum( 
              for $A in /object[1]/state[1]/OrderDetails[1]/object/state[1] 
                      return ($A/ItemPrice[1] * $A/Quantity[1]))}  </OrderTotal>
  <CustomerId type="guid">
   {xs:string(/object[1]/state[1]/CustomerId[1])}
  </CustomerId>
  <Details type="collection">
   {/object[1]/state[1]/OrderDetails[1]/object}
  </Details>
 </state>
</object>') as DocumentData

from documents
where
CollectionName = 'Order'  and
(documentdata.exist('

/object/state[(fn:sum( 
        for $A in /object[1]/state[1]/OrderDetails[1]/object/state 
             return ($A/Quantity[1] * $A/ItemPrice[1])) > xs:decimal(10)) and

/object[1]/state[1]/ShippingDate[1][@type="null"] and
(/object[1]/state[1]/OrderTotal[1] > xs:decimal(100000000))]

')) = 1