Massive parallelism – F# in the cloud?


I’m still trying to learn a bit of F# and I thought of a quite nice experiment.
Since F# supports quotations (for you C# devs, think Linq Expressions on roids) wouldn’t it be possible to serialize such quotation and pass it to a webservice and execute that code there?

Imagine the following code for a fractal calculation:

for y = 0 to 100000 do
    CloudExec(
            <@
            let Fractal xx yy =
                .....fractal calculation....

            for x = 0 to 100000 do
                Fractal x y
            @>)

If “CloudExec” passes the code quote for individual scanlines of the fractal to the cloud, we could get some massive parallelism.
It would be just like PLinq but instead of executing a delegate in multiple threads, we would execute blocks of code on multiple threads on multiple machines (please ignore how naïve the code sample above is).

The biggest problem as far as I can tell would be to pass a result set back to the client in some way (that is missing in the sample code).
Input data doesn’t seem to be a problem since values defined outside the quotes are represented as constants in the quote.

It would ofcourse madness to expose such services to the public since you could pass in any code you want, but maybe it would work in an isolated environment.

Have anyone done such thing already?

//Roger

F# Pipelining in C#


Here is one such example where F# developers try to make it look like F# can do things that C# can not.

http://lorgonblog.spaces.live.com/blog/cns!701679AD17B6D310!165.entry

F# code
F# code that apparently is much easier to read than C# code:

HttpGet "http://www-static.cc.gatech.edu/classes/cs2360_98_summer/hw1"
|> fun s -> Regex.Replace(s, "[^A-Za-z']", " ")
|> fun s -> Regex.Split(s, " +")
|> Set.of_array
|> Set.filter (fun word -> not (Spellcheck word))
|> Set.iter (fun word -> printfn "   %s" word)

C# code
My attempt to accomplish the same in C#.

HttpGet("http://www-static.cc.gatech.edu/classes/cs2360_98_summer/hw1")
    .Transform(s => Regex.Replace(s, "[^A-Za-z']", " "))
    .Transform(s => Regex.Split(s, " +"))
    //.AsParallel()  // [EDIT] now with parallelism support
    .Distinct()
    .Where(word => !Spellcheck(word))
    .Process(word => Console.WriteLine("  {0}", word))
    .Execute();

I think that’s fairly similar?

OK, I cheated, “Transform”,”Process” and “Execute” does not exist out of the box in C#.
You would have to write those extension methods yourself.
However, they are reusable and fits nicely into a util lib.

public static class Extensions
{
    public static TOut Transform<TIn, TOut>(this TIn self, Func<TIn, TOut> selector)
    {
        return selector(self);
    }

    public static IEnumerable<T> Process<T>(this IEnumerable<T> self, Action<T> action)
    {
        return self.Select(item =>
        {
            action(item);
            return item;
        });
    }

    public static ParallelQuery<T> Process<T>(this ParallelQuery<T> self, Action<T> action)
    {
        return self.Select(item =>
        {
            action(item);
            return item;
        });
    }

    public static void Execute<T>(this IEnumerable<T> self)
    {
        self.ToList();
    }

    public static void Execute<T>(this ParallelQuery<T> self)
    {
        self.ToList();
    }
}

Either way, I hope this shows that you can accomplish the same thing using good old C# instead of F#.

I still don’t get F#


I think that Microsoft are trying to sell F# to us as something new and awesome, but I’m having serious problems seeing the benefits over C#.

But F# can do function currying!
Well, so can C#.

string Foo(int a,bool b)
{
    //do stuff
}

void UseCurry()
{
  Func<int,string> curriedFooWithTrue = a => Foo(a,true);

     //invoke curried function.
  var res = curriedFooWithTrue(123);
}

F# can do pipelining!
Well, so can C#


var listOfInts = new List<int> {1,2,3,4,5,6};
Func<int,int> squared = x => x*x;
var squaredListOfInts = listOfInts.Select(x => squared).ToList();

F# can use tuples!
Well, they are built into .NET 4 as a generic type so they are available for all .NET languages with generics support.

F# can do tail recursion.

OK, you got me, it can.
Now let me know the last time you really needed that?
All tail recursive algorithms can be implemented as iterative.
But sure, syntactic sugar is nice to have.

F# makes it easier to write async code.

This was one of the arguments at a demo of F# at PDC 2008.
They showed how it was made possible by using PLinq wrapped up in a C# assembly.

Maybe I’ve misunderstood every example I’ve seen, but most of them can be done in C# with pretty much the same amount of code.

What I would like to see is a really good F# example that would be very hard or impossible to accomplish with C#.
If F# is just slightly better than C# on some tasks, then the cost of bringing F# competence into a project will always outweight the slight benefits it brings.

Another argument is that it targets a completely different problem area.
OK, show us where F# shines without lying about what C# can and can not do.

Anyone got such example?

My predictions for 2010


I predict that the C# 4 “dynamic” keyword will be the most abused feature.
It will be used for everything from ducktyped dependency injection, dynamic dictionaries and about a million Rails like frameworks.
All in a very non refactor friendly way.
…It will rarely be used to interop with dynamic .NET languages.

The least used will be the co/contra variance feature.
In 2011 not even the C# team will remember it exists.

The most frustrating feature will be code contracts.
Developers will cry out in pain when they realize that it might not be so easy as they expected it to be.

I’m just guessing :-)