Lisp Debugger


I’m still working on my Lisp language clone.
Today I started to add debugging support to it.
I made the AST aware of the original source code, so each node can reference back to the place it was parsed from, and I also added a bit of call stack features.

I’m pretty satisfied with it so far, it almost feels like a real language now ;-)

Anyway, here is a screenshot of the not so well designed GUI.

 Debugger
Click for full size.

Downloads are available at: 

Source: http://www.puzzleframework.com/roger/mylisp.zip
Binaries: http://www.puzzleframework.com/roger/mylispbin.zip

(The code is written in C#3, VS.NET 2008 solution)

Lisp weekend


I’ve been reading up a bit on functional programming the last few week, the reason is just to comprehend the new features and possibilities in .NET 3.5 as much as possible.

Anyway, I got a bit carried away and started to read about Lisp, and decided to learn what it’s all about.
So what better way to learn a language than to make your own parser for it is there? ;-)

I started to hammer away on a simple parser, and once the parser was done, I couldn’t stop, so I began writing an engine too.
So after a few hours of Aha moments, I finally got my very own Lisp(ish) code executor and a bit more understanding for the language. ;-)

Well, enough blabbering, here are a few samples of whats currently possible in my still un-named language.

Hello world:

(print 'Hello world!')

Simple function and call:

(defun Mul (x y) 
 (* x y))       

(print (Mul 2 3))

Variables:

(let my-var 'hello lisp') 
(let my-int 123) 
(let my-double 123.456) 
(let half-pi (/ pi 2)) 
(let my-arr (arr 1 2 3 4 5 6 7))

Loops:

(foreach item my-arr 
 (print item))      

(for i 1 20 
 (print i))      

(let i 0) 
(while (< i 20) 
 ((print i) (++ i)))

Lambdas:

(let my-lambda (lambda (x y) (* x y))) 
(my-lambda 2 3)

Delegates:

(let my-delegate Mul) // delegate to Mul 
(let print other-print-func) //redirect the print function to "other-print-func"

Objects:

(let form (new Form)) 
(set form Text 'hello windows forms') 
(let button (new Button)) 
(set button Text 'my button') 
(set-event button Click MyButtonClick) 
(list-add (get form Controls) button) 
(call form Show)

List comprehensions:

(foreach item (select (lambda (concat 'transformed: ' item '!')) 
              (where (lambda (> (get item Length) 3)) 
              (list 'foo' 'bar' 'roger' '.net' 'lisp')))       

      (print item))

The next step will be to make it possible to define your own classes.
Im thinking of emitting true .NET classes and let the methods redirect the calls to the engine.
Thus making it possible to redefine the behaviour of a method in runtime.

That, and find some reason to use it :-P

Making a custom Linq engine


You can download the complete source here
NOTE: The download contains much more than this post.

Today I’m going to show how to make a custom linq engine.

So, why would you want to create your own engine?

Well replicating the standard in mem engine is pretty pointless except for educational purposes.
But you might need custom engines for your own frameworks where you want to query your own special DB or media of some sort.
We use this in NPerist, we transform the linq query into our own NPath DSL and then run the NPath query through NPersist into the database.

Anyway, back to the sample.
Lets start by creating a simple console application and add the following code in the Main method:

using System.Linq; 
... 
... 
... 
//setup test data 
List<string> source = new List<string>() { 
    "public", 
    "protected internal", 
    "void", 
    "static", 
    "for", 
    "string", 
    "int", 
    "long", 
    "List<T>" 
};                  

//run a simple query 
var result = from item in source 
             where item.Length > 1 
             select item + " hello linq"; //display the result                

Console.WriteLine(); 
Console.WriteLine("Result:------------------------"); 
foreach (var item in result) 
{ 
    Console.WriteLine("{0}", item); 
}

Nothing fancy, we simply init a list with a few strings, and then we run a standard linq query on it.

Now lets start writing our own Linq engine.
To do this we need a few delegates:

// T = item type 
// IN = item type IN 
// OUT = item type OUT  
  
//delegate to handle the where clause 
public delegate bool WhereFunc<T>(T type);                

//delegate to handle the select clause 
public delegate OUT SelectFunc<IN, OUT>(IN item);

We also need two extension methods:

namespace MyLinqEngine; 
{ 
public static class Sequence 
{ 
    //extension method that will handle where clauses 
    public static List<T> Where<T> 
    (this List<T> source, WhereFunc<T> matchDelegate) 
    { 
        //create a result list 
        List<T> result = new List<T>();         
        //scan all items in the source 
        foreach (T item in source) 
        { 
            //check for match 
            bool match = matchDelegate(item);             
            if (match) 
            { 
                //add match to result 
                result.Add(item); 
            } 
        } 
   
        return result; 
    }     
    
    //extension method that will handle select clauses 
    public static List<OUT> Select<IN, OUT> 
    (this List<IN> source, SelectFunc<IN, OUT> selectorDelegate ) 
    { 
        //create a result list 
        //since the result will have the exact same size as 
        //the source we can init it with a fixed size 
        List<OUT> result = new List<OUT>(source.Count);                  
        //scan each item in the source 
        foreach (IN item in source) 
        { 
            //transform the items in the source to an 
            //item of the given out type 
            OUT resultItem = selectorDelegate(item); 
            result.Add(resultItem); 
        }         
        return result; 
    } 
} 
}

And that’s it!
That’s all you need in order to create the simplest form of custom Linq engine.

The default Linq extensions are activated by the “using System.Linq;” at the top of your program.cs class file.
We can now remove that line and replace it with “using MyLinqEngine;” in order to activate our own linq extensions.

You’ve now got your very own debuggable Linq engine.

Happy coding.

//Roger

You can download the complete source here
NOTE: The download contains much more than this post.

ORM and C#3 anonymous types


Here goes my first blog post in 1+ years.

Nowadays I’m playing around with VS 2008 trying to abuse all the new features as much as possible.

Earlier today I was talking to Mats Helander about making a NPersist v3, for .NET 3.5.
We were discussing new features such as supporting anonymous types instead of our old “Tabular” concept.

After a while I started to think about the possibilities to use anonymous types to create read only views for stored procs,
and I came up with a quite slick solution ( in my own opinion OFC ;-) )

Anyway, here is what I came up with:

//create a prototype for our result 
var prototype = new { FirstName = "" , Age = 0 , Email = "" };      

//fill a list with items of the above type with data from an sproc 
var res = MyDal.ExecuteStoredProc(prototype, "sp_MyStoredProc", p1, p2, p3);      

//present the data in our top notch UI 
foreach (var item in res) 
{ 
    Console.WriteLine("FirstName = {0}", item.FirstName); 
}

What I do here is:
I create an object of an anonymous type, which will act as a prototype for our result.
The anonymous type contains our properties and the type for each property.

Then I pass the prototype object into my DAL and ask my DAL to execute an stored procedure with some args.
The DAL executes procedure and fill a generic list of my anonymous type with the result.
And then return the filled list the consumer.

This allows the consumer to call a stored procedure and get the result in an objectified way.
Making it possible to work with his data in a typed way w/o adding typed datasets or entity classes.
OK, the anonymous type is a sort of entity class too.
But since it can be designed per use case, you do not have to clutter the project with loads of view specific classes.

Well that’s it for now.
Full sample code will be provided soon.

//Roger

Code mangling AOP vs. Runtime Proxy AOP


 THIS IS AN OLD POST FROM MY OLD BLOG (2006) 

AOP is gaining momentum in .NET and there are starting to pop up quite a few AOP frameworks.
A comparison of (some of) those can be found here:
List of existing approaches

As you can see there are a few different approaches to accomplish AOP in .net
the ones that I know are:

  • ContextBoundObject – Inherit ContextBound
  • Source transformation – Extra compile step
  • IL instrumentation – Extra compile step
  • .NET profiling API – Modify the Jitted IL
  • Runtime Subclass proxies – Reflection emit subclasses
  • Runtime Interface proxies – Reflection emit proxies with interfaces

These approaches can be divided into two categories:

  • Inheritance based (Runtime proxies)
  • Code mangling (all the others)

(Ok ContextBound might be a separate category but since its too limited Ill just ignore it ;-) )

So what is the difference between these two approaches?

Code mangling can be very powerful and can modify your code in ways that is not supported by inheritance.
e.g. you could add interception to non virtual members, you could intercept field access and you can change base class of other classes.

Runtime proxies can only intercept ctors, virtual members or interface members.
So runtime proxies are in most cases considered to be the less of the two.
But here is my take on it:

If we take a look at good’ol OO, we have constructs like private, sealed/final, virtual, override etc.
Those constructs all have a purpose, to let the author of a class specify things like:

“It’s all OK to alter the behaviour of this method and things should work all OK if you do because its virtual”

Or:

“You may not alter this method because nasty things might happen”

(e.g. sealed or private etc)

No one has ever questioned those constructs in OOP.
So why should different rules apply to AOP?

the goal is the same as in inheritance, to alter or extend behaviour / functionality of a class.

So in my opinion inheritance based AOP is the way to go because its 100% OO compatible.
It doesn’t break the above rules.

While you might be able to do cool hacks with code mangling, you are also opening up for a whole lot of problems.
e.g., it requires much more of a consumer of a virtual method than a consumer of a non virtual method.

Why?

Because when you consume a non virtual method you know exactly what might happen, you know what exceptions that might occur and you know what the expected behaviour is.
When consuming a virtual method, you don’t know what might happen because you might call the method on your base class or on a subclass and the subclass could have been provided from someone else.
so when you consume the virtual method you might have to add some extra exception handling or make your code a bit more generic because you can’t be certain that things behave exactly as when you call the base implementation.

So if AOP would allow you to break the fundamental OO rules, you could alter the behaviour of a non virtual method and introduce behaviours that is not expected by the consumer.
and I think that is VERY bad.
If the author of the consuming code wrote his code knowing that it was a non virtual method he called, he did not prepare his code for anything else, and the AOP’ed variant of the provider method might screw things up big time.

And since the entire framework is designed w/o AOP in mind, I bet there is plenty of code in both your applications and in the framework itself that simply expects those non virtual methods to behave as they did when the code was written.

So in short, I don’t think AOP should be allowed to break the fundamental OO rules because most of the code you use is designed for those rules.
(However I think its a completely different story with things like AspectJ where the language itself is designed for AOP and those who use it know that every kind of method might be intercepted.)
The next thing is debugging.

Code mangling can accomplish lots of weird stuff like changing the base class of a type or introduce fields.
That might seem like a cool feature but since we all write our code in .NET languages that is NOT designed for AOP, e.g. C# or VB.NET
How are we supposed to debug such code?

The code executing is not the same code you have in your source files.
For me, that is just madness.

In runtime proxies we do not have those problems because the proxies are just dumb objects redirecting your calls to your interceptors, mixins and base methods.
so you will never ever need to debug the proxies themselves since they just redirect the calls.
and your base class and interceptors are still normal .NET code which you can debug just fine.
However there is a scenario where I do agree it is nice to be able to intercept non virtuals.
Debug aspects.
e.g. tracing, code coverage, interaction tests and various mock aspects.
Such aspects is a special case for me since they do pretty much the same as your IDE debugger.
in the debugger you can step into a private method, you can view all private variables etc.
Debug aspects does the same but in a different way.
Well that’s it :-)

And as stated in the beginning, this is just my take on it.
I’m not a fanatic so I could convert if someone just provides me some valid arguments why breaking OO rules is good in AOP :-)

//Roger

Query Objects vs. DSL


 THIS IS AN OLD POST FROM MY OLD BLOG (2006)

 Most O/R Mappers today support some sort of Query Objects.
Query Objects are often some sort of semi fluent combination of objects and methods where you can build a query.
 
Query objects also seem to be the preferred way to build queries in code because they are “Typed”.
Being “Typed” comes with two major benefits, you get intellisense (to some extent) and you get compiler warnings if you do something way too funky with your query.
 
When Mats Helander started to build NPersist, he went for a DSL approach instead of QO’s and invented NPath which object query language similar to SQL in syntax.
 
So why go for an untyped DSL instead of typed QO?
 
Well, one thing that QO’s does not do is to verify your logic, you need unit tests for that.
And once you’ve got unit tests, the benefit of being strictly typed is reduced because with unit tests you can clearly see if your untyped code does work or not.
 
QO’s still got the benefit of intellisense,
But let’s look at the readability of two queries:
 
(the queries come from : http://www.mygenerationsoftware.com/phpbb2/viewtopic.php?t=107)
 
SQL:

SELECT * FROM Employees 
WHERE LastName LIKE 'A%' AND LastName IS NOT NULL OR 
( City LIKE 'Ind%' OR HireDate BETWEEN '1/1/95' AND '4/4/04' )

[edit: added npath sample]

NPath:

SELECT * FROM Employee 
WHERE LastName LIKE 'A%' AND LastName != NULL OR 
( City LIKE 'Ind%' OR HireDate BETWEEN #1/1/95# AND #4/4/04# )

NOTE: npath specifies classes and properties , not tables and fields
See: *** Missing Page *** : Old blog is gone
[/edit]
 

dOOdas ORM:

Dim emps As Employees = New Employees 
emps.Where.LastName.Value = "A%" 
emps.Where.LastName.Operator = WhereParameter.Operand.Like 
Dim wp As WhereParameter = emps.Where.TearOff.LastName 
wp.Operator = WhereParameter.Operand.IsNotNull 
emps.Query.AddConjunction WhereParameter.Conj.Or 
emps.Query.OpenParenthesis() 
emps.Where.City.Conjuction = WhereParameter.Conj.And 
emps.Where.City.Value = "Ind%" 
emps.Where.City.Operator = WhereParameter.Operand.Like 
emps.Where.HireDate.Conjuction = WhereParameter.Conj.Or 
emps.Where.HireDate.Operator = WhereParameter.Operand.Between 
emps.Where.HireDate.BetweenBeginValue = "1/1/95" 
emps.Where.HireDate.BetweenEndValue = "4/4/04" 
emps.Query.CloseParenthesis()

Are QO’s really sane?
If the whole purpose of an O/R Mapper is to reduce code and enable RAD, shouldn’t it be easier to query your DB?
I was a LLBL gen pro user a few years ago and loved most parts of it, but I just couldn’t stand the QO’s.
Just making a query with a few conditions and some sorting would generate queries similar to the one above.
Even if Frans has excellent documentation and the classes got intellisense and all, I still had to spend way too much time
just to make a simple query that should take a few seconds to write.
(I have to add that I haven’t looked at LLBL Gen since and Frans might have altered the API or added some DSL, I’m just stating what I experienced with it about 2 years ago.
my arguments are also targeted against QO’s in general, not only the ones in LLBL Gen)

Later on I got in contact with Mats and saw NPath for the first time, and I loved it, NPath is the #1 reason I teamed up with Mats and started co-develop NPersist.
With NPath I can write queries that are similar to SQL but instead of targeting your DB schema you target your domain classes and properties, and instead of creating joins you just traverse property paths.

So even if NPath is untyped I still find it way more productive than any QO’s and I get a smaller code base which is easy to read and since I do unit tests I can verify that my untyped queries does work as intended.
It would be very interesting to hear other opinions on this topic and why QO’s still is the most common way to make queries in O/R Mappers?
Is it because of the intellisense?
or because most people don’t do unit tests?
or just because it looks cool with some huge object graph?
or because most O/R vendors don’t know how to make a decent parser?

Fire away! :-)

[edit]
Just added this sample incase anyone is interested in NPath and in mem querying:
http://web.archive.org/web/20070103130842/http://blogs.wdevs.com/phirephly/archive/2006/02/03/12096.aspx
[/edit]

NAspect – the first debuggable AOP framework for .NET


THIS IS AN OLD POST FROM MY OLD BLOG (2006)

There are quite a few AOP frameworks for .NET out there today.
But all (?) of them suffer from the same problem, its hard or impossible to debug them.

I myself would pick debugging over AOP any day, so forcing our users to debug via coredbg or such tool was not an option when we started to build NAspect.

NAspect supports an intact callstack when stepping from your consumer code into your interceptors and finally into your real provider method.
Here is a few screenshots:

Place a breakpoint in your consumer code:

Single step into the method, notice how we ended up in our first interceptor instead of the provider method:

Single step into the proceed method, now we arrive at the real provider method.
Note that the call stack is intact and shows all steps from your consumer to your interceptor to your provier method.

This is as far as I know not possible with any other AOP framework for .NET today.

Another problem area when dealing with AOP is to know what interceptors that actually got attached to your method.
You can ofcourse step through the entire call flow and check the callstack, but that is pretty messy if you got a few hundred methods with a few interceptors each.

NAspect support VS.NET 2005 DebuggerVisualizers.
Instead of trying to figure out if your config file works or single step through every method to see the interceptors, we can simply enter debug mode and visualize the AOP’ed instance:

 

On the left side of the visualizer you get a list of all adviced methods in your type,
On the right side you get a list of interceptors for the currently selected method.

The next step will be to build a debugger visualizer that shows an UML graph of the proxied instance, showing what mixins and baseclass it relates to.

//Roger