Tuesday, October 27, 2009

Expression Trees .NET 4.0 Part 3

This is the third part of a series of posts with regards to the new features of Expression Trees in .NET 4.0.  In this post I will try to make things a little bit more difficult than the previous two posts.  I will take an example from MSDN and I will try to make it work using the new features from .NET 4.0.  The main idea of this exercise is to use Expression Trees in order to build Dynamic Queries.  So the .NET 3.5 code is:

string[] companies = { "Consolidated Messenger", "Alpine Ski House", "Southridge Video", "City Power & Light",
"Coho Winery", "Wide World Importers", "Graphic Design Institute", "Adventure Works",
"Humongous Insurance", "Woodgrove Bank", "Margie's Travel", "Northwind Traders",
"Blue Yonder Airlines", "Trey Research", "The Phone Company",
"Wingtip Toys", "Lucerne Publishing", "Fourth Coffee" };

// The IQueryable data to query.
IQueryable<String> queryableData = companies.AsQueryable<string>();

// Compose the expression tree that represents the parameter to the predicate.
ParameterExpression pe = Expression.Parameter(typeof(string), "company");

// ***** Where(company => (company.ToLower() == "coho winery" || company.Length > 16)) *****
// Create an expression tree that represents the expression 'company.ToLower() == "coho winery"'.
Expression left = Expression.Call(pe, typeof(string).GetMethod("ToLower", System.Type.EmptyTypes));
Expression right = Expression.Constant("coho winery");
Expression e1 = Expression.Equal(left, right);

// Create an expression tree that represents the expression 'company.Length > 16'.
left = Expression.Property(pe, typeof(string).GetProperty("Length"));
right = Expression.Constant(16, typeof(int));
Expression e2 = Expression.GreaterThan(left, right);

// Combine the expression trees to create an expression tree that represents the
// expression '(company.ToLower() == "coho winery" || company.Length > 16)'.
Expression predicateBody = Expression.OrElse(e1, e2);

// Create an expression tree that represents the expression
// 'queryableData.Where(company => (company.ToLower() == "coho winery" || company.Length > 16))'
MethodCallExpression whereCallExpression = Expression.Call(
typeof(Queryable),
"Where",
new Type[] { queryableData.ElementType },
queryableData.Expression,
Expression.Lambda<Func<string, bool>>(predicateBody, new ParameterExpression[] { pe }));
// ***** End Where *****

// ***** OrderBy(company => company) *****
// Create an expression tree that represents the expression
// 'whereCallExpression.OrderBy(company => company)'
MethodCallExpression orderByCallExpression = Expression.Call(
typeof(Queryable),
"OrderBy",
new Type[] { queryableData.ElementType, queryableData.ElementType },
whereCallExpression,
Expression.Lambda<Func<string, string>>(pe, new ParameterExpression[] { pe }));
// ***** End OrderBy *****

// Create an executable query from the expression tree.
IQueryable<string> results = queryableData.Provider.CreateQuery<string>(orderByCallExpression);

// Enumerate the results.
foreach (string company in results)
Console.WriteLine(company);

/* This code produces the following output:

Blue Yonder Airlines
City Power & Light
Coho Winery
Consolidated Messenger
Graphic Design Institute
Humongous Insurance
Lucerne Publishing
Northwind Traders
The Phone Company
Wide World Importers
*/


In case you want to check the original post in MSDN check here



So now it is time to see how can we do the same thing using .NET 4.0.  I have created a small console application with the following code:



using System;
using System.Collections.Generic;
using System.Linq;
using System.Linq.Expressions;
using System.Text;
using System.Reflection;

namespace ConsoleDynamic_Queries
{
class Program
{
static void Main(string[] args)
{
string[] companies = { "Consolidated Messenger", "Alpine Ski House", "Southridge Video", "City Power & Light",
"Coho Winery", "Wide World Importers", "Graphic Design Institute", "Adventure Works",
"Humongous Insurance", "Woodgrove Bank", "Margie's Travel", "Northwind Traders",
"Blue Yonder Airlines", "Trey Research", "The Phone Company",
"Wingtip Toys", "Lucerne Publishing", "Fourth Coffee" };

var items = FilterCompanies(companies, "coho winery");

foreach (string item in items)
Console.WriteLine("Items is: {0}", item);

Console.ReadLine();

}

private static List<string> FilterCompanies(string[] companies, string filter)
{
//Input Data
var inputValue = Expression.Parameter(typeof(string[]), "inputValue");
var filterValue = Expression.Parameter(typeof(string), "filterValue");
//Return Variable
var result = Expression.Variable(typeof(List<string>), "result");


var breakLoop = Expression.Label();
//Local Variables for the Loop part
var counter = Expression.Variable(typeof(int), "counter");
var arrayLength = Expression.Variable(typeof(int), "arrayLength");
var currentValue = Expression.Variable(typeof(string), "currentValue");

var process = Expression.Lambda<Func<string[], string, List<string>>>
(
Expression.Block
(
new[] { result},
Expression.Assign
(
result,
Expression.New(typeof(List<string>))
),
Expression.Block
(
new[] { counter, arrayLength, currentValue },
Expression.Assign
(
counter,
Expression.Constant(0)
),
Expression.Assign
(
arrayLength,
Expression.ArrayLength(inputValue)
),
Expression.Loop
(
Expression.Block
(
Expression.IfThen
(
Expression.IsTrue
(
Expression.Equal
(
counter,
arrayLength
)
),
Expression.Break(breakLoop)
),
Expression.Block
(
Expression.Assign
(
currentValue,
Expression.Call(Expression.ArrayAccess(inputValue, counter), typeof(string).GetMethod("ToLower", new Type[] { }))
),
Expression.Block
(
Expression.IfThenElse
(
Expression.GreaterThan
(
Expression.Property(currentValue, typeof(string).GetProperty("Length")),
Expression.Constant(16)
),
Expression.Call
(
result,
typeof(List<string>).GetMethod("Add"),
currentValue
),
Expression.IfThen
(
Expression.Equal
(
currentValue,
filterValue
),
Expression.Call
(
result,
typeof(List<string>).GetMethod("Add"),
currentValue
)
)
)
)

),
Expression.PostIncrementAssign(counter)
),
breakLoop
),
Expression.Block
(
Expression.Call
(
result,
typeof(List<string>).GetMethod("Sort", new Type[] {})
)
)
),
result
),
inputValue,
filterValue

).Compile();


return process(companies, filter);
}
}
}


So by running the code above I got exactly the same results with the .NET 3.5 implementation!!!  Keep in mind that for the second implementation you need to have VS2010 Beta 2 installed in your development environment.



I am not a smoker and this the SmokingCode!

Thursday, October 22, 2009

Expression Trees .NET 4.0 Part 2

In my previous post I showed an example of some of the new extensions that were added to the Expression Tress in .NET 4.0.  In this post I will show you how to create a simple Switch/Case Expression.  So in that case I will be passing two integers to the expression and then using an Expression.Switch expression which will decide whether the values are equal or not.  So here is the method code:

private static string CompareIntegersForEqualitySwitch(int value1, int value2)
{
    var result = Expression.Variable(typeof(string), "result");
    var inputValue1 = Expression.Parameter(typeof(int), "inputValue1");
    var inputValue2 = Expression.Parameter(typeof(int), "inputValue2");

    var compare = Expression.Lambda<Func<int, int, string>>
        (
            Expression.Block
            (
                new[] { result },
                Expression.Block
                (
                    Expression.Assign
                    (
                        result,
                        Expression.Constant(String.Empty)
                    )
                ),
                Expression.Block
                (
                    Expression.Switch
                    (
                        inputValue1,
                        Expression.Assign(result, Expression.Constant("Not Equal")),//default body
                        new []
                        {
                            Expression.SwitchCase(Expression.Assign(result, Expression.Constant("Equal")), new [] {inputValue2})
                        }
                    )
                ),
                result
            ),
            inputValue1,
            inputValue2

        ).Compile();

    return compare(value1, value2);

}

The points of interest here are:

1.  Expression.Switch which is the starting point/expression of the switch statement.  The first expression (inputValue1) is an Expression.Parameter that contains the integer that we will be testing for equality.

2.  There is a default body expression which in this case is returning “Not Equal” as result.

3.  There is an array of Expression.SwitchCase that contains all possible case statements and assigns the appropriate value to the result Expression.Variable.  Of course, if none of those case statements is valid, the default statement will be valid.

I am not a smoker and this is the SmokingCode!

Wednesday, October 21, 2009

Expression Trees .NET 3.5 Vs .NET 4.0 Part 1

Expression Trees is not a new technology.  In fact, they were first introduced with the release on .NET 3.0 which means that have been around for approximately 3 years.  Basically, Expression Trees represent language-level code in the form of data.  This data is stored on a tree shaped structure where each node represents an expression.  You can think of Expression Trees as a form of binary trees where each node contains zero, one or two children.  Based on this Binary Tree implementation the application can quickly search and find the value that is looking for.    If this is a totally new topic for you I suggest you read the Introduction to Expression Trees by Charlie Calvert in order to obtain a very clear idea about Expression Trees. 

I have been using Expression Trees for some time now, mainly in order to create a LINQ provider that connects to an HRMS application.  The development environment that I use is VS2008 and everything looks great so far.  Then a few days ago VS2010 Beta 2 was released and I started reviewing the enhancements/changes there, regarding Expression Trees. Well dear friend, I was stunned!!!  Microsoft people did delivered this time.  A significant amount of additional functionality that basically allows you to take full advantage of the upcoming DLR.  So this is the first of a series of small posts with examples regarding the enhancements/changes between Expression Trees in .NET 3.5 and .NET 4.0.

The first example is a console application created with VS2010 Beta2 that takes two numbers as input, compares them and returns a string with the comparison result.  First let’s take a look at the .NET 3.5 way of doing the comparison using expressions:

private static string WhatIsGreater(int value1, int value2)
{
    Func<int, int, string> compare = (inValue1, inValue2) =>
    {
        if (inValue1 == inValue2)
            return "Equal";
        if (inValue1 > inValue2)
            return "Value1 is Greater than Value2";
        if (inValue1 < inValue2)
            return "Value2 is Greater than Value1";

        return null;
    };

    return compare(value1, value2);

}

Nothing new here.  Just a method that returns a lambda expression.  Well see now the VS2010 approach for implementing the same thing using all the new extension methods of the Expressions.

private static string WhatIsGreaterNew(int value1, int value2)
{
    var result = Expression.Variable(typeof(string), "result");
    var inputValue1 = Expression.Parameter(typeof(int), "inputValue1");
    var inputValue2 = Expression.Parameter(typeof(int), "inputValue2");

    var compare = Expression.Lambda<Func<int, int, string>>
        (
            Expression.Block
            (
                new[] { result },
                Expression.Block
                (
                    Expression.Assign
                    (
                        result,
                        Expression.Constant(String.Empty)
                    )
                ),
                Expression.Block
                (
                    Expression.IfThen
                    (
                        Expression.Equal(inputValue1, inputValue2),
                        Expression.Assign
                        (
                                result,
                                Expression.Constant("Equal")
                        )
                    ),
                    Expression.IfThen
                    (
                        Expression.GreaterThan(inputValue1, inputValue2),
                        Expression.Assign
                        (
                                result,
                                Expression.Constant("Value1 is Greater than Value2")
                        )
                    ),
                    Expression.IfThen
                    (
                        Expression.LessThan(inputValue1, inputValue2),
                        Expression.Assign
                        (
                                result,
                                Expression.Constant("Value2 is Greater than Value1")
                        )
                    )
                ),
                result
            ),
            inputValue1,
            inputValue2

        ).Compile();

    return compare(value1, value2);
}

Hm!  A little bit more different, right?  So let’s describe the main differences here:

1.  We declare two Expression.Parameters that we use in order to compare the values inside the expression.  As you can see these parameters come right after the main Expression body.

2.  We declare an Expression.Variable that we will use in order to return the result of the comparison.

3.  The flow inside the Expression body is a follows:

3.1 declare the variable and initialize it.  Notice the Expression.Assign for initialization purpose

3.2 Compare the expression using simple if statements, an example using Expression.Switch will be posted soon, and assign the return value with the process result.

What is very important to notice is that while in .NET 3.5 it was all about expressions, in .NET 4.0 it is about expressions and statements.  This is now really powerful!  We can mutate variables, express sequence of operations and finally do dynamic dispatch using DLR to its maximum.

I am not a smoker and this is the SmokingCode!

Tuesday, October 20, 2009

ADO.NET Performance Boost

This is something interesting that we discovered during our PSR tests.  So we had a table that had a char(5) as primary key and which was accessed very frequently in order to retrieve data from the table.  By reviewing the execution plans for all select statements we realized that every select on that table was performing an Index Scan instead of an Index Seek.  Cheers! There is room for improvement!  As a result, I started looking into the application code in order to find which part of the application was actually sending the SQL request.  So I came across a method that looked more or less like this:

 

image

Hm! Nothing strange so far, right!  Well, not exactly because the developer who wrote this piece of code forgot something fundamental about .NET String and ADO.NET.  You see System.String by default is Unicode.  As a result in the example above when you don’t specify the DBParameter type ADO.NET will default it to nvarchar for the actual SQL request execution.  So the T-SQL emitted is:

image

which will result to the following execution plan:

image

Oops! There you got it.  So how do you fix it?  Well it is really easy.  Just specify the DBParameter type to be in accordance with the primary key type or else:

image

So with this small change the query emitted becomes:

image

which gives us an optimized execution plan that performs Index Seek instead of Index Scan

image

So this really small change gave us a 15% performance boost.  Oh well! One problem solved, one thousand more waiting to be solved.

I am not a smoker and this is the SmokingCode!