LINQ Expression Trees and the Specification Pattern

Over the past couple of months I have tried to immerse myself in domain-driven design, which includes learning about its purpose, the methodology, and the domain patterns presented in Evans’ book and built upon in many other venues (blogs, conferences, etc.). While I have not worked on a full-fledged DDD project, I have fiddled with a lot of patterns. One of these is the Specification pattern, which says to introduce a predicate-like Value Object into the domain layer whose purpose is to evaluate whether an object meets some criteria. From what I’ve read in Evans’ book, specification objects typically have an isSatisfiedBy method that takes a domain object and returns a boolean. The specification therefore encapsulates a predicate that can be used to test an object to see if it satisfies the criteria.

image

The problem that Evans later calls out is that of querying a data store using specification objects as filters. Because using the specification to filter records from the database requires that those records be selected and reconstituted into objects, it can be inefficient for some applications to use specification objects as is. (Imagine using a specification object on one million rows in the Customer table just to find the gold Customers!) Surely we can do better.

Ideas

One idea in the book is to allow a repository to help with the implementation and utilize double dispatch to keep the separation of domain and infrastructure in tact. Application code calls a method on a repository to query for objects based on a specification. That repository passes itself to a method on the specification object, so the specification can utilize the repository’s power to query for the objects that fulfill the criteria, and then return that data to the application.

image

Another alternative is to harness the power of LINQ and expression trees to represent the predicate that the specification object encapsulates. This means that we can (1) use the expression trees in the infrastructure to let the data store take care of filtering and (2) still represent our rule in one location without resorting to compromises in the repository API.

Expression trees are abstract syntax trees that can represent the predicates that specification objects strive to encapsulate. With these expression trees, certain O/R mappers like LINQ to SQL, the Entity Framework, and LLBLGen Pro can determine the intent of the code and translate it into the corresponding T-SQL code to run against the database.

Creating an expression tree is very simple. In fact, if you’ve used any of the O/R mappers I mentioned above, you’ve probably used them already. Here’s an example of an expression tree being used in LINQ to SQL to generate the WHERE clause in the corresponding T-SQL query below.

NorthwindDataContext db = new NorthwindDataContext();

db.Products.Single(p => p.ProductName == "Aniseed Syrup");

SELECT [t0].[ProductID], [t0].[ProductName], [t0].[SupplierID], [t0].[CategoryID], [t0].[QuantityPerUnit], [t0].[UnitPrice], [t0].[UnitsInStock], [t0].[UnitsOnOrder], [t0].[ReorderLevel], [t0].[Discontinued]

FROM [dbo].[Products] AS [t0]

WHERE [t0].[ProductName] = @p0

– @p0: Input NVarChar (Size = 13; Prec = 0; Scale = 0) [Aniseed Syrup]

– Context: SqlProvider(Sql2008) Model: AttributedMetaModel Build: 3.5.30729.1

Normally the lambda expression ‘p => p.ProductName == "Aniseed Syrup"’ would be treated as a Func<Product, bool>. However, in this particular usage the compiler infers that it is an Expression<Func<Product, bool>>. The difference means that the LINQ to SQL library no longer has a method pointer. Instead, it has a tree which represents what that method does. LINQ to SQL can visit the nodes in this tree and translate what it finds into SQL without ever invoking the code itself. A simple Func does not have that capability; it is simply a method pointer, like any other delegate type.

I hope you start to see how expression trees and the specification pattern can be very powerful together. If in addition to exposing an isSatisfiedBy method on the specification object, we add something which exposes the raw Expression, the repository can compose this Expression into the query and filter the results using the infrastructure. Let’s look at some code.

For this example, let’s continue to use the Products table from Northwind. The specification we implement here will tell us whether a product is a low stock product i.e. whether the number of units in stock for a particular product falls below a certain threshold. That threshold is defined in another system, so we will feed that data to the specification.

Let’s start with the basics. Here’s the base class for all Specifications. Instead of using IsSatisfiedBy, we expose a method which returns an expression tree of type Expression<Func<T, bool>>.

public abstract class Specification<T>

{

    public abstract Expression<Func<T, bool>> IsSatisfied();

}

The Expression class is in the System.Linq.Expressions namespace which is a part of System.Core.dll. Remember, this is just a different representation of IsSatisfiedBy; instead of keeping the logic embedded in a method in the specification object, we package the logic in an expression tree. The predicate still receives an object and returns a boolean. Other classes, like the ProductRepository, can now leverage this expression tree to optimize the query it sends to the database.

public partial class ProductRepository : IProductRepository

{

    public IQueryable<Product> SelectSatisfying(Specification<Product> specification)

    {

        return this.context.Products.Where(specification.IsSatisfied());

    }

}

Here we use the Entity Framework to select the Products that match a certain Product Specification. (The field "context" is the ObjectContext, in this case.) However, we could switch this for any data access technology that can leverage expression trees and retrieve similar results.

The next step is to implement the actual specification.

public class LowStockSpecification : Specification<Product>

{

    public LowStockSpecification(int lowStockThreshold)

    {

        this.LowStockThreshold = lowStockThreshold;

    }

 

    public int LowStockThreshold

    {

        get;

        private set;

    }

 

    public override Expression<Func<Product, bool>> IsSatisfied()

    {

        return p => p.UnitsInStock < this.LowStockThreshold;

    }

}

Evans says that specifications should be value objects, so I’ve taken that to heart and made this class immutable. This allows us to make some optimizations with specifications (caching the expression tree, introducing an IsSatisfiedBy by reusing the logic in the expression tree, etc.) if we would like.

This final code snippet shows how to leverage the specification and repository together.

public class ProductReorderingService

{

    private IProductRepository productRepository;

 

    public ProductReorderingService(IProductRepository productRepository)

    {

        this.productRepository = productRepository;

    }

 

    public void ReorderLowStockProducts()

    {

        LowStockSpecification spec = new LowStockSpecification(5);

        foreach (var p in this.productRepository.SelectSatisfying(spec))

        {

            // Reorder product

        }

    }

}

Composing Specifications

One property of specifications is that they can be combined to form more interesting predicates. This would allow our ProductRepository to support queries that involve multiple specification instances—for example, a filter that checks for units with low stock OR units whose stock is below their re-order level. The most common implementation I’ve seen of this requirement involves three new classes, AndSpecification<T>, OrSpecification<T>, and NotSpecification<T>. While it’s easy enough to implement these when all you worry about is IsSatisfiedBy (e.g. spec1.IsSatisfiedBy(o) && spec2.IsSatisfiedBy(o) for the AndSpecification<T>), it’s actually a bit tricky to do this with expressions.

Fortunately, it’s not impossible, and Colin Meek has it documented on his blog post about combining predicates in the Entity Framework, but the concepts apply more generally to any provider that can use expression trees. Be careful though; if you’re using the Entity Framework you will have to copy more code than you would with LINQ to SQL. I am not sure about LLBLGen Pro.

If you use the extension methods that Colin provides for AND’ing and OR’ing expression trees together, you’ll end up with these implementations of AndSpecification<T> and OrSpecification<T>:

public class AndSpecification<T> : Specification<T>

{

    private Specification<T> spec1;

    private Specification<T> spec2;

 

    public AndSpecification(Specification<T> spec1, Specification<T> spec2)

    {

        this.spec1 = spec1;

        this.spec2 = spec2;

    }

 

    public override Expression<Func<T, bool>> IsSatisfied()

    {

        return this.spec1.IsSatisfied().And(this.spec2.IsSatisfied());

    }

}

public class OrSpecification<T> : Specification<T>

{

    private Specification<T> spec1;

    private Specification<T> spec2;

 

    public OrSpecification(Specification<T> spec1, Specification<T> spec2)

    {

        this.spec1 = spec1;

        this.spec2 = spec2;

    }

 

    public override Expression<Func<T, bool>> IsSatisfied()

    {

        return this.spec1.IsSatisfied().Or(this.spec2.IsSatisfied());

    }

}

We’ll have to write the NotSpecification<T> ourselves, but this is not as involved as And and Or, even with the Entity Framework. We essentially take the body of the expression tree from the original specification and negate the result. Using the patterns you can read about in Colin’s blog post, we can use the following class as our NotSpecification<T>.

public class NotSpecification<T> : Specification<T>

{

    private Specification<T> originalSpec;

 

    public NotSpecification(Specification<T> originalSpec)

    {

        this.originalSpec = originalSpec;

    }

 

    public override Expression<Func<T, bool>> IsSatisfied()

    {

        Expression<Func<T, bool>> originalTree = this.originalSpec.IsSatisfied();

        return Expression.Lambda<Func<T, bool>>(

            Expression.Not(originalTree.Body),

            originalTree.Parameters.Single()

        );

    }

}

This is all well and good, but doesn’t this tie my domain to my infrastructure?

I think you can find arguments for both viewpoints. The specification pattern allows you to encapsulate a predicate to determine whether an object matches a condition. My opinion is whether that predicate is exposed as a method or an expression tree, the intent is preserved and there is one place where the criteria for a specification are checked. It does require you to use infrastructure that can utilize expression trees, but I would say that there is nothing about expression trees that tie them to the infrastructure layer directly. The details of the underlying data store have not leaked into the domain layer. If I had a provider that could use expression trees for XML or an object database store, then my domain layer would not change.

I enjoy learning about DDD and what other folks have done in this area. I’d love to hear your feedback.

14 Comments

  1. Hello David,

    I investigated the specification pattern in combination with linq and I got stuck on some techincal issues. This is the first post I read that explains it in good detail. Thanks very much to take the time to put in on digital paper!

    If you are serious about DDD than the specification pattern is essential and I hope it will get first class support in the Entity Framework best practices. Encapsulating business logic in 1 place and be able to use it while performing TDD is something very valuable.

    First chance I get I will test out your code

    PS Are you from Belgium/Holland? your name sounds very dutch :)

    • David DeWinter

      Glad the post is useful. Let me know if you are able to improve upon it when you start writing code!

      (And although my name sounds Dutch I myself am not!)

    • David DeWinter

      I haven’t developed in Java for some time now, but a quick search reveals the Quaere project that is aiming to develop something similar to LINQ in Java. However, I am not sure there is anything that is quite as rich as the .NET implementation (which allows translations of these expression trees to any data source, like a relational database) that would allow you to compose these expressions into a SQL query.

  2. marco souza

    Hello, I am following your example, but ta experiencing an error when I try
    make the following code

    this.spec1.IsSatisfied return (). Or (this.spec2.IsSatisfied ());

  3. Unai

    This And, Or operators using Invoke method in expressions and this method is not supported in EF 1 or EF 4

    Best regards,
    Unai

    • David DeWinter

      Unai, this is why Colin’s method rebinds the parameters in the expression manually instead of using Expression.Invoke for EF.

  4. peter

    I’m assuming from the code above that Product is a domain object. Is this also the “Product” in entity framework tables..??
    Are you not tying yourself to the storage implementation?

    • David DeWinter

      No, since Product is a domain object I can write it myself without worrying about what O/RM or storage engine I use.

  5. peter

    Hi! again!
    so “Product” is NOT on your entity model?
    So your Specification returns an expression not a bool (as per the original intention of the specification pattern)??

    Your specifications are tied to Implementations and not interfaces correct?

    How would you do it would interfaces?
    I’m more interested in the repository layer

    Thanks

  6. This post was very useful. I’m a Java Developer getting started with .NET and I was curious about how to implement the specification pattern on .NET. The LINQ framework and C# language are more flexible than Java Persistence API to implement this pattern.

    Thanks.

Leave a Reply

Your email address will not be published. Required fields are marked *