Grouping Assertions in Tests

Date Published: 28 September 2021

Grouping Assertions in Tests

Although it's generally considered a best practice to assert only one thing per test, sometimes one logical "thing" may require multiple assertion statements to be executed. In such cases, it's helpful to be able to see all of the parts of the larger assertion that failed, so you don't end up chasing after them one by one.

As an example, let's say you have a class that is initialized with its constructor. You want to write tests that verify that the class's properties are set correctly based on what is passed into the constructor. Here's a class we can use to demonstrate this:

using Ardalis.GuardClauses;

namespace MultipleAssertions; // new C# 10 syntax

public class Account
{
    public Account(string name, string number, decimal balance)
    {
        Name = Guard.Against.NullOrEmpty(name, nameof(name));
        Number = Guard.Against.NullOrEmpty(number, nameof(number));
        Balance = Guard.Against.NegativeOrZero(balance, nameof(balance));
    }

    public string Name { get; }
    public string Number { get; }
    public decimal Balance { get; }
}

In this class you can see that in addition to setting the properties based on the inputs to the constructor, some guard clauses are also applied to ensure the inputs are valid (using Ardalis.GuardClauses). But let's back up a moment and consider this class with the same constructor being empty:

    public Account(string name, string number, decimal balance)
    {
    }

Clearly in this case, any tests that were expecting property values to be properly initialized from the constructor will fail. And in fact, all three of the properties will fail, not just one of them. It would be good to know that.

Testing Multiple Conditions with Many Tests

Now, we could follow the "one assertion per test" best practice to the letter, in which case we might have separate tests for AccountConstructorInitializesNameProperty and AccountContructorInitializesNumberProperty, etc. but this gets very verbose and tedious. I'm not even going to show it here, but it would achieve the goals of this article, since every unset property would be idenfitied by its own failing test. If you have the patience to write your tests this way, you don't necessarily need test grouping.

Testing Multiple Conditions with Many Assert Statements

If it were me, I would probably just write one test like the following one (using xUnit), which checks all three of the properties in one test method with a name that suggests it's logically testing the controller and its initialization behavior as one concept.

public class Account_Constructor
{
    private readonly string _testName = "Acme";
    private readonly string _testNumber = "12345";
    private readonly decimal _testBalance = 123.50m;

    [Fact]
    public void SetsAssociatedProperties_NoGrouping()
    {
        var account = new Account(_testName, _testNumber, _testBalance);

        // Individual assertions will only catch the first problem
        Assert.Equal(_testName, account.Name);
        Assert.Equal(_testNumber, account.Number);
        Assert.Equal(_testBalance, account.Balance);
    }
}

If we run this test against the empty constructor version of Account, we'll get this result:

test failure message

Note that only the first assertion is hit, since it immediately exits the test. Once we fix that problem, the next run of the test will reveal another problem. We'll fix that one, and once more run the test to reveal the third problem. This kind of "whack-a-mole" response to test failures is common for tests that use multiple assertions.

Manually Group Multiple Test Assertions

If you know how assertions work, you can work around the default behavior shown above. In most test frameworks, assertions throw exceptions. Tests that complete without throwing an exception pass, and those that throw fail. The test results shown are simply generated by the exception messages thrown. Knowing this, we can group multiple exceptions together using try-catch blocks for each assertion and adding any caught exceptions to a collection, which is then thrown at the end of the test.

[Fact]
public void SetsAssociatedProperties_ManualGrouping()
{
    var account = new Account(_testName, _testNumber, _testBalance);

    var exceptions = new List<Exception>();
    try
    {
        Assert.Equal(_testName, account.Name);
    }
    catch (Exception ex)
    {
        exceptions.Add(ex);
    }
    try
    {
        Assert.Equal(_testNumber, account.Number);
    }
    catch (Exception ex)
    {
        exceptions.Add(ex);
    }
    try
    {
        Assert.Equal(_testBalance, account.Balance);
    }
    catch (Exception ex)
    {
        exceptions.Add(ex);
    }

    if(exceptions.Any())
    {
        throw new AggregateException(exceptions.ToArray()); 
    }
}

In this version, the same three assertions exist, but now each one is wrapped in a try-catch block. Any exception caught in a catch block is added to a collection, but otherwise ignored. At the end of the test, if any exceptions were detected, an AggregateException is thrown, failing the test and listing each of the underlying failures. With the constructor's contents removed (no property initialization), this test yields the following result:

test failure message - manual grouping

This output isn't ideal, but it does at least list all of the assertion failures. The code we had to write for the test to get here, though... Let's just say there are better ways.

Group Assertions with FluentAssertions AssertionScope

If you install the popular FluentAssertions package, in addition to getting more readable assertion statements, you also get access to its AssertionScope class. You can use this with the package's assertions to perform the kind of grouping we're looking for, as this example shows:

[Fact]
public void SetsAssociatedProperties_FluentAssertionsAssertionScopes_FluentAsserts()
{
    var account = new Account(_testName, _testNumber, _testBalance);

    using (new AssertionScope())
    {
        account.Name.Should().Be(_testName);
        account.Number.Should().Be(_testNumber);
        account.Balance.Should().Be(_testBalance);
    }
}

When this test fails, the test results show:

test failure message - assertion scope

Here you can see all three of the assertion failures with minimal code and minimal noise.

It's worth noting, though, that the AssertionScope doesn't work with standard xUnit assertions. This test will still fail on the first Assert.Equal statement:

[Fact]
public void SetsAssociatedProperties_FluentAssertionsAssertionScopes_XUnitAsserts()
{
    var account = new Account(_testName, _testNumber, _testBalance);

    using (new AssertionScope())
    {
        // doesn't work with xUnit asserts
        Assert.Equal(_testName, account.Name);
        Assert.Equal(_testNumber, account.Number);
        Assert.Equal(_testBalance, account.Balance);
    }
}

Group Assertions with NUnit AssertMultiple

If you're using NUnit, its AssertMultiple feature provides the same behavior as AssertionScopes. The syntax is similar (from the linked docs):

[Test]
public void ComplexNumberTest()
{
    ComplexNumber result = SomeCalculation();

    Assert.Multiple(() =>
    {
        Assert.AreEqual(5.2, result.RealPart, "Real part");
        Assert.AreEqual(3.9, result.ImaginaryPart, "Imaginary part");
    });
}

Group Assertions using Assert.All

I don't much care for the syntax of this one from a readability standpoint, but it works. This is modified from an answer given by Nick Giampietro on Stack Overflow:

[Fact]
public void SetsAssociatedProperties_AssertAll()
{
    var account = new Account(_testName, _testNumber, _testBalance);

    var expectations = new List<Tuple<object, object>>()
    {
        new(_testName, account.Name),
        new(_testName, account.Number),
        new(_testBalance, account.Balance),
    };
    Assert.All(expectations, pair => Assert.Equal(pair.Item1, pair.Item2));
}

The test output for this version is:

test failure message - assert.all

As you can see the output is a bit more verbose than the much cleaner AssertionScope version, but it does show all of the failures.

SatisfyAllConditions and Shouldly

Another popular assertion package, Shouldy, supports grouping assertions using its SatisfyAllConditions method (h/t Matt Frear). The syntax looks like this:

[Fact]
public void SetsAssociatedProperties_AssertAll()
{
    var account = new Account(_testName, _testNumber, _testBalance);

    account.ShouldSatisfyAllConditions(
        () => account.Name.ShouldBe(_testName),
        () => account.Number.ShouldBe(_testNumber),
        () => account.Balance.ShouldBe(_testBalance));
}

test failure message - ShouldSatisfyAllConditions

Similar to FluentAssertions, Shouldly's output shows all of the failures in a nicely formatted manner.

Summary and Resources

There are times when it makes sense to logically group multiple assertions together in order to test a larger-grained activity. When you do so, if you simply use individual asserts you're likely to have to keep re-running the test when a failure occurs as you fix each assertion one by one. Using multiple assertions with AssertionScope or Assert.Multiple provides a better way to write these kinds of tests.

Steve Smith

About Ardalis

Software Architect

Steve is an experienced software architect and trainer, focusing on code quality and Domain-Driven Design with .NET.


Ardalis

Copyright © 2021