Automated Testing

Introduction

ABP Framework has been designed with testability in mind. There are some different levels of automated testing;

• Unit Tests: You typically test a single class (or a very few classes together). These tests will be fast. However, you generally need to deal with mocking for the dependencies of your service(s).
• Integration Tests: You typically test a service, but this time you don't mock the fundamental infrastructure and services to see if they properly working together.
• UI Tests: You test the UI of the application, just like the users interact with your application.

Unit Tests vs Integration Tests

Integration tests have some significant advantages compared to unit tests;

• Easier to write since you don't work to establish mocking and dealing with the dependencies.
• Your test code runs with all the real services and infrastructure (including database mapping and queries), so it is much closer to the real application test.

While they have some drawbacks;

• They are slower compared to unit tests since all the infrastructure is prepared for each test case.
• A bug in a service may make multiple test cases broken, so it may be harder to find the real problem in some cases.

We suggest to go mixed: Write unit or integration test where it is necessary and you find effective to write and maintain it.

The Application Startup Template

The Application Startup Template comes with the test infrastructure properly installed and configured for you.

The Test Projects

See the following solution structure in the Visual Studio:

There are more than one test project, organized by the layers;

• Domain.Tests is used to test your Domain Layer objects (like Domain Services and Entities).
• Application.Tests is used to test your Application Layer (like Application Services).
• EntityFrameworkCore.Tests is used to test your custom repository implementations or EF Core mappings (this project will be different if you use another Database Provider).
• Web.Tests is used to test the UI Layer (like Pages, Controllers and View Components). This project does exists only for MVC / Razor Page applications.
• TestBase contains some classes those are shared/used by the other projects.

HttpApi.Client.ConsoleTestApp is not an automated test application. It is an example Console Application that shows how to consume your HTTP APIs from a .NET Console Application.

The following sections will introduce the base classes and other infrastructure included in these projects.

The Test Infrastructure

The startup solution has the following libraries already installed;

• xUnit as the test framework.
• NSubstitute as the mocking library.
• Shouldly as the assertion library.

While you are free to replace them with your favorite tools, this document and examples will be base on these tooling.

The Test Explorer

You can use the Test Explorer to view and run the tests in Visual Studio. For other IDEs, see their own documentation.

Open the Test Explorer

Open the Test Explorer, under the Tests menu, if it is not already open:

Run the Tests

Then you can click to the Run All or Run buttons to run the tests. The initial startup template has some sample tests for you:

Run Tests In Parallel

The test infrastructure is compatible to run the tests in parallel. It is strongly suggested to run all the tests in parallel, which is pretty faster then running them one by one.

To enable it, click to the caret icon near to the settings (gear) button and select the Run Tests In Parallel.

Unit Tests

For Unit Tests, you don't need to much infrastructure. You typically instantiate your class and provide some pre-configured mocked objects to prepare your object to test.

Classes Without Dependencies

In this simplest case, the class you want to test has no dependencies. In this case, you can directly instantiate your class, call its methods and make your assertions.

Example: Testing an Entity

Assume that you've an Issue entity as shown below:

using System;
using Volo.Abp.Domain.Entities;

namespace MyProject.Issues
{
    public class Issue : AggregateRoot<Guid>
    {
        public string Title { get; set; }
        public string Description { get; set; }
        public bool IsLocked { get; set; }
        public bool IsClosed { get; private set; }
        public DateTime? CloseDate { get; private set; }

        public void Close()
        {
            IsClosed = true;
            CloseDate = DateTime.UtcNow;
        }

        public void Open()
        {
            if (!IsClosed)
            {
                return;
            }

            if (IsLocked)
            {
                throw new IssueStateException("You can not open a locked issue!");
            }

            IsClosed = true;
            CloseDate = null;
        }
    }
}

Notice that the IsClosed and CloseDate properties have private setters to force some business rules by using the Open() and Close() methods;

• Whenever you close an issue, the CloseDate should be set to the current time.
• An issue can not be re-opened if it is locked. And if it is re-opened, the CloseDate should be set to null.

Since the Issue entity is a part of the Domain Layer, we should test it in the Domain.Tests project. Create an Issue_Tests class inside the Domain.Tests project:

using Shouldly;
using Xunit;

namespace MyProject.Issues
{
    public class Issue_Tests
    {
        [Fact]
        public void Should_Set_The_CloseDate_Whenever_Close_An_Issue()
        {
            // Arrange

            var issue = new Issue();
            issue.CloseDate.ShouldBeNull(); // null at the beginning

            // Act

            issue.Close();

            // Assert

            issue.IsClosed.ShouldBeTrue();
            issue.CloseDate.ShouldNotBeNull();
        }
    }
}

This test follows the AAA (Arrange-Act-Assert) pattern;

• Arrange part creates an Issue entity and ensures the CloseDate is null at the beginning.
• Act part executes the method we want to test for this case.
• Assert part checks if the Issue properties are same as we expect to be.

[Fact] attribute is defined by the xUnit library and marks a method as a test method. Should... extension methods are provided by the Shouldly library. You can directly use the Assert class of the xUnit, but Shouldly makes it much comfortable and straightforward.

When you execute the tests, you will see that is passes successfully:

Let's add two more test methods:

[Fact]
public void Should_Allow_To_ReOpen_An_Issue()
{
    // Arrange

    var issue = new Issue();
    issue.Close();

    // Act

    issue.Open();

    // Assert

    issue.IsClosed.ShouldBeFalse();
    issue.CloseDate.ShouldBeNull();
}

[Fact]
public void Should_Not_Allow_To_ReOpen_A_Locked_Issue()
{
    // Arrange

    var issue = new Issue();
    issue.Close();
    issue.IsLocked = true;

    // Act & Assert

    Assert.Throws<IssueStateException>(() =>
    {
        issue.Open();
    });
}

Assert.Throws checks if the executed code throws a matching exception.

See the xUnit & Shoudly documentations to learn more about these libraries.

Classes With Dependencies

If your service has dependencies and you want to unit test this service, you need to mock the dependencies.

Example: Testing a Domain Service

Assume that you've an IssueManager Domain Service that is defined as below:

using System;
using System.Threading.Tasks;
using Volo.Abp;
using Volo.Abp.Domain.Services;

namespace MyProject.Issues
{
    public class IssueManager : DomainService
    {
        public const int MaxAllowedOpenIssueCountForAUser = 3;

        private readonly IIssueRepository _issueRepository;

        public IssueManager(IIssueRepository issueRepository)
        {
            _issueRepository = issueRepository;
        }

        public async Task AssignToUserAsync(Issue issue, Guid userId)
        {
            var issueCount = await _issueRepository.GetIssueCountOfUserAsync(userId);

            if (issueCount >= MaxAllowedOpenIssueCountForAUser)
            {
                throw new BusinessException(
                    code: "IM:00392",
                    message: $"You can not assign more" +
                             $"than {MaxAllowedOpenIssueCountForAUser} issues to a user!"
                );
            }

            issue.AssignedUserId = userId;
        }
    }
}

IssueManager depends on the IssueRepository service, that will be mocked in this example.

Business Rule: The example AssignToUserAsync doesn't allow to assign more than 3 (MaxAllowedOpenIssueCountForAUser constant) issues to a user. If you want to assign an issue in this case, you first need to unassign an existing issue.

The test case below tries to make a valid assignment:

using System;
using System.Threading.Tasks;
using NSubstitute;
using Shouldly;
using Volo.Abp;
using Xunit;

namespace MyProject.Issues
{
    public class IssueManager_Tests
    {
        [Fact]
        public async Task Should_Assign_An_Issue_To_A_User()
        {
            // Arrange

            var userId = Guid.NewGuid();

            var fakeRepo = Substitute.For<IIssueRepository>();
            fakeRepo.GetIssueCountOfUserAsync(userId).Returns(1);

            var issueManager = new IssueManager(fakeRepo);

            var issue = new Issue();

            // Act

            await issueManager.AssignToUserAsync(issue, userId);

            //Assert

            issue.AssignedUserId.ShouldBe(userId);
            await fakeRepo.Received(1).GetIssueCountOfUserAsync(userId);
        }
    }
}

• Substitute.For<IIssueRepository> creates a mock (fake) object that is passed into the IssueManager constructor.
• fakeRepo.GetIssueCountOfUserAsync(userId).Returns(1) ensures that the GetIssueCountOfUserAsync method of the repository returns 1.
• issueManager.AssignToUserAsync doesn't throw any exception since the repository returns 1 for the currently assigned issue count.
• issue.AssignedUserId.ShouldBe(userId); line checks if the AssignedUserId has the correct value.
• await fakeRepo.Received(1).GetIssueCountOfUserAsync(userId); checks if the IssueManager called the GetIssueCountOfUserAsync method exactly one time.

Let's add a second test to see if it prevents to assign issues to a user more than the allowed count:

[Fact]
public async Task Should_Not_Allow_To_Assign_Issues_Over_The_Limit()
{
    // Arrange

    var userId = Guid.NewGuid();

    var fakeRepo = Substitute.For<IIssueRepository>();
    fakeRepo
        .GetIssueCountOfUserAsync(userId)
        .Returns(IssueManager.MaxAllowedOpenIssueCountForAUser);

    var issueManager = new IssueManager(fakeRepo);

    // Act & Assert

    var issue = new Issue();

    await Assert.ThrowsAsync<BusinessException>(async () =>
    {
        await issueManager.AssignToUserAsync(issue, userId);
    });

    issue.AssignedUserId.ShouldBeNull();
    await fakeRepo.Received(1).GetIssueCountOfUserAsync(userId);
}

For more information on the mocking, see the NSubstitute documentation.

It is relatively easy to mock a single dependency. But, when your dependencies grow, it gets harder to setup the test objects and mock all the dependencies. See the Integration Tests section that doesn't require mocking the dependencies.

Tip: Share the Test Class Constructor

xUnit creates a new test class instance (IssueManager_Tests for this example) for each test method. So, you can move some Arrange code into the constructor to reduce the code duplication. The constructor will be executed for each test case and doesn't affect each other, even if they work in parallel.

Example: Refactor the IssueManager_Tests to reduce the code duplication

using System;
using System.Threading.Tasks;
using NSubstitute;
using Shouldly;
using Volo.Abp;
using Xunit;

namespace MyProject.Issues
{
    public class IssueManager_Tests
    {
        private readonly Guid _userId;
        private readonly IIssueRepository _fakeRepo;
        private readonly IssueManager _issueManager;
        private readonly Issue _issue;

        public IssueManager_Tests()
        {
            _userId = Guid.NewGuid();
            _fakeRepo = Substitute.For<IIssueRepository>();
            _issueManager = new IssueManager(_fakeRepo);
            _issue = new Issue();
        }

        [Fact]
        public async Task Should_Assign_An_Issue_To_A_User()
        {
            // Arrange            
            _fakeRepo.GetIssueCountOfUserAsync(_userId).Returns(1);

            // Act
            await _issueManager.AssignToUserAsync(_issue, _userId);

            //Assert
            _issue.AssignedUserId.ShouldBe(_userId);
            await _fakeRepo.Received(1).GetIssueCountOfUserAsync(_userId);
        }

        [Fact]
        public async Task Should_Not_Allow_To_Assign_Issues_Over_The_Limit()
        {
            // Arrange
            _fakeRepo
                .GetIssueCountOfUserAsync(_userId)
                .Returns(IssueManager.MaxAllowedOpenIssueCountForAUser);

            // Act & Assert
            await Assert.ThrowsAsync<BusinessException>(async () =>
            {
                await _issueManager.AssignToUserAsync(_issue, _userId);
            });

            _issue.AssignedUserId.ShouldBeNull();
            await _fakeRepo.Received(1).GetIssueCountOfUserAsync(_userId);
        }
    }
}

Keep your test code clean to create a maintainable test suite.

Integration Tests

You can also follow the web application development tutorial to learn developing a full stack application, including the integration tests.

The Integration Test Infrastructure

ABP Provides a complete infrastructure to write integration tests. All the ABP infrastructure and services will perform in your tests. The application startup template comes with the necessary infrastructure pre-configured for you;

The Database

The startup template is configured to use in-memory SQLite database for the EF Core (for MongoDB, it uses Mongo2Go library). So, all the configuration and queries are performed against a real database and you can even test database transactions.

Using in-memory SQLite database has two main advantages;

• It is faster compared to an external DBMS.
• It create a new fresh database for each test case, so tests doesn't affect each other.

Tip: Do not use EF Core's In-Memory database for advanced integration tests. It is not a real DBMS and has many differences in details. For example, it doesn't support transaction and rollback scenarios, so you can't truly test the failing scenarios. On the other hand, In-Memory SQLite is a real DBMS and supports the fundamental SQL database features.

The Seed Data

Writing tests against an empty database is not practical. In most cases, you need to some initial data in the database. For example, if you write a test class that query, update and delete the products, it would be helpful to have a few products in the database before executing the test case.

ABP's Data Seeding system is a powerful way to seed the initial data. The application startup template has a YourProjectTestDataSeedContributor class in the .TestBase project. You can fill it to have an initial data that you can use for each test method.

Example: Create some Issues as the seed data

using System.Threading.Tasks;
using MyProject.Issues;
using Volo.Abp.Data;
using Volo.Abp.DependencyInjection;

namespace MyProject
{
    public class MyProjectTestDataSeedContributor
        : IDataSeedContributor, ITransientDependency
    {
        private readonly IIssueRepository _issueRepository;

        public MyProjectTestDataSeedContributor(IIssueRepository issueRepository)
        {
            _issueRepository = issueRepository;
        }

        public async Task SeedAsync(DataSeedContext context)
        {
            await _issueRepository.InsertAsync(
                new Issue
                {
                    Title = "Test issue one",
                    Description = "Test issue one description",
                    AssignedUserId = TestData.User1Id
                });

            await _issueRepository.InsertAsync(
                new Issue
                {
                    Title = "Test issue two",
                    Description = "Test issue two description",
                    AssignedUserId = TestData.User1Id
                });

            await _issueRepository.InsertAsync(
                new Issue
                {
                    Title = "Test issue three",
                    Description = "Test issue three description",
                    AssignedUserId = TestData.User1Id
                });

            await _issueRepository.InsertAsync(
                new Issue
                {
                    Title = "Test issue four",
                    Description = "Test issue four description",
                    AssignedUserId = TestData.User2Id
                });
        }
    }
}

Also created a static class to store the User Ids:

using System;

namespace MyProject
{
    public static class TestData
    {
        public static Guid User1Id = Guid.Parse("41951813-5CF9-4204-8B18-CD765DBCBC9B");
        public static Guid User2Id = Guid.Parse("2DAB4460-C21B-4925-BF41-A52750A9B999");
    }
}

In this way, we can use these known Issues and the User Ids to perform the tests.

Example: Testing a Domain Service

AbpIntegratedTest<T> class (defined in the Volo.Abp.TestBase package) is used to write tests integrated to the ABP Framework. T is the Type of the root module to setup and initialize the application.

The application startup template has base classes in each test project, so you can derive from these base classes to make it easier.

See the IssueManager tests are re-written as integration tests

using System.Threading.Tasks;
using Shouldly;
using Volo.Abp;
using Xunit;

namespace MyProject.Issues
{
    public class IssueManager_Integration_Tests : MyProjectDomainTestBase
    {
        private readonly IssueManager _issueManager;
        private readonly Issue _issue;

        public IssueManager_Integration_Tests()
        {
            _issueManager = GetRequiredService<IssueManager>();
            _issue = new Issue
            {
                Title = "Test title",
                Description = "Test description"
            };
        }

        [Fact]
        public async Task Should_Not_Allow_To_Assign_Issues_Over_The_Limit()
        {
            // Act & Assert
            await Assert.ThrowsAsync<BusinessException>(async () =>
            {
                await _issueManager.AssignToUserAsync(_issue, TestData.User1Id);
            });

            _issue.AssignedUserId.ShouldBeNull();
        }
        
        [Fact]
        public async Task Should_Assign_An_Issue_To_A_User()
        {
            // Act
            await _issueManager.AssignToUserAsync(_issue, TestData.User2Id);

            //Assert
            _issue.AssignedUserId.ShouldBe(TestData.User2Id);
        }
    }
}

• First test method assigns the issue to the User 1, which has already assigned to 3 issues in the Data Seed code. So, it throws a BusinessException.
• Second test method assigns the issue to User 2, which has only 1 issue assigned. So, the method succeeds.

This class typically locates in the .Domain.Tests project since it tests a class located in the .Domain project. It is derived from the MyProjectDomainTestBase which is already configured to properly run the tests.

Writing such an integration test class is very straightforward. Another benefit is that you won't need to change the test class later when you add another dependency to the IssueManager class.

Example: Testing an Application Service

Testing an Application Service is not so different. Assume that you've created an IssueAppService as defined below:

using System.Collections.Generic;
using System.Threading.Tasks;
using Volo.Abp.Application.Services;

namespace MyProject.Issues
{
    public class IssueAppService : ApplicationService, IIssueAppService
    {
        private readonly IIssueRepository _issueRepository;

        public IssueAppService(IIssueRepository issueRepository)
        {
            _issueRepository = issueRepository;
        }

        public async Task<List<IssueDto>> GetListAsync()
        {
            var issues = await _issueRepository.GetListAsync();

            return ObjectMapper.Map<List<Issue>, List<IssueDto>>(issues);
        }
    }
}

(assuming you've also defined the IIssueAppService and IssueDto and created the object mapping between Issue and the IssueDto)

Now, you can write a test class inside the .Application.Tests project:

using System.Threading.Tasks;
using Shouldly;
using Xunit;

namespace MyProject.Issues
{
    public class IssueAppService_Tests : MyProjectApplicationTestBase
    {
        private readonly IIssueAppService _issueAppService;

        public IssueAppService_Tests()
        {
            _issueAppService = GetRequiredService<IIssueAppService>();
        }

        [Fact]
        public async Task Should_Get_All_Issues()
        {
            //Act
            var issueDtos = await _issueAppService.GetListAsync();

            //Assert
            issueDtos.Count.ShouldBeGreaterThan(0);
        }
    }
}

It's that simple. This test method tests everything, including the application service, EF Core mapping, object to object mapping and the repository implementation. In this way, you can fully test the Application Layer and the Domain Layer of your solution.

Dealing with Unit of Work in Integration Tests

ABP's unit of work system controls the database connection and transaction management in your application. It seamlessly works while you writing your application code, so you may not aware of it.

In the ABP Framework, all the database operations must be performed inside a unit of work scope. When you test an application service method, the unit of work scope will be the scope of your application service method. If you are testing a repository method, the unit of work scope will be the scope of your repository method.

In some cases, you may need to manually control the unit of work scope. Consider the following test method:

public class IssueRepository_Tests : MyProjectDomainTestBase
{
    private readonly IRepository<Issue, Guid> _issueRepository;

    public IssueRepository_Tests()
    {
        _issueRepository = GetRequiredService<IRepository<Issue, Guid>>();
    }

    public async Task Should_Query_By_Title()
    {
        IQueryable<Issue> queryable = await _issueRepository.GetQueryableAsync();
        var issue = queryable.FirstOrDefaultAsync(i => i.Title == "My issue title");
        issue.ShouldNotBeNull();
    }
}

We are using _issueRepository.GetQueryableAsync to obtain an IQueryable<Issue> object. Then, we are using the FirstOrDefaultAsync method to query an issue by its title. The database query is executed at this point, and you get an exception indicating that there is no active unit of work.

To make that test properly working, you should manually start a unit of work scope as shown in the following example:

public class IssueRepository_Tests : MyProjectDomainTestBase
{
    private readonly IRepository<Issue, Guid> _issueRepository;
    private readonly IUnitOfWorkManager _unitOfWorkManager;

    public IssueRepository_Tests()
    {
        _issueRepository = GetRequiredService<IRepository<Issue, Guid>>();
        _unitOfWorkManager = GetRequiredService<IUnitOfWorkManager>();
    }

    public async Task Should_Query_By_Title()
    {
        using (var uow = _unitOfWorkManager.Begin())
        {
            IQueryable<Issue> queryable = await _issueRepository.GetQueryableAsync();
            var issue = queryable.FirstOrDefaultAsync(i => i.Title == "My issue title");
            issue.ShouldNotBeNull();
            await uow.CompleteAsync();
        }
    }
}

We've used the IUnitOfWorkManager service to create a unit of work scope, then called the FirstOrDefaultAsync method inside that scope, so we don't have the problem anymore.

Note that we've tested the FirstOrDefaultAsync to demonstrate the unit of work problem. Normally, as a good principle, you should write tests only your own code.

Working with DbContext

In some cases, you may want to directory work with the Entity Framework's DbContext object to perform database operations in your test methods. In this case, you can use IDbContextProvider<T>service to obtain a DbContext instance inside a unit of work.

The following example shows how you can create a DbContext object in a test method:

public class MyDbContext_Tests : MyProjectDomainTestBase
{
    private readonly IDbContextProvider<MyProjectDbContext> _dbContextProvider;
    private readonly IUnitOfWorkManager _unitOfWorkManager;

    public IssueRepository_Tests()
    {
        _dbContextProvider = GetRequiredService<IDbContextProvider<MyProjectDbContext>>();
        _unitOfWorkManager = GetRequiredService<IUnitOfWorkManager>();
    }

    public async Task Should_Query_By_Title()
    {
        using (var uow = _unitOfWorkManager.Begin())
        {
            var dbContext = await _dbContextProvider.GetDbContextAsync();
            var issue = await dbContext.Issues.FirstOrDefaultAsync(i => i.Title == "My issue title");
            issue.ShouldNotBeNull();
            await uow.CompleteAsync();
        }
    }
}

Just like we've done in the Dealing with Unit of Work in Integration Tests section, we should perform DbContext operations inside an active unit of work.

For MongoDB, you can use the IMongoDbContextProvider<T> service to obtain a DbContext object and directly use MongoDB APIs in your test methods.

UI Tests

In general, there are two types of UI Tests;

Non Visual Tests

Such tests completely depends on your UI Framework choice;

• For an MVC / Razor Pages UI, you typically make request to the server, get some HTML and test if some expected DOM elements exist in the returned result.
• Angular has its own infrastructure and practices to test the components, views and services.

See the following documents to learn Non Visual UI Testing;

• Testing in ASP.NET Core MVC / Razor Pages
• Testing in Angular
• Testing in Blazor

Visual Tests

Visual Tests are used to interact with the application UI just like a real user does. It fully tests the application, including the visual appearance of the pages and components.

Visual UI Testing is out of the scope for the ABP Framework. There are a lot of tooling in the industry (like Selenium) that you can use to test your application's UI.