Design, Test and Develop like it's heaven on Earth - Part 12

Published on 8/21/2019

Now that we’ve gotten the syndication service sorted out we need to move on to the next item in the social media processing pipeline, the queue. In the previous parts we’ve only dealt with the IBlogSyndicationQueue. Now we need to implement it.

Sticking to a simple collection

In part 7 we briefly discussed various queues. For this one I’m going to keep it simple and employ a rather dull FIFO using a built-in collection ConcurrentQueue. It is necessary to use the concurrent version of Queue, since ASP .NET runs in an asynchronous environment. This means that multiple request threads can enqueue at the same time. I’ll start with defining the concrete class implementation from the abstraction.

public class BlogSyndicationQueue : IBlogSyndicationQueue
{
    public Task<IBlogSyndication> DequeueAsync()
    {
        throw new NotImplementedException();
    }

    public Task EnqueueAsync(IBlogSyndication blogSyndication)
    {
        throw new NotImplementedException();
    }
}

And we create our matching test class and the two obvious test methods.

[TestClass]
public class BlogSyndicationQueueTests
{
    [TestMethod]
    public async Task EnqueueAsync_Verify()
    {

    }

    [TestMethod]
    public async Task DequeueAsync_Verify()
    {

    }
}

We want to test that we enqueue something successfully, and that we dequeue something successfully. So I’m sitting here wondering how I can assert that something was enqueued? By trying to dequeue it of course!

[TestMethod]
public async Task EnqueueAsync_DequeueAsync_Verify()
{
    //arrange
    var expected = new Mock<IBlogSyndication>();
    var queue = new BlogSyndicationQueue();

    //act
    await queue.EnqueueAsync(expected.Object);
    var actual = await queue.DequeueAsync();

    //assert
    Assert.IsNotNull(actual);
    Assert.AreEqual(expected, actual);
}

We only need one unit test now, because this single test covers and invokes both the methods on the abstraction. In order to pass this test, I complete the concrete queue implementation.

public class BlogSyndicationQueue : IBlogSyndicationQueue
{
    readonly ConcurrentQueue<IBlogSyndication> _queue = new ConcurrentQueue<IBlogSyndication>();

    public async Task<IBlogSyndication> DequeueAsync()
    {
        return await Task.Run(() =>
        {
            if (_queue.TryDequeue(out IBlogSyndication blogSyndication))
            {
                return blogSyndication;
            }

            return null;
        });
    }

    public async Task EnqueueAsync(IBlogSyndication blogSyndication)
    {
        await Task.Run(() => _queue.Enqueue(blogSyndication));
    }
}

This passes the test, but because the ConcurrentQueue’s methods aren’t async I use the Task API to wrap those calls. This doesn't mean the collection isn't thread safe. It certainly is, but it manages thread safety with 'lock' in its own implementation.

Hosted Concrete

This concludes the development of the BlogSyndicationQueue and we can move onto the process that will dequeue and take the syndication items to the various social media sites. For this, I’m mostly taking a cue from the IHostedService article. The method we need to test is ExecuteAsync. In order to test it, we need to inject a mock of the queue that we can set up to Dequeue() items from. Our unit test is starting to take form.

[TestClass]
public class SyndicationHostedServiceTests
{
    [TestMethod]
    public async Task ExecuteAsync_Verify()
    {
        //arrange
        Mock<IBlogSyndicationQueue> queueMock = new Mock<IBlogSyndicationQueue>();
        Mock<ILoggerFactory> loggerFactoryMock = new Mock<ILoggerFactory>();
        SyndicationHostedService service = new SyndicationHostedService(queueMock.Object, loggerFactoryMock.Object);
        var cancellationToken = new CancellationToken();

        //act
        await service.StartAsync(cancellationToken);

        //
        queueMock.Verify(x => x.DequeueAsync());
    }
}

Let’s provide that concrete implementation, SyndicationHostedService, to fulfill this unit test. In the reference article, it’s not abstracted - we don’t deal with it on an acute basis in our other code. It is simply set up as a service within the web-hosted service and it runs on its own. So this time we just build the concrete implementation.

public class SyndicationHostedService : BackgroundService
{
    readonly IBlogSyndicationQueue _queue;
    readonly ILogger _logger;

    public SyndicationHostedService(IBlogSyndicationQueue queue, ILoggerFactory loggerFactory)
    {
        _queue = queue;
        _logger = loggerFactory.CreateLogger<SyndicationHostedService>();
    }

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        IBlogSyndication syndicationItem = _queue.DequeueAsync();
    }
}

Now that we pass our test on dequeuing, we need to test that we’re doing something with the dequeued item. Let’s switch back to red and adjust our unit test.

[TestMethod]
public async Task ExecuteAsync_Verify()
{
    //arrange
    var syndication = new Mock<IBlogSyndication>();
    var queueMock = new Mock<IBlogSyndicationQueue>();
    queueMock.Setup(x => x.DequeueAsync())
        .Returns(syndication.Object);
    var loggerMock = new Mock<ILogger>();
    var loggerFactoryMock = new Mock<ILoggerFactory>();
    loggerFactoryMock.Setup(x => x.CreateLogger(It.IsAny<string>()))
        .Returns(loggerMock.Object);
    var service = new SyndicationHostedService(queueMock.Object, loggerFactoryMock.Object);
    var cancellationToken = new CancellationToken();

    //act
    await service.StartAsync(cancellationToken);

    //assert
    queueMock.Verify(x => x.DequeueAsync());
    loggerMock.Verify(x => x.Log(LogLevel.Information, It.IsAny<EventId>(), It.IsAny<object>(), It.IsAny<Exception>(), It.IsAny<Func<object, Exception, string>>()));
    syndication.Verify(x => x.Process());
}

A few things have changed here. I introduced a mock for IBlogSyndication and I set up the queue mock to return it as part of the dequeue call. Then I added a verify check to a method called Process() on IBlogSyndication. This method doesn’t exist, so I used code completion to create it on the abstraction. I included verifying for logging. To pass this test, we need to do some more work on the service host.

protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
    IBlogSyndication syndicationItem = _queue.Dequeue();
    _logger?.LogInformation($"Dequeue IBlogSyndication for title '{syndicationItem?.Blog?.Title}'");
    syndicationItem?.Process();
}

Building a robust service

This is essentially the most basic implementation of this hosted service. Dequeue an item and process it. But we’re missing some additional functionality, like supporting many items to dequeue, and handling any errors that might occur during the processing of individual items.

Looping

We need to change our unit tests to test for multiple calls.

[TestMethod]
public void ExecuteAsync_Verify_WithLoop()
{
    //arrange
    var cancellationTokenSource = new CancellationTokenSource();
    var syndication = new Mock<IBlogSyndication>();
    var queueMock = new Mock<IBlogSyndicationQueue>();
    queueMock.Setup(x => x.DequeueAsync())
        .ReturnsAsync(syndication.Object);
    var loggerMock = new Mock<ILogger>();
    var loggerFactoryMock = new Mock<ILoggerFactory>();
    loggerFactoryMock.Setup(x => x.CreateLogger(It.IsAny<string>()))
        .Returns(loggerMock.Object);
    var service = new SyndicationHostedService(queueMock.Object, loggerFactoryMock.Object);

    //act
    Task.Run(() => service.StartAsync(cancellationTokenSource.Token));
    Thread.Sleep(1000);
    cancellationTokenSource.Cancel();

    //assert
    queueMock.Verify(x => x.DequeueAsync(), Times.AtLeastOnce());
    loggerMock.Verify(x => x.Log(LogLevel.Information, It.IsAny<EventId>(), It.IsAny<object>(), It.IsAny<Exception>(), It.IsAny<Func<object, Exception, string>>()), Times.AtLeastOnce());
    syndication.Verify(x => x.ProcessAsync(), Times.AtLeastOnce());
}

The test remains largely unchanged, apart from how the act section is written to run the task and move on to cancel the call using the CancellationTokenSource. And then the verify calls were all changed to assert that there where multiple invokes.

Exception Handling

Here we write a second unit test since we need to set up an exception for the IBlogSyndication mock, and this would affect our current unit test.

[TestMethod]
public void ExecuteAsync_Verify_WithException()
{
    //arrange
    var cancellationTokenSource = new CancellationTokenSource();
    var syndication = new Mock<IBlogSyndication>();
    syndication.Setup(x => x.ProcessAsync())
        .ThrowsAsync(new Exception());
    var queueMock = new Mock<IBlogSyndicationQueue>();
    queueMock.Setup(x => x.DequeueAsync())
        .ReturnsAsync(syndication.Object);
    var loggerMock = new Mock<ILogger>();
    var loggerFactoryMock = new Mock<ILoggerFactory>();
    loggerFactoryMock.Setup(x => x.CreateLogger(It.IsAny<string>()))
        .Returns(loggerMock.Object);
    var service = new SyndicationHostedService(queueMock.Object, loggerFactoryMock.Object);

    //act
    Task.Run(() => service.StartAsync(cancellationTokenSource.Token));
    Thread.Sleep(1000);
    cancellationTokenSource.Cancel();

    //assert
    queueMock.Verify(x => x.DequeueAsync(), Times.AtLeastOnce());
    loggerMock.Verify(x => x.Log(LogLevel.Information, It.IsAny<EventId>(), It.IsAny<object>(), It.IsAny<Exception>(), It.IsAny<Func<object, Exception, string>>()), Times.AtLeastOnce());
    syndication.Verify(x => x.ProcessAsync(), Times.AtLeastOnce());
    loggerMock.Verify(x => x.Log(LogLevel.Error, It.IsAny<EventId>(), It.IsAny<object>(), It.IsAny<Exception>(), It.IsAny<Func<object, Exception, string>>()), Times.AtLeastOnce());
}

Again, the test is very similar apart from the additional setup to throw, and the additional verify call to the logger for LogLevel.Error. We can pass this test by adding exception handling in our service.

protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
    while (!stoppingToken.IsCancellationRequested)
    {
        IBlogSyndication syndicationItem = await _queue.DequeueAsync();
        _logger?.LogInformation($"Dequeue IBlogSyndication for title '{syndicationItem?.Blog?.Title}'");
        try
        {
            await syndicationItem?.ProcessAsync();
        }
        catch (Exception ex)
        {
            _logger?.LogError(ex, $"Error occured processing IBlogSyndication for title '{syndicationItem?.Blog?.Title}'");
        }
    }
}

This pretty much completes the hosted service implementation also, and having a glance back at the reference article, we can see that we ended up with a very similar set of code for this service.

Conclusion

In this part, we concluded the implementation of the last of the syndication infrastructure elements, which enables us to now accept a blog entry and hand it off to a variety of adaptors to syndicate to various different social media outlets. Even though we didn’t see any new concepts, we did have an opportunity to implement our own version of the documented pattern using our now established test-driven techniques. Something else that the keen reader would have noticed by now is that our unit tests in the last few blog entries are more complicated and consists of more lines of code than our actual concrete implementations. This will mostly be the case, and it serves to illustrate to us how many dependencies we have, how it creates complications for us, and how code typically has more use cases to consider than the single purpose it was intended for.