C# Advent Calendar 2018 – Can I Put All Of My Smoke Tests For A REST API In One Test? Yes!

Last year, I had a great time contributing to the C# Advent Calendar with a post on combining API and UI automation. I was excited to sign up once again for the 2nd Annual C# Advent Calendar! Check out the page here with the full calendar: https://crosscuttingconcerns.com/The-Second-Annual-C-Advent

This year, I want to talk about smoke tests for your REST APIs. I’m sure this could fit with non-REST APIs, but that’s where I’m focusing rather than all of the “what-if”s.

First, what do I mean by “smoke tests”? Smoke tests are really just checks to be sure the system is up and running, before you do more in-depth testing. I’ve heard two different origins for this term – one in hardware, and the other in plumbing. In hardware, when you plug something in – if there’s smoke, there’s a problem.

I like the plumbing definition better. I don’t remember where I first heard it, but I heard that plumbers, when done with a job, will run smoke through the pipes to make sure there are no leaks (since it’s much cleaner than running water through if there are leaks). I like this one better because it is pretty much what we’re doing – lightly touching things to make sure there aren’t any leaks!

So with that definition in mind, there’s still some vagueness around what I mean by “smoke tests”. So let me clarify that more: I define a smoke test for an API specifically as non-destructive, non-additive checks to ensure endpoints are operational. An example would be to call the GET method on all endpoints and ensure you get a 200 back. However, maybe your API will give a 200 even if the call failed (why?! fix that!). So for that, a smoke test would be to call the GET method and verify a certain piece of data is in the response.

I want my smoke tests to be able to be run whenever, and wherever, I need them. In Production after a deployment, to ensure we didn’t break anything. Or after maintenance on an adjacent system, to ensure *they* didn’t break anything.

So my smoke tests are all going to do the same thing, do a GET request to a specific endpoint, then expect a certain response back. If I have 3 endpoints, each with one or two GET requests to test (GET all items, GET single item for example), plus the query parameters, that’s a lot of duplicated code. We can absolutely use data-driven testing techniques here. For that, I like to use NUnit’s TestCaseData feature.

To demonstrate, let’s use the Gitter API – it’s pretty well documented and has a lot of variety for us to work with. We’ll add smoke tests for the Groups endpoint first, but we need to get the basics in place for our test.

First, we need a class for our test data. This class can be used for any of our tests, so we can give it the generic name of “TestData”. In it, we’ll define an IEnumerable which will specifically house our smoke test data, so we’ll call that “SmokeTests”.
The format of the TestCaseData is as below: first, the data we’re sending to the test, and second, the data the test will send back that we’re going to check. So we’re sending the endpoint we want to hit, and expecting a 200 OK response. I’ve added test data for the two Groups endpoints available.

public class TestData
{
    public static IEnumerable SmokeTests
    {
        get
        {
            yield return new TestCaseData("/v1/groups").Returns("OK");
            yield return new TestCaseData($"/v1/groups/{_testGroupId}/rooms").Returns("OK");
        }
    }
}

 

Let’s see what our test would look like.
The first thing you might notice, is that our Test has another attribute TestCaseSource. This is defining where our test data is coming from (the class) and specifying the actual source (the name of the IEnumerable).
Next you may notice that the test method is actually taking in a parameter. It’s taking in the data sent from TestCaseData.

So we set up the URL to use the data we want, we make our request, and then we actually return the status code. Note that there is not an Assert statement in this test. TestCaseData will check what is returned against what we told it we expect, and that is what will tell us it passes or fails.

[Test, TestCaseSource(typeof(TestData), "SmokeTests")]
public string GitterApiSmokeTests(string endpoint)
{
    //Arrange
    var url = $"{_baseUri}/{endpoint}?access_token={_accessToken}";
    var client = new RestClient(url);
    var req = new RestRequest(Method.GET);

    //Act
    IRestResponse response = client.Execute(req);

    //Assert
    return response.StatusCode.ToString();
}

 

Now that we have this established, all we need to do to add more tests is to add more TestCaseData! Here’s the SmokeTests data with the additional checks of the rooms endpoint.

public class TestData
{
    public static IEnumerable SmokeTests
    {
        get
        {
            yield return new TestCaseData("/v1/groups").Returns("OK");
            yield return new TestCaseData($"/v1/groups/{_testGroupId}/rooms").Returns("OK");
            yield return new TestCaseData("/v1/rooms").Returns("OK");
            yield return new TestCaseData("/v1/rooms?q=testing").Returns("OK");
            yield return new TestCaseData($"/v1/rooms/{_testRoomId}/users").Returns("OK");
            yield return new TestCaseData($"/v1/rooms/{_testRoomId}/users?q=hilary").Returns("OK");
            yield return new TestCaseData($"/v1/rooms/{_testRoomId}/users?skip=3").Returns("OK");
            yield return new TestCaseData($"/v1/rooms/{_testRoomId}/users?limit=5").Returns("OK");
        }
    }
}

 

The really cool thing is, in Test Explorer, they all show up as individual tests. That way, you can see which TestCaseData actually failed, and which ones passed. You can pass additional arguments to the TestCaseData to give it a nicer name as well.

Now you’re probably thinking – just because an endpoint with a query parameter returns a 200 doesn’t mean it actually returned the thing it was querying for. In that case, I would probably have two tests (I know, I said just one!) – one for checking the 200 responses, and one for checking the quantity of items returned or some data in the response. You could technically put it all in one, but that is making a test that’s way more complicated than it needs to be. Keep it simple!

So let’s look at what the complete test and TestCaseData would look like (removing the query params as we’ll put those in a separate smoke test).

[Test, TestCaseSource(typeof(TestData), "SmokeTests")]
public string GitterApiSmokeTests(string endpoint)
{
    //Arrange
    var url = $"{_baseUri}/{endpoint}?access_token={_accessToken}";
    var client = new RestClient(url);
    var req = new RestRequest(Method.GET);

    //Act
    IRestResponse response = client.Execute(req);

    //Assert
    return response.StatusCode.ToString();
}

public class TestData
{
    public static IEnumerable SmokeTests
    {
        get
        {
            yield return new TestCaseData("/v1/groups").Returns("OK");
            yield return new TestCaseData($"/v1/groups/{_testGroupId}/rooms").Returns("OK");
            yield return new TestCaseData("/v1/rooms").Returns("OK");
            yield return new TestCaseData($"/v1/rooms/{_testRoomId}/users").Returns("OK");
            yield return new TestCaseData($"/v1/rooms/{_testRoomId}/chatMessages").Returns("OK");
            yield return new TestCaseData($"/v1/rooms/{_testRoomId}/chatMessages/{_testMessageId}").Returns("OK");
            yield return new TestCaseData($"/v1/user").Returns("OK");
            yield return new TestCaseData($"/v1/user/{_userId}/rooms").Returns("OK");
            yield return new TestCaseData($"/v1/user/{_userId}/rooms/{_testRoomId}/unreadItems").Returns("OK");
            yield return new TestCaseData($"/v1/user/{userId}/orgs").Returns("OK");
            yield return new TestCaseData($"/v1/user/{_userId}/repos").Returns("OK");
            yield return new TestCaseData($"/v1/user/{_userId}/channels").Returns("OK");
        }
    }
}

 

There you go, 12 tests for the four endpoints of Gitter’s API, all in one test.
You can do a lot of stuff with TestCaseData, and this is really just scratching the surface. I hope you find this useful!
Thanks for stopping by, and enjoy the rest of the C# Advent Calendar!