Category Archives: QA Related

Posts related to Software Quality Assurance

C# Advent Calendar – Combining Integration and UI Automation in C#

I signed up for the C# Advent Calendar with this post – be sure to check out the rest of the blog posts in the calendar! This post assumes you’ve got some experience with C# and Visual Studio, so I won’t be over-explaining some things. Comment if you get stuck or have questions!

If you’re here for the first time, welcome! I’m a “career QA”, been in the QA/testing field for just over 10 years now. I’ve been doing test automation with C# for several years, including CodedUI, Selenium, and integration tests (using NUnit or MSTest) for REST services, as well as unit testing. So I’m kicking off the C# Advent Calendar with a testing topic. Hope you enjoy!


My talk on Testing RESTful Web Services is all about evangelizing integration testing. Integration tests are sometimes missed between unit tests and GUI tests – developers may think it’s the QA’s job, and QA may think it’s the developer’s job, so they can fall through the cracks.

no_integration_test.gif

So why combine integration and UI tests? These are useful to see if the API and the UI are speaking the same language. They are often developed at the same time by different people, so it’s nice to have a safety net of tests that can be run whenever to verify things are still working properly.

I wrote a very simple web API for funsies (using this tutorial) and because it has an API and a web UI, it seemed like a great subject for this mixed tests treatment! My code can be found at my github, here: https://github.com/g33klady/MontyPythonApi.

Our test will:

  • do a GET via the API for an item
  • launch the UI and search for the same item
  • compare the responses

Creating The Class Library For The Tests

Because this is pretty small, I’m just going to use a single project for my tests. In a larger application I might split them out. I’ll create a Class Library for all of my testing stuff. The first thing I do in general is add all of the NuGet Packages I need.

NuGetPackages

I then set up my structure.

For the UI tests, I know I’ll be using the Page Object Model so I create a folder for those, and add my HomePage class. I also know I’ll need some utility methods, so I add a Utilities class. I’ll wait to add the test classes until I’m ready for them.

ProjectStructure1

Setting Up The Page Objects

In essence the Page Object Model is decoupling the definitions of the elements on a page from the tests that use/manipulate them. This keeps the tests maintainable and not as brittle.

Our page is pretty simple. I’ve pointed out the elements we’ll want to put into the class. We’ll just be typing a number into the text field, clicking Search, and reading the response in the div below that.

webpage_objectsIndicated

Each of my elements has an ID – easily identifiable elements helps us QA folks use the elements in automation. If I have to use Xpath, for example, the tests can be more brittle than if I tell the automation “find this thing with this unique identifier”.

This is what my class looks like with the text field identified:

using OpenQA.Selenium;
using OpenQA.Selenium.Support.PageObjects;
 
namespace MontyPythonApi.Tests.PageObjects
{
	public class HomePage
	{
		[FindsBy(How = How.Id, Using = "prodId")]
		public IWebElement ProductIdInput { get; private set; }
	}
}

The FindsBy is how Selenium will be finding the element, and then the property is of type IWebElement (so Selenium can find it) and I can call it whatever I want. I like to use the type of element it is in the name to make it more clear, especially when there’s tons of elements in there.

I’ll add the rest of the page elements, and then initializing the elements via the PageFactory (part of Selenium.Support.PageObjects):

using OpenQA.Selenium;
using OpenQA.Selenium.Support.PageObjects;
 
namespace MontyPythonApi.Tests.PageObjects
{
	public class HomePage
	{
		[FindsBy(How = How.Id, Using = "prodId")]
		public IWebElement ProductIdInput { get; private set; }
 
		[FindsBy(How = How.Id, Using = "searchButton")]
		public IWebElement SearchButton { get; private set; }
 
		[FindsBy(How = How.Id, Using = "product")]
		public IWebElement ProductDisplayOutput { get; private set; }
 
		public HomePage(IWebDriver browser)
		{
			PageFactory.InitElements(browser, this);
		}
	}
}

Any methods specific to this page will also go here. We can add them as we need them.

Setting Up The Utility Methods

For our integration tests, we’ll need to make HTTP Web Requests, and then get the response back and deserialize it. I prefer to deserialize it in the test itself, but if you want to do it in the utilities be my guest 😀

Our utility class looks like this (the formatting sucks – check out the code instead here):

using System;
using System.Net.Http;
using System.Text;
 
namespace MontyPythonApi.Tests
{
	public class Utilities
	{
		public static HttpResponseMessage SendHttpWebRequest(string url, string method, string content = null)
		{
			using (var httpClient = new HttpClient())
			{
				var httpMethod = new HttpMethod(method);
 
				using (var httpRequestMessage = new HttpRequestMessage { RequestUri = new Uri(url), Method = httpMethod })
				{
					if (httpMethod != HttpMethod.Get && content != null)
					{
						httpRequestMessage.Content = new StringContent(content, Encoding.UTF8, "application/json");
					}
					return httpClient.SendAsync(httpRequestMessage).Result;
				}
			}
		}
 
		public static string ReadWebResponse(HttpResponseMessage httpResponseMessage)
		{
			using (httpResponseMessage)
			{
				return httpResponseMessage.Content.ReadAsStringAsync().Result;
			}
		}
	}
}

Now We Can Write Our Test!

Now to the good stuff.

I start with the SetUp method, which runs prior to every test. This is where I set the browser driver up – what Selenium uses to make the browser bend to its will.

using NUnit.Framework;
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
 
namespace MontyPythonApi.Tests
{
	[TestFixture]
	public class IntegrationAndUiTests
	{
                private IWebDriver browser;
		private static string webUrl;
		private static string apiUrl;
		private static string baseUrl;
 
		[SetUp]
		public void Setup()
		{
			browser = new ChromeDriver();
			baseUrl = "http://localhost:20461";
			webUrl = baseUrl + "/index.html";
			apiUrl = baseUrl + "/api/products";
		}
        } 
}

I instantiate the browser driver (using IWebDriver from Selenium) in the SetUp method because each test will need a new instance of it. Like all good tests, these need to be atomic.

We need a TearDown method as well, to clean up after ourselves.

[TearDown]
public void TearDown()
{		
	browser.Quit();
}

I’m going to set up my test to first make the call to the API, then use an Assert to verify that I got a 200 OK response back. This isn’t necessary but I find it useful – my tests fail here rather then trying to deserialize later on, so I know where things went wrong more quickly.

So we know our test is going to make a call to the API about a product, then verify the data in the web UI. We have a requirement that if a product has a discount price, it’s the only price that will display in the web UI. We set up our test data so that the product with ID = 1 has a discount price, so that will be the subject of our test.

[Test]
public void ProductsDiscountPriceDisplaysOnWebPage()
{
	//API call
	var uri = apiUrl + "/1"; //get the product with ID = 1
	var apiResponse = Utilities.SendHttpWebRequest(uri, "GET");
	Assert.That(apiResponse.IsSuccessStatusCode, 
		"Did not get success status code; got " + 
                apiResponse.StatusCode.ToString());
}

Our test has started to take shape. We are making our call, and getting the response back. We’re verifying  it got a 200 OK back but nothing else with it yet.

Let’s deserialize the response, so we can get the discount price value. To do that, I need to add the reference to the API project first, so I can use the model.

Here’s my code now, after having deserialized the response, using the utility method to read the response content into a string:

[Test]
public void ProductsDiscountPriceDisplaysOnWebPage()
{
        //API call
	var uri = apiUrl + "/1"; //get the product with ID = 1
	var apiResponse = Utilities.SendHttpWebRequest(uri, "GET");
	Assert.That(apiResponse.IsSuccessStatusCode, 
		"Did not get success status code; got " + 
		apiResponse.StatusCode.ToString());
	Models.Product product = JsonConvert.DeserializeObject(Utilities.ReadWebResponse(apiResponse));
}

Now we have the data that is returned for the product. We can now launch the web browser with Selenium and get the data returned in the web UI.

I need to have the browser go to the url, and then instantiate my page object:

//WebUI
browser.Navigate().GoToUrl(webUrl);
PageObjects.HomePage page = new PageObjects.HomePage(browser);

Now I can use the properties of page to interact with the elements on the page.

//WebUI
browser.Navigate().GoToUrl(webUrl);
PageObjects.HomePage page = new PageObjects.HomePage(browser);
page.ProductIdInput.SendKeys("1");
page.SearchButton.Click();

I’m essentially typing 1 into the search field, and clicking the search button.

Next I need to read the display of the data as it came back for that product. It comes back in a string with the format <Name> : $<Price> so I’ll need to parse it. Because this is something I’ll be doing on this page for more tests, I’ll add this utility method to the HomePage class.

public string GetPriceFromDisplayText(string displayText)
{
        decimal result;
	Regex r = new Regex("\\$(.*)");
	Match m = r.Match(displayText);
        decimal.TryParse(m.Groups[1].Value, out result);
	return result;
}

Definitely could hit some exceptions along the way, but for now we’ll leave it as-is.

Now our Web call looks like this:

//WebUI
browser.Navigate().GoToUrl(webUrl);
PageObjects.HomePage page = new PageObjects.HomePage(browser);
page.ProductIdInput.SendKeys("1");
page.SearchButton.Click();
var displayedPrice = page.GetPriceFromDisplayText(page.ProductDisplayOutput.Text);

And we can finally add that assert statement to check our values! Here’s our full test:

[Test]
public void ProductsDiscountPriceDisplaysOnWebPage()
{
	//API call
	var uri = apiUrl + "/1"; //get the product with ID = 1
	var apiResponse = Utilities.SendHttpWebRequest(uri, "GET");
	Assert.That(apiResponse.IsSuccessStatusCode, 
		"Did not get success status code; got " + 
		apiResponse.StatusCode.ToString());
	Models.Product product = JsonConvert.DeserializeObject(Utilities.ReadWebResponse(apiResponse));
 
	//WebUI
	browser.Navigate().GoToUrl(webUrl);
	PageObjects.HomePage page = new PageObjects.HomePage(browser);
	page.ProductIdInput.SendKeys("1");
	page.SearchButton.Click();
	var displayedPrice = page.GetPriceFromDisplayText(page.ProductDisplayOutput.Text);
 
	Assert.AreEqual(product.DiscountPrice, displayedPrice, "The prices don't match!");
			
}

Let’s run it and see what we get! Because the API and the tests live in the same solution, we’ll need to open a new instance of Visual Studio to run the tests locally.

Time For Some Results

And… our test fails!

TestResult

There’s a bug in the UI code that should display the discount code. Would we have found this otherwise? Probably with a good unit test, but this is a nice way to combine our tests and see how the application really behaves.

I’m leaving the bug there in the repo, so you can follow along. All of the code demonstrated above is there.

Let me know what you think – is this something you could use? Is this too much overhead for your project? Is this useful?

Advertisements

Using Fiddler AutoResponder for quick service or response mocking

In my Testing  RESTful Web Services talk, I touch on service virtualization briefly, and I include a short demo of using Fiddler‘s AutoResponder. I’ve been meaning to look into other (free) tools but haven’t had the chance.

After a question on the Testers Slack, I realized I could do a short blog post on how I demo’d AutoResponder in my talk. It gives a quick idea of the tool and one way you could use it. I love Fiddler*, and there’s so much more you can do than what I’ve been able to use. Please don’t take this post as a “this is the only way you can do things” – I’ll be the first to admit that even though I’ve used Fiddler for years, I feel like there’s so much more to know about it!

*Note: I am not paid by Telerik/Progress for my endorsement; I just love the tool 😀

Read the rest of this entry

Coding the right thing: not just for developers

This tweet came across my feed recently and I emphatically agreed:

coding_thinktweet

It occurs to me, however, that we as testers sometimes don’t see ourselves falling into the category of needing such advice. There’s the oft repeated “coding the right thing rather than coding the thing right”:

codingrightthing_tweet

We throw together a script that walks through the application, that performs a particular use case, and we run it and we have this sense of security that if it passes, we are ensuring good code is going out the door.

But are we, as testers, doing due diligence with our code, the same way we expect the developers to? Are we doing design reviews to ensure we’re actually coding the right thing? Are we getting code reviews? Are we reviewing and refactoring our own code as the application code changes? Are we pair programming? From what I’ve seen, some of us are. Many are not.

resized_philosoraptor-meme-generator-if-testers-test-to-assure-quality-then-who-assures-the-quality-of-the-tester-15551b

We cannot advocate that “test code is as important as application code” if we don’t treat it as such. We can teach almost anyone how to write a script to automate use cases of an application. We should be teaching them how to think more – what will this code actually do? What does it accomplish? If it passes, what does it prove? Am I actually asserting anything, or am I just having it click buttons and fill in fields with no actual check in place? It is one thing to write a script to assist you in manual testing – perhaps to get you to a certain point so that you can get to the real testing quicker. It is another to write such a script and treat it as the test itself.

If you write test automation, I challenge you to reconsider what your code actually accomplishes, what a passing check means. Reconsider the design and architecture choices made. What did you mean to do? What did you miss?

I challenge you to treat your code as if it is application code. I challenge you to treat your code as you want the developers you work with to treat theirs. And remember that test automation is not testing (at least not until the machines take over) – you need a brain to test.

On being a (not so newbie) speaker

A few months ago, I received an email from someone that attended my very first conference talk, at QA or the Highway in February. They were from $Company, and lead a QA team there. They thought that my talk would be great for their team to hear, and asked that I come to speak to the team. Instant freakout! That was my first time speaking at a conference! Holy shit!

I replied that I was interested for sure, and we set a day and time. I got my talk ready, by making a few changes from the conference, updating a few points and adding some things.

I also started setting up a drinkup for the time I would be there, to catch up with testers in the Columbus area. That turned out to be an excellent decision. And Mr. Keith Klain introduced me to the Per Scholas folks in Columbus, and asked that I stop by and speak to them as well – a casual conversation about being a tester and just being a IT professional. And that he’d pay for the drinks! Bonus!

So now I was on a mini speaking tour. Time for the Imposter Syndrome to kick in…

I drove down on Father’s Day, checked in to the hotel, and went to the bar. The original one we chose was closed, so we moved on to another – we’re testers, we adapt well to adverse situations!

Now for the conversation at the bar. I met up with Damian Synadinos and Diana Wendruff – two awesome folks in the test arena in the Columbus area. We talked about why I was in town, and who I was speaking to. Damian offered some advice that proved incredibly valuable to me – this could be a “quick fix” by management, try to make everyone get along by bringing in this speaker. Some of the folks there in QA are *not* testers by profession, they probably don’t want to be there. That, in itself, was extremely helpful. I had not thought about it yet, but here’s what I realized:

speaking at a conference, people have chosen to be there or at least chose to be at your session, and can walk out at any time if they feel it doesn’t apply to them. Speaking at a company, people are most likely made to attend your talk. They have other stuff to do, and if they don’t want to be there or it doesn’t apply to them, they can’t leave – they can only sit there. You’re giving it your all, showing your passion to them, and some of them just don’t care. Maybe even resent you.

AND THAT’S OK!

Speaking in front of an audience like that was intensely different than at the conference. At the conference, the attendees were engaged and interested, and they laughed at my jokes! At this company, I was just like “management” to most of them – throwing a message at them that they didn’t care about or believe in.

Now I’m not saying every person that I spoke to there felt this way – there were a few attentive faces. But for the most part, it was the “ugh, I’d rather be doing my job right now” face. Without that realization – without that conversation at the bar! – I wouldn’t have been prepared. I would right now probably still be crying from the experience 😛 Lesson here – always get drinks with your buddies! Or something…

 

Then I went, directly after, to Per Scholas. Per Scholas is an amazing program that provides IT training and job placement to low income folks. This audience was the complete opposite of who I spoke to earlier in the day – they were excited to have someone come in to speak to them that was doing in the real world something that they could do in the very near future. I gave a quick rundown of my resume essentially, and opened it up for questions. Some great, intelligent questions and excellent conversation. They asked for my contact information and business cards (nobody at $Company did!) which I happily provided – I’m happy to answer any of their questions privately, or connect with them on LinkedIn. They are an awesome group of women and men that have a very bright future ahead of them, thanks to the Per Scholas program.

A few of the points I tried to make there:

  • You absolutely do NOT have to know how to code to be a great tester
  • However, knowing how code comes together will help you (the process of dev)
  • When I was hiring for testers, I was looking for:
    • Passion for software quality – beta testing games? providing bug reports for software you use every day?
    • Excited by puzzles – if you have to figure something out, you don’t get frustrated easily
    • Intelligent and willing to learn – I don’t care if you have a degree, but if you’re not willing to learn buh-bye
    • Admit when you’re wrong or don’t know – srsly, don’t lie or inflate your abilities. It won’t end well
    • Some technical aptitude – you have to be able to figure out how to troubleshoot. Or Google well 😛
  • Network network network! Look for meetup groups. Networking is how I got into this industry!

 

All in all, my mini speaking tour was great. I gained a lot of experience speaking in front of a not-so-accepting crowd, and got to meet the amazing Per Scholas organizers and students, and of course drinks with some great testers (paid for by Keith Klain!).

I’ll be speaking at the Quicken Loans internal Technology Conference in July, and then again at CAST in August. Then I think I’m done for the year! And I’m getting married next year, so probably not doing much next year… Though I started 2014 with the hopes of speaking at a single conference, and am ending up speaking at 3 plus privately for a company and Per Scholas. So who knows!

Testers meet up in Detroit – let’s do this!

Hey all – let’s get this going. If you’re interested in speaking – either full session, leading a workshop, or doing a lightning talk, let me know.
I’ll get started on the location, day and time. I’d like to do this monthly!
W00t w00t let’s do this!

 

ETA: Here’s the meetup group!

http://www.meetup.com/Motor-City-Software-Testers/

2013 Year in Review

Last year I wrote up a post with my 2013 goals and resolutions. I wanted to review what I had actually done, so here we are!
Achieve a better work/life balance. This will be the most difficult of all of my goals…

  • With my new job, I think I’ve gotten this a bit more under control. It’s easy for me to work 50+ hour weeks, but I try to keep myself to 45 hours when it’s not busy time

Attend CAST, KalamazooX, IT in the D, and maybe some other conferences too

  • I attended KalamazooX on the initial list. I also went to StarWest, which wasn’t even on my radar before because it was something I couldn’t personally afford. Thanks to my new job this year, I got to go!

Submit a talk to a conference and/or user group

  • Done! 3 or 4 times, even!

Actually do above talk

  • I did the lightning talk version at the Great Lakes Area .NET User Group. I was also accepted to do the full version of the talk at QA or the Highway in February 2014 😀 I also did a series of 99 second talks on the same topic (dev vs. qa) for Software Testing Club (note – they’re horrible!)

Submit articles to QA publication

  • I was published! In the 10th issue of The Testing Planet I wrote about Leadership (actually The 10 Habits of a Highly Ineffective Leader, which pulls from my experience over the years)

Attend at least one GiveCamp; I didn’t attend any this year and I’m jonesing for it!

  • Was on the planning committee and attended the Ann Arbor GiveCamp. Huzzah!

Assist in coordination of and/or attend Detroit GiveCamp

  • I was part of this, but the whole thing got sidetracked :/

Seek out new opportunities…

  • I got a new job! At Quicken Loans! Which I LOVE!

Finally write that Android app I’ve been thinking about for a while

  • Nope, didn’t happen

Write more code in general. Ruby, Java, C, and of course my favorite, Perl.

  • A bit, in C# with Coded UI tests. And VBScript for QTP (blech)

Continue to be involved in the QA community

  • Yeah, for the most part, this is happening 😛

Hire some people that actually know WTF they’re doing

  • I left there. I hope they have good QA 😛

Actually go to the gym I have a membership for >.<

  • Yeah, I went like a few times. Still paying for it, still don’t go *shrug*

Be a better leader

  • Constant work in progress on this one

 

So 2013 has treated me pretty well in my professional career. I look forward to speaking at QA or the Highway and whatever else this year throws my way!