Software Testing and Its Many Incarnations

I started out in my career as a test automation engineer and manual software tester in Quality Assurance.

Testing is paramount to developing high quality software. The meme “I don’t always test my code, but when I do, I test in production” is funny but also, unfortunately, very true. It is rare to have a fully-functional QA team these days. Developers are expected to test their own code.

It is important everyone is aware of the different types of software testing, who is responsible for which kind, and what they are.

Below is a description of what I have experienced in my few years as both a test automation engineer and a manual software tester.

There are generally two types of testing: white-box testing and black-box testing.

White-Box Testing

White-box testing is when the tester is able to view the source code for the application under test (AUT). White-Box testing is done in unit and integration testing.

Black-Box Testing

Black-box testing is just the opposite. Typically software developers perform white-box testing via unit and integration tests, whereas testers and end-users typically perform black-box testing via functional and acceptance testing.

Functional Testing

Functional Testing is a type of testing that makes sure the software behaves as the design and/or requirements, user stories, use cases, etc. dictate. A good tester is going to test boundary conditions, negative tests, and positive tests.

Boundary Conditions

Suppose I have a textbox that is supposed to accept a five digit zip code. Some ways I could test the boundary conditions on this field would be to enter less than five digits or more than five digits and see if the system accepts it.

Negative Tests

This is pretty much like testing boundary conditions only with negative testing the tester is making sure the application fails gracefully and does not return a Yellow Screen of Death (YSOD) or completely crashes the system when unexpected input is entered.

So, taking the zip code textbox example, let’s say I enter special characters. As a user, I would expect to get back a user-friendly error message such as “Please enter a five digit, numeric, zip code.”. If instead, I get a YSOD, then this test fails.

Positive Tests

This is just making sure the AUT is functioning correctly based on the requirements, user stories, use cases, etc. Any deviation from the requirements, user stories, use cases, etc. is a defect.

There are different types of functional testing as well.

Smoke Testing

When an iteration is released to Quality Assurance (QA) for testing, the team typically first does what’s called “Smoke Testing”. This is simply making sure the application works enough to get through it. There could be defects, but if the pages load (if it is a web application), then it would pass smoke testing. Otherwise, if there is an issue such as not being able to login to the system or a page simply does not load, then it fails.

Functional Requirements Testing

The next step that QA does is perform functional testing, which is just exercising the requirements, user stories, or use cases to make sure the iteration under test functions correctly. Positive and negative testing is done in this phase (and throughout, really).

Test Automation

If the QA team has a test automation engineer on the team, he or she will begin developing automated tests that will perform both smoke and regression testing. Regression testing is just testing the previous iteration’s functionality to make sure the current iteration did not break anything in the last.

Automated testing is using a software tool (CASE tool) to perform a pre-defined set of steps on a system. In my experience, with QuickTest Professional, it was as easy as recording actions, putting in checkpoints to make sure the expected values are on the screen, and provide a pass/fail criteria. It has been at least five years since I have used QuickTest Professional or any test automation tool. I am sure they are much more sophisticated now.

System Testing

Towards the end of the project, system testing is performed. This is to make sure that all end-to-end processes work per the requirements, user stories, or use cases.

During this entire process, the developers are performing tests of their own, in the way of unit testing and integration testing.

Unit Testing

Unit testing is a form of white-box testing to make sure code meets its design and behaves as intended. I think of it as “code that tests code”. In other words, the developer writes his or her own code that will test code within the system and either pass or fail based on what is expected from a set of inputs.

Integration Testing

The next is integration testing. This can be done by either the developer or the tester. It is testing combined software modules and testing them as a group to make sure new modules didn’t break old or vice versa. This is very much like regression testing.

User Acceptance Testing

The final form of testing is User Acceptance Testing (UAT). This can be thought of as beta testing and is performed by the client to make sure the application functions correctly and meets their needs.

3 thoughts on “Software Testing and Its Many Incarnations

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s