Posts Tagged ‘Acceptance Testing’

Getting Started with Visual Studio Test and Lab Manager

November 16th, 2009

My buddy Charles Sterling, Program Manager at Microsoft has posted a new webcast video showing the basics of how to create a test plan, test case, recording a test, filing a bug and playing back the test using the Beta 2 version of Visual Studio 2010.

You can see the video here:

How to create record and playback Test Cases in Visual Studio Beta2

This new central place for QA teams to create and manage test plans and tests is a great addition to Visual Studio for traditional QA teams.  However my primary interest is in how could an Agile team use this new tool to move testing forward in the Agile process.

Firstly I think that this tool could be a good place for Product Owners and Test Professionals to collaborate on the Acceptance Tests for an upcoming iteration.  The User Stories could be the basis for creating the actual test cases that would be used to validate the delivery of the anticipated business value from the User Story.

During Sprint (Iteration) planning the team could use the test case to understand the intent of the story and get clear on what the real acceptance criteria is.  I would expect that more detail and more tests would be added during the planning session as clarity is improved.

While working the story in the Sprint the team would continue to update the tests with recorded steps of the newly developed feature and run the acceptance tests to monitor the Sprint progress.

When the Acceptance Tests all passed, the User Stories would be functionally complete and they would provide a safety net for performance tuning, refactoring and integration.

I hope you have noticed that I am completely ignoring the coded automation of the Acceptance Tests.  I am a big proponent of automating Acceptance Tests.  However in my experience the road to fully automated Acceptance Tests starts with understanding the process of writing the Acceptance Tests and the Agile team using them to drive development.

I’m sure I will come back to the topic of Automated Acceptance Tests later, but if your team is looking for an easy way to get your QA professionals involved in your Agile team, using Visual Studio Test and Lab Manager to author Acceptance Tests is a great place to start!

  • Share/Save/Bookmark

Tags: , , , ,
Posted in Code Quality, Scrum, TFS | Comments (0)

Bugs, bugs, BUGS!

July 7th, 2009

bugI’m sure that you all know the story of how the term “bug” got associated with a computer failure.  It was a moth in a computer which caused a hardware failure.  However, since then the term bug  has become synonymous with a software error.  There is a bug in the program.  Yes, there is a bug in the program, every program.  Many software development organizations spend a large portion of their time and effort finding and fixing bugs.  However the situation rarely improves.

This leads organizations to start tracking bugs and applying all sorts of analysis and pressure to fix them.   How do we know which bug to fix first?  The priority (how important we think it is) is assigned and the severity (who and what does it affect) is assessed.  Then when we figure out what to fix based on priority and severity, then who should fix it?  All too often the analysis is that the primary development team is too busy and important to fix bugs, so let’s create a bug team and have them fix all the bugs.

Quite often the situation gets worse again.  More bugs are found by customers and the QA team.  The backlog for the bug team gets bigger and the time from bug discovery to fix delivery to customers increases.

If you have been around software for any time at all, you will recognize this pattern.  Lots of bugs being tacked separately from new requirements and an increasing slice of all development effort being allocated to the effort to attempt to slow the erosion of software quality.

I believe we need a different approach.

We have to understand what caused the bug in the first place.  We have to make sure our goals are clear and that we are applying the right people and talent and money to reach our goals.

Should the goal of the software development organization be to deliver quality software?

Trick question.  Of course we need quality software, but quality should not be the goal.  The goal should be the delivery of business or customer value.  That’s why we are building software.  If we deliver poor quality then we will have to spend more time and effort fixing it so that we can deliver on the solution we promised.  However the focus should be on the delivery of business value.

If we turn away from tracking and prioritization of bugs and turn towards the delivery of business value quickly and effectively, we can change the entire landscape of software development.

We need to understand where the bugs come from.  Contrary to the belief of some, the code does not just rot.  The gremlins don’t attack source control at night.  Except for the occasional corruption of source and binaries from hardware failure, most of the time the bug was written by a developer.

I think there are two kinds of bugs.  The first is just a mistake.  Someone used the wrong variable, used a wrong value, called the wrong method.  Stuff happens.  This type is blocking functionality that was assumed to be working.  Hopefully an Agile team using TDD will find these before the end of the iteration.  If not, there are few things of higher priority.  Track it if you have to, but get it on the next sprint backlog in the form of a story.

The other kind is when the team did not understand ( or was not given ) the full scope of the requirement.  This one often includes unrecognized interaction with other systems.  This type is much more common and the priority cannot be determined by the development team.  The Product Owner must make sure that they understand the value of the missing or wrong functionality and present that story back to the team for sizing and prioritization.

We will probably need a system to track the errors that are found, but we need to quickly decide which type it is.  The first type is the failure to really deliver what we said we delivered.  The second is the failure of the Product Owner to deliver the requirements to the development team in such a way that they are able to deliver an appropriate solution.

In either case, I think that the work needs to be prioritized alongside the other stories.  What is the business  (customer) value?  How much effort and risk is there in completing this story?  If you must use a bug tracking system to be responsible and responsive, then do so.  However your decisions to deliver business value should be contained in a story that the team can deliver.

This, by definition obviates the need for a “bug” team.  The delivery of business value is the responsibility of the whole team.  The team can and must learn from its mistakes.   How did that error get into production (the mistake type)?  What tests should have been in place to make sure that the code was correct.  What about that requirement did we not understand ( the second type)?  How can we better understand next time?  How will we make sure that we understand it now?  These are all questions that the team should investigate and try to answer.

The first type of bug is lessened by unit testing by developers.  A test first approach using Test Driven Development is best.  TDD all but eliminates this type of bug.

The second type of bug is lessened by an Acceptance Test Driven approach to requirements understanding.  Virtually all requirements can be tested with Automated Acceptance Testing.  Basically what we are doing is repeating back to the Product Owner the requirements in the form of a test that says,  “when this happens, the code will respond with this behavior.”  The Acceptance Test can be written before the object code is even named.  The test can fail on day one of the iteration.

Scrum and Agile are about delivering business value quickly and repeatably.   Prioritization of business value must drive the project.  Prioritize all work together and drive the process with Acceptance Tests and Test Driven Development.  Your team will deliver more business value quicker and will deliver higher quality software in the process.

  • Share/Save/Bookmark

Tags: ,
Posted in Code Quality | Comments (0)

SEO Powered by Platinum SEO from Techblissonline