This is my first attempt at a series of “what’s wrong with this test case code” entries. Let me know what you think of this idea and if you’re interested in seeing more examples.
A False Positive is defined as a test case that has been logging a passing result; however, the test case logic is incorrect, masking a product bug. In other words, if the test case writer were to run the test case manually, he or she would see the product bug.
Can you identify why the test case below could contain a false positive? Remember, you’re looking for the logic error.
You can make the following assumptions (for the sake of this example)
- The current line in the text file contains at least one whitespace
- Each individual method is bug free and does what the comment above says
‘ put the contents of the current line into a string var
Dim currentLine As String = vsEditor.Cursor.GetCurrentLineText()
‘ find out where in the line the whitespace is
Dim whitespaceLocation As Integer = currentLine.IndexOf(" ")
‘ move to the left until we’re at the whitespace location
vsEditor.Cursor.Move(CursorUnits.Character, whitespaceLocation)
‘ press delete
vsEditor.SendKeys("{DELETE}")
‘ get the new contents of the line
currentLine = vsEditor.Cursor.GetCurrentLineText()
‘ if the whitespace is gone, the test case passes
‘ otherwise, the test case fails
If currentLine.Substring(whitespaceLocation, 1) <> " " Then
LogPass()
Else
LogFailure("Failed to delete whitespace…")
End If
Got it figured out? If not, here’s a hint…
One common mistake when writing test cases is to verify against the side effects of what we’re expecting to happen. In the above code, the side effect is that the whitespace is gone from that location. In order to correctly verify that some desired action has occurred properly, the test case needs to simulate what the desired action is really doing.
Ready for the answer?
We’re testing that pressing the delete key will remove the whitespace, a single character, in front of the cursor. But let’s say delete is broken, and it deletes the whitespace and the next character. If our line of text was, “A lazy brown dog” and pressing delete deleted “ l”, the string would become “Aazy brown dog”. The second character isn’t a whitespace, yet it would pass our test case logic.
A better test case would simulate what the delete key is doing, for example
‘———————————————————————
InitScenario("Verify user can delete whitespace in middle of line")
‘———————————————————————
‘ put the contents of the current line into a string var
Dim currentLine As String = vsEditor.Cursor.GetCurrentLineText()
‘ find out where in the line the whitespace is
Dim whitespaceLocation As Integer = currentLine.IndexOf(" ")
‘ move to the left until we’re at the whitespace location
vsEditor.Cursor.Move(CursorUnits.Character, whitespaceLocation)
‘ press delete
vsEditor.SendKeys("{DELETE}")
‘ get the new contents of the line
Dim currentLineAfterDelete As String = vsEditor.Cursor.GetCurrentLineText()
‘ Remove the whitespace from the string variable
‘ and verify this is what the current line really is
If currentLineAfterDelete = currentLine.Remove(whitespaceLocation, 1) Then
LogPass()
Else
LogFailure("Failed to delete whitespace…")
End If
View original comments