MSTest Complains if the TestMethod has parameters

May 28, 2008 at 3:18 PM
Hi,

MS Test complains if I put input parameters for my test methods.

[PexMethod]
        [ExpectedException(typeof(Exception),"String is null or empty")]
        public void should_throw_exception_if_the_passed_string_is_null(string text)
        {
            Palindrome palindrome = new Palindrome();
            Assert.IsTrue(palindrome.IsPalindrome(text));
        }

Also, the above Pex assumption fails but there are no details of why is it failing?

Coordinator
May 28, 2008 at 8:27 PM
[I don't know what your meant with your MS Test statement, so please elaborate on this]

First of all, Pex will generate tests using your PUT with different arguments. At a minimum, this will include
1) a test where "text" is null
2) a test where IsPalindrome() returns true
3) a test where IsPalindrome() returns false

ExpectedException requires that the expected exception be derived from Exception, so your Palindrome class should throw an "ArgumentException" instead of Exception.

To truly check the behavioud of IsPalindrome, you need to evaluate the text twice: Once for the "expected" result, and then by calling IsPalindrome to get the actual result. As an example, here is an implementation of the Palindrome class I used [NOTE: This implementation is incorrect, by design ]

 

public bool IsPalindrome ( string text )

 

{

 

if ( string.IsNullOrEmpty ( text ) )

 

{

 

throw new ArgumentException ( "String is null or empty" );

 

}

 

var length = text.Length - 1;

 

 

var middle = length / 2;

 

 

var result = true;

 

 

//NOTE THIS IMLEMENTATION IS ERRONEOUS
for
( var i = 0; result && ( i < middle ); i += 1 )

 

{

result = text [ i ] == text [ length - i ];

}

 

return result;

 

}




Then, to test the behaviour I use a different approach to determine if the text is a Palindrome in the PUTs:

 

 

using

 

System;

 

 

 

 

using

 

System.Linq;

 

 

 

 

using

 

System.Text;

 

 

 

 

using

 

Microsoft.Pex.Framework;

 

 

 

 

using

 

Microsoft.VisualStudio.TestTools.UnitTesting;

 

 

 

 

using

 

TestProject1;

 

 

 

 

namespace

 

TestProject1

 

{

[

TestClass ]

 

[

PexClass ( typeof ( Palindrome ) ) ]

 

 

public partial class PalindromeTest

 

 

 

 

 

{

 

 

 

#region

 

Methods

 

[

PexMethod ]

 

 

public void TestIsPalindrome ( [ PexAssumeUnderTest ] Palindrome target, string text )

 

{

 

var result = target.IsPalindrome ( text );

 

 

PexValue.AddForValidation ( "result", result );

 

 

// Use a different approach to determine if text is a palindrome
var
x = new StringBuilder ( );

 

x.Append ( text.Reverse ( ).ToArray ( ) );

 

var reversed = x.ToString ( );

 

 

Assert.AreEqual ( text == reversed, target.IsPalindrome ( text ) );

 

}

[

PexMethod ]

 

[

ExpectedException ( typeof ( ArgumentException ), "String is null or empty" ) ]

 

 

public void should_throw_exception_if_the_passed_string_is_null ( string text )

 

{

 

var palindrome = new Palindrome ( );

 

 

var isPalindrome = palindrome.IsPalindrome ( text );

 

 

var x = new StringBuilder ( );

 

x.Append ( text.Reverse ( ).ToArray ( ) );

 

var reversed = x.ToString ( );

 

 

Assert.AreEqual ( text == reversed, isPalindrome );

 

}

 

 

 

#endregion

 

 

 

 

}

}

Then, if you run PexIt on class PalindromeTest it will generate several specific test cases that trigger the ArgumentException, demonstrate cases where IsPalindrome returns the correct result, and also demonstrate cases where IsPalindrome returns an incorrect result.


For the cases where an argument exception is thrown, you need to specifically tell Pex it is allowed in the PexAssemblyInfo.cs file:

using

 

Microsoft.Pex.Framework.Validation;

 

using

 

System;

 

[

assembly: PexAllowedExceptionFromAssembly ( typeof ( ArgumentException ), "TestProject1" )]

 





May 29, 2008 at 1:47 AM

>>>> For the cases where an argument exception is thrown, you need to specifically tell Pex it is allowed in the PexAssemblyInfo.cs file:

Why do I have to specify that an exception will be thrown. What about if one of the input causes the exception to be thrown and not the other one? 
May 29, 2008 at 2:30 AM
What is the purpose of these tests? 

  [TestMethod]
        [ExpectedException(typeof(AssertFailedException))]
        [PexGeneratedBy(typeof(TestStringService))]
        public void should_be_able_to_reverse_a_stringString_20080528_212754_003()
        {
            this.should_be_able_to_reverse_a_string("\0\u0001");
        }

        [TestMethod]
        [ExpectedException(typeof(AssertFailedException))]
        [PexGeneratedBy(typeof(TestStringService))]
        public void should_be_able_to_reverse_a_stringString_20080528_212754_004()
        {
            this.should_be_able_to_reverse_a_string("\0\0\0");
        }

What exactly are we passing to the method? 
Coordinator
May 29, 2008 at 11:10 AM

The first test passes a 2 character string containing a null followed by a non-null. This identified a bug in the IsPalindrome implementation where not all characters were checked (off by one).

The second test passes a 3-character string where all characters are the null character. This test case is the first test case in the sequence of cases generated that caused the "for" loop body to be executed (the preceeding (in the entire file, your excerpt) generated case passed in only a single character string, which was too short to trigger execution of the loop body.


Coordinator
May 29, 2008 at 11:20 AM

This is the difference between "Allowed" and "Expected".

Allowed tells Pex that some test cases may trigger an exception, which is correct behaviour. These test cases in turn will be decorated with [ExpectedException].

NOTE: In the code actually generated by Pex, the test case for ("\0\u0001") had the ExpectedException attribute, but the test case for ("\0\0\0") does not.

May 29, 2008 at 12:53 PM
>>> The first test passes a 2 character string containing a null followed by a non-null. This identified a bug in the IsPalindrome implementation where not all characters were checked (off by one).

The second test passes a 3-character string where all characters are the null character. This test case is the first test case in the sequence of cases generated that caused the "for" loop body to be executed (the preceeding (in the entire file, your excerpt) generated case passed in only a single character string, which was too short to trigger execution of the loop body.



So, how do I pass these tests where the parameters sent are null character or characters. Seems like I can just pass them using Fix it and then it won't be passed again. Do I want to do that?
Coordinator
May 29, 2008 at 5:02 PM
Keep in mind that the IsPalindrome implementation I used was deliberateley flawed, and the presence of null characters should probably be allowed in the string. So the developer would first fix the bug in IsPalindrome, and then see what the Pex generated tests report.