pex
A library and tool for generating .pex (Python EXecutable) files
I am playing with Pex and one of the parameters it passes into my method is "\0".
What does that mean? My guess is an empty string ("") based on the content of my method. However, if it is the same then why not just use "" instead of "\0"?
Anyone know what it is?
Source: (StackOverflow)
I love the idea of Pex - autogenerating unit tests through static code analysis - but the tests that are actually generated by the tool are horrible, ugly, tightly coupled to Pex modules, difficult to read and understand etc.
Is a tool like that really suitable (in its current state) for use in an enterprise environment, where the emphasis must be on ease of maintenance?
Or have I misunderstood the intended use of Pex?
Source: (StackOverflow)
I'm trying to use Pex to test some code. I have an abstract class with four concrete implementations. I have created factory methods for each of the four concrete types. I had also created one for the abstract type, except as this nice thread explains, Pex will not use the abstract factory method, nor should it.
The problem is that some of my code depends on the four concrete types being all there are (since it is very, very unlikely that any more subclasses will be created), but Pex is breaking the code by using Moles to create a stub.
How can I force Pex to use one of the factory methods (any one, I don't care) to create instances of the abstract class without ever creating Moles stubs for that abstract class? Is there a PexAssume directive that will accomplish this? Note that some of the concrete types form a type of tree structure, so say ConcreteImplementation derives from AbstractClass, and ConcreteImplementation has two properties of type AbstractClass. I need to ensure that no stubs are used anywhere in the tree at all. (Not all the concrete implementations have AbstractClass properties.)
Edit:
It appears that I need to add some more information on how the class structure itself works, though remember that the goal is still how to get Pex not to stub classes.
Here are simplified versions of the abstract base class and the four concrete implementations thereof.
public abstract class AbstractClass
{
public abstract AbstractClass Distill();
public static bool operator ==(AbstractClass left, AbstractClass right)
{
// some logic that returns a bool
}
public static bool operator !=(AbstractClass left, AbstractClass right)
{
// some logic that basically returns !(operator ==)
}
public static Implementation1 Implementation1
{
get
{
return Implementation1.GetInstance;
}
}
}
public class Implementation1 : AbstractClass, IEquatable<Implementation1>
{
private static Implementation1 _implementation1 = new Implementation1();
private Implementation1()
{
}
public override AbstractClass Distill()
{
return this;
}
internal static Implementation1 GetInstance
{
get
{
return _implementation1;
}
}
public bool Equals(Implementation1 other)
{
return true;
}
}
public class Implementation2 : AbstractClass, IEquatable<Implementation2>
{
public string Name { get; private set; }
public string NamePlural { get; private set; }
public Implementation2(string name)
{
// initializes, including
Name = name;
// and sets NamePlural to a default
}
public Implementation2(string name, string plural)
{
// initializes, including
Name = name;
NamePlural = plural;
}
public override AbstractClass Distill()
{
if (String.IsNullOrEmpty(Name))
{
return AbstractClass.Implementation1;
}
return this;
}
public bool Equals(Implementation2 other)
{
if (other == null)
{
return false;
}
return other.Name == this.Name;
}
}
public class Implementation3 : AbstractClass, IEquatable<Implementation3>
{
public IEnumerable<AbstractClass> Instances { get; private set; }
public Implementation3()
: base()
{
Instances = new List<AbstractClass>();
}
public Implementation3(IEnumerable<AbstractClass> instances)
: base()
{
if (instances == null)
{
throw new ArgumentNullException("instances", "error msg");
}
if (instances.Any<AbstractClass>(c => c == null))
{
thrown new ArgumentNullException("instances", "some other error msg");
}
Instances = instances;
}
public override AbstractClass Distill()
{
IEnumerable<AbstractClass> newInstances = new List<AbstractClass>(Instances);
// "Flatten" the collection by removing nested Implementation3 instances
while (newInstances.OfType<Implementation3>().Any<Implementation3>())
{
newInstances = newInstances.Where<AbstractClass>(c => c.GetType() != typeof(Implementation3))
.Concat<AbstractClass>(newInstances.OfType<Implementation3>().SelectMany<Implementation3, AbstractUnit>(i => i.Instances));
}
if (newInstances.OfType<Implementation4>().Any<Implementation4>())
{
List<AbstractClass> denominator = new List<AbstractClass>();
while (newInstances.OfType<Implementation4>().Any<Implementation4>())
{
denominator.AddRange(newInstances.OfType<Implementation4>().Select<Implementation4, AbstractClass>(c => c.Denominator));
newInstances = newInstances.Where<AbstractClass>(c => c.GetType() != typeof(Implementation4))
.Concat<AbstractClass>(newInstances.OfType<Implementation4>().Select<Implementation4, AbstractClass>(c => c.Numerator));
}
return (new Implementation4(new Implementation3(newInstances), new Implementation3(denominator))).Distill();
}
// There should only be Implementation1 and/or Implementation2 instances
// left. Return only the Implementation2 instances, if there are any.
IEnumerable<Implementation2> i2s = newInstances.Select<AbstractClass, AbstractClass>(c => c.Distill()).OfType<Implementation2>();
switch (i2s.Count<Implementation2>())
{
case 0:
return AbstractClass.Implementation1;
case 1:
return i2s.First<Implementation2>();
default:
return new Implementation3(i2s.OrderBy<Implementation2, string>(c => c.Name).Select<Implementation2, AbstractClass>(c => c));
}
}
public bool Equals(Implementation3 other)
{
// omitted for brevity
return false;
}
}
public class Implementation4 : AbstractClass, IEquatable<Implementation4>
{
private AbstractClass _numerator;
private AbstractClass _denominator;
public AbstractClass Numerator
{
get
{
return _numerator;
}
set
{
if (value == null)
{
throw new ArgumentNullException("value", "error msg");
}
_numerator = value;
}
}
public AbstractClass Denominator
{
get
{
return _denominator;
}
set
{
if (value == null)
{
throw new ArgumentNullException("value", "error msg");
}
_denominator = value;
}
}
public Implementation4(AbstractClass numerator, AbstractClass denominator)
: base()
{
if (numerator == null || denominator == null)
{
throw new ArgumentNullException("whichever", "error msg");
}
Numerator = numerator;
Denominator = denominator;
}
public override AbstractClass Distill()
{
AbstractClass numDistilled = Numerator.Distill();
AbstractClass denDistilled = Denominator.Distill();
if (denDistilled.GetType() == typeof(Implementation1))
{
return numDistilled;
}
if (denDistilled.GetType() == typeof(Implementation4))
{
Implementation3 newInstance = new Implementation3(new List<AbstractClass>(2) { numDistilled, new Implementation4(((Implementation4)denDistilled).Denominator, ((Implementation4)denDistilled).Numerator) });
return newInstance.Distill();
}
if (numDistilled.GetType() == typeof(Implementation4))
{
Implementation4 newImp4 = new Implementation4(((Implementation4)numReduced).Numerator, new Implementation3(new List<AbstractClass>(2) { ((Implementation4)numDistilled).Denominator, denDistilled }));
return newImp4.Distill();
}
if (numDistilled.GetType() == typeof(Implementation1))
{
return new Implementation4(numDistilled, denDistilled);
}
if (numDistilled.GetType() == typeof(Implementation2) && denDistilled.GetType() == typeof(Implementation2))
{
if (((Implementation2)numDistilled).Name == (((Implementation2)denDistilled).Name)
{
return AbstractClass.Implementation1;
}
return new Implementation4(numDistilled, denDistilled);
}
// At this point, one or both of numerator and denominator are Implementation3
// instances, and the other (if any) is Implementation2. Because both
// numerator and denominator are distilled, all the instances within either
// Implementation3 are going to be Implementation2. So, the following should
// work.
List<Implementation2> numList =
numDistilled.GetType() == typeof(Implementation2) ? new List<Implementation2>(1) { ((Implementation2)numDistilled) } : new List<Implementation2>(((Implementation3)numDistilled).Instances.OfType<Implementation2>());
List<Implementation2> denList =
denDistilled.GetType() == typeof(Implementation2) ? new List<Implementation2>(1) { ((Implementation2)denDistilled) } : new List<Implementation2>(((Implementation3)denDistilled).Instances.OfType<Implementation2>());
Stack<int> numIndexesToRemove = new Stack<int>();
for (int i = 0; i < numList.Count; i++)
{
if (denList.Remove(numList[i]))
{
numIndexesToRemove.Push(i);
}
}
while (numIndexesToRemove.Count > 0)
{
numList.RemoveAt(numIndexesToRemove.Pop());
}
switch (denList.Count)
{
case 0:
switch (numList.Count)
{
case 0:
return AbstractClass.Implementation1;
case 1:
return numList.First<Implementation2>();
default:
return new Implementation3(numList.OfType<AbstractClass>());
}
case 1:
switch (numList.Count)
{
case 0:
return new Implementation4(AbstractClass.Implementation1, denList.First<Implementation2>());
case 1:
return new Implementation4(numList.First<Implementation2>(), denList.First<Implementation2>());
default:
return new Implementation4(new Implementation3(numList.OfType<AbstractClass>()), denList.First<Implementation2>());
}
default:
switch (numList.Count)
{
case 0:
return new Implementation4(AbstractClass.Implementation1, new Implementation3(denList.OfType<AbstractClass>()));
case 1:
return new Implementation4(numList.First<Implementation2>(), new Implementation3(denList.OfType<AbstractClass>()));
default:
return new Implementation4(new Implementation3(numList.OfType<AbstractClass>()), new Implementation3(denList.OfType<AbstractClass>()));
}
}
}
public bool Equals(Implementation4 other)
{
return Numerator.Equals(other.Numerator) && Denominator.Equals(other.Denominator);
}
}
The heart of what I am trying to test is the Distill method, which as you can see has the potential to run recursively. Because a stubbed AbstractClass is meaningless in this paradigm, it breaks the algorithm logic. Even trying to test for a stubbed class is somewhat useless, since there is little I can do about it other than throw an exception or pretend that it is an instance of Implementation1. I would prefer not to have to rewrite the code under test to accommodate a specific testing framework in that way, but writing the test itself in such a way as never to stub AbstractClass is what I am trying to do here.
I hope it is apparent how what I am doing differs from a type-safe enum construct, for instance. Also, I anonymized objects for posting here (as you can tell), and I did not include all methods, so if you're going to comment to tell me that Implementation4.Equals(Implementation4) is broken, don't worry, I'm aware that it is broken here, but my actual code takes care of the issue.
Another edit:
Here is an example of one of the factory classes. It is in the Factories directory of the Pex-generated test project.
public static partial class Implementation3Factory
{
[PexFactoryMethod(typeof(Implementation3))]
public static Implementation3 Create(IEnumerable<AbstractClass> instances, bool useEmptyConstructor)
{
Implementation3 i3 = null;
if (useEmptyConstructor)
{
i3 = new Implementation3();
}
else
{
i3 = new Implementation3(instances);
}
return i3;
}
}
In my factory methods for these concrete implementations, it is possible to use any constructor to create the concrete implementation. In the example, the useEmptyConstructor parameter controls which constructor to use. The other factory methods have similar features. I recall reading, though I cannot immediately find the link, that these factory methods should allow the object to be created in every possible configuration.
Source: (StackOverflow)
Suppose there is a method like this (C#):
public static int Add(int x, int y)
{
return x + y;
}
If the sum does not fit into the int data type it is probably an error situation which is worth a unit test. Is Pex able to identify such errors and generate unit tests for those?
Source: (StackOverflow)
After much playing i have managed to get TFS to be able to run builds on all of my projects except the test projects. These have a moles assembly in them.
The errors i am getting are
C:\Program Files\Microsoft Moles\bin\Microsoft.Moles.targets (79):
The command ""C:\Program Files\Microsoft Moles\bin\moles.exe"
@"C:\Builds\2\cv2\DevBranchBuild\Sources\CV.BL.Tests\obj\Release\Moles\moles.args""
exited with code -1002.
The warning is
C:\Windows\Microsoft.NET\Framework\v4.0.30319\Microsoft.Common.targets (1360):
Could not resolve this reference.
Could not locate the assembly "CV.DAL.Moles, Version=1.0.0.0, Culture=neutral, processorArchitecture=MSIL".
Check to make sure the assembly exists on disk.
If this reference is required by your code, you may get compilation errors.
CV.DAL is the assembly i am moling.
I am using the latest version of pex and moles.
I also have the CV.DAL.moles file checked into TFS (not the assembly).
Everything builds perfectly fine on my local machine. The machine I am using to do the builds has exactly the same configuration. By exactly I mean that one is 32 bit and the other is 64 bit :o (serious oversight there).
Any ideas?
This makes pex and moles utterly useless as it is breaking gated checkins when merging branches which is a great shame as i invested a decent amount of time into them.
EDIT
Which molesAssemblies directories?
The install folder for moles or the directories under the project?
The installation folder has he same files and file sizes on both machines.
Source: (StackOverflow)
I have a simple method with a single condition like this.
if (String.IsNullOrEmpty(FirstName))
{
success = false;
}
return success;
When I run Pex it generates only one test case which assigns Null to FirstName property and the other with assigns "\0" to the FirstName.
Why is it not generating a third method which will assign string.Empty to the FirstName property?
Source: (StackOverflow)
I am interested in finding out more about the use of Pex, the testing framework for .NET.
I am aware that the current version of Pex is only 0.11, but is anyone aware of an open source project that currently makes use of it?
Thanks, MagicAndi.
Update - Results So Far
Source: (StackOverflow)
I am aware that the current release of PEX allows use with the NUnit framework.
The thing is I can't configure it to use the NUnit framework.
In the Options -> PEX: What Exactly do i have to put in the "TestFramework" and "TestFrameworkDirectory".
I Have tried many combinations but failed to achieve any success.
Source: (StackOverflow)
Hi:
I've enjoyed playing around with Pex and Moles...now it's time to use it in earnest to cover an legacy app as much as possible.
We've started by converted to .NET 3.5 (the client can't yet host in .NET 4.0)
We noticed that we are having trouble with getting Moles to work.
Seems to work if the Test Project assembly is a .NET 4.0 -- but when converted the Unit Tests assembly framework back to .NET 3.5 (after installing Visual Studio 2010 SP1 to give that ability) it stops working.
Keeps complaining with "you can only run tests using the default host adapter from a test assembly that targets .NET framework 3.5"
If the client wants everything to be in .NET 3.5 -- including the unit tests...any ideas?
Source: (StackOverflow)
I'm trying to start using Pex, and I have certain code that I want it to ignore testing.
I create configuration sections for config files using the Configuration Section Designer addin. Unfortunately the code generated is not quite perfect, because it doesn't do tests for nulls and other nice checks. However, for now at least I want the code to be ignored when running pex explorations as I can't change the code without it being overwritten in future, and it's a known fault that we can work around.
I found the PexInstrumentMarkedByand the PexCoverageFilterMarkedBy attributes, which seem like they may do the job (of ignoring code with the GeneratedCodeAttribute), but as far as I cam see I would need to put those in my assembly, and thus have a reference to the Pex framework in my operational assembly... not going to happen.
Does anyone have better ideas?
Source: (StackOverflow)
I have been trying using Pex in a F# project but I have faced several issues on which I would appreciate some help:
Visual Studio Pex addin worked for half an hour and not for complex project exploration.
After that each time I run a Pex Exploration from within Visual Studio 2010 I get a !warning! [metadata] no explorations found after applying all filters; did you forget a [PexClass] or [PexMethod] attribute? error message. If I copy and paste the command run by visual studio when doing the exploration and remove some attributes (like sourceFilter ...) and run it from a command console then I get some generated tests. Any idea why visual studio generates an invalid command?
After a while the Microsoft.Pex.Framework.dll either disappears from the .Net Reference list and I have to reference it manually using its file path or I can't even reference it any more and I have to uninstall/reinstall Pex. (A reference to C:\Program Files (x86)\Microsoft Moles\PublicAssemblie\Microsoft.Pex.Framework.dll could not be added. Please make sure that the file is accessible, and that it is a valid assembly or COM component.)
All these issues don't occur when I work on a C# project.
Issues 1 2 3 occurred with the Academic version.
Using the version available for MSDN Subscriber I never got it working within visual studio (Visual Studio 2010 Ultimate NO SP1). However it worked from the command prompt. I did not face problem 3 with this version.
Has anyone managed to get Pex working with F# within visual studio ? If yes can you you tell me how you did it ? Which versions (VS and Pex) do you use ?
Source: (StackOverflow)
When it comes to testing I've backed into it. I'm now a huge fan, and often do something like TDD. In VS 2010 for design and regressions I've just used MSTEST. This is great - all my projects are open source and connected to my work anyway (physics research @ university). I then got into Pex which helped a lot find all sorts of edge cases I'd not been looking for. In general, this has been a revolution in how I code. And this has introduced me to Moles. Also very cool (but I'm still learning).
Unfortunately, I have access only to the Pro version of VS (that is what my university gets as a site license). The result is that Moles no longer works, and the upgrade, Fakes won't either. And I would imagine that the next version of Pex will rely on Fakes, make it also an Ultimate-only option.
So, I find myself now looking for open source tools that have similar functionality. Is there an article somewhere or a resource that compares some of the more mature OSS projects out there? I find myself at a bit of a loss (i.e. too many options). While I love this mocking ability, it is a tool, and not what I want to spend my spare time playing with (which is where most of my .NET programming occurs).
I assume that Pex will be impossible to replace without spending real money.
And while I'm at it, it seems like perhaps I should look to see if there are other testing frameworks that are better than MSTEST. MSTEST is pretty good, but is starting to feel a bit limited (I've not explored any new features in VS2012, if there are any).
Thanks a lot for the help, in advance, and really, normally I try to ask a more specific question, but I'm a bit at sea here.
Source: (StackOverflow)
I don't see any current information about the plans to make Pex work on Visual Studio 2012.
Source: (StackOverflow)