adrift in the sea of experience

Friday, February 25, 2011

Executing visual studio 2010 unit tests without installing visual studio

In a previous post I already explained how to get your unit tests running on a build machine with mstest.exe, without having to install Visual Studio 2008. We recently upgraded to Visual Studio 2010 so I had to repeat the exercise.

This time I wrote a batch script which does most of the work. Just run the script on a machine where Visual Studio 2010 is installed, and it will create a "mstest" folder with the necessary binaries and registry entries. Copy the folder to your build machine, prepare the registry by executing mstest.reg and you're good to run mstest.exe like this:

mstest /noisolation /testcontainer:"path\to\testproject.dll"

This will return an error code if any of the tests fail.

As an anonymous commenter on my previous post pointed out, you should be careful of the license implications. In our case it is most likely OK because we have a site license. But even so it can be useful to avoid a full Visual Studio install: it saves a lot of disk space, and you don't have to spend time babysitting the installation. Or multiple installations, if you have a cluster of build machines!

update: thanks to Frederic Torres for pointing out that the script doesn't work on a 64-bit Windows, and suggesting a fix! It should work now. Apparently there are still problems with 64-bit machines. But I don't have a 64-bit machine without VS2010 to test with, so I'll have to leave it as an exercise for the reader...

9 comments:

Thomas Schreiner said...

Thanks for the Post. But I can't figure out where on the BuildServer I have to put the MSTest directory. That results in the inability to build our solution on the buildserver as msbuild does not find the MsTest-Assemblies and fails.

So where do I put the mstest folder or how do I tell MsBuild where the MSTest assemblies are located?

Thanks,
Thomas

Wim Coenen said...

@Thomas: in my case I have a batch script which invokes mstest.exe, logs the output and checks the return code. I don't use an msbuild task to execute the tests.

Wim Coenen said...

@Thomas: I'm not sure which msbuild task you are using, but with "Exec" you should be able to do the same thing as my batch script.

Sneha Saggam said...

This solution worked for me. But i am running mstest with testlist and testmetadata switches. When I run mstest without visual studio on some other box, I get the error that /testlist and /testmetadata are invalid switches.

Wim Coenen said...

@Sneha: to enable those switches, you can copy the registry entries at HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VisualStudio\10.0\Licenses

You can use the export/import in regedit to copy them all at once.

Sneha Saggam said...

@Wim - I was able to run the fix on a 32-bit machine, but the fix is not working on a 64-bit machine.

Thanks,
Sneha

Wim Coenen said...

@Sneha: on a 64-bit machine, the registry entries for 32-bit applications are in another location:
HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\10.0\Licenses

So if you export the entries from a 32-bit machine, you have to edit the file to insert "Wow6432Node\" in the keys before importing on a 64-bit machine.

That being said, I don't have a 64-bit machine without Visual Studio so I can't test any of this.

Sneha Saggam said...

@Wim - Changing the registry keys made it work on a 64-bit machine.
Thanks for the information.

Sneha

Armando Jr. said...

I'm having the following problem:
Failed to initialize the unit test extension 'CodedUITestAttribute': Failed to create an instance of the TestClassExtensionAttribute ('Microsoft.VisualStudio.TestTools.UITesting.CodedUITestAttribute, Microsoft.VisualStudio.QualityTools.CodedUITestFramework, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a') for the unit test extension 'CodedUITestAttribute': Could not load file or assembly 'Microsoft.VisualStudio.QualityTools.CodedUITestFramework, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.
bin\release\prjcco.dll
Unable to load the test container 'bin\release\prjcco.dll' or one of its dependencies. Error details: System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.VisualStudio.TestTools.UITesting, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.

can u help me?
i try set CopyLocal=true, but don't work!!! :/