How to fix the scenario of No tests to execute via MSTEST in TFS build log?

On June 13, 2012, I wrote a post about How to fix the scenario of No tests to execute via MSTEST in TFS Build 2010?  But I forgot to mention another important thing. You might find this post helpful to you if you have the same issue.

On June 12, 2012, I created a test project that contains unit tests and web tests.  In Visual Studio 2010, I created a new test list called BVT and put all of the tests in BVT and then I checked in the code to TFS source control. After I kicked off a new build in TFS, the build succeeded, but all of the tests in BVT never got executed in the build log (No Tests to execute.)

I logged in to the build server and opened the .sln file of my test project. The interesting part was that all of the tests in BVT were not enabled. I needed to manually click refresh in the test list editor to make them enabled. How did I make those tests enabled in BVT Test List without clicking refresh manually? 

After long investigation , I must check in the bin folder and can’t check in the obj folder. This is  very important!  If I put more tests in BVT, I need to check in bin folder again to make those tests changed. 

After I kicked off a new build again, all tests passed in the build log. After logging to the build server again, all tests were enabled in BVT Test List as well.

Coded UI tests have the huge limitation for Silverlight Application and cross browser testing

For Silverlight 5 Application, Coded UI test is not supported in Visual Studio 2010, Visual Studio 11 and Visual Studio 2012 RC.

For Silverlight 4 Application in Firefox, Coded UI test is only supported in Firefox 3.5 and 3.6.  For higher version of Firefox, it is not supported.

For Silverlight Application in Chrome, Coded UI test is not supported.

For Silverlight Application in Safari, Silverlight is never supported on Windows platform. It is only supported on Mac OS. However, Coded UI test is not supported for Safari.

 As a SDET, I would like to say this is not good news for Coded UI test because it has the huge limitation for Silverlight Application and cross browser testing.

Telerik Test Studio can handle Silverlight 5 Application in different browser easily.

Coded UI tests don’t support Silverlight 5 at this moment

I have used Telerik Test Studio to do Silverlight UI Automation more than 2 years. It is a very powerful tool. In my last Microsoft Project, I tried to use Coded UI tests for Silverlight 4 applications, and I couldn’t record and run tests very well. However, Telerik Test Studio worked very well. My current project is related to Silverlight 5, and Telerik Test Studio is still a powerful tool.

Today I would like to try Coded UI tests to test Silverlight 5, but the non-supported dialog box appears as follows:

After doing some research, Coded UI tests in Visual Studio 2010 and Visual 11 don’t support Silverlight 5. I give up Coded UI tests and still use my favorite testing tool -Telerik Test Studio.

Just one thought: If I can’t use Coded UI Tests for Silverlight 5, I don’t think Microsoft Test Manager can play an important role in the testing tool.

Unhandled Error in Silverlight Application - ESRI.ArcGIS.Client.Layer.OnInitializationFailed

Our Silverlight Project is based on ArcGIS Map Services and the site is for internal use only in the company.

I can use my local machine to access the sites outside the company successfully such as google.com , but for Windows Server 2008 R2 machines,  I can only browse the internal sites including our Silverlight Application.

When I launch our Silverlight Application, the site is good and has no JavaScript error. But when I run my Silverlight UI automation on Windows Server 2008 R2 machine, the site has the following JavaScript error and it is related to ArcGIS error.

Message: Unhandled Error in Silverlight Application A security exception occurred while trying to connect to the REST endpoint. Make sure you have a cross domain policy file available at the root for your server that allows for requests from this application.   at ESRI.ArcGIS.Client.Layer.OnInitializationFailed(EventArgs e)
   at ESRI.ArcGIS.Client.ArcGISTiledMapServiceLayer.<>c__DisplayClass1a.b__18()

After investigation, our map service needs to have the Internet connection first. When I get the values of proxy server configuration from my QA manager and configure it correctly in IE, the JavaScript error is gone after launching Silverlight Application again.

MSBUILD needs to build the solution file successfully in the Build Controller

Every time when I go to MSDN forum, I see some people asking why after they queue a new build in TFS 2010 and TFS build fails due to some errors. That’s because there are missing components in the Build Controller. The environment in the Build Controller is different from the one in your local machine.

The best way for you is to login to the Build Controller and validate if you can use MSBUILD to build the solution file successfully.

cd C:\Windows\Microsoft.NET\Framework\v4.0.30319\

msbuild C:\Test\TestA\TestA.sln

If the above simple MSBUILD command line has the error, whatever you queue a new build in TFS, TFS build will fail.

When you see the errors, you can copy the missing components from your local machine to Build Controller and try it again.

How to fix the scenario of No tests to execute via MSTEST in TFS Build 2010?

Currently I am working on TFS build 2010. I believe many of you have the same situation - the Build Controller doesn't have Visual Studio 2010 installed because it is the license issue.

In order to run some of my BVTs (2 unit tests and 1 coded web test) via mstest from TFS build, I need to install Visual Studio Test Agent 2010 to get mstest.exe first per the MSDN forum.

After kicking off a new TFS build, the interesting part is that TFS build can only execute 2 unit tests. This coded web test is never executed and the test result never appears. I can only see 2 unit tests passed in the TFS activity log (the section: Run MSTest for Metadata File).

I log in to the Build Controller and use the following command line, but this coded web test cannot be found; but 2 unit tests passed.

MSTest.exe /nologo /testmetadata:"C:\\test\qa\qatfs.vsmdi" /testlist: BVTTFS

Test XXX cannot be found.

No Tests to execute.

In my local machine, I add one basic webtest (record google.com), put all project files in the Build Controller and execute this basic webtest via the following command line:

MSTest.exe /nologo /testmetadata:"C:\\test\qa\qatfs.vsmdi" /test:webtest1

This time it shows the error.

Failed to load tests from 'c:\test\qa\qatfs\webtest1.webtest': Microsoft.VisualStudio.TestTools.TestManagement.InvalidStorageExtensionException: File extension specified '.webtest' is not a valid test extension.

   at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.GetTestTypeInfosForExtension(String ext)

   at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.GetTestTypesNotManaged InStorage(String storage)

   at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.LoadTestsFromTipsHelper(IEnumerable`1 locations, ProjectData projectData)

   at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.LoadTests(IEnumerable`

1 locations, ProjectData projectData, TestConflictHandler vetoingHandler)

   at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.LoadTests(String location, ProjectData projectData, TestConflictHandler vetoingHandler)

   at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.LoadTestLinkStorageHelper.LoadTests(String fullStoragePath, ProjectData projectData)

   at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.SimpleLoadTestLinkStorageHelper.Load(String fullStoragePath, ProjectData projectData)

Test webtest1 cannot be found.

Test webtest1 cannot be found.

Starting execution...

No tests to execute.

It can explain why "No tests to execute" appears in my early testing.

After some testing, the above error of Microsoft.VisualStudio.TestTools.TestManagement.Tmi.XXXXX is gone after I install Visual Studio Test Controller in the Build Controller. It means that MSTEST can't execute webtest and coded web tests nicely if you only install Visual Studio Test Agent 2010. It could also explain why the log in TFS build has the information of No Tests to execute. 

Next time if you see "No tests to execute" in TFS Build, don't worry.  Installing Visual Studio Test Controller and Visual Studio Test Agent together in your Build Controller should solve your issue. 

Besides, does it mean if you want to execute your web tests via MSTEST without Visual Studio IDE install in the Build Controller ,Visual Studio Test Controller and Visual Studio Test Agent need to be installed in the same machine? From my perspective, the answer is yes.

How to handle Windows Authentication in Telerik Test Studio?

Our developers add Windows Authentication in our Silverlight Login Page. In my local machine, when I launch the Silverlight Application, no Windows Authentication popup appears. If I login to another machine, the Windows Authentication popup appears and using my username and password to login is required.

How to I handle Silverlight UI Automation via Telerik Test Studio? After doing some research, I use the following code to handle this scenario in IE 8 successfully.

var windowsAuth = LogonDialog.CreateLogonDialog(ActiveBrowser, "UserName","Password", DialogButton.OK);
Manager.DialogMonitor.AddDialog(windowsAuth);