15 comments
.NET 
Recently I ran into the following situation at a customer that is implementing Team Foundation Server as the ALM tool of choice.
A BI team had developed a so called “screening framework” which they use to automatically test the SSIS packages they build. The screening framework they build allowed them to specify, store and run tests using a web interface. However the team also started using Team Foundation Server (TFS) and Microsoft Test Manager (MTM) and they really wanted to integrate their custom testing framework with TFS.
As the testers were also using MTM to run manual tests it would be great if the testers were able to run the automated tests from within MTM also by simply clicking the “Run” button. TFS allows you to link a piece of code to your test case, also known as an associated automation, so achieving the above should be quite straight forward. This would also allow them to use their custom in a “Lab Build” in the future, something like this should work:
[TestMethod]
public void RunScreeningWithId1()
{
ScreeningTest test = new ScreeningTest(1);
bool result = test.Run();
Assert.IsTrue(result);
}
However this would mean that for every test you want to run a new test method must be created. TFS allows you to enter parameters in a test case, wouldn’t it be great if we could pass these parameter to our test method? We could then create a single test method to run a test and use the test parameter to pass the id of the actual test.
Since Visual Studio 2012 Update 1 the Test Controller and Test Agent post a wealth of TFS related information in the TestContext class, as my colleague Marcel de Vries describes in this post. So lets take my previous example to the next level and retrieve the test case associated with our test method from TFS.
When a test method is executed TFS using a test agent the TestContext object’s property dictionary is filled with TFS related information, Marcel posted an elaborate table but we are interested in the following 3 properties for this scenario:
| Property | Value |
| __Tfs_TestCaseId__ | 1 |
| __Tfs_TeamProject__ | TestProject |
| __Tfs_TfsServerCollectionUrl | http://localhost:8080/tfs/DefaultCollection |
We can access these properties, retrieve the test case from TFS and print the values for a parameter using the following piece of code:
[Test Method]
public void TestMethod()
{
//just to be safe
if(TestContext.Properties["__Tfs_TestCaseId__"] == null)
throw new ArgumentNullException("__Tfs_TestCaseId__");
if(TestContext.Properties["__Tfs_TeamProject__"] == null)
throw new ArgumentNullException("__Tfs_TeamProject__");
if(TestContext.Properties["__Tfs_TfsServerCollectionUrl__"] == null)
throw new ArgumentNullException("__Tfs_TfsServerCollectionUrl__");
int testCaseId = Convert.ToInt32(TestContext.Properties["__Tfs_TestCaseId__"]);
string teamProject = TestContext.Properties["__Tfs_TeamProject__"];
string collectionUrl = TestContext.Properties["__Tfs_TfsServerCollectionUrl__"];
//We'll get to this method in a bit
ITestManagementTeamProject testManagementTeamProject = GetTeamProjectTestManagementTeamProject(collectionUrl, teamProject)
ITestCase testCase = testManagementTeamProject.TestCases.Find(testCaseId);
//We'll get to this method in a bit
PrintParameterValues(testCase, "parameter1");
}
In the example above I used this helper method to connect to TFS and retrieve an instance of “ITestManagementTeamProject”:
private static ITestManagementTeamProject GetTeamProjectTestManagementTeamProject(string tpcUri, string teamProjectName)
{
TfsTeamProjectCollection collection = TfsTeamProjectCollectionFactory.GetTeamProjectCollection(new Uri(tpcUri));
collection.Connect(Microsoft.TeamFoundation.Framework.Common.ConnectOptions.IncludeServices);
ITestManagementService service= collection.GetService<ITestManagementService>();
ITestManagementProject project = service.GetTeamProject(teamProjectName);
return project;
}
And this helper method to print the parameter values:
private void PrintParameterValues(ITestCase testCase, string parameterName)
{
foreach(DataRow row in testCase.DefaultTableReadOnly.Rows)
{
string value = row[parameterName];
Console.WriteLine(parameterName + " value: " + value;
}
}
The code above is quite straight forward, using the Client Object Model of TFS we are able to easily retrieve the required information from TFS. One thing to keep in mind when using this code, the test agent executing this code must run under an account that has at least read permission on the team project you are trying to read test cases from.
To wrap up this post, for the BI team I made a wrapper class wrapping ITestCase and exposing a method to retrieve the parameter values for a specific parameter. Also I created an extension method for the TestContext object so they could do something like the following:
TestCaseWrapper wrapper = TestContext.GetTfsTestCase()
wrapper.GetParameterValues("parametername")
Update:
There seem to be some issues with how TFS publishes information into the TestContext, this leads to issues when you run multiple tests at once as indicated by Ish (thanks for the heads-up!). Basically you will only ever get one TestCaseId back in the TestContext, even if you run multiple tests as part of a test run. A colleague of mine ran into the same issue a few weeks aggo and already found a workaround for it. Check out his post:
15 comments
Hello Team,
I have created automation script in coded UI and associated that script to multiple test cases in MTM and now run those scripts on the basis of parameters passed through MTM.
Now when I am running all the test cases from MTM than it is taking the last test case parameter and run all scripts on that parameter.
Also, if I run single single script from the MTM it is running fine and perfect.
Please advise how I do that to run all the test cases with their specific parameter.
I have used the above concept and it is working fine for running single test case.
Regards,
Ish Grover
Ish Grover
Hello Ish,
What exactly are you trying to accomplish? Do you want to run all test cases from MTM (so select multiple automated “Test Case” items and run them in bulk) or do you want to have 1 “Test Case” item executing multiple tests in it’s automation. Can you maybe attach an example of what your are trying to do? Then I can have a look for you.
Sander Aernouts
Thanks for your reply.
Please find my scenario what I am trying to accomplish.
I have 3 test cases in MTM i.e. Scenario1, scenario2, scenario3.
Scenario1 has parameter :ScenarioID:4
Scenario2 has parameter :ScenarioID:5
Scenario3 has parameter :ScenarioID:6
When I select all the test cases and run those test cases with options.
Test cases run on test agent successfully. But all the test cases are running with parameter of Scenario3 i.e. parameter with value 6.
However when i am running single single test cases than it is taking its parameter only.
I dont know how to resolve this issue. Please help.
Ish
Hello Ish,
Just to make sure I understand you fully, you have 3 seperate test cases using a @ScenarioID parameter with different values. If you run these tests seperately it will use the correct value for ScenarioID but if you select all tree test cases and run them at the same time only one of the 3 values will be passed down into your code. I assume you are reusing the same [TestMethod] for all tree test cases?
Could you possible share a snippet of your code with me? Looks like the test context is not reloaded an thus always returns the same TestCaseID and thus will only retrieve parameters from on of your test cases, which would be strange. Could you also please verify that you have 3 seperate test case work items linked to the same [TestMethod] using the associated automation tab on the test case work item.
Sander Aernouts
I am working on 2013 of both the version.
Please find my scenario what I am trying to accomplish.
I have 3 test cases in MTM i.e. Scenario1, scenario2, scenario3.
Scenario1 has parameter :ScenarioID:4
Scenario2 has parameter :ScenarioID:5
Scenario3 has parameter :ScenarioID:6
When I select all the test cases and run those test cases with options.
Test cases run on test agent successfully. But all the test cases are running with parameter of Scenario3 i.e. parameter with value 6.
However when i am running single single test cases than it is taking its parameter only.
Ish Grover
Yes. Your assumptions are correct.
Yes I am reusing the same test method in all the test cases.
Yes I have verified that, my all test cases have the same test method associated with it.
Please find the code snippet:
[TestMethod]
public void RunScenario()
{
int testCaseId = Convert.ToInt32(TestContext.Properties[“__Tfs_TestCaseId__”]);
string teamProject = TestContext.Properties[“__Tfs_TeamProject__”].ToString();
string collectionUrl = TestContext.Properties[“__Tfs_TfsServerCollectionUrl__”].ToString();
//We’ll get to this method in a bit
ITestManagementTeamProject testManagementTeamProject = GetTeamProjectTestManagementTeamProject(collectionUrl, teamProject);
ITestCase testCase = testManagementTeamProject.TestCases.Find(testCaseId);
CarletonRegression(PrintParameterValues(testCase));//this method is sending the parameter to other method and on this basis code will work.
}
Ish Grover
Just for good measure, do have the latest update of VS2012 agents/TFS installed? Should be Update for if they released that for the agents for VS also.
Do you use a singleton pattern or statics in your CarletonRegression method? I have never seen the parameters “hanging” between different test runs.
Sander Aernouts
I am also not understanding this issue, all the time my code should have to use the different parameter.
My carletonRegression method is not static and this method is created in the same class where RunScenario is created.
Ish Grover
Hi Ish,
I checked your issue with a colleague of mine and guess what, he just fixed this problem a few weeks ago. Apparently the TestContext stalls after the first batch of information is pushed into it, meaning that you will only ever get the information of one Test Case in the TestContext. There is a way arround this by using the TestRunId and polling TFS for the TestRuns that are in progress for your machine, this will then give you the ID of the test case that is being executed. As the results are published to TFS async there will be a delay in your test runs, I believe it takes about 30 seconds for the results to be pushed into TFS.
The solution is quite extensive but it should give you exactly what you are looking for, check out his blog post here: https://blogs.infosupport.com/switching-browser-in-codedui-or-selenium-tests-based-on-mtm-configuration-part-2/, kudos to Marcel for figuring this one out!
Sander Aernouts
Thanks for your reply.
I am still trying for our case but it is not working properly.
Can you please create a sample code for me. I will be thankful to you.
Ish Grover
Thanks for your help.
I tried by myself and now it is working at my end.
Thanks a lot again.
Ish Grover
Ish, Can you please share your peice of code with me. I am trying to achieve some thing similar on MTM 2013. email id is singhai.saurabh@gmail.com
Thanks
Saurabh
Hi Ish,
I am trying to achieve same functionality on MTM 2012. Can you please share your piece of code with me. email id : aamirsyed10@live.com
Thanks
Aamir
Hello,
nice idea, thank you.
One question – how can I load Tfs data to the TestContext in the 1st place? Do i need to reference any TeamFoundation library first to be able to get Tfs related data?
Robert
Hi Robert,
TFS, or actually the Test Agent, does this for you when you run your tests using as automated tests from within an MTM test plan. You will need to add your tests as associated automation to a test case for this to work.
Sander Aernouts