When using TestNG factories, the result tree only shows one instance of each class and test methods executed instead of one node for each class instance and its accompanying test methods. Furthermore, the summary number of tests executed is also incorrect. Interestingly, the Statistics tab shows each test method executed, albeit without any indication to which class instance they belong.
Expected behavior: The result tree should show one node for each class instance. If the class implements ITest, then each class instance node should have the return value of getTestName() appended to it. (This is standard TestNG behavior.) The summary number of tests should reflect the correct number of tests executed.
Steps to reproduce:
1) Create a project and add the attached source to it.
2) Include the testng jar as a project dependency.
3) Right-click the class in the attached source file (FactoryProblemTestngPluginTest) and choose Run.
The Run frame comes up. Note that the Test Results tree only has one test method in it. (There should be two.) Also, the summary reads: "Done: 1 of 1 Failed: 1", which is also incorrect.
4) Click on the Statistics tab. There you will see both tests.
You can also bring up the index.html file in the test-output directory of your project. From there, click Custom Suite and then click Results (on the lower right of the frame). In the right frame, you will see both tests. If you hover on the method names, you will see the class instance name appendages.