Yesterday, march 6th 2012, my colleague Barri Jansen and I attended Valori's "thema avond" (theme night). The subject for the evening was "New generation software for automated testing". The event was held at Microsoft headquarters in the Netherlands, which is located almost on top of the runway at Amsterdam's Schiphol airport.
During the evening there where 2 presentations. One was from Microsoft and it was about Visual Studio 10 & Team Foundation Server, and the other one was from TrendIC and it was about Citratest.
The first presentation was given by Gerard van der Pol and Jan Willem van Buuren both from Microsoft. Their subject was Test support within Visual Studio and Team Foundation Server.
The product Test Professional 2010 was being demoed. They showed the point of view of both tester and developer.
Testers can test software manually with a sort of help screen / script window on the left side of the screen while the application being tested remains visible on the right (there is also an option to start automated tests and report the results back). Testers have the ability to mark each step in a test case as passed or failed. If a test step fails then the tool can add screenshots and/or video of the whole test case into a bug report.
This bug report is called a rich report because of the extra information it contains. It will indicate to developers which build was being tested but also which applications / services were running on the computer, which OS and browser it ran on, etc. All automatically. Developers get lots of useful information to facilitate fixing the problem.
The bug report is accessible from Visual Studio, the developer software. The bug can be fixed from there and the tester will get an automatic notification that the issue is ready to be re-tested. The tester opens the bug report and can simply 'fast forward' the test to the point of the previous failure. This means he doesn't have to enter the previous steps all over again. It goes without saying that this Microsoft application connects with their family of applications. As an example, it uses Sharepoint to share reports with team members and managers.
For more information and screen shots please visit the Microsoft website.
The second presentation was held by René Hazelaar and Marcel Diepenbroek both from TrendIC. Their subject was Citratest. Interfacing within difficult applications.
The main line / joke in the whole presentation was "If Marcel's mother can do it, Citratest can automate it". So if Marcel can tell his mother to click on a icon on the desktop (or go to Start -> Tools, etc), Citratest can also do it. It uses OCR (Optical Character Recognition) to recognize images and text and identifies similar images on screen.
Citratest use the GUI to navigate through applications. It runs native on Windows but it can open VMware and Citrix or similar applications so it can also be used on a huge range of Operating systems and browsers. The major advantage of Citratest is that it doesn't need to know what programming language the application was written in. It only uses the visible areas and can do anything a normal user would do. That means the tests won't fail the instant something under the hood has changed.
The application also offers some standard reports. A useful report we were shown was individual test step duration. Using it, you can easily compare specific test steps. So instead of just an overview of total duration time, you can spot potential bottlenecks.
For more information and screenshots please visit the TrendIC website.
Derk-Jan de Grood of Valori captured some parts of the evening on video which you can view below (Dutch only).
Both tools are meant for different user groups. Microsoft's Test Professional is more suitable for development companies with in-house testing capabilities. If you are running Microsoft products already this tool will fit in nicely. If you want to use only the Test Professional product and integrate it with some other (open source) software / frameworks you may face some considerable challenges. Have you integrated Microsoft's Test Professional with open source software? Please let us know why and how in the comments section below.
Citratest on the other hand seems to be meant for user story testing, acceptance testing and possibly integration testing. The tool looks really promising as far as functionality and features are concerned but falls short in regards to look & feel and possibly also general usability. However if it's only used by "techies" (as they call technical testers) that shouldn't be a big problem.
Both tools aren't cheap but if you want good software you should be ready to pay the price.
We would like to thank the people at Valori again for inviting us and for their hospitality. We also would like to thank the people at Microsoft for their inspiring location, their presentation room and their splendid catering!
Please share your thoughts and experiences with us using the comments section