Automated Report Generation for CI Execution Lists using TC-Shell & TQL
This Article will provide step-by-step instructions to automate the generation/print of a Test Execution Report that is limited only to the Execution lists identified for Continuous Integration.
In a Tosca CI setup we often need a report generated automatically post the Execution completion. But when there are multiple Execution lists that needs to run, this might not be straight-forward to locate all relevant Execution lists in order to Print/Generate a Report.
If Tosca CI is set-up via the ToscaCIRemoteExecutionService, then we know that the Execution Lists can be identified using the property "ContinuousIntegration" and any other custom property that was defined in the CITestExecutionConfiguration.xml file. Please refer the link(s) in Reference section for more details on how to set-up Tosca CI via the ToscaCIRemoteExecutionService.
The idea is to create a Test Case that will trigger a .bat file. This Test Case will be run as part of the CI Execution lists but strictly as the last one to be run. And when the .bat file runs, it will execute the TC-Shell commands along with a TQL enclosed within it.
This will retreive the CI Execution Lists and Print a Report out of them.
Please follow the below steps to achieve this.
Make sure all pre-requisite activities to Create automated reports via TCShell are performed as explained in this page.
Prepare the TC-Shell script file (.tcs file) that will be executed when the .bat file is triggered. Lets name it as Test.tcs for example.
This Test.tcs file will contain the below commands.
For "=>SUBPARTS:ExecutionList[ContinuousIntegration==\"True\" AND TestforBuildA==\"True\"]" TaskOnAll "Print Report... ExecutionEntries with ActualLog"
- The first line instructs Tosca to navigate to the Parent Execution List Folder ExecutionLists.
- The second line contains a TQL that fetches all Execution Lists having the properties ContinuousIntegration and TestforBuildA set to True. Then the task Print Report... ExecutionEntries with ActualLog is executed on all of the TQL Result objects using the command TaskOnAll. The ExecutionEntries with ActualLog is a standard Execution Report.
- The Last line contains the path where the Report needs to go and its name.
This is how the Properties will look like on the Execution List.
Refer the link(s) in Reference section for more details on Steering Tosca Commander via the Command Line (TCShell.exe).
Prepare the .bat file that will open the Tosca workspace via TCShell.exe and execute the commands from the Test.tcs file created in the above step. We'll name this file as PrintReport.bat for example.
The PrintReport.bat will contain the below commands.
TCShell.exe -workspace "C:\Tosca_Projects\Tosca_Workspaces\Tricentis_Local\Tricentis_Local.tws" -login "Admin" "Admin" "C:\Temp\Test.tcs"
The command timeout 60 will wait for 60 seconds before executing the next command. This wait will be required if Tosca CI connects to the same Tosca workspace that we are connecting via TC-Shell. This provides the CI process with some time to complete the whole logging and Checkin activities.
Create a Test Case in Tosca using the TBox Start Program module that will trigger the above PrintReport.bat file. Make sure the "WaitForExit" option on the TBox Start Program test step is set to "False" or left blank, so that the test case will be completed immediately after the PrintReport.bat file is triggered. Sample below.
Make sure the Test case created in the above step is run at the end of the CI Execution. This can be achieved by placing this Test Case as the last Execution Entry in the last-in-sequence CI Execution List.
Here's how it will work:
- At the end of the last CI Execution List's run, the PrintReport.bat file will be triggerred.
- This will run all scripts from the Test.tcs file to fetch required CI Execution Lists and will Print a standard Report out of them.
***The Property name and its value in the Test.tcs file created in Step 1 can be parameterised. One way to do it is via Tosca "TBox Read/Create File" module and make Tosca to create this .tcs file for you. Keep in mind that all special characters in the commands will need to be escaped accordingly when creating the .tcs file via Tosca.*** Subset of a sample test case to achieve this is attached in this page for reference!
***In case Tosca CI is implemented via Distributed Execution (DEX), you can utilise the below provided TQL to filter the Execution lists that are linked to a specific Test Event.***
For "=>SUBPARTS:TestEvent[Name=="TESTEVENTNAME"]->ExecutionListLinks->AllReferences:ExecutionList" TaskOnAll "Print Report... ExecutionEntries with ActualLog"
Set-up Tosca CI via the ToscaCIRemoteExecutionService:
Steering Tosca Commander via the Command Line (TCShell.exe):
Create automated reports via TCShell: