This site requires JavaScript to be enabled
IE BUMPER

Product
Version
IncidentLookup using list
Language
How can we help?

Solution Suggestions

Please enter your question to get suggestions.
  Help
IE BUMPER
Categories
Boolean Operators
OR or vertical bar symbol (|)
Finds a match if either the terms exist in a document (a union using sets).
AND
Finds a match if both terms exist in a document (an intersection of sets).
NOT, minus (-), or exclamation point (!)
Excludes documents that contain the term after NOT (a difference of sets).
Wildcards
Asterisk (*)
The asterisk symbol performs a multi character wildcard search.
Percent sign (%)
The percent sign performs a single character wildcard search.
Phrases
Quotation marks (")
Use quotation marks to search for an exact phrase with multiple terms.
For examples and more information look at our Knowledge Base: Advanced Search On The Support Portal

Automated Report Generation for CI Execution Lists using TC-Shell & TQL

This Article will provide step-by-step instructions to automate the generation/print of a Test Execution Report that is limited only to the Execution lists identified for Continuous Integration.

Situation

In a Tosca CI setup we often need a report generated automatically post the Execution completion. But when there are multiple Execution lists that needs to run, this might not be straight-forward to locate all relevant Execution lists in order to Print/Generate a Report.

Solution

If Tosca CI is set-up via the ToscaCIRemoteExecutionService, then we know that the Execution Lists can be identified using the property "ContinuousIntegration" and any other custom property that was defined in the CITestExecutionConfiguration.xml file. Please refer the link(s) in Reference section for more details on how to set-up Tosca CI via the ToscaCIRemoteExecutionService.

The idea is to create a Test Case that will trigger a .bat file. This Test Case will be run as part of the CI Execution lists but strictly as the last one to be run. And when the .bat file runs, it will execute the TC-Shell commands along with a TQL enclosed within it.

This will retreive the CI Execution Lists and Print a Report out of them.

 

Please follow the below steps to achieve this.

 

Step 1:

Make sure all pre-requisite activities to Create automated reports via TCShell are performed as explained in this page.

Prepare the TC-Shell script file (.tcs file) that will be executed when the .bat file is triggered. Lets name it as Test.tcs for example.

This Test.tcs file will contain the below commands.

 

jumpToNode "/BI-Testing/Execution/ExecutionLists"

For "=>SUBPARTS:ExecutionList[ContinuousIntegration==\"True\" AND TestforBuildA==\"True\"]" TaskOnAll "Print Report... ExecutionEntries with ActualLog"
C:\\Temp\\TestExecutionReport.pdf

 

  • The first line instructs Tosca to navigate to the Parent Execution List Folder ExecutionLists.
  • The second line contains a TQL that fetches all Execution Lists having the properties ContinuousIntegration and TestforBuildA set to True. Then the task Print Report... ExecutionEntries with ActualLog is executed on all of the TQL Result objects using the command TaskOnAll. The ExecutionEntries with ActualLog is a standard Execution Report.
  • The Last line contains the path where the Report needs to go and its name.

 

This is how the Properties will look like on the Execution List.

 

Refer the link(s) in Reference section for more details on Steering Tosca Commander via the Command Line (TCShell.exe).

 

Step 2:

Prepare the .bat file that will open the Tosca workspace via TCShell.exe and execute the commands from the Test.tcs file created in the above step. We'll name this file as PrintReport.bat for example.

The PrintReport.bat will contain the below commands.

 

timeout 60

TCShell.exe -workspace "C:\Tosca_Projects\Tosca_Workspaces\Tricentis_Local\Tricentis_Local.tws" -login "Admin" "Admin" "C:\Temp\Test.tcs"
 

The command timeout 60 will wait for 60 seconds before executing the next command. This wait will be required if Tosca CI connects to the same Tosca workspace that we are connecting via TC-Shell. This provides the CI process with some time to complete the whole logging and Checkin activities.

 

Step 3:

Create a Test Case in Tosca using the TBox Start Program module that will trigger the above PrintReport.bat file. Make sure the "WaitForExit" option on the TBox Start Program test step is set to "False" or left blank, so that the test case will be completed immediately after the PrintReport.bat file is triggered. Sample below.

Step 4:

Make sure the Test case created in the above step is run at the end of the CI Execution. This can be achieved by placing this Test Case as the last Execution Entry in the last-in-sequence CI Execution List.

Here's how it will work:

  • At the end of the last CI Execution List's run, the PrintReport.bat file will be triggerred.
  • This will run all scripts from the Test.tcs file to fetch required CI Execution Lists and will Print a standard Report out of them.

***The Property name and its value in the Test.tcs file created in Step 1 can be parameterised. One way to do it is via Tosca "TBox Read/Create File" module and make Tosca to create this .tcs file for you. Keep in mind that all special characters in the commands will need to be escaped accordingly when creating the .tcs file via Tosca.*** Subset of a sample test case to achieve this is attached in this page for reference!

 

***In case Tosca CI is implemented via Distributed Execution (DEX), you can utilise the below provided TQL to filter the Execution lists that are linked to a specific Test Event.***

TQL:

 

For "=>SUBPARTS:TestEvent[Name=="TESTEVENTNAME"]->ExecutionListLinks->AllReferences:ExecutionList" TaskOnAll "Print Report... ExecutionEntries with ActualLog"
C:\\Temp\\TestExecutionReport.pdf

 

 

[opt.] Reference

Set-up Tosca CI via the ToscaCIRemoteExecutionService:

https://support.tricentis.com/community/manuals_detail.do?lang=en&version=12.2.0&url=continuous_integration/set_up_ci_remote.htm

 

Steering Tosca Commander via the Command Line (TCShell.exe):

https://support.tricentis.com/community/manuals_detail.do?lang=en&version=12.2.0&url=tosca_commander/tcshell.htm

 

Create automated reports via TCShell:

https://support.tricentis.com/community/manuals_detail.do?lang=en&version=12.2.0&url=reporting/print_report.htm

 

Attachments
0 0

Ravi Kumar Sankaranarayanan

Created: 2019-07-23 12:39:03

Hi Arun,

Thanks for the article. I am able to follow upto Step3. In Step4, it is mentioned as below:

Make sure the Test case created in the above step is run at the end of the CI Execution. This can be achieved by placing this Test Case as the last Execution Entry in the last-in-sequence CI Execution List.

Do i need to create a new execution list and place this test case? If yes, how to trigger that execution list during CI execution from Jenkins?

Not sure on how to do that. Could you please share some screenshots for reference. It would be more helpful.

Thanks,

Ravi

Arun Jothi

2019-07-23 11:55:05 , Updates: 1, Last Update: 2019-07-23 11:56:23

Hi Ravi,

You don't necessarily need a new Execution list, but can add an additional test case to your current Execution list.

I have attached a new document that has step by step instructions to set-up Tosca CI via Jenkins and trigger the execution lists.

More details can be found in the Tosca Manual. Click here.

Please let me know if this helps.

 

Thanks,

Arun Jothi

0 0

Ravi Kumar Sankaranarayanan

Created: 2019-07-24 11:53:56 , Updates: 2, Last Update: 2019-07-24 12:08:43

Hi Arun Jothi,

As suggested, Tried placing the print report test case in the current CI Execution list. 

Tried as below:

Created 1 CI execution list (ContinuousIntegration property to True), having 1  test case

Placed the PrintReport testcase in that Execution list as the last entry. Now it have 2 test cases entry

Triggered CI from Jenkins

Now, getting the Execution report which is having 2 Test case entries (including the one we have for print report test case).

Kindly help on the same. Attached execution report for reference.

Thanks,

Ravi

 

Arun Jothi

2019-07-26 01:14:23 , Updates: 4, Last Update: 2019-07-26 08:16:43

Hi Ravi,

One way to exclude the Print report test case from your Report is to create 2 Execution lists, 1 for your actual test cases and another for the Report generation Test case.

But add an additional custom property only to your Execution list containing the Actual Test Cases and then add the same to the TQL in the Test.tcs file (As explained in Step1). like this:

 

jumpToNode "/BI-Testing/Execution/ExecutionLists"

For "=>SUBPARTS:ExecutionList[ContinuousIntegration==\"True\" AND TestforBuildA==\"True\" AND NewProperty==\"True\"]" TaskOnAll "Print Report... ExecutionEntries with ActualLog"
C:\\Temp\\TestExecutionReport.pdf

 

This way the Report generation Execution list will run as part of your CI Execution but won't appear on the Execution Report.

 

Let me know if this helps.

 

Cheers!

Arun Jothi

0 0

Ravi Kumar Sankaranarayanan

Created: 2019-07-26 08:37:55

Hi Arun Jothi,

Tried with the 1st approach as you suggested.

Added custom property only to Report Generation execution list and added the same property in xml file as well. But this time, execution happens only for the Report Generation execution list ant not the actual Execution list.

Could you please provide the update TQL query (the other approach). Appreciate your help on this.

Thanks,

Ravi

Arun Jothi

2019-07-26 08:21:27

Hi Ravi,

I have updated my previous reply. Please give it a try and let me know.

Cheers!

0 0

Ravi Kumar Sankaranarayanan

Created: 2019-07-29 08:49:11

Hello Arun Jothi,

This time, it works for me. Thanks for your great support.

Also you mentioned in the article as below:

In case Tosca CI is implemented via Distributed Execution (DEX), you can write a TQL to filter the Execution lists that have the property "ContinuousIntegration" as "True" and also linked to a specific Test Event.

Yes, We have implemented the Tosca CI via DEX as well. Able to trigger Test Event successfully from Jenkins. But not sure how to get Tosca Standard report for the particular Test Event. We have 3 Execution lists placed in that Test Event. It would be a great help if you share the TQL query that you mentioned above. Appreciate your help.

Thanks,

Ravi 

Arun Jothi

2019-08-09 03:58:16

Good to know know that it helped. Will get back to you shortly with the TQL for your requirement.

0 0

Ravikumar S

Created: 2019-08-12 07:27:06

Thanks. We ae waiting for that TQL update. Please help on the same.

Arun Jothi

2019-11-26 00:00:31 , Updates: 1, Last Update: 2019-11-26 00:01:11

Sorry about the late response. But here's the TQL that's required for the Tosca CI via DEX setup.

=>SUBPARTS:TestEvent[Name=="TESTEVENTNAME"]->ExecutionListLinks->AllReferences:ExecutionList

 

I have updated the article with the details as well.

0 0

Poonam Nagarajan

Created: 2019-10-10 08:08:02

Hi,

Kindly update if the automated report generation for Execution lists under Test Event for DEX Execution 

We have a requirement to print report after DEX execution of the testevent

Regards,

Poonam

 

Arun Jothi

2019-11-26 00:02:15

Hi Poonam,

I have updated the article with the TQL details for Tosca CI via DEX.

Let me know how you go.

 

Thanks,

Arun Jothi

0 0

Raghavendra Aithal

Created: 2020-09-07 14:41:13 , Updates: 1, Last Update: 2020-09-07 14:41:57

Hi Arun,

I am able to download report at end of execution, by having a different Execution list.

I need to send the report in email once generated using Standard Modules->XEngines->Mail modules.

Having Test cases, Report & Email generation in same workspace, could you please assist how to achieve this.

Thanks,

Raghavendra

 

0 0

Neha Parihar

Created: 2021-01-14 00:45:11

Hi Arun,

 

I tried using the TQL that you have for DEX, but it does return any results. Could you help me.

 

Thanks

Neha