Automating tests with Squash DEVOPS

Squash AUTOM allows you to launch the execution of the iteration’s automated tests from the Squash TM interface. How can I do the same thing from my continuous integration chain?

I have Squash TM, with a defined campaign and iteration, including at least one automated test case.

I additionally have Squash DEVOPS, and a continuous integration chain (it can be Jenkins, GitLab CI, GitHub Actions, or something else).

Structure of a workflow (or “Planned Execution as Code” or “PEaC”)

A workflow is a YAML file that must contain at least two elements: a name and a job.

metadata:
  name: my first workflow
jobs:
  my_first_job:
    ...

A workflow can contain other elements, but these other elements are not essential.

The name identifies the workflow. It will be used to recognize my workflow.

The job indicates what to do.

My first job

Here, I want to run an iteration from my test case management tool, Squash TM.

My job is going to make use of a plugin from the Squash orchestrator, tm.squashtest.org/tm.generator@v1, which will talk to my Squash TM.

It takes a few parameters that can be divided into two categories:

1. What it should do

2. The coordinates of the Squash TM instance it will talk to

What the job should do

This plugin can do two things: (1) execute the test cases of an iteration and (2) execute the test cases of a test suite.

The first parameter, testPlanType, can take two values, Iteration or TestSuite.

The second parameter, testPlanUuid, identifies the iteration or test suite that will be executed. This unique identifier can be found in the Information block of my iteration, in Squash TM.

The coordinates of the Squash TM instance

The coordinates of the Squash TM instance with which it will discuss consist of three parts: its address, the account, and the password to use.

In summary

The job definition will look like this:

my_first_job:
  generator: tm.squashtest.org/tm.generator@v1
  with
    testPlanType: Iteration
    testPlanUuid: 1e2ae123-6b67-44b2-b229-274ea17ad489
    squashTMUrl: https://mySquashTMInstance.org/squash
    squashTMAutomatedServerLogin: tfserver
    squashTMAutomatedServerPassword: tfserver

It is not perfect (as I have the account and even worse the password of my Squash TM instance, I can’t store this file in my favorite source code manager), but it will allow me to check that everything is ok.

My first iteration launch

My workflow is therefore a file with the following content:

metadata:
  name: my first workflow
jobs:
  my_first_job:
    generator: tm.squashtest.org/tm.generator@v1
    with:
      testPlanType: Iteration
      testPlanUuid: 1e2ae123–6b67–44b2-b229–274ea17ad489
      squashTMUrl: https://mySquashTMInstance.org/squash
      squashTMAutomatedServerLogin: tfserver
      squashTMAutomatedServerPassword: tfserver

I can give it any name I want. By convention, I will give it the name ‘Squashfile’ (but that is just it, a convention).

So I will be able to launch the execution of my iteration’s test cases via the following command from my computer (my Squash orchestrator instance has been deployed with docker-compose and is accessible via the URL http://orchestrator.squash):

curl -X POST \
    --data-binary @Squashfile \
    -H "Authorization: Bearer <yourtoken>" \
    -H "Content-type: application/x-yaml" \
    http://orchestrator.squash/workflows

This command will return the identifier of my workflow. I could use it to follow its progress.

To see the progress, I can run the following command:

curl -X POST \
    -H "Authorization: Bearer <yourtoken>" \
    http://orchestrator.squash/workflows/<workflow_id>/status

If all goes well, once my tests are run, I should see the results in Squash TM, associated with my iteration.

Addition to my continuous integration chain

If all went well, I am ready to add this iteration’s execution step to my continuous integration chain.

(1) I trigger the execution of the workflow, (2) the orchestrator makes a request to Squash TM, (3) Squash TM returns the list of test cases, (4) after execution, the orchestrator publishes the results in Squash TM, (5) the continuous integration chain can continue.

To do this, I must do two things: remove the secrets (here, the password that allows to connect to Squash TM, and the token if I use the curl command to trigger the operations) and add my workflow in my source code manager.

I will then only have to add a step in my continuous integration chain to trigger the execution of the iteration.

For the secrets, I will use environment variables. Depending on the tool I am using for my continuous integration, this may have a slightly different name, but the mechanism should always be present.

So, I will define two secrets: ORCHESTRATOR_TOKEN and SQUASHTM_PWD.

I will then modify my workflow to reference this environment variable (SQUASHTM_PWD):

metadata:
  name: my first workflow
jobs:
  my_first_job:
    generator: tm.squashtest.org/tm.generator@v1
    with:
      testPlanType: Iteration
      testPlanUuid:
      squashTMUrl: https://mysquashtminstance.org/squash
      squashTMAutomatedServerLogin: tfserver
      squashTMAutomatedServerPassword: ${{ variables.SQUASHTM_PWD }}

The notation ${{ variables.SQUASHTM_PWD }} will, during the execution phase, be replaced by the value of the expression: here, the content of the environment variable SQUASHTM_PWD.

I could also parameterize the UUID of the iteration, or even the type of test plan.

I can now add this file in my code manager.

git add Squashfile
git commit -m "Add workflow Squash DEVOPS"

All I need to do now is add a step to my continuous integration pipeline. Since I am using Jenkins, my pipeline will look like this:

pipeline {
  agent { label 'agent' }
  environment {
    ORCHESTRATOR_TOKEN = credentials('squash-devops-token')
    SQUASHTM_PWD = credentials('squash-tm-password')
  }
  stages{
    stage('Build'){
      // ...
    }
    // ...
    stage('Automated tests'){
      sh '''
      curl -X POST \
        -H "Authorization: Bearer ${ORCHESTRATOR_TOKEN}" \
        -H "Content-type: application/x-yaml" \
        -F workflow=@Squashfile \
        -F variables="SQUASHTM_PWD=${SQUASHTM_PWD}" \
        http://orchestrator.squash/workflows
      '''
    }
    // ...
  }
}

I did not use the Jenkins plugin here, nor opentf-ctl, so that the example would be easily adaptable if I used another tool for my continuous integration chain, but, in practice, I would use it.

From now on, every time my continuous integration chain is triggered, my iteration’s tests are run, and I can see the results in Squash TM. And if I change the composition of that iteration, subsequent runs will include those changes.

Martin Lafaix
Architect

Translation by Thibaut Lefaucheur

Learn more:

SQUASH - Squash DEVOPS to extend CI/CD for continuous testing

SQUASH - Roadmap & releases Squash DEVOPS