Jenkins
Jenkins and Testing Results
info
Jenkins is one of the most popular CI applications and is used by thousands of companies all over the world. Jenkins is used to define the application build and test processes. QA teams also use Jenkins to run/schedule functional tests across a variety of tools and frameworks. However, the results of these tests can only be viewed in Jenkins, and it is not really possible to aggregate results across multiple jobs, retain long history, and delegate test failures to your team.
Micro Focus ALM + Jenkins + Railflow for ALM
info
By using the Railflow for ALM, you can easily integrate Jenkins testing jobs with Micro Focus ALM and automatically export all testing reports to ALM. Aggregating result from all your Jenkins jobs into ALM allows teams to look at test trends, manage defects via Railflow automation and much more.
Jenkins Plugin
Plugin approach
Railflow for ALM Jenkins plugin is a commercial product. As such, it is not available in the Jenkins plugin directory. Railflow for ALM Jenkins plugin (hpi file) can be downloaded from the website's download section.
Prerequisites
- Jenkins version 2.332.1 or newer
Installation
- Download Railflow for ALM Jenkins Plugin
- Deploy the plugin via
Manage Jenkins→Manage Plugins→Advanced Settings→Deploy Plugin
3. Click on the Deploy button and restart Jenkins for the plugin to take effect.
Configuration
To configure plugin, navigate to Manage Jenkins → Configure System page of the Jenkins UI and scroll down to the Railflow ALM Global Settings section:
Licensing
info
Railflow for ALM provides two license activation models. If your network allows outbound calls to our api endpoint, then option 1: License Key (online activation) will work. If you are unable to reach our api endpoint, pick option 2: License File (offline activation).
- License Key (online activation): Select
License Keyoption and copy/paste the license key from your email and clickActivate License
- License File (offline activation): If your Jenkins instance does not have outbound internet access, you may use this option to upload the license activation from your email. The license file has a .skm extension.
ALM Instances Configuration
Railflow for ALM plugin configuration section allows you to centrally defined one or multiple ALM servers. Once configured, they can be easily referenced in the Jenkins jobs.
Freestyle Job Configuration
If you are using Jenkins UI based (freestyle job) jobs to run some test frameworks/tools, you can use the Railflow for ALM plugin's UI controls to easily integrate with ALM by following these steps.
Add and configure the Jenkins job's post-build action: Railflow Plugin: ALM Test Results Processor. Railflow's post-build action allows you to specify the test framework, test results location, and various other ALM-related configurations.
Post-Build Action Reference
Both the Pipeline and Freestyle job depend on the Railflow Plugin: ALM Test Results Processor post-build action to process test results and post them to ALM. This action provides users a host of options to integrate Jenkins with ALM in a variety of ways and across any testing tool and framework. The reference below describes all the options and their usage.
tip
You can add multiple Railflow post-build actions using Add More button. This may be needed if you have multiple reports to process and want to map each report to different places in ALM.
| Key | Description |
|---|---|
| ALM Server Instance | (required) The name of the corresponding ALM Instance in the global configuration |
| ALM Domain Name | (required) The name of the ALM domain |
| ALM Project Name | (required) The name of the ALM project |
| Fail build if upload unsuccessful | If checked, the build will be marked as failed if for any reason the plugin could not upload the results. This could be due to Railflow issues, ALM issues, network issues, etc. |
| ALM Username | If specified, it overrides ALM username defined in Global Configuration |
| ALM Password | If specified, it overrides ALM password defined in Global Configuration |
| Results File Pattern | (required) The file path to the test report file(s) generated during the build. Ant-style patterns such as **/surefire-reports/*.xml can be used. E.g. use target/surefire-reports/*.xml to capture all XML files in target/surefire-reports directory. |
| Test Report Format | (required) Results file format |
| Test Plan Path | (required) Path to a Test Plan folder where test results should be exported to. If it does not exist, Railflow will create it automatically |
| Test Lab Path | (required) Path to a Test Lab folder where test set should be created. If it does not exist, Railflow will create it automatically |
| Test Set | (required) Name of a test set in ALM Test Lab where test results will be exported. If it does not exist, Railflow will create it automatically |
| Test Fields | (optional) Values for Test fields in ALM Test Plan can be specified in this field. The format is [Test field label]=[value]. Each field name\value pair should start with the new line. E.g.: Custom Field One=foo Custom Field Two=bar |
| Test Set Fields | (optional) Values for Test Set fields in ALM Test Lab can be specified in this field. The format is [Test set field label]=[value]. Each field name\value pair should start with the new line. E.g.: Custom Set Field One=foo Custom Set Field Two=bar |
| Run Fields | (optional) Values for Run fields in ALM Test Lab can be specified in this field. The format is [Run field label]=[value]. Each field name\value pair should start with the new line. E.g.: Custom Run Field One=foo Custom Run Field Two=bar |
| Design Step Fields | (optional) Values for Design Step fields in ALM Test Plan can be specified in this field. The format is [Design step field label]=[value]. Each field name\value pair should start with the new line. E.g.: Custom Step Field One=foo Custom Step Field Two=bar |
| Run Step Fields | (optional) Values for Run Step fields in ALM Test Lab can be specified in this field. The format is [Run step field label]=[value]. Each field name\value pair should start with the new line. E.g.: Custom Step Field One=foo Custom Step Field Two=bar |
| Test Instance Fields | (optional) Values for Test Instance fields in ALM Test Lab can be specified in this field. The format is [Test instance field label]=[value]. Each field name\value pair should start with the new line. E.g.: Custom Instance Field One=foo Custom Instance Field Two=bar |
| Upload Mode | Railflow Upload mode:
|
| Disable Grouping | Disabling Grouping will ignore report test structure and upload all test cases into a single ALM TestPlan folder as defined in the Test Plan Path. |
| ALM Auto Defect Management | Whether Railflow ALM Auto Defect Management feature is enabled or not |
| Defect Create Policy | Defect creation policy:
|
| Create Status | Status in ALM which will be assigned to a newly created defect |
| Create Defect Fields | Values for fields of a newly created defect. The format is [Defect field label]=[value]. Each field name\value pair should start with the new line. E.g.: Custom Defect Field One=foo Custom Defect Field Two=bar |
| Reopen Status | Status in ALM which will be assigned to a reopened defect. Effective only if create policy is Reopen |
| Reopen Defect Fields | Values for fields of a reopened defect. Effective only if create policy is Reopen. The format is [Defect field label]=[value]. Each field name\value pair should start with the new line.E.g.: Custom Defect Field One=foo Custom Defect Field Two=bar |
| Severity | Value for Severity property of a defect |
| Resolve Policy | Defect resolving policy:
|
| Resolve Status | Status in ALM which will be assigned to a resolved defect |
| Resolve Defect Fields | Values for fields of a resolved defect (being set when defect is being resolved). The format is [Defect field label]=[value]. Each field name\value pair should start with the new line. E.g.: Custom Defect Field One=foo Custom Defect Field Two=bar |
| Debug Log | If checked, Railflow for ALM will create .railflow-alm.log file in the workspace root folder |
Pipeline Job Configuration
If you are using Jenkins Pipelines to run some test frameworks/tools, you can access the same plugin functionality within the Jenkinsfile:
pipeline {
agent {
label "linux"
}
stages {
stage('Build') {
steps {
echo '********BUILD STEP**********'
git branch: 'master', credentialsId: '*****', url: 'https://...junit-demo.git'
sh 'mvn test'
}
}
}
post {
always {
echo '********UPLOAD RESULT INTO ALM**********'
railflowAlm (
almServerName: 'ALM 15',
domain: 'DEFAULT',
project: 'demo',
overriddenUserName: 'user',
overriddenPassword: 'password',
debugLogEnabled: true,
failIfUploadFailed: true,
uploadConfigurations: [[
resultFilePattern: '**/target/surefire-reports/*.xml',
reportFormat: 'JUNIT',
testPlanPath: 'Subject\\Test Plan Folder',
testLabPath: 'Root\\Test Lab Folder',
testSetName: 'Test Set',
testSetFields: 'Test set field = set field',
designStepFields: 'Design step field = design step value',
disableGrouping: true,
runFields: 'Run field = run field value',
runStepFields: 'Run step field = run step value',
testFields: '''Test field 1 = test 1
Test field 2 = test 2''',
testInstanceFields: 'Instance field = instance value',
uploadMode: 'FULL',
defectHandlingConfiguration: [
createFields: 'Defect field = created',
createPolicy: 'REOPEN',
createStatus: 'New',
reopenFields: 'Defect field = reopened',
reopenStatus: 'Reopen',
resolveFields: 'Defect field = fixed',
resolvePolicy: 'CLOSE',
resolveStatus: 'Fixed',
severity: '2-Medium']
]])
}
}
}
railflowAlm parameters
| Key | Description |
|---|---|
| almServerName | (required) The name of the corresponding ALM Instance in the global configuration |
| domain | (required) The name of the ALM domain |
| project | (required) The name of the ALM project |
| overriddenUserName | If specified, it overrides ALM username defined in Global Configuration |
| overriddenPassword | If specified, it overrides ALM password defined in Global Configuration |
| failIfUploadFailed | If checked, the build will be marked as failed if for any reason the plugin could not upload the results. This could be due to Railflow issues, ALM issues, network issues, etc. |
| debugLogEnabled | If true, Railflow for ALM will create .railflow-alm.log file in the workspace root folder |
uploadConfigurations parameters
| Key | Description |
|---|---|
| resultFilePattern | (required) The file path to the test report file(s) generated during the build. Ant-style patterns such as **/surefire-reports/*.xml can be used. E.g. use target/surefire-reports/*.xml to capture all XML files in target/surefire-reports directory. |
| reportFormat | (required) Results file format, available values:
|
| testPlanPath | (required) Path to a Test Plan folder where test results should be exported to. If it does not exist, Railflow will create it automatically |
| testLabPath | (required) Path to a Test Lab folder where test set should be created. If it does not exist, Railflow will create it automatically |
| testSetName | (required) Name of a test set in ALM Test Lab where test results will be exported. If it does not exist, Railflow will create it automatically |
| testFields | (optional) Values for Test fields in ALM Test Plan can be specified in this field. The format is [Test field label]=[value]. Each field name\value pair should start with the new line. E.g.: Custom Field One=foo Custom Field Two=bar |
| testSetFields | (optional) Values for Test Set fields in ALM Test Lab can be specified in this field. The format is [Test set field label]=[value]. Each field name\value pair should start with the new line. E.g.: Custom Set Field One=foo Custom Set Field Two=bar |
| runFields | (optional) Values for Run fields in ALM Test Lab can be specified in this field. The format is [Run field label]=[value]. Each field name\value pair should start with the new line. E.g.: Custom Run Field One=foo Custom Run Field Two=bar |
| designStepFields | (optional) Values for Design Step fields in ALM Test Plan can be specified in this field. The format is [Design step field label]=[value]. Each field name\value pair should start with the new line. E.g.: Custom Step Field One=foo Custom Step Field Two=bar |
| runStepFields | (optional) Values for Run Step fields in ALM Test Lab can be specified in this field. The format is [Run step field label]=[value]. Each field name\value pair should start with the new line. E.g.: Custom Step Field One=foo Custom Step Field Two=bar |
| testInstanceFields | (optional) Values for Test Instance fields in ALM Test Lab can be specified in this field. The format is [Test instance field label]=[value]. Each field name\value pair should start with the new line. E.g.: Custom Instance Field One=foo Custom Instance Field Two=bar |
| uploadMode | Railflow Upload mode:
|
defectHandlingConfiguration parameters
| Key | Description |
|---|---|
| createPolicy | Defect creation policy:
|
| createStatus | Status in ALM which will be assigned to a newly created defect |
| createFields | Values for fields of a newly created defect. The format is [Defect field label]=[value]. Each field name\value pair should start with the new line. E.g.: Custom Defect Field One=foo Custom Defect Field Two=bar |
| reopenFields | Status in ALM which will be assigned to a reopened defect. Effective only if create policy is REOPEN |
| reopenFields | Values for fields of a reopened defect. Effective only if create policy is REOPEN. The format is [Defect field label]=[value]. Each field name\value pair should start with the new line.E.g.: Custom Defect Field One=foo Custom Defect Field Two=bar |
| severity | Value for Severity property of a defect |
| resolvePolicy | Defect resolving policy:
|
| resolveStatus | Status in ALM which will be assigned to a resolved defect |
| resolveFields | Values for fields of a resolved defect (being set when defect is being resolved). The format is [Defect field label]=[value]. Each field name\value pair should start with the new line. E.g.: Custom Defect Field One=foo Custom Defect Field Two=bar |
ALM Export Details
Railflow creates a very rich and flexible integration between Jenkins and Micro Focus ALM. Based on Railflow configuration, ALM entities can be created or updated automatically as part of your CICD process.
Jenkins console output
Automatic TestPlan Entries Creation
Automatic Test Set and Test Run Creation
Automatic Defect Creation
