Running UI based performance tests for continuous integration using Selenium Grid + JMeter + Jenkins with reports.

Now you can run JMeter tests part of your continuous integration. More over you can run your browser related load tests  on remote machines using Selenium Grid and Jenkins Performance Plugin. The following pages will help you to achieve this.

Create JMeter tests which runs with Selenium Grid.

Configuring Jenkins to trigger and run JMeter test

Find more information in my talk at the Selenium Conference Austin 2017 found below.

Create JMeter tests which runs with Selenium Grid.

 

  • Create a new Test Plan in JMeter.
  • Add a Thread group.
  • Make sure the Selenium related plugins are install in the JMeter tool, else get them added with the help of the “Plugin Manager” (found under “Options”).

    Screen Shot 2017-02-15 at 10.33.52

  • Right Click on the Thread Group Goto Controller -> Add Simple Controller
  • Go to Config Element -> Add Remote WebDriver Config to the Simple Controller.

    Screen Shot 2017-02-15 at 10.34.28.png

  • Go to Sampler -> Add WebDriver Sampler to the Simple Controller under Remote WebDriver Config

    Screen Shot 2017-02-15 at 11.02.50.png

  • Once the the WebDriver Sampler is added write the Selenium code in the Sampler to run your automated tests. Add an assertion, and the Listeners like Summary Report and other Listeners.

    Screen Shot 2017-02-15 at 11.03.21.png

  • Make sure you set up the hub in your machine and connect them to the nodes, and add the node machine IP Address in the Selenium Grid URL space in Remote Webdriver Config, As shown below.

    Eg:- http://<IP_ADDRESS&gt;:<NODE PORT>/wd/hub

    Screen Shot 2017-02-15 at 14.45.30.png

  • Now you can run the tests on the node machines.

 

Configuring Jenkins to trigger and run JMeter test

 

  • Install Jenkins.
  • Download the Performance Plug-in (performance.hpi) for Jenkins from https://wiki.jenkins-ci.org/display/JENKINS/Performance+Plugin.
  • Unload Jenkins. Save the performance.hpi in the Plugins folder in the Jenkins. eg:- /Users/Shared/Jenkins/Home/plugins/
  • Load Jenkins. Login and goto Manage Jenkins -> Manage Plugins -> Installed (Tab) and check if the performance plugin is available.

    Screen Shot 2017-02-22 at 11.40.22

  • In the your Jmeter /bin/user.properties add the following line
        #jmeter.save.saveservice.output_format=xml
  • Now create a JMeter script and check it is working fine. This script can be the script created with the Remote WebDriver Config mentioned above.

     

  • Now try to run it with the following command on the terminal and check if the tests are running fine
         sh jmeter.sh -Jjmeter.save.saveservice.output_format=xml -n -t /<jmeter_script_location>/TestScript.jmx -l /<jmeter_test_result_location>/TestResult1.jtl
    
    
  • Go to Jenkins and Click on Create a new Job / New Item then select Freestyle project -> name it (eg:- JMeterSeleniumGridJenkins)

    Screen Shot 2017-02-21 at 12.36.17.png

  • Navigate to the project and Click on Configure.

    Screen Shot 2017-02-21 at 13.06.23.png

  • Scroll down to Build and click on Add a build step -> Execute shell.

    Screen Shot 2017-02-21 at 14.17.57.png

  • Add the following script in the command box.
          cd /<apache_jmeter_folder_location>/bin/
          sh jmeter.sh -Jjmeter.save.saveservice.output_format=xml -n -t /<jmeter_script_location>/TestScript.jmx -l /<jmeter_test_result_location>/TestResult1.jtl

    Screen Shot 2017-02-21 at 14.35.40.png

  • Scroll down to Post Build Actions and select Publish Performance test Result

    Screen Shot 2017-02-21 at 14.37.10

  • Select JMeter for for the Performance Report and give the path /<jmeter_test_result_location>/TestResult1.jtl

    Screen Shot 2017-02-21 at 14.37.33.png

  • You can change the threshold etc according to your need. Then click on save.

    Screen Shot 2017-02-21 at 14.38.10.png

  • Then click on Build now in the project. And click on the running build and check the Console Output.

    Screen Shot 2017-02-21 at 16.35.11.png

  • In case issues with the run please check the Read and Write option of the folders of accessing.
  • You should have the Performance reports displayed as well.

    Screen Shot 2017-02-21 at 16.35.20.png

 

Distributed Testing and Test Reporting with Selenium

As we are moving into the agile world, continuous integration has a major role to play.

So how do we cater for a complete test on every sprint or every release? We can use Selenium for Test Automation. When we use a continuous integration approach it would be helpful to use Selenium Grid. It allows you to run your tests on different machines against different browsers in parallel. Essentially, Selenium-Grid supports distributed test execution.

This helps you to run your automated tests on various different machines, operating systems and browsers at the same time. This saves time and would help  to run your testing in a nightly build.

Extent Reports will go hand in hand with Selenium Grid as it will help you retrieve all test results including Test Evidences into a comprehensive report. This webinar would give you a quick look of how to use both Selenium Grid and Extent Reports.