Tips for those who use Selenium – WebDriver Sampler with JMeter

Many people find it very challenging to convert their normal Selenium Tests scripts to adhere the WebDriver Samplers in JMeter. If you are very much used to the fact of writting Selenium scripts in a day to day basis, writing them on the WebDriver Sampler can not be very tedious. I faced a few challenges in the use of Selenium in JMeter when working on client projects. Following are 5 of those challenges and ways on how I over came them.

  1. Overcome unwanted crashes due to delay in page load
    The test scripts usually tend to fail when the Selenium Driver tries to search for a WebElement way before the page could completely load. For this we usually use the ExplicitWait() function in the normal selenium scripting world. In the WebDriver Sampler you could use the following.

    var pkg = JavaImporter(org.openqa.selenium)
    var support_ui = JavaImporter(org.openqa.selenium.support.ui.WebDriverWait)
    var ui = JavaImporter(org.openqa.selenium.support.ui)
    var wait = new support_ui.WebDriverWait(WDS.browser, 7000)
    //Here 7000 is the amount of milliseconds that code wait till it sees the specified username element
    
    wait.until(ui.ExpectedConditions.visibilityOfElementLocated(pkg.By.ByXPath("//input[@name='username']")))

     

  2. Can screenshots be taken for my JMeter Selenium Tests? Yes, with unique names by using parameters.
    You would like to take screenshots of your tests and verify them at the end of the test as the JMeter tests run so quickly and you can have it part of your test execution. This can be achieved in the following manner.

    var fileUtils = JavaImporter(org.apache.commons.io)
    var io = JavaImporter(java.io)
    var location = 'D:/Users/cthalayasingam/Documents/jmeterTest/'
    
    var screenshot = WDS.browser.getScreenshotAs(pkg.OutputType.FILE)
    fileUtils.FileUtils.copyFile(screenshot, new io.File(location+'messages1.png'))
  3. Can we use the JMeter User Defined Variables or JMeter Functions in the Selenium WebDriver Sampler Script? Yes, Can be achieved in the following ways (the first method is more preferred)
    If the JMeter variable is called cDate and you need to use it in send keys or some specific place in the code as shown below.

    fromDate.sendKeys(['${cDate1}'])

    The other is to mention the variable in the parameters field and then call it as shown in the following manner.

    Screen Shot 2017-08-31 at 12.40.52

  4. Checking if an element is present or displayed.
    WDS.browser.findElement(pkg.By.ByXPath("//div[contains(@class,'column2')]//div[contains(@class, 'dashlet')]//div[contains(.,'Resend Messages')][1]")).isDisplayed())
  5. Logging info messages.
    WDS.log.info('Messages all are deleted');
  6. How can we perform Drag And Drop function?
    var actions = new org.openqa.selenium.interactions.Actions(WDS.browser)
    
    wait.until(ui.ExpectedConditions.visibilityOfElementLocated(pkg.By.ByXPath("//ul[@class='availableList']//div[contains(@title,'Resend Messages')]")))
    
    var resendDashlet = WDS.browser.findElement(pkg.By.ByXPath("//ul[@class='availableList']//div[contains(@title,'Resend Messages')]"))
    var col2 = WDS.browser.findElement(pkg.By.ByXPath("//ul[@id='template_x002e_customise-dashlets_x002e_customise-user-dashboard_x0023_default-column-ul-2']"))
    
    //the element resendDashlet needs to be dragged and dropped into col2
    
    actions.clickAndHold(resendDashlet).moveToElement(col2).release(col2).build().perform()
    
    

     

There are many out there who had a few queries on how can the above be achieved. Hope the above tips were helpful. If there are more will keep updating this page or create more.

Running UI based performance tests for continuous integration using Selenium Grid + JMeter + Jenkins with reports.

Now you can run JMeter tests part of your continuous integration. More over you can run your browser related load tests  on remote machines using Selenium Grid and Jenkins Performance Plugin. The following pages will help you to achieve this.

Create JMeter tests which runs with Selenium Grid.

Configuring Jenkins to trigger and run JMeter test

Find more information in my talk at the Selenium Conference Austin 2017 found below.

Create JMeter tests which runs with Selenium Grid.

 

  • Create a new Test Plan in JMeter.
  • Add a Thread group.
  • Make sure the Selenium related plugins are install in the JMeter tool, else get them added with the help of the “Plugin Manager” (found under “Options”).

    Screen Shot 2017-02-15 at 10.33.52

  • Right Click on the Thread Group Goto Controller -> Add Simple Controller
  • Go to Config Element -> Add Remote WebDriver Config to the Simple Controller.

    Screen Shot 2017-02-15 at 10.34.28.png

  • Go to Sampler -> Add WebDriver Sampler to the Simple Controller under Remote WebDriver Config

    Screen Shot 2017-02-15 at 11.02.50.png

  • Once the the WebDriver Sampler is added write the Selenium code in the Sampler to run your automated tests. Add an assertion, and the Listeners like Summary Report and other Listeners.

    Screen Shot 2017-02-15 at 11.03.21.png

  • Make sure you set up the hub in your machine and connect them to the nodes, and add the node machine IP Address in the Selenium Grid URL space in Remote Webdriver Config, As shown below.

    Eg:- http://<IP_ADDRESS&gt;:<NODE PORT>/wd/hub

    Screen Shot 2017-02-15 at 14.45.30.png

  • Now you can run the tests on the node machines.

 

Configuring Jenkins to trigger and run JMeter test

 

  • Install Jenkins.
  • Download the Performance Plug-in (performance.hpi) for Jenkins from https://wiki.jenkins-ci.org/display/JENKINS/Performance+Plugin.
  • Unload Jenkins. Save the performance.hpi in the Plugins folder in the Jenkins. eg:- /Users/Shared/Jenkins/Home/plugins/
  • Load Jenkins. Login and goto Manage Jenkins -> Manage Plugins -> Installed (Tab) and check if the performance plugin is available.

    Screen Shot 2017-02-22 at 11.40.22

  • In the your Jmeter /bin/user.properties add the following line
        #jmeter.save.saveservice.output_format=xml
  • Now create a JMeter script and check it is working fine. This script can be the script created with the Remote WebDriver Config mentioned above.

     

  • Now try to run it with the following command on the terminal and check if the tests are running fine
         sh jmeter.sh -Jjmeter.save.saveservice.output_format=xml -n -t /<jmeter_script_location>/TestScript.jmx -l /<jmeter_test_result_location>/TestResult1.jtl
    
    
  • Go to Jenkins and Click on Create a new Job / New Item then select Freestyle project -> name it (eg:- JMeterSeleniumGridJenkins)

    Screen Shot 2017-02-21 at 12.36.17.png

  • Navigate to the project and Click on Configure.

    Screen Shot 2017-02-21 at 13.06.23.png

  • Scroll down to Build and click on Add a build step -> Execute shell.

    Screen Shot 2017-02-21 at 14.17.57.png

  • Add the following script in the command box.
          cd /<apache_jmeter_folder_location>/bin/
          sh jmeter.sh -Jjmeter.save.saveservice.output_format=xml -n -t /<jmeter_script_location>/TestScript.jmx -l /<jmeter_test_result_location>/TestResult1.jtl

    Screen Shot 2017-02-21 at 14.35.40.png

  • Scroll down to Post Build Actions and select Publish Performance test Result

    Screen Shot 2017-02-21 at 14.37.10

  • Select JMeter for for the Performance Report and give the path /<jmeter_test_result_location>/TestResult1.jtl

    Screen Shot 2017-02-21 at 14.37.33.png

  • You can change the threshold etc according to your need. Then click on save.

    Screen Shot 2017-02-21 at 14.38.10.png

  • Then click on Build now in the project. And click on the running build and check the Console Output.

    Screen Shot 2017-02-21 at 16.35.11.png

  • In case issues with the run please check the Read and Write option of the folders of accessing.
  • You should have the Performance reports displayed as well.

    Screen Shot 2017-02-21 at 16.35.20.png

 

Distributed Testing and Test Reporting with Selenium

As we are moving into the agile world, continuous integration has a major role to play.

So how do we cater for a complete test on every sprint or every release? We can use Selenium for Test Automation. When we use a continuous integration approach it would be helpful to use Selenium Grid. It allows you to run your tests on different machines against different browsers in parallel. Essentially, Selenium-Grid supports distributed test execution.

This helps you to run your automated tests on various different machines, operating systems and browsers at the same time. This saves time and would help  to run your testing in a nightly build.

Extent Reports will go hand in hand with Selenium Grid as it will help you retrieve all test results including Test Evidences into a comprehensive report. This webinar would give you a quick look of how to use both Selenium Grid and Extent Reports.

 

Comparing content value of a file with response JMeter

When working with data related systems, you will have to upload multiple documents as well as download documents and check if the downloaded documents is the same as you expected it to be.

When you call the API call for download and you have the same document in your system or your JMeter package then you can compared the content with the file that you have with you. As always you will have to use the response assertion to compare the file with the response you receive when you download the file from the application (you are testing).

The following call needed to be added in the response assertion of the Download API Call.

${__FileToString(/Users/testing-excellence/Perf/blog/json_file.txt,,)}

screen-shot-2017-02-02-at-3-26-48-pm

You can also a CSV Data config if you will have to compare the response with multiple downloads. Say you know that the download is going to be in order or you would like to make a specific call with a json value and you will need to make the call multiple times then you can use the above call for help.

You can have a CSV file with the names of the files(without extensions) with the respective json you need to to use the following call in the body

${__FileToString(/Users/testing-excellence/Perf/blog/${__eval(${json_file})}.txt,,)}
Where json_file as the CSV Data config variable.
screen-shot-2017-02-02-at-15-12-57

Uploading a document from same source document with jMeter.

This is another script that you can use to upload the same document to your system using jMeter. This is another way of doing as I have mentioned here.

import java.text.*;
import java.io.*;
import java.lang.*;
import org.apache.commons.io.FileUtils;

//Take the fileLocation from the user defined variable
//fileLocation = “/Users/cthalayasingam/Documents/jmeterTest/PerformanceTests/”;
fileLocation = vars.get(“fileCreatedPath”);
// The string portrays the current location of the document (Source file)
mainFile = “images.png”;
//The string portrays the location of the document which is renamed.
System.out.println(System.currentTimeMillis());
start = System.currentTimeMillis();
fileName = start + “images.png”;
//Save the name of the document with the time so that can be DELETED later
vars.put(“fileName1”, fileName);
//Get current document location
File file = new File( fileLocation + mainFile);
//Create location for renamed location
File file2 = new File( fileLocation + fileName);
System.out.println(fileName);
//Copy the file to the document to the new location
FileUtils.copyFile(file, file2);

After you have entered the above script in the Bean Shell pre processor. Then put the script in the delete script found below in a BeanShell Post Processor, that applies the delete for the document you have copied to the new destination.

import java.text.*;
import java.io.*;
import java.util.*;
import java.lang.*;

csvDir = vars.get(“fileCreatedPath”);
fileName2 = vars.get(“fileName1”);
File file1 = new File( csvDir + fileName2);
if (file1.exists()) {
file1.delete();
}

Configuring Appium Node Server with Selenium Grid

Introduction

Mobile devices are widely used in the world today. Therefore mobile application require vigorous testing. As android is the common mobile operating system we should be able to test it on multiple mobile operating systems.There can be multiple platforms (Android, iOS) and multiple devices which a Mobile App can support. However it involves a lot of resources and time for development to cover all of the tests across these devices. Now we can configure our Appium Node Servers with Selenium Grid.

How to configure and run test on Nodes?

Follow the following steps to configure the Android on Appium Node to Selenium Grid.

Step 1

Install the hub in the machine you have your code in the following manner.

java -jar selenium-server-standalone-2.47.1.jar -role hub

Step 2

Host 2 Appium Server Instances for 2 Mobile Devices. To configure 2 Running AVDs with Selenium Grid we need to create interfaces of the Appium Server. We have to run 2 instances of the Appium Server each associated with one of the Mobile devices and on another end connected to Selenium Grid.

 

Setting up an Appium Node instance for one device (Tab 3) to register with Selenium Grid Hub

  1.  Creating the Appium node configuration for the Tab 3 Device (appium_node_Tab3.json) using similar contents as shown below in the JSON file –

 

appium_node_Tab3.json
{
"capabilities":
     [
       {
          "deviceName": "3230e0d3d2d070dd",
         "version":"4.4.2",
         "maxInstances": 3,
         "platformName":"ANDROID"
       }
     ],
"configuration":
{
   "cleanUpCycle":2000,
   "timeout":30000,
   "proxy": "org.openqa.grid.selenium.proxy.DefaultRemoteProxy",
   "host": "192.168.1.100",
   "port": 4724,
   "maxSession": 6,
   "register": true,
   "registerCycle": 5000,
   "hubPort": 4444,
   "hubHost": "127.0.0.1"
}
}

 

Note: devicename is one of the keys of the Appium node which we are configuring. Here the JSON describes that the Appium node will be associated with the S4 device and the device will be listening for commands coming on the Appium node server address (URL key in JSON) on port 4723.

 

On the other side, hubPort and hubHost parameters decide which port and host machine Selenium Grid is hosted on and in turn registers the Appium node with it.

 

2. Open Terminal and navigate to the location where Appium was installed. Run the following command to host the Appium node instance:

 

Appium.app/Contents/Resources/node_modules/appium/bin/appium.js –nodeconfig /Users/deranthika/Desktop/appium_node_Tab3.json -p 4724 -U 3230e0d3d2d070dd

Once this is up and running you will see the notification in the Selenium grid console as an Appium node is registered as shown below:

In the same way create the Appium node JSON configuration file for the S4 device as shown below and host another instance of the Appium node server –

 

appium_node_S4.json
{
"capabilities":
     [
       {
          "deviceName": "f68a5d03",
         "version":"5.0.1",
         "maxInstances": 3,
         "platformName":"ANDROID"
       }
     ],
"configuration":
{
   "cleanUpCycle":2000,
   "timeout":30000,
   "proxy": "org.openqa.grid.selenium.proxy.DefaultRemoteProxy",
   "host": "192.168.1.100",
   "port": 4723,
   "maxSession": 6,
   "register": true,
   "registerCycle": 5000,
   "hubPort": 4444,
   "hubHost": "127.0.0.1"
}
}

 

Appium.app/Contents/Resources/node_modules/appium/bin/appium.js –nodeconfig /Users/deranthika/Desktop/appium_node_S4.json -p 4723 -U f68a5d03

Note: When hosting another instance of the Appium Node server for the S4 device we need to host it on a port other than the one that was used by the Appium server previously hosted for the Tab3 device i.e. 4724. So we choose 4723 as the port to host the Appium server for the S4 device. Always select a port from 4723 and above.

Step 3

Setting the Desired Capability in the Test Automation Script

 

Regardless of which programming language library we choose to write automation scripts, we need to set the desired capabilities for the Remote Webdriver.

Make sure you add the following code snippet in your before class.

 

Appium Node Server with Selenium Grid
@Before
    public void Androidsetup() throws MalformedURLException
    {
        DesiredCapabilities Androidcapability=new DesiredCapabilities();
            Androidcapability.setCapability("deviceName", "f68a5d03");
            Androidcapability.setCapability("platformName", "Android");
            Androidcapability.setCapability("platformVersion", "5.0.1");
            File file =new File("/Users/deranthika/MobileAPKs/Alfresco/Alfresco_v1.6_apkpure.com.apk");
            Androidcapability.setCapability("app", file.getAbsolutePath());
            androiddriver= new RemoteWebDriver(new URL("http://192.168.1.100:4723/wd/hub"),Androidcapability);
            DesiredCapabilities Androidcapability2=new DesiredCapabilities();
            Androidcapability2.setCapability("deviceName", "3230e0d3d2d070dd");
            Androidcapability2.setCapability("platformName", "Android");
            Androidcapability2.setCapability("platformVersion", "4.4.2");
            File file2 =new File("/Users/deranthika/MobileAPKs/Alfresco/Alfresco_v1.6_apkpure.com.apk");
            Androidcapability2.setCapability("app", file.getAbsolutePath());
            androiddriver= new RemoteWebDriver(new URL("http://192.168.1.100:4724/wd/hub"),Androidcapability2);
}

 

Now you would be able to run the tests on both the Tab3 and S4. Once you run the test and concluded you will see the following in the hub terminal which will show you the connections etc.

 

 

 

Configuring iOS device

Configure the Hub
Step 1
Download selenium server standalone jar to your computer

Step 2
Navigate to the location of  selenium server standalone located and run it as hub

daz:~ deranthika$ cd /Users/deranthika/Downloads
daz:Downloads deranthika$ java -jar selenium-server-standalone-2.48.2.jar -port 4444 -role hub
10:56:29.203 INFO - Launching Selenium Grid hub
2016-04-27 10:56:29.917:INFO::main: Logging initialized @1109ms

 

Configure Node
Step 1
Creating appium node configuration for iPad using json (create a json script as below and save in your computer eg: iOS.json)

{
  "capabilities":
  [
    {
      "browserName": "iPad"
    , "platform": "MAC"
    , "version": "9.1"
    "maxInstances": 1
    }
  ],
  "configuration":
  {
    "cleanUpCycle":2000,
    "timeout":30000,
    "proxy": "org.openqa.grid.selenium.proxy.DefaultRemoteProxy",
    "host": "192.168.1.100",
    "port": 4723,
    "maxSession": 1,
    "register": true,
    "registerCycle": 5000,
    "hubPort": 4444,
    "hubHost": "127.0.0.1"
    }
}

Note: browserName is your device name and version is the iOS version of the device. You can get it both after connect iOS device to the computer. Then open xcode, Go the Window> Devices and select the device.

hub: your selenium hub url
url: appium doamin: port number
host: appium domain
port: appium port
hubPort: selenium hubport
hubHost: selenium hub host

Step 2
Run the node
To run node, you need to navigate to the location where appium installed. Then run below command

daz:Downloads deranthika$ cd /Applications/
daz:Applications deranthika$ Appium.app/Contents/Resources/node_modules/appium/bin/appium.js --nodeconfig /Users/deranthika/Desktop/ios.json -p 4723 -U c332fce71e40f09de19998dc763afea33d0a6b24

Where “/Users/deranthika/Desktop/ios.json” is ios.json saved location
-p 4723: appium port
-U c332fce71e40f09de19998dc763afea33d0a6b24: device UDID (You can get it the same way used to get device name and version)

 

If you able to run correctly you’ll be able to see below message

You can see configuration data on hub console

http://localhost:4444/grid/console

Set Desired Capabilities in Automation Script

@Before
    public void iOSsetup() throws MalformedURLException
    {  
        iOScapability.setCapability("platformName", "iOS");
        iOScapability.setCapability("platformVersion", "9.1");
        iOScapability.setCapability("deviceName", "iPad");
        iOScapability.setCapability("app", "com.zaizi.Sensefy");
        iOScapability.setCapability("udid", "c332fce71e40f09de19998dc763afea33d0a6b24");
        driver=new IOSDriver(new URL("http://192.168.1.100:4723/wd/hub"),iOScapability); 
    }

Then you can run the script as maven. And you don’t need to start appium server

Deleting a particular CSV file using Jmeter

You might come across situations where you will have to delete a document before running a process. Say you have a situation where you have to create a document with all the data in that is needed for the particular test. Every time you run the test you would need create new documents. For that case you would need to delete the previously created document. If that is the case how do you delete the document. Following is the way to do so.

  1. Initially Add a BeanShell PreProcessor in the start of the Thread if you need to Delete the document in the beginning of the Thread.Screen Shot 2016-04-11 at 11.24.58Screen Shot 2016-04-11 at 11.26.02
  2. Once you have added the Beanshell PreProcessor, now you can add the following script. to delete the document.Screen Shot 2016-04-11 at 11.47.52
    import java.text.*;
    import java.io.*;
    import java.util.*;
    downloadDoc = "download.csv";
    csvDir = vars.get("fileLocation");
    File file1 = new File( csvDir + downloadDoc);
    if (file1.exists()) {
    file1.delete(); 
    }

Now that the the Delete script is ready to be run and now you can delete the document you want to delete from the specific directory.

 

Randomly selecting values from .csv file for Jmeter Tests im on the

There might be instances where you will have to grab dynamic values from .csv file to run your Jmeter Performance tests. You can not use the CSV Data Config as the point here to select values randomly from the .csv file

The following script in a BeanShell PreProcessor will help to randomly select values from the CSV file that has the values. It randomly selects each row from the the CSV you can later break down the row as you prefer, in case a row has multiple values.

import java.text.*;
import java.io.*;
import java.util.*;

String [] params = Parameters.split(“,”);
csvTest = “download.csv”;
csvDir = vars.get(“fileLocation”);
ArrayList strList = new ArrayList();

try {
File file = new File( csvDir + csvTest);

if (!file.exists()) {
throw new Exception (“ERROR: file ” + csvTest + ” not found in ” + csvDir + ” directory.”);
}

BufferedReader bufRdr = new BufferedReader(new FileReader(file));
String line = null;

while((line = bufRdr.readLine()) != null) {
strList.add(line);
}

bufRdr.close();

Random rnd = new java.util.Random();
vars.put(“NodeId”,strList.get(rnd.nextInt(strList.size())));
//String NodeId= strList.get(rnd.nextInt(strList.size()));
//System.err.println(“Parts jjj “+q);
//String[] parts = q.split(“,”);

//System.err.println(“Parts 0 “+parts[0]);
//System.err.println(“NodeName “+parts[0]);
System.err.println(“NodeId “+NodeId);

//vars.put(“NodeId”,NodeId);
//String correctName = parts[0].replace(“%20″,””);
//vars.put(“name”,correctName);
}
catch (Exception ex) {
IsSuccess = false;
log.error(ex.getMessage());
System.err.println(ex.getMessage());
}
catch (Throwable thex) {
System.err.println(thex.getMessage());
}

Screen Shot 2016-04-08 at 11.14.15 AM (2).png

You can now use this scripts and randomly select values from CSV file when running your Jmeter Tests.