Are Your Selenium Scripts Ready For Parallel Execution?

If I ask you how can you run your Selenium scripts in parallel (Assuming you are using Selenium – Java with TestNG) , probably you will answer just add a “parallel” attribute with a desired value (methods, classes, tests or instances) and set a thread count in TestNG XML and you are all set to run your scripts in parallel.

Correct answer. Now ask a question yourself that “Are your selenium scripts ready for parallel run?”

Let’s see some basics concepts about Parallel execution in TestNG.

• We may have multiple @Test annotated methods (tests) in a TestNG class.
• Similarly we can have multiple TestNG classes with multiple @Test annotated methods in it.
• We can have multiple “test” tags consisting multiple TestNG classes in a TestNG suite.
• We can have multiple suites i.e. testng.xml files as well.
• We can have multiple methods in an instance (Factory method)

We can run out tests in parallel using TestNG in multiple ways :-

1. Run @Test methods in parallel
2. Run TestNG classes in parallel
3. Run <test> tags of TestNG.xml in parallel
4. Run instances in parallel
5. Run testng.xml files in parallel

I would like to give extract from Official TestNG document how parallel execution is performed in TestNG here. They have perfectly summarized clearly.

• parallel=”methods”: TestNG will run all your test methods (@Test annotated methods) across suite in separate threads. Dependent methods will also run in separate threads but they will respect the order that you specified.
• parallel=”tests”: TestNG will run all the test methods in the same <test> tag in the same thread, but each <test> tag will be in a separate thread. This allows you to group all your classes that are not thread safe in the same <test> and guarantee they will all run in the same thread while taking advantage of TestNG using as many threads as possible to run your tests.
• parallel=”classes”: TestNG will run all the methods in the same class in the same thread, but each class will be run in a separate thread.
• parallel=”instances”: TestNG will run all the methods in the same instance in the same thread, but two methods on two different instances will be running in different threads.

If we have 10 @Test methods and we want to run these methods in parallel and we provide thread count as 2, then these 2 threads will divide test methods between them based on availability. If T1 picks a method say M1 for execution and T2 picks a method say M2 for execution, upcoming method will be picked by thread which finishes execution first.

Let’s implement a simple parallel run mechanism using Threads concept in Java.

You are writing automated scripts for an e-Commerce application and you perform different tests like placing an order, cancelling an order etc. As a part of prerequisite, you are creating a user before starting actual tests. You have created a utility to register a user. Your different tests will be using same resource or method to register a user.

RegisterUser.class

package ThreadLocalUsageInSelenium.RegisterUserParallelRunProblem;

import java.util.Random;

/*
 * This class has a method to register user. Implementing this class to run as a Thread.
 */
public class RegisterUser implements Runnable {

	// Using it as a class variable as I need to use same registered user in
	// multiple methods.
	// Registered user in createUser() method and getting the name in
	// getUserWithDelay() methods.
	String registeredUserName = "No Value";

	@Override
	public void run() {
		createUser();
		try {
			getUserWithDelay();
		} catch (InterruptedException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}

	}

	// Creating a user and setting value as global variable
	private void createUser() {
		System.out.println(Thread.currentThread().getId()
				+ " - Before starting registration, value of registeredUserName :" + registeredUserName);
		System.out.println(Thread.currentThread().getId() + " - Registering a user.");
		registeredUserName = Thread.currentThread().getId() + " - User" + new Random().nextInt(999);
		System.out.println(Thread.currentThread().getId() + " - After registration, value of registeredUserName :"
				+ registeredUserName);
	}

	// Retrieving registered user name
	private void getUserWithDelay() throws InterruptedException {
		Thread.sleep(5000);
		System.out.println(Thread.currentThread().getId() + " - After some delay , value of registeredUserName :"
				+ registeredUserName);

	}

}

Now creating two threads which are sharing resource as below:-

RegisterUserTest.java

package ThreadLocalUsageInSelenium.RegisterUserParallelRunProblem;

public class RegisterUserTest {

	public static void main(String[] args) throws InterruptedException {
		RegisterUser registerUser = new RegisterUser();
		// Creating two threads and both are using same instance of RegisteredUser class
		Thread thread1 = new Thread(registerUser);
		Thread thread2 = new Thread(registerUser);
		// Starting first thread
		thread1.start();
		// Giving a pause
		Thread.sleep(2000);
		// Starting another thread
		thread2.start();
		
	}
}

Let’s run the RegisterUserTest.java and observe output:-

 10 - Before starting registration, value of registeredUserName :No Value
 10 - Registering a user.
 10 - After registration, value of registeredUserName :10 - User447
 11 - Before starting registration, value of registeredUserName :10 - User447
 11 - Registering a user.
 11 - After registration, value of registeredUserName :11 - User839
 10 - After some delay , value of registeredUserName :11 - User839
 11 - After some delay , value of registeredUserName :11 - User839

Let’s understand how did execution go step by step:-

1. An instance of RegisteredUser were created.
2. Same instance of RegisteredUser were passed to two different threads.
3. First thread ( Thread ID 10 above) started execution first and started execution of createUser() method. It checked default value of class variable “registeredUserName” which was currently “No Value”. Now creates a user and set value of “registeredUserName”.
4. Second Thread (Thread ID 11 above) also started execution just after Thread 1 , and started execution of createUser() method. It also checked default value of class variable “registeredUserName” which was NO more as “No Value”. In stead of it had value set by Thread 1. Thread 2 also created a user and set value of “registeredUserName”.
5. Both threads waited for 5 seconds simultaneously and fetched registeredUserName value. Expectation is that both threads will print registered user name created by them but they printed the user name registered by last thread.

Reason:-

Class variables are shared by threads created from same object.

In above example, we have two threads but they shared class variable “registeredUserName ” and both worked on same reference which leads to data inconsistently.

So are you making sure that your tests are thread safe and data generated by a test running by a thread should not be used by another test running by another thread?

Let’s see similar example in a Selenium script which we generally do.

We have a class named “WebDriverFactory.java” which initializes a chrome browser and give you a setter method and a getter method to set and get driver instance respectively.

WebDriverFactory.java

package ThreadLocalUsageInSelenium;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;

import io.github.bonigarcia.wdm.WebDriverManager;

public class WebDriverFactory {
	
	private WebDriver driver;
	
	public void setDriver() {
		WebDriverManager.chromedriver().setup();
		driver = new ChromeDriver();
	}
	
	public WebDriver getDriver()
	{
		return driver;
	}
}

A class named TestCases which contains two normal TestNG tests. This class has a class variable of type “WebDriverFactory” which is used to set and get driver instance in tests. For each test BeforeMethod will be called to initialize driver and AfterMethod will close browser.

TestCases.java

package ThreadLocalUsageInSelenium.WebDriverParallelRunProblem;

import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Test;

public class TestCases {

	/*
	 * When we run this particular class methods in parallel, each method may
	 * execute by a different thread based on thread count. So here class variable
	 * "webDriverFactory" which is used in both tests will be shared by all threads
	 * and you will see unexpected execution.
	 * 
	 * I have given sleep just to observe execution. You will see two browsers
	 * (because thread count is 2 in xml) are launched but execution will not be as
	 * expected. Our expectation is that it should launch two browsers in parallel
	 * and execute test 1 in one and test 2 in another and close respective browser
	 * on completion of test. But because of sharing of class variable
	 * "webDriverfactory" between threads, execution will not be as expected.
	 * Different run may give you different output.
	 * 
	 * At last you will see both threads pointing to same browser instance and other
	 * will be left unattended.
	 */
	WebDriverFactory webDriverFactory;

	@BeforeMethod
	public void setUpBrowser() {
		webDriverFactory = new WebDriverFactory();
		webDriverFactory.setDriver();
	}

	@Test
	public void test1() throws InterruptedException {

		webDriverFactory.getDriver().get("https://www.google.com/");
		Thread.sleep(5000);
		System.out.println(
				"Title printed by " + Thread.currentThread().getId() + " - " + webDriverFactory.getDriver().getTitle());
		webDriverFactory.getDriver().close();
	}

	@Test
	public void test2() throws InterruptedException {
		webDriverFactory.getDriver().get("https://www.facebook.com/");
		Thread.sleep(5000);
		System.out.println(
				"Title printed by " + Thread.currentThread().getId() + " - " + webDriverFactory.getDriver().getTitle());
		webDriverFactory.getDriver().close();
	}

	@AfterMethod
	public void tearDown() {
		webDriverFactory.getDriver().close();
	}
}

Let’s run methods in parallel using testng xml:-

You should observe while running that it is not running as we expected. It will launch two browsers but both tests may be executed in same browser and another will be left unattended. You can play around by placing sleep before starting and may observe different output. You know the reason well now.

Console Output:-

Starting ChromeDriver 80.0.3987.106 (f68069574609230cf9b635cd784cfb1bf81bb53a-refs/branch-heads/3987@{#882}) on port 48054
 Only local connections are allowed.
 Please protect ports used by ChromeDriver and related test frameworks to prevent access by malicious code.
 Starting ChromeDriver 80.0.3987.106 (f68069574609230cf9b635cd784cfb1bf81bb53a-refs/branch-heads/3987@{#882}) on port 35731
 Only local connections are allowed.
 Please protect ports used by ChromeDriver and related test frameworks to prevent access by malicious code.
 Title printed by 12 - Google

We can solve above problem by converting class variable in to local as below:-

package ThreadLocalUsageInSelenium.WebDriverParallelRunProblem;

import org.testng.annotations.Test;

import ThreadLocalUsageInSelenium.SolvingWebDriverParallelRunProblem.WebDriverFactory;


public class WebDriverTestWithoutClassVariable {
	

	@Test
	public void test1()
	{
		WebDriverFactory webDriverFactory = new WebDriverFactory();	
		webDriverFactory.setDriver();
		webDriverFactory.getDriver().get("https://www.google.com/");
		System.out.println("Title printed by "+Thread.currentThread().getId()+" - "+webDriverFactory.getDriver().getTitle());
		webDriverFactory.getDriver().close();
	}
	
	@Test
	public void test2()
	{
		WebDriverFactory webDriverFactory = new WebDriverFactory();	
		webDriverFactory.setDriver();
		webDriverFactory.getDriver().get("https://www.facebook.com/");
		System.out.println("Title printed by "+Thread.currentThread().getId()+" - "+webDriverFactory.getDriver().getTitle());
		webDriverFactory.getDriver().close();
	}
}

Run about tests in parallel and you can see expected output:-

Starting ChromeDriver 80.0.3987.106 (f68069574609230cf9b635cd784cfb1bf81bb53a-refs/branch-heads/3987@{#882}) on port 17791
 Only local connections are allowed.
Please protect ports used by ChromeDriver and related test frameworks to prevent access by malicious code.
Starting ChromeDriver 80.0.3987.106 (f68069574609230cf9b635cd784cfb1bf81bb53a-refs/branch-heads/3987@{#882}) on port 28438
Only local connections are allowed.
Please protect ports used by ChromeDriver and related test frameworks to prevent access by malicious code.
Title printed by 11 - Google
Title printed by 12 - Facebook – log in or sign up

But above solution is not optimal as you can not setup prerequisite and post requisite. Also states can not be shared between tests.

We will solve this problem using ThreadLocal concept which is recommended as well. We will see that in next post.

You can clone the above example from my repo.

If you have any doubt, feel free to comment below.
If you like my posts, please like, comment, share and subscribe.
#ThanksForReading
#HappyLearning

You can find all Selenium related post here.
You can find all API manual and automation related posts here.
You can find frequently asked Java Programs here.