Storing Web Table With Pagination Data Into List Of Map – Java

Name	Position	Office	Age	Start date	Salary
Tiger Nixon	System Architect	Edinburgh	61	2011/04/25	$320,800
Garrett Winters	Accountant	Tokyo	63	2011/07/25	$170,750
Ashton Cox	Junior Technical Author	San Francisco	66	2009/01/12	$86,000
Cedric Kelly	Senior Javascript Developer	Edinburgh	22	2012/03/29	$433,060
Airi Satou	Accountant	Tokyo	33	2008/11/28	$162,700
Brielle Williamson	Integration Specialist	New York	61	2012/12/02	$372,000
Herrod Chandler	Sales Assistant	San Francisco	59	2012/08/06	$137,500
Rhona Davidson	Integration Specialist	Tokyo	55	2010/10/14	$327,900
Colleen Hurst	Javascript Developer	San Francisco	39	2009/09/15	$205,500
Sonya Frost	Software Engineer	Edinburgh	23	2008/12/13	$103,600
Jena Gaines	Office Manager	London	30	2008/12/19	$90,560
Quinn Flynn	Support Lead	Edinburgh	22	2013/03/03	$342,000
Charde Marshall	Regional Director	San Francisco	36	2008/10/16	$470,600
Haley Kennedy	Senior Marketing Designer	London	43	2012/12/18	$313,500
Tatyana Fitzpatrick	Regional Director	London	19	2010/03/17	$385,750
Michael Silva	Marketing Designer	London	66	2012/11/27	$198,500
Paul Byrd	Chief Financial Officer (CFO)	New York	64	2010/06/09	$725,000
Gloria Little	Systems Administrator	New York	59	2009/04/10	$237,500
Bradley Greer	Software Engineer	London	41	2012/10/13	$132,000
Dai Rios	Personnel Lead	Edinburgh	35	2012/09/26	$217,500
Jenette Caldwell	Development Lead	New York	30	2011/09/03	$345,000
Yuri Berry	Chief Marketing Officer (CMO)	New York	40	2009/06/25	$675,000
Caesar Vance	Pre-Sales Support	New York	21	2011/12/12	$106,450
Doris Wilder	Sales Assistant	Sidney	23	2010/09/20	$85,600
Angelica Ramos	Chief Executive Officer (CEO)	London	47	2009/10/09	$1,200,000
Gavin Joyce	Developer	Edinburgh	42	2010/12/22	$92,575
Jennifer Chang	Regional Director	Singapore	28	2010/11/14	$357,650
Brenden Wagner	Software Engineer	San Francisco	28	2011/06/07	$206,850
Fiona Green	Chief Operating Officer (COO)	San Francisco	48	2010/03/11	$850,000
Shou Itou	Regional Marketing	Tokyo	20	2011/08/14	$163,000
Michelle House	Integration Specialist	Sidney	37	2011/06/02	$95,400
Suki Burks	Developer	London	53	2009/10/22	$114,500
Prescott Bartlett	Technical Author	London	27	2011/05/07	$145,000
Gavin Cortez	Team Leader	San Francisco	22	2008/10/26	$235,500
Martena Mccray	Post-Sales support	Edinburgh	46	2011/03/09	$324,050
Unity Butler	Marketing Designer	San Francisco	47	2009/12/09	$85,675
Howard Hatfield	Office Manager	San Francisco	51	2008/12/16	$164,500
Hope Fuentes	Secretary	San Francisco	41	2010/02/12	$109,850
Vivian Harrell	Financial Controller	San Francisco	62	2009/02/14	$452,500
Timothy Mooney	Office Manager	London	37	2008/12/11	$136,200
Jackson Bradshaw	Director	New York	65	2008/09/26	$645,750
Olivia Liang	Support Engineer	Singapore	64	2011/02/03	$234,500
Bruno Nash	Software Engineer	London	38	2011/05/03	$163,500
Sakura Yamamoto	Support Engineer	Tokyo	37	2009/08/19	$139,575
Thor Walton	Developer	New York	61	2013/08/11	$98,540
Finn Camacho	Support Engineer	San Francisco	47	2009/07/07	$87,500
Serge Baldwin	Data Coordinator	Singapore	64	2012/04/09	$138,575
Zenaida Frank	Software Engineer	New York	63	2010/01/04	$125,250
Zorita Serrano	Software Engineer	San Francisco	56	2012/06/01	$115,000
Jennifer Acosta	Junior Javascript Developer	Edinburgh	43	2013/02/01	$75,650
Cara Stevens	Sales Assistant	New York	46	2011/12/06	$145,600
Hermione Butler	Regional Director	London	47	2011/03/21	$356,250
Lael Greer	Systems Administrator	London	21	2009/02/27	$103,500
Jonas Alexander	Developer	San Francisco	30	2010/07/14	$86,500
Shad Decker	Regional Director	Edinburgh	51	2008/11/13	$183,000
Michael Bruce	Javascript Developer	Singapore	29	2011/06/27	$183,000
Donna Snider	Customer Support	New York	27	2011/01/25	$112,000

Above Web Table will look like as below :-

So we need to go through all pages of web table and fetch data. Pagination implementation may be different in each table. For example :- In above table, you see a “Next” button which takes you next page of web table. It gets disable when there is no further page i.e if you are at page No 6, Next button will be disabled. We can use this condition to decide for number of iteration in code.

There can be another way where you can find the total pagination count and then iterate from Page 1 to N where N is total pages count.

How to wait till a page is loaded in web table

This is very important in reading rows and columns of a particular page. When you click on Next or any page number directly, it takes some time to load, If we do not wait till data for particular row is displayed, we may end up with either reading data of previous page in stead of next page or StaleElementReferenceException (Depends how you are using locator).

So for above table, when you are on a page say 1, you will see class name consists “current” as shown below:-

We need to wait till “current” is added to class name of desired page. Another way is to wait till we get updated entries text. If we are on page 1, we see “Showing 1 to 10 of 57 entries” and when we are on page 2, we see “Showing 11 to 20 of 57 entries”.

Step by step logic

1. Get the table headers and store in a List as we need to maintain insertion order.
2. We will store each row in to a LinkedHashMap and all LinkedHashMap in to a List.
3. Get first row of a page. Get all cells of first row and store cell data in to a Map where header name is key and cell is value. We need to iterate header list of Step 1 simultaneously.
4. Repeat the same process for all rows of first page.
5. Now we need to go to next page. We will get the current page first then click on Next button and wait till the next page gets “current” in its class attribute. The same we need to do in loop till we get last page.

Complete Java Code

package WebTablesExamples;

import java.time.Duration;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;
import org.testng.annotations.Test;

import io.github.bonigarcia.wdm.WebDriverManager;

public class ReadTableDataWithPaginationInListOfMap {

        @Test
        public void readTableDataInListOfMap() throws InterruptedException {
                // Browser initialization
                WebDriverManager.chromedriver().setup();
                WebDriver driver = new ChromeDriver();
                String fileURL = System.getProperty("user.dir");
                driver.get(fileURL + "/src/test/resources/htmlFiles/WebTablePagination.html");

                // Let's get header first
                String headerLoc = "//table[@id='dtBasicExample']//tr//th";
                List allHeadersEle = driver.findElements(By.xpath(headerLoc));
                List allHeaderNames = new ArrayList();
                for (WebElement header : allHeadersEle) {
                        String headerName = header.getText();
                        allHeaderNames.add(headerName);
                }

                // Each row will be a key value pair. So we will use LinkedHashMap so that order
                // can be retained.
                // All map will be added to a list.
                List> allTableData = new ArrayList>();
                boolean flag = true;

                while (flag) {
                        // Get total rows count
                        String rowLoc = "//table[@id='dtBasicExample']//tbody/tr";
                        List allRowsEle = driver.findElements(By.xpath(rowLoc));
                        for (int i = 1; i <= allRowsEle.size(); i++) {
                                // Getting specific row with each iteration
                                String specificRowLoc = "//table[@id='dtBasicExample']//tbody/tr[" + i + "]";
                                // Locating only cells of specific row.
                                List allColumnsEle = driver.findElement(By.xpath(specificRowLoc))
                                                .findElements(By.tagName("td"));
                                // Creating a map to store key-value pair data. It will be created for each
                                // iteration of row
                                LinkedHashMap eachRowData = new LinkedHashMap<>();
                                // Iterating each cell
                                for (int j = 0; j < allColumnsEle.size(); j++) {
                                        // Getting cell value
                                        String cellValue = allColumnsEle.get(j).getText();
                                        // We will put in to map with header name and value with iteration
                                        // Get jth index value from allHeaderNames and jth cell value of row
                                        eachRowData.put(allHeaderNames.get(j), cellValue);
                                }
                                // After iterating row completely, add in to list.
                                allTableData.add(eachRowData);

                        }
                        if ((driver.findElement(By.id("dtBasicExample_next")).getAttribute("class").contains("disabled"))) {
                                flag = false;
                                break;
                        } else {
                                // Before clicking Checking current page
                                int currentPage = Integer.parseInt(driver.findElement(By.xpath("//a[@class='paginate_button current']")).getText().trim());
                                System.out.println("We are at page "+currentPage+" now.");
                                driver.findElement(By.id("dtBasicExample_next")).click();
                                // Now we know for which page I need to wait
                                String customLoc = "//a[contains(@class,'paginate_button') and text()='"+(currentPage+1)+"']";
                                // Waiting till class attribute contains current
                                new WebDriverWait(driver, Duration.ofSeconds(10)).until(ExpectedConditions.attributeContains(By.xpath(customLoc), "class", "current"));
                        }
                }

                System.out.println(allTableData);
                driver.quit();

        }

}

Output

We are at page 1 now.
We are at page 2 now.
We are at page 3 now.
We are at page 4 now.
We are at page 5 now.
[{Name=Airi Satou, Position=Accountant, Office=Tokyo, Age=33, Start date=2008/11/28, Salary=$162,700}, 
{Name=Angelica Ramos, Position=Chief Executive Officer (CEO), Office=London, Age=47, Start date=2009/10/09, Salary=$1,200,000}, 
{Name=Ashton Cox, Position=Junior Technical Author, Office=San Francisco, Age=66, Start date=2009/01/12, Salary=$86,000}, 
{Name=Bradley Greer, Position=Software Engineer, Office=London, Age=41, Start date=2012/10/13, Salary=$132,000}, 
{Name=Brenden Wagner, Position=Software Engineer, Office=San Francisco, Age=28, Start date=2011/06/07, Salary=$206,850}, 
{Name=Brielle Williamson, Position=Integration Specialist, Office=New York, Age=61, Start date=2012/12/02, Salary=$372,000}, 
{Name=Bruno Nash, Position=Software Engineer, Office=London, Age=38, Start date=2011/05/03, Salary=$163,500}, 
{Name=Caesar Vance, Position=Pre-Sales Support, Office=New York, Age=21, Start date=2011/12/12, Salary=$106,450}, 
{Name=Cara Stevens, Position=Sales Assistant, Office=New York, Age=46, Start date=2011/12/06, Salary=$145,600}, 
{Name=Cedric Kelly, Position=Senior Javascript Developer, Office=Edinburgh, Age=22, Start date=2012/03/29, Salary=$433,060}, 
{Name=Charde Marshall, Position=Regional Director, Office=San Francisco, Age=36, Start date=2008/10/16, Salary=$470,600}, 
{Name=Colleen Hurst, Position=Javascript Developer, Office=San Francisco, Age=39, Start date=2009/09/15, Salary=$205,500}, 
{Name=Dai Rios, Position=Personnel Lead, Office=Edinburgh, Age=35, Start date=2012/09/26, Salary=$217,500}, 
{Name=Donna Snider, Position=Customer Support, Office=New York, Age=27, Start date=2011/01/25, Salary=$112,000}, 
{Name=Doris Wilder, Position=Sales Assistant, Office=Sidney, Age=23, Start date=2010/09/20, Salary=$85,600}, 
{Name=Finn Camacho, Position=Support Engineer, Office=San Francisco, Age=47, Start date=2009/07/07, Salary=$87,500}, 
{Name=Fiona Green, Position=Chief Operating Officer (COO), Office=San Francisco, Age=48, Start date=2010/03/11, Salary=$850,000}, 
{Name=Garrett Winters, Position=Accountant, Office=Tokyo, Age=63, Start date=2011/07/25, Salary=$170,750}, 
{Name=Gavin Cortez, Position=Team Leader, Office=San Francisco, Age=22, Start date=2008/10/26, Salary=$235,500}, 
{Name=Gavin Joyce, Position=Developer, Office=Edinburgh, Age=42, Start date=2010/12/22, Salary=$92,575}, 
{Name=Gloria Little, Position=Systems Administrator, Office=New York, Age=59, Start date=2009/04/10, Salary=$237,500}, 
{Name=Haley Kennedy, Position=Senior Marketing Designer, Office=London, Age=43, Start date=2012/12/18, Salary=$313,500}, 
{Name=Hermione Butler, Position=Regional Director, Office=London, Age=47, Start date=2011/03/21, Salary=$356,250}, 
{Name=Herrod Chandler, Position=Sales Assistant, Office=San Francisco, Age=59, Start date=2012/08/06, Salary=$137,500}, 
{Name=Hope Fuentes, Position=Secretary, Office=San Francisco, Age=41, Start date=2010/02/12, Salary=$109,850}, 
{Name=Howard Hatfield, Position=Office Manager, Office=San Francisco, Age=51, Start date=2008/12/16, Salary=$164,500}, 
{Name=Jackson Bradshaw, Position=Director, Office=New York, Age=65, Start date=2008/09/26, Salary=$645,750}, 
{Name=Jena Gaines, Position=Office Manager, Office=London, Age=30, Start date=2008/12/19, Salary=$90,560}, 
{Name=Jenette Caldwell, Position=Development Lead, Office=New York, Age=30, Start date=2011/09/03, Salary=$345,000}, 
{Name=Jennifer Acosta, Position=Junior Javascript Developer, Office=Edinburgh, Age=43, Start date=2013/02/01, Salary=$75,650}, 
{Name=Jennifer Chang, Position=Regional Director, Office=Singapore, Age=28, Start date=2010/11/14, Salary=$357,650}, 
{Name=Jonas Alexander, Position=Developer, Office=San Francisco, Age=30, Start date=2010/07/14, Salary=$86,500}, 
{Name=Lael Greer, Position=Systems Administrator, Office=London, Age=21, Start date=2009/02/27, Salary=$103,500}, 
{Name=Martena Mccray, Position=Post-Sales support, Office=Edinburgh, Age=46, Start date=2011/03/09, Salary=$324,050}, 
{Name=Michael Bruce, Position=Javascript Developer, Office=Singapore, Age=29, Start date=2011/06/27, Salary=$183,000}, 
{Name=Michael Silva, Position=Marketing Designer, Office=London, Age=66, Start date=2012/11/27, Salary=$198,500}, 
{Name=Michelle House, Position=Integration Specialist, Office=Sidney, Age=37, Start date=2011/06/02, Salary=$95,400}, 
{Name=Olivia Liang, Position=Support Engineer, Office=Singapore, Age=64, Start date=2011/02/03, Salary=$234,500}, 
{Name=Paul Byrd, Position=Chief Financial Officer (CFO), Office=New York, Age=64, Start date=2010/06/09, Salary=$725,000}, 
{Name=Prescott Bartlett, Position=Technical Author, Office=London, Age=27, Start date=2011/05/07, Salary=$145,000}, 
{Name=Quinn Flynn, Position=Support Lead, Office=Edinburgh, Age=22, Start date=2013/03/03, Salary=$342,000}, 
{Name=Rhona Davidson, Position=Integration Specialist, Office=Tokyo, Age=55, Start date=2010/10/14, Salary=$327,900}, 
{Name=Sakura Yamamoto, Position=Support Engineer, Office=Tokyo, Age=37, Start date=2009/08/19, Salary=$139,575}, 
{Name=Serge Baldwin, Position=Data Coordinator, Office=Singapore, Age=64, Start date=2012/04/09, Salary=$138,575}, 
{Name=Shad Decker, Position=Regional Director, Office=Edinburgh, Age=51, Start date=2008/11/13, Salary=$183,000}, 
{Name=Shou Itou, Position=Regional Marketing, Office=Tokyo, Age=20, Start date=2011/08/14, Salary=$163,000}, 
{Name=Sonya Frost, Position=Software Engineer, Office=Edinburgh, Age=23, Start date=2008/12/13, Salary=$103,600}, 
{Name=Suki Burks, Position=Developer, Office=London, Age=53, Start date=2009/10/22, Salary=$114,500}, 
{Name=Tatyana Fitzpatrick, Position=Regional Director, Office=London, Age=19, Start date=2010/03/17, Salary=$385,750}, 
{Name=Thor Walton, Position=Developer, Office=New York, Age=61, Start date=2013/08/11, Salary=$98,540}, 
{Name=Tiger Nixon, Position=System Architect, Office=Edinburgh, Age=61, Start date=2011/04/25, Salary=$320,800}, 
{Name=Timothy Mooney, Position=Office Manager, Office=London, Age=37, Start date=2008/12/11, Salary=$136,200}, 
{Name=Unity Butler, Position=Marketing Designer, Office=San Francisco, Age=47, Start date=2009/12/09, Salary=$85,675}, 
{Name=Vivian Harrell, Position=Financial Controller, Office=San Francisco, Age=62, Start date=2009/02/14, Salary=$452,500}, 
{Name=Yuri Berry, Position=Chief Marketing Officer (CMO), Office=New York, Age=40, Start date=2009/06/25, Salary=$675,000}, 
{Name=Zenaida Frank, Position=Software Engineer, Office=New York, Age=63, Start date=2010/01/04, Salary=$125,250}, 
{Name=Zorita Serrano, Position=Software Engineer, Office=San Francisco, Age=56, Start date=2012/06/01, Salary=$115,000}]

You can clone code from my git repo.

If you have any doubt, feel free to comment below.
If you like my posts, please like, comment, share and subscribe.
#ThanksForReading
#HappyLearning

Find all Selenium related post here, all API manual and automation related posts here and find frequently asked Java Programs here.

Many other topics you can navigate through menu.