How to Get Text Data from Google Maps Using R Selenium

In this article, we’ll explore how to extract text data from Google Maps using the R Selenium package. We’ll delve into the details of the code, discuss potential issues, and provide examples to help you overcome common challenges.

Introduction to R Selenium

R Selenium is a popular package in R that allows you to automate web browsers for tasks such as data scraping, testing, and automation. It provides an easy-to-use interface for interacting with web pages, making it a great tool for extracting data from websites like Google Maps.

The package relies on the Selenium WebDriver technology, which allows us to interact with web browsers programmatically. We can use different browser types, including Chrome, Firefox, PhantomJS, and Internet Explorer, depending on our needs.

Setting Up R Selenium

Before we dive into the code, let’s set up our R environment for R Selenium.

First, install the required packages:

# Install R Selenium
install.packages("RSelenium")

Next, load the necessary libraries:

library(RSelenium)
library(XML)
library(xlsx)

Preparing the Data

In this section, we’ll prepare our data for extraction. We’ll use an example dataset with Google Maps URLs.

Create a new R script or modify an existing one and add the following code:

# Load libraries
library(RSelenium)
library(XML)
library(xlsx)

# Create a test dataset with Google Maps URLs
test <- read.xlsx("C:\\Selenium_Tool\\Segmentaion_Files\\test.xlsx", 1)

# Format the addresses for use in Google Maps
test$addr <- paste(test$CUST_ADDRESS, test$CUST_CITY, test$CUST_STATE, sep = ",")

This code loads the necessary libraries and creates a test dataset with Google Maps URLs. We’ll format the addresses to ensure they’re correctly structured.

Creating the R Selenium Connection

Next, we’ll create an R Selenium connection using the rsDriver() function.

# Create an R Selenium connection
rd <- rsDriver(port = 4567L, browser = c("chrome", "firefox", "phantomjs", "internet explorer"),
              version = "latest", chromever = "latest", geckover = "latest",
              iedrver = NULL, phantomver = "2.1.1", verbose = TRUE, check = TRUE)

In this code snippet, we create an R Selenium connection using the rsDriver() function. We specify the port number, browser types, and versions to use.

Navigating Google Maps

Now that we have our R Selenium connection established, let’s navigate to the Google Maps URL.

# Create a remote driver object
remDr <- rd[["client"]]

# Navigate to the first Google Maps URL
remDr$navigate(test$URL[1])

In this code, we create a remote driver object from our R Selenium connection and navigate to the first Google Maps URL.

Finding Elements on the Page

Next, we’ll use the findElements() method to find elements on the page.

# Find elements using 'class' attribute
webElem <- remDr$findElements(using = 'class', 'section-listbox')

In this code snippet, we find elements using the findElements() method with the class name section-listbox.

Extracting Text Data

Now that we’ve found the elements on the page, let’s extract the text data.

# Get the first element
elem <- webElem[1]

# Extract the text data
test$result[i] <- elem$getElementText()[[1]]

In this code, we get the first element and extract its text data using the getElementText() method.

Handling Errors

When working with R Selenium, it’s essential to handle errors that may occur during execution. Let’s add some error checking to our code.

# Loop through each Google Maps URL
for (i in 1:length(test$CUST_ID)) {
    tryCatch(
        expr = {
            # Navigate to the Google Maps URL
            remDr$navigate(test$URL[i])
            
            # Find elements using 'class' attribute
            webElem <- remDr$findElements(using = 'class', 'section-listbox')
            
            # Get the first element
            elem <- webElem[1]
            
            # Extract the text data
            test$result[i] <- elem$getElementText()[[1]]
        },
        error = function(e) {
            # Handle errors
            cat("Error occurred for URL:", test$URL[i], "\n")
        }
    )
}

In this code snippet, we use tryCatch() to catch any errors that may occur during execution. We navigate to the Google Maps URL, find elements using the findElements() method, get the first element, and extract its text data.

Closing the R Selenium Connection

Finally, let’s close our R Selenium connection.

# Close the remote driver object
remDr$close()

# Stop the server
rd[["server"]]$stop()

In this code snippet, we close the remote driver object using remDr$close() and stop the server using rd[["server"]]$stop().

Last modified on 2023-12-03