Obtaining Real Time Port Closures for the United States using Selenium
Ports may be closed for a variety of reasons. Terrorist attacks, severe weather, and marine casualty events are some examples. Generally information on port closure is disseminated by the Captain of the Port, and for vessel owners and charterers the information is usually obtained through an intermediary such as an agent.
In this article, I demonstrate automatically obtaining port closure information using Selenium.
Introduction
I came around to the question of port closures by investigating maritime security levels (MARSEC) and discovered that — in addition to the lack of a MARSEC API — there does not appear to be an endpoint for dissemination of port closure information.
Port closures are important because they will cause vessel delays. This impacts commerciality — and also the environment as you will have a collection of vessels waiting around burning fuel within the vicinity of the port.
Any voyage management system should be collecting an endpoint for these closures. And some may: I’m unclear as I don’t have access to every VMS on the market. So, for now, I will demonstrate the best way I could find to automatically obtain port closure information using Selenium.
Method
Step 1 — Obtain ChromeDriver
Because the port closure tables within the U.S. Coast Guard’s Homeport are rendered in Javascript, you will need ChromeDriver (or similar) and Selenium to interact with the rendered port closure table within in each area of responsibility. We’ll use sector San Francisco as an example.
Step 2-Identify Port Closure Table within Homeport
The port closure table attribute within San Francisco sector’s Homeport page is ports46.
Step 3-Delay Timer and Strip
It takes awhile for the Homeport page to load. You can delay the retrieval of the element by 10 seconds using:
wait = WebDriverWait(driver, 10)The port closure table can then be stripped:
table = wait.until(EC.presence_of_element_located((By.ID, 'ports46')))
# Extract table rows using Selenium
rows = table.find_elements(By.TAG_NAME, 'tr')
# Initialize an empty list to store the table data
table_data = []
# Iterate through the rows and extract data from each cell
for row in rows:
cells = row.find_elements(By.TAG_NAME, 'td')
row_data = [cell.text.strip() for cell in cells]
table_data.append(row_data)Which will result in the following outcome:
/Users/jordantaylor/PycharmProjects/marsec/venv/bin/python /Users/jordantaylor/PycharmProjects/marsec/main.py
[]
['ANCHORAGE 9 (SOUTH SFB)', 'Open', '', '2017-05-22']
['BODEGA BAY', 'Open', '', '2017-05-22']
['CARQUINEZ STRAIT (& FACILITIES)', 'Open', '', '2017-05-22']
['CORTE MADERA CHANNEL', 'Open', '', '2017-05-22']
['CRESCENT CITY HARBOR', 'Open', '', '2017-05-22']
['HALF MOON BAY', 'Open', '', '2017-05-22']
['HUMBOLDT BAY', 'Open', '', '2017-05-22']
['MONTEREY HARBOR', 'Open', '', '2017-05-22']
['MOSS LANDING', 'Open', '', '2017-05-22']
['MOTCO', 'Open', '', '2017-05-22']
Process finished with exit code 0Step 4-Wrapping it up
The above example is for sector San Francisco, but other sectors can be scraped by changing the locator. Only the first tab is scraped as well, for sake of article brevity I’ll leave it to the reader to build code to scrape the remainder. Further, if anyone wants to build a scraper for all or some of the other domestic U.S., I will include the extensions at the end of this article.
What next? You can automate above process, commit to a database, and generate a program interface.
Final code example (note that below was cleaned using GPT 3.5):
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
# URL of the webpage to scrape
url = 'https://homeport.uscg.mil/port-directory/san-francisco#'
# Initialize the Selenium WebDriver (in this case, using Chrome)
driver = webdriver.Chrome()
# Navigate to the same webpage using Selenium
driver.get(url)
try:
# Wait for the element with ID 'ports46' to become visible
wait = WebDriverWait(driver, 10)
table = wait.until(EC.presence_of_element_located((By.ID, 'ports46')))
# Extract table rows using Selenium
rows = table.find_elements(By.TAG_NAME, 'tr')
# Initialize an empty list to store the table data
table_data = []
# Iterate through the rows and extract data from each cell
for row in rows:
cells = row.find_elements(By.TAG_NAME, 'td')
row_data = [cell.text.strip() for cell in cells]
table_data.append(row_data)
# Print the extracted table data
for row in table_data:
print(row)
finally:
# Close the Selenium WebDriver
driver.quit()Conclusion
It should be mentioned as well that I am aware that above is a messy, and possibly untenable, way of obtaining automated port closure data. But it was the best I could find.
The bigger question is why the U.S. Coast Guard does not offer endpoints for this data directly from their repository, which would be a heavy value proposition. If they do, and I missing it, please do leave a comment.
References
(n.d.). Selenium WebDriver. Retrieved December 7, 2023, from https://www.selenium.dev/
Port Directory Content. (n.d.). Port Directory Content. Retrieved December 7, 2023, from https://homeport.uscg.mil/port-directory/san-francisco#
WebDriver for Chrome — Version Selection. (n.d.). ChromeDriver. Retrieved December 7, 2023, from https://chromedriver.chromium.org/downloads/version-selection
Appendix — Locators for all U.S. Domestic Ports
Parent extension: https://homeport.uscg.mil
/port-directory/boston
/port-directory/buffalo-(buffalo-and-cleveland)
/port-directory/charleston
/port-directory/columbia-river
/port-directory/corpus-christi
/port-directory/d8-gulf-of-mexico
/port-directory/delaware-bay
/port-directory/detroit
/port-directory/duluth
/port-directory/guam
/port-directory/honolulu
/port-directory/houma
/port-directory/houston-galveston
/port-directory/jacksonville
/port-directory/key-west
/port-directory/lake-michigan
/port-directory/long-island-sound
/port-directory/los-angeles-long-beach
/port-directory/lower-mississippi-river-(memphis)
/port-directory/maryland-ncr
/port-directory/miami
/port-directory/mobile
/port-directory/new-orleans
/port-directory/new-york
/port-directory/north-carolina
/port-directory/northern-new-england-(portland-maine)
/port-directory/ohio-valley
/port-directory/pittsburgh
/port-directory/port-arthur-and-lake-charles
/port-directory/prince-william-sound-(valdez)
/port-directory/san-diego
/port-directory/san-francisco
/port-directory/san-juan
/port-directory/sault-ste-marie
/port-directory/savannah
/port-directory/seak-southeast-alaska-(juneau)
/port-directory/seattle-(puget-sound)
/port-directory/southeastern-new-england-(providence)
/port-directory/st-petersburg
/port-directory/upper-mississippi-river-(st-louis)
/port-directory/virginia
/port-directory/western-alaska-(anchorage)
