Getting Real-time Stock Price Data with Python Selenium
Read this article in Portuguese
Have you ever wonder how to get stock prices from trustworthy sources, stoling only a few bucks?
On this tutorial I’m gonna to show you how to extract stock data prices with Python Selenium from tradingview.com, let’s move it on!
First of all, you should have python installed on your computer and I highly recommend that you use Jupyter Notebook to get instant feedback of what you’re coding.
Go to tradingview.com, select any stock and go to “Full-featured chart”, in this page you can see a trending line of the chosen stock and also you can see bellow some stock names:
As we’re interested on how to extract these data, the tradingview offers download these data in csv format if you have a monthly subscription of its services (currently $ 15 / month), so make sure to follow these steps to download it:
Aaaaaaand voilà, click on this button to download the CSV file
Ok, we’ve got the CSV file with the stock price data, but you won’t be in front of your computer 24/7 just clicking on this button, right?
So, let’s automate this task with Selenium!
First, import packages and get our credentials of tradingview.com
import time
from selenium import webdriver# Import your username and password of tradingview.com
from credentials import get_credentials
credentials = get_credentials()
username = credentials['username']
password = credentials['password']
To automate this task we could use some python libraries: Scrapy, Beautiful Soup and Selenium
But, as the CSV download button is a javascript method, we should have problem with Scrapy and Beautiful Soup, so let’s use Selenium that deals perfectly with these type of tasks!
We need a webdriver to integrate our code with the browser, download chromedriver from this url: https://chromedriver.chromium.org/downloads
# You should download chromedriver and place it in a high hierarchy folder
chromedriver_path = "C:/webdrivers/chromedriver.exe"# This is the generic url that I mentioned before
url = "https://www.tradingview.com/chart/GZ2VoO8U/#signin"
Open the browser
def open_browser(chromedriver_path):
chrome_options = webdriver.ChromeOptions()
preferences = {"download.prompt_for_download": False,
"download.default_directory":r"C:\Users\Felip\Documents\Projects\Trading\source\\",
"download.directory_upgrade": True,
"profile.default_content_settings.popups": 0,
"profile.default_content_setting_values.notifications": 2,
"profile.default_content_setting_values.automatic_downloads": 1
}chrome_options.add_experimental_option("prefs",preferences)
driver = webdriver.Chrome(executable_path=chromedriver_path,
chrome_options=chrome_options)
return driver
Site login
def site_login(username,password,url,driver):
driver.get(url)
driver.find_element_by_class_name("tv-dialog__close").click() driver.find_element_by_name('username').send_keys(username)
driver.find_element_by_name('password').send_keys(password)
driver.find_element_by_class_name('tv-button__loader').click()
time.sleep(10) # ensure the page loads (bottleneck)
return driver
Get CSV
def get_csv(driver):
driver.find_element_by_xpath('//div[@title="Export screener data to a CSV file"]').click()Run
driver_1 = open_browser(chromedriver_path)
driver_2 = site_login(username,password,url,driver_1)
get_csv(driver_2)That’s it, simple as that ! If you get all these steps correctly, you should have a CSV file with stock price data, all that you need to do now it automate this task to run every 10 seconds and generate a new csv file of refreshly data! Afterwards, you should place every new data into a large dataframe and get your analysis or ML to work !
If at any point you are stuck or confused on a topic or concept, feel free to ask for help and I will do my best.
You can also watch my YouTube video, where I pass through all these steps
Good luck !
