Comment by SURAJSINGH RAJPUT on "Using Selenium to Download Corporate Action Data on Indian Stocks"

Great work dude, keep up the good work.

BSE seems to have changed the HTML code slightly. Need to update those XPath selectors.

This is the worst part of a programmed scraper, I have a scraper for Screener.in that scrapes EPS and Sales data and there too I face this very same issue every few months.

Changes:

Purpose selector id has changed from @id="ContentPlaceHolder1_ddlPurpose" to @id="ddlPurpose"
Submit button id changed from @id="ContentPlaceHolder1_btnSubmit" to @id="btnSubmit"
Download link selector changed from //[@id="ContentPlaceHolder1_lnkDownload1"]/img to //[@id="lnkDownload"]/i

Hi SURAJSINGH RAJPUT thanks for highlighting this change in the website. I am actually working on a much better Corporate Actions Scraper at the moment and will update this blog and my Youtube channel with the new method very soon. Just having a bit of time off to release some pressure!

Yash Roongta Yeah that's very important too.

That's pretty good to know. I will be waiting eagerly for the post.

Thanks Yash, had to do few changes outlined above ...worked beaut. Need to see correlation of NSE ones to BSE, hopefully smooth :D

Btw, ElementClickInterceptedException: element click intercepted: Element is not clickable at point (46, 901) while selected_purpose.click(). Had to get rid of it by adding WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, f'//*[@id="ddlPurpose"]/option[@value=\"{corp_act}\"]'))).click()

Search Hashnode