Compare Proposal

Nothing to compare.

Web scraping using python / selenium / bs4

  • Posted at : 27 days ago
  • Post Similar Project


Posted By -


Projects Posted : 2
Projects Paid : 0
Services Purchased : 0
Total Spent :
Feedbacks : 0 %

Project Details show (+) hide (-)

I collect stock market ticker symbols (and their associated company names) for private research. I prefer to gather these from the stock exchanges around the world. Some exchanges make it very easy to gather this information by providing downloadable spreadsheets or CSV files, others present the data in compact HTML tables, while still others spread this information out over a fairly large number of pages that require considerable navigation to view.

I am looking for a person with extensive experience with Python (2.7), Beautiful Soup and Selenium. The task is to write script to automate the process of navigating sites (using selenium) and extracting and assembling [ticker symbol, company name] tuples from the HTML using BS4, and ultimately dumping the collected data to a flat file (preferably JSON).

Here is an example of a national stock exchange site for which such a script needs to be written:

The interface consists of three levels. The top level allows a user to select the starting letter of a company name [A-Z,0-9]. 

Each of these hyperlinked starting characters leads to the second level of the interface that displays a table of companies. Each row in this table displays a company's name, but not its ticker. 

To see the ticker, the user must click on the name hyperlink to be taken to a "Company detail" page (the third level of the interface). This detail page shows one or more alphanumeric ticker symbols (called the "Trading Code") along with other useful company information to be parsed.

I need this script to run on a headless Chrome browser running on Linux (Ubuntu).

This 3-level tree must be traversed by using Selenium since some of the hrefs fire JS events.

If this is something you have extensive experience with, please apply! Make sure you have an excellent command of the three requisite technologies before applying. 

I am looking for one or more people to whom I can assign many such automated data harvesting tasks of varying degrees of complexity. Please base your quote on the particular example described above.