The Scope and Objective: 

The objective is to webscrape financial website named SEC Edgar;Purpose is to gather categorical  symbols from sublinks on webpage and make custom  wathclists created from the information on the  internet and monitored in java or csharp or dotnet  open source project; All webscrapings will result in multiple GUI watchlists. it is up to you to choose from one of the following open source projects to bind your scrapings to. You might choose jbooktrader , or humaitrader, or jstock.org or from another open framework in java backend, for example it might be: http://www.cyanspring.com/index.php look under develop and downloads and features; It is java based and has connector to IB broker as well..and seems to have a nice small frontend. 

you should consider hooking the java program of your choice to JPython; This would allow us to call in python deeplearners and make forecasts of portfolios or watchlists. It would mean additional bonuses to you by making predictor plugins in the weeks and months to come. 

Please do not accept this task order if you cannot or dont intend to complete it ...or if you dont wish to be around in the future to extend the project. 

If  you prefer dotnet or vb or chsarp frontend..request some examples i have harvested to be sent to you...

You are required to make fullstack windows app running under windows 10. I may wish to have web version of this later..so please keep that in mind as you are creating project. 

You will make a windows desktop app. it will open and enduser can chose manually to run a script. This script results in a multithreaded multilink harvesting of symbosl by scraping a site and its internal links as listed down below in this document file attached to this project. 

You will harvest many symbol lists from the sublinks on the website and thru the different penney stock candidate lists.as desribed in .doc file below. 

You will customize the open source framework you chose ...ie jstock.org, or jbooktrader or cyanspring to allow for many watchlists ; these will popoulate multiple gui windows in your scraper..and will be timestamped and each category t hat is scraped will contain proper nomenclature of what it represents. This would preferably be setup as a spreadsheet like order of things. You would list category of the scraping at top of the gui. You would then position Symbol Date and rank of the symbol in the spreadsheet like gui tabs. You will be webscraping Sec Insider trading reports from SEC Edgar  site. I have attached document file below to describe what tabs need to be scraped. 

You will also have to allow importation feature of some of my large 300mb end of day datasets into the inmemory database or database module..so that those symbols historical datas can be called into the watchlists . So importation of CSV files is mandatory.

The watchlists should be made so that I can copy and paste out symbols of my choice from any watchlist. 

Perhaps you create some hot keys.. Key can be hit...Shift down arrow or or ctl key plus left click mouse,..for me to highlight and thus copy and paste out symbol names to notepad or other program enviroments. I need the watchlist to be toggleable as to on or off status as it pertains to updates thru data feeds.. Each list should be capable of being updated in realtime data feed. Enduser can choose the time frame of seconds to minutes to hourly or daily in order to have realtime live update on the portfolio monitor portion of the program gui interface. 

I also need ability to copy and paste any custom symbol list I have made...into a blank gui window. Exanple: If I paste in 20 symbols of my liking into a hot grid window..then the gui is then toggleable as to on or off status for realtime feed. After all watchlists are made.. i need ability to make any watchlist live or not. I need a toggle to turn any watchlist on or off.

Additionally ...two additional unused portfolio watchlists need to be made available to me:

Called: "Long Candidates Watchlist" and "Short Candidates Watchlist"

These guis will allow me to paste any number of symbols onto the watchlist gui..and they become populated on the watchlst grid. I will be allowed to turn on or off the realtime feed to these two types of Candidate lists..just in case I have decided to go long or buy some symbols or if I go short to sell some symbols.. any gui watchlist I choose can become fed with realtime data if I toggle it on or off.

This is important in case i come up with some symbols that i wish to follow and who may or may not be part of any prior scraping on your pat.. Thus , two custom watchlists from which i can watch for price changes and alerts or signals.

I need IB brokers data feed to work > I need windows executable in 21 days...I need this all to work flawlessly. 

Then in the future..I have some other project codes i need embedding. These will be centered around alerting and signal generation and paging and sms texting. I might wish to embed some mechanical system logics into the program for buy, sell and hold alerts. I can let you know about that a little later after we see what you do here and the quality of work. They would be additional extensions and additional funds to you. So please make multithreaded, safe array codes with pluggable architecture as it pertains to data feeds, scanning filters, and reports. 

Please note..if performed well..i have another data scrape and extensions to do. Please allow program to use 32gb memory or higher if avaialble..Please allow for multithreaded safe array support. Please make active realtime feed. 

I would like to embed a free reatltime and two to three professional realtime feeds . I have codes for these in python, java and csharp...thanks.. 

See attached document for website and sublinks to scrape. Please choose a frontend gui to use. If you wish to make in chsarp or dotnet. Please contact me at site and I can send you some private folders for use as frontend as well. 

PLEASE SEE DOCX FILE ATTACHED AS DESCRIPTION OF WEBSITE TO HARVEST.
You will have to email me for frontend if you are awarded project.

ric