Selenium

Selenium module can be used in dynamic page web scraping on Windows Systems. It supports Firefox, IE, Chrome and Remote.

step 1 install selenium ：

pip install selenium

step 2 install webdriver and put it in the project's working directory：

For example, I use Chrome, so I need download a ChromeDriver. Download link： https://sites.google.com/a/chromium.org/chromedriver/home

step 3 start from opening a dynamic page web url:

browser = webdriver.Chrome()
browser.get('http://money.cnn.com/search/index.html?sortBy=date&primaryType=mixed&search=Search&query=trade%20war')
time.sleep(10)

For detailed use, refer to the Selenium documentation: https://selenium-python-zh.readthedocs.io/en/latest/index.html