Web crawler no.1

This weekend, I try to programming a application to get some data from website “https://zhihu.com”

This is my first time to make a web crawler.

I learned something about selenium.

First, It’s not easy to make a selenium and scrapy environment in Windows.

Lots of thing should be download and you need to install Ixml manually.

This is the officially guide but it’s not enough.

Installation guide - Scrapy 1.2.1 documentation

You also need this one to finish the install

Installing lxml, libxml2, libxslt on Windows 8.1

The source code is here:

DeronLee/tracklikeonzhihu

I will update the stats about this application and explain the code files one by one