Shailesh Jain
Jul 29, 2017 · 1 min read

Thank you for this article. I’m looking at building a topical crawler (across multiple domains) which can basically create an index of web pages based on topics. Any recommendations?

    Shailesh Jain

    Written by

    Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
    Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
    Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade