rss Spider,Web crawler
Design and coding with all the love in the world by ShaneLau.
fork on [github](https://github.com/kissliux/rssSpider) ,I like star.
Web crawler, use NodeJs grab the RSS news, fetching the contents of the include title,released time, description, source,content and images . At the same time for the client to provide the standard news service interface.
Use nodejs captured above to network news, then insert into mongodb . How the RSS news and links to the url address and grab a summary of the news text and the news pictures For a news client, without pictures is a fatal blow, illustrated to attract users.
What’s included
This project grabbed headlines news source,url,description,images,content. Provide list query and single news query services.
Environment
nodejs、mongodb
Usage
npm install -d
node app.js
spide will begin after five seconds. cpoy to broswer
http://localhost:8002
-
Test rss : http://www.163.com/rss **
if you running successfully, you will see this.
![app](/img/rssspide/0.png)
index page
![rss spide index page ](/img/rssspide/1.png)
http://127.0.0.1:8002/list
![news list](/img/rssspide/2.png)
post detail page
![list ](/img/rssspide/3.png)