crawlee

爬取网页数据的一个工具crawlee, 地址： https://crawlee.dev

安装

npx crawlee create my-crawler

cd my-crawler

npm start

示例

import { CheerioCrawler } from 'crawlee';

const crawler = new CheerioCrawler({
    async requestHandler({ $, request }) {
        const title = $('title').text();
        console.log(`The title of "${request.url}" is: ${title}.`);
    }
})

await crawler.run(['https://www.baidu.com']);