用scrapy 下载图片的快捷方式

爬虫文件xx.py

import scrapy
 
class ZolSpider(scrapy.Spider):
    name = 'zol'
    allowed_domains = ['zol.com.cn']
    start_urls = ['https://desk.zol.com.cn/bizhi/9691_117173_2.html']
 
    def parse(self, response):
        img_url = response.xpath('//img[@id="bigImg"]/@src').getall()
        yield {
            'image_urls':img_url
        }

2.settings.py 设置路径,打开下载通道

ITEM_PIPELINES = {
#    'image.pipelines.ImagePipeline': 300,
   'scrapy.pipelines.images.ImagesPipeline':200
}
 
# 图片下载路径
IMAGES_STORE= 'D:\python_reptile\scrapy中imagepipeline的使用\image\img'

3.pipeline.py,关键

from scrapy.pipelines.images import ImagesPipeline
 
class ImagePipeline(ImagesPipeline):
    def get_media_requests(self, item, info):
        return Request(item.get('key'))

这篇好文章是转载于：学新通技术网