How to callback on 301 redirect without crawling in scrapy?

I am scraping a search result page where in some cases a 301 redirect will be triggered. In that case I do not want to crawl that page, but I need to call a different callback function, passing the redirect URL string to it.

I belive it should be possible to do it along the rules, but could not figure out how to:

rules = (
            callback= 'parse_item'

Is this possible with scrapy 2.4.x

Read more here:

Content Attribution

This content was originally published by Merlin at Recent Questions - Stack Overflow, and is syndicated here via their RSS feed. You can read the original post over there.

%d bloggers like this: