AR
Anil Kumar
No. I'm getting response if I open the urls in browser. I'm not able to find the error.
then what happens to those URLs in Scrapy?
Size: a a a
2020 March 06
AK
I'm not able to find those missing urls.
AK
How to find those urls. The urls will be around 50000
AR
I don't understand the question, sorry
AK
I'm scraping data from 50000 urls. But for some urls the data is missing and I'm getting response code as 200 for the missing data urls.
AR
save the responses for pages that don't contain data, look at them, if they are some kind of error pages, detect that in the spider and retry them
AK
But when I the open the missing data url in the browser, I'm getting all the data.
AR
so?
2020 March 07
AR
покаж либу
AR
и чо говорит?
AR
может там твистед внутри?
AR
оба на асинкио, может поэтому
AR
да