-
Notifications
You must be signed in to change notification settings - Fork 50
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Scraper only scrapes the first review page #5
Comments
Hi! I have the same issue, have you figured out how to fix it yet? |
Hi, I am also facing same issue. Please let me know if you figured out how to fix it. |
I believe when you place the URLs in a TXT file, it reads the new line (\n) character when could mess with the URL. I found that if I strip off the last character of the URL being read or place the URLs in a python list, it worked fine for me. |
I wanted to start with just an asin, get to the first page of reviews, and then keep going to the next page. |
The thing is Amazon restricts from scraping ,so when you make too many frequent requests this happens and scrapes only the first page and repeats this only so i would suggest use the request.sessions this might work and it worked for me |
Hi, first of all thank you for the code!
I am however having the problem that when scraping multiple pages of reviews for the same product, only the first page gets scraped. The other pages get "scraped" too and show up in the data, but the actual reviews extracted from them are the ones from the first page.
Does anyone know how to fix this?
Thank you!
The text was updated successfully, but these errors were encountered: