Size: a a a

2021 April 30

MA

Murilo Andrade in Scrapy
I have a question related to a button that when clicking, it takes me to another page. How do I do this with scrapy?
источник

MA

Murilo Andrade in Scrapy
before that, login with login and password authentication. This I managed to do, now inside the site I need to click on that button and it takes me to a page.
источник

МС

Михаил Синегубов... in Scrapy
F12 -> Network
See the requests that go to the page.
But, this is not always required. Most often, you can skip this step and immediately request a page with the data.

scrapy.httpRequest - for GET requests: https://docs.scrapy.org/en/latest/topics/request-response.html#request-objects
scrapy.http.FormRequest  - for POST requests: https://docs.scrapy.org/en/latest/topics/request-response.html#formrequest-objects
источник

МС

Михаил Синегубов... in Scrapy
The button calls POST or GET request
источник

МС

Михаил Синегубов... in Scrapy
I hope I'm writing clearly. :))
Using translator
источник

MA

Murilo Andrade in Scrapy
button behavior is get request
источник

МС

Михаил Синегубов... in Scrapy
ok
yield scrapy.Request(url='some url', callback=self.some_callback_func)


link, which you must go, is known for?
источник

MA

Murilo Andrade in Scrapy
I did this with selenium webdriver, but there when I put login and password and click, I request the url directly with driver.get ("https: // URL")
источник

МС

Михаил Синегубов... in Scrapy
ok, have you already logged in through scrapie? Or does the request for authorization need the same?

@trefto , @wrar42 - спасите-помогите, знаний языка не хватает 😂
источник

AR

Andrey Rahmatullin in Scrapy
я ничо не понял :))
источник

К

Кирилл in Scrapy
Вроде нужно нажать на кнопку чтоб куда-то перебросило
источник

МС

Михаил Синегубов... in Scrapy
я то же  :).
человеку хочет "кликнуть по кнопке после авторизации", но, я не пойму, он сам запрос на авторизацию сделал или нет
источник

MA

Murilo Andrade in Scrapy
Yes, I managed to log in
источник

МС

Михаил Синегубов... in Scrapy
о, авторизовался, помогите человеку :)
источник

К

Кирилл in Scrapy
so, just send another request from the login callback
источник

MA

Murilo Andrade in Scrapy
which command would I give?
источник

К

Кирилл in Scrapy
you are using selenium, right? Are you using scrapy+selenium or pure selenium?
источник

MA

Murilo Andrade in Scrapy
I have the code in selenium, now I'm doing it with pure scrapy
источник

К

Кирилл in Scrapy
okay, then you should send the first request for login, after that send another request form login's callback
источник

MA

Murilo Andrade in Scrapy
import scrapy

class BotEcacSpider(scrapy.Spider):
   name = 'bot_ecac'

   start_urls = ['https://cav.receita.fazenda.gov.br/autenticacao/login']


   def parse(self, response, **kwargs):

       return scrapy.FormRequest.from_response(
           response,
           formdata={
               'NI': '*******',
               'Senha': '*******',
               'CodigoAcesso': '********',
           },
           callback=self.after_login

       )

   def after_login(self, response):
       response.meta.get("https://www3.cav.receita.fazenda.gov.br/extratodirpf/#/")
       title = response.xpath('//*[@id="informacao-perfil"]/div[1]/span/text()').get()
       print(title)
источник