Find all potential initiating GET requests

[Sim4n6]

I would like to identify all potential initiating GET requests using CodeQL in python language.
I have this one HTTP::Client::Request A data-flow node that makes an outgoing HTTP request.
as in the documentation

But does it identify a GET request performed by requests modules or another module? Please, how do I proceed to catch them all ?

[hvitved]

@github/codeql-python : Could you help out here, please?

[RasmusWL]

Hi @Sim4n6. We don't currently model the request method, but Http::Client::Request will give you all outgoing requests, so you should be able to use the code below to find any modeled HTTP client request

private import python
private import semmle.python.dataflow.new.DataFlow
private import semmle.python.Concepts

select any(Http::Client::Request r)

I'm very interested to hear why you only care about GET requests.

If you really need this, I'd be open to help you figure out how we can add the request method to our modeling. However, notice that there will be cases like the one below where it's not possible to know what method will be used

def func(method):
    requests.request(method, "https://github.com")

[Sim4n6]

I'm very interested to hear why you only care about GET requests.

My point is to identify ways of retrieving remote files (download).
I mean, using select any(Http::Client::Request r) I was not able to catch wget.download() or urllib.request.urlretrieve(). Which seems to be strange.

where it's not possible to know what method will be used

Indeed, I don't think it is required. I will though stick with the select any(Http::Client::Request r) . Since a POST request from a client for instance may be used to download a file due to the server-side way of handling the HTTP verb.

Thank you for your reply @RasmusWL.

[RasmusWL]

I mean, using select any(Http::Client::Request r) I was not able to catch wget.download() or urllib.request.urlretrieve(). Which seems to be strange.

Our library modeling might not cover all libraries. Happy to accept PRs if you care to add this modeling 👍

[Sim4n6]

I see, thank you very much I will be working on that soon.

[Sim4n6]

Sorry, man I completely forget about this one.