lighthousebot not failing PRs when score < minScore

[ankeetmaini]

I debugged this by running the frontend server in my local and the reason is when I add lighthousebot as a collaborator, the request isn't accepted by the bot and hence it doesn't have the rights and fails. It can only post a comment.

I even checked github api to add a webhook and a handler in server to auto accept requests but at this time there isn't a way we can automate this process.

[ebidel]

If you're trying to run your own CI server, the lighthouse bot won't work with that setup. That's because you don't have the correct .oauth_token credentials. Instead, I'd get an oauth token for yourself, follow the instructions at https://github.com/ebidel/lighthouse-ci#running-your-own-ci-server, and it will post as you.

[ankeetmaini]

No, I created my own .oauth_token, it's just that if I don't accept the collaborator request github api doesn't allow to change the status.

See this PR -> ankeetmaini/react-hn#11, here your bot posts a comment but fails to mark the PR as a failure.

The one I tried with my own .oauth_token passes because I accepted the collaborator request from my bot account visible in the screenshot perf-bot ankeetmaini/react-hn#9

[ankeetmaini]

So I'm guessing that the lighthousebot account needs to accept collaborator request from all the repos. Correct me if I am wrong?

[ebidel]

I see what you mean! Didn't realize there was an invitation link to collaborate on repos. I accepted your invite.

Hmm, do you know of a way to work around that? Email invites are not going to scale well :(

[ankeetmaini]

I checked the GitHub API if we can add a handler in the front end server to auto accept any invite which we can hook into via git webhooks. But at this time there doesn't exist an API. I can raise a feature request but not sure if it'll be taken up! Or we can ask repository owners to give their .oauth_token and use that? In a set like you're doing for the webpage test request id? But it'll grow with time, might make server slow and we'd need to move this to a redis client.

…

On Mon, Sep 18, 2017, 10:53 PM Eric Bidelman ***@***.***> wrote: I see what you mean! Didn't realize there was an invitation link to collaborate on repos. I accepted your invite. Hmm, do you know of a way to work around that? Email invites are not going to scale well :( — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#8 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGWDl8pgIY5SJ_ZGaoMbZ4KmPA3mOXNDks5sjqb6gaJpZM4PZufR> .

[ebidel]

An Authorization header with the user's oauth token would be great, but that would that work?
Wouldn't that also post as the user and not the lighthouse bot profile icon?

[ankeetmaini]

Yes the moment we depend upon the user's oauth it'll be posted as his comment. We can add the icon in the comment body? :-D

…

On Mon, Sep 18, 2017, 11:44 PM Eric Bidelman ***@***.***> wrote: An Authorization header with the user's oauth token would be great, but that would that work? Wouldn't that also post as the user and not the lighthouse bot profile icon? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#8 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGWDl1FwU0R9uv5OVrgHOb0Mzq5Nce7pks5sjrMfgaJpZM4PZufR> .

[ankeetmaini]

Also we can save the user's . oauth_token in a secret environment variable (Travis) which the lighthouse-ci will be able to access and send to the frontend server!

…

On Tue, Sep 19, 2017, 12:07 AM Ankeet Maini ***@***.***> wrote: Yes the moment we depend upon the user's oauth it'll be posted as his comment. We can add the icon in the comment body? :-D On Mon, Sep 18, 2017, 11:44 PM Eric Bidelman ***@***.***> wrote: > An Authorization header with the user's oauth token would be great, but > that would that work? > Wouldn't that also post as the user and not the lighthouse bot profile > icon? > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <#8 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AGWDl1FwU0R9uv5OVrgHOb0Mzq5Nce7pks5sjrMfgaJpZM4PZufR> > . >

[ebidel]

Those might be an option. Without a proper Github Integration, I do like that users see the official name and icon:

Otherwise, they're going to see themselves and their name. Could be confusing.

Right now the volume of collaborator emails is low. I was just ignoring them :(

[ankeetmaini]

Let me know if I can help in resolving this!

[schoenwaldnils]

Hi, I'm having the same issue.

Even when I dont use my own CI server.

https://travis-ci.org/schoenwaldnils/timeline/builds/287505971
schoenwaldnils/timeline#1

The comment is there, but I don't get a failing test.

[ebidel]

@schoenwaldnils That setup looks good to me. You've got the bot as a collaborator on the repo and using an API as the Travis env https://github.com/ebidel/lighthouse-ci#get-an-api-key?

[schoenwaldnils]

It works now. I might had to wait until @lighthousebot accepts the collaborator-request.

[riderx]

3 days and lighthouse still didn't accept :/ how can i change that ?

[EoinTraynor]

This is obviously still an ongoing issue.
What is the expected duration for lighthouse to accept my collaborator-request? It's been 6 days since I have requested it.

[krzkaczor]

If anyone still needs it lighthouse-keeper supports failing PRs if score is too low, as well as tracking score changes between PRs and many other things. Cheers!