Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove pulls trigger #450

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

Remove pulls trigger #450

wants to merge 6 commits into from

Conversation

nora-codecov
Copy link
Contributor

@nora-codecov nora-codecov commented Dec 5, 2024

Part of the quest to reduce locks by eliminating triggers https://github.com/codecov/internal-issues/issues/1029

The trigger is deleted by these changes. The behavior is replaced in a cron job on worker codecov/worker#947.

What the trigger did: every time a pull was updated, if it was transitioning from OPEN to any other state, it set the flare field to null.
flare is an Archive field, so the value is either in _flare on the object or in our Archive storage.

Replacement behavior: see FlareCleanupTask codecov/worker#947
Once the trigger is removed, when the pull transitions from OPEN to any other state, leave flare alone in order to reduce locks and wait time.
The cron job runs overnight, gets all pulls that are non-OPEN and have flare (either in our db or in Archive storage), clears it (either in our db or in Archive storage) so flare is null.

more about flare:

Copy link

codecov bot commented Dec 6, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 89.97%. Comparing base (abc6b36) to head (741151b).

Current head 741151b differs from pull request most recent head 2331c4a

Please upload reports for the commit 2331c4a to get more accurate results.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #450      +/-   ##
==========================================
- Coverage   90.58%   89.97%   -0.61%     
==========================================
  Files         401      324      -77     
  Lines       12508     9199    -3309     
  Branches     2103     1632     -471     
==========================================
- Hits        11330     8277    -3053     
+ Misses       1069      859     -210     
+ Partials      109       63      -46     
Flag Coverage Δ
shared-docker-uploader ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@Swatinem Swatinem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’m unsure related to the should_write_to_storage changes. I don’t think those are needed necessarily?

Comment on lines 429 to 433
if self.state != PullStates.OPEN.value:
# while a pull is OPEN, we check whether to write_to_storage
# when a pull is no longer OPEN, we no longer want the flare in storage.
# The nightly cron job cleans up the value in storage (see FlareCleanupTask in worker)
return False
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this would rather mean that the flare is being stored in the JSON field of the table.
I’m unsure what would happen if you change this value based on the state. will is copy over data from storage to the database? or will that only happen if you save a new value in the field?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good call - I think I was thinking that if this came back False it wouldn't be saved anywhere, but you're right, this would just force it onto the db field. Removing.

@@ -147,7 +147,7 @@ def delete_file(self, bucket_name, path):
except ClientError:
raise

def delete_files(self, bucket_name, paths=[]):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This default value is dangerous and unnecessary. fixed the occurrences I found across the project.

@nora-codecov nora-codecov force-pushed the nora/1029 branch 3 times, most recently from 710ef8b to 741151b Compare December 28, 2024 02:22
…rchiveService, remove default value from paths
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants