Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update_dq improvement in findsat_mrt #201

Merged
merged 11 commits into from
Mar 14, 2024

Conversation

dvstark
Copy link
Contributor

@dvstark dvstark commented Feb 29, 2024

Description

This PR is to add in functionality to acstools.utils_findsat_mrt.updatedq to expand the input mask to match the size of the original DQ array if necessary. This is needed if a satellite trail was identified and masked on a binned image, which is common. Users could do this manually, but it is needed frequently enough that it made sense to include. This step has also been designed to handle images with odd dimensions,(e.g., WFC3/UVIS are 2051x4096) which ensures the code is applicable to other imaging data.

Warnings have been added into acstools.findsat_mrt.TrailFinder.rebin step to inform users if their data has odd dimensions and that rows/columns were being trimmed as needed to allow rebinning (this was happening already, but users may not have been aware). These trimmed rows/columns are added back in during the update_dq step after the satellite mask is enlarged.

@dvstark dvstark requested review from pllim and jryon February 29, 2024 16:09
@pllim
Copy link
Collaborator

pllim commented Feb 29, 2024

Lemme close/reopen and see.

@pllim pllim closed this Feb 29, 2024
@pllim pllim reopened this Feb 29, 2024
@pllim
Copy link
Collaborator

pllim commented Feb 29, 2024

You can ignore the triage failure in this PR. That is my bad.

@dvstark
Copy link
Contributor Author

dvstark commented Feb 29, 2024

just to put all comments in one space: I had asked what the triage step is and what might be causing it to fail

Copy link
Collaborator

@pllim pllim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You do have a minimal test in https://github.com/spacetelescope/acstools/blob/master/acstools/tests/test_wfc_findsat_mrt.py . If you can expand your test suite to cover this new option, would prevent regression in the future, but I'll leave that up to you.

acstools/findsat_mrt.py Outdated Show resolved Hide resolved
acstools/findsat_mrt.py Outdated Show resolved Hide resolved
acstools/utils_findsat_mrt.py Outdated Show resolved Hide resolved
acstools/utils_findsat_mrt.py Outdated Show resolved Hide resolved
acstools/utils_findsat_mrt.py Outdated Show resolved Hide resolved
acstools/utils_findsat_mrt.py Outdated Show resolved Hide resolved
acstools/utils_findsat_mrt.py Show resolved Hide resolved
acstools/utils_findsat_mrt.py Outdated Show resolved Hide resolved
Copy link
Contributor

@jryon jryon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran a few basic tests with different sizes of masks paired with ACS and WFC3 data. The results look good and make sense, except for the case when the mask is larger than the DQ array.

acstools/utils_findsat_mrt.py Show resolved Hide resolved
@dvstark
Copy link
Contributor Author

dvstark commented Mar 12, 2024

I'm unsure what to do about the error in the "Python 3.12 with remote data and dev dependencies" CI test:

py312-test-devdeps: commands[3] /home/runner/work/acstools/acstools/.tmp/py312-test-devdeps> pytest --pyargs acstools /home/runner/work/acstools/acstools/doc --remote-data -v
ImportError while loading conftest '/home/runner/work/acstools/acstools/conftest.py'.
../../conftest.py:10: in <module>
    from acstools.version import version
../../acstools/__init__.py:20: in <module>
    from . import satdet  # noqa
../../acstools/satdet.py:152: in <module>
    import skimage
../../.tox/py312-test-devdeps/lib/python3.12/site-packages/skimage/__init__.py:128: in <module>
    from ._shared import geometry
geometry.pyx:1: in init skimage._shared.geometry
    ???
E   ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
py312-test-devdeps: exit 4 (1.01 seconds) /home/runner/work/acstools/acstools/.tmp/py312-test-devdeps> pytest --pyargs acstools /home/runner/work/acstools/acstools/doc --remote-data -v pid=2518
  py312-test-devdeps: FAIL code 4 (303.45=setup[272.30]+cmd[9.07,20.65,0.42,1.01] seconds)
  evaluation failed :( (303.59 seconds)
Error: Process completed with exit code 4.

Some new issue with skimage and satdet?

@pllim
Copy link
Collaborator

pllim commented Mar 12, 2024

You can ignore the devdeps failure for now. Maybe scikit-image nightly wheel needs time to catch up to numpy nightly wheel. With numpy 2.0 in beta now, hopefully soon. I'll give it a few days and up to a week.

@dvstark
Copy link
Contributor Author

dvstark commented Mar 13, 2024

I've done the updates and am ready for this PR to be reviewed again. Thanks for the feedback so far!

Copy link
Contributor

@jryon jryon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple more very minor comments

acstools/utils_findsat_mrt.py Show resolved Hide resolved
acstools/utils_findsat_mrt.py Outdated Show resolved Hide resolved
Copy link
Contributor

@jryon jryon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy with this!

@dvstark
Copy link
Contributor Author

dvstark commented Mar 14, 2024

@pllim The Wheel-building step is being skipped. Do we need to make sure that passes before merging this in?

@pllim pllim added the Build wheels Build wheels for PR label Mar 14, 2024
@pllim
Copy link
Collaborator

pllim commented Mar 14, 2024

Does not hurt. I added the label to enable it.

Copy link
Collaborator

@pllim pllim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since Jenna approved, I have nothing much to add, except one Sphinx formatting fix.

acstools/utils_findsat_mrt.py Outdated Show resolved Hide resolved
Co-authored-by: P. L. Lim <2090236+pllim@users.noreply.github.com>
@pllim pllim merged commit d58bbf8 into spacetelescope:master Mar 14, 2024
12 checks passed
@pllim
Copy link
Collaborator

pllim commented Mar 14, 2024

Thanks, all!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants