Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tme series data don't overlap after filtering in TMS #9

Open
pritamd47 opened this issue Feb 9, 2023 · 0 comments
Open

Tme series data don't overlap after filtering in TMS #9

pritamd47 opened this issue Feb 9, 2023 · 0 comments
Assignees
Labels
bug Something isn't working edge case

Comments

@pritamd47
Copy link
Member

TMS can receive time-series that overlap in time when passed into the function, but may get unaligned after filtering out the suspected erroneous points. This case is not handled in TMS currently.

area['filtered_area'] = deviation_from_sar(area['area'], sar['area'], AREA_DEVIATION_THRESHOLD)
area.rename({'area': 'unfiltered_area'}, axis=1, inplace=True)
# area.rename({'filtered_area': 'area'}, axis=1, inplace=True)
area_filtered = area.dropna(subset=['filtered_area'])
area_filtered.loc[:, 'days_passed'] = area_filtered.index.to_series().diff().dt.days
area_filtered.loc[:, 'trend'] = area_filtered['filtered_area'].diff()/area_filtered['days_passed']
sar, area_filtered = clip_ts(sar, area_filtered, which='left')

Example:
Using default Area-Deviation threshold (5%), for the Taylor park reservoir (Gunnison, CO) in the test suite, the threshold is very low (0.37 sq. km.). This causes most of the areas to get filtered out and the resultant dataframe's max date is less than the SAR's min date. This results in an empty dataframe when clip_ts is used. Currently, the hotfix was resolved by increasing the AREA_DEVIATION_THRESHOLD value for smaller reservoirs thru 03a1b8c. The new default thresholds are as follows:

  • 5% for reservoirs with area >= 100 sq. km.
  • 10% if 100 > area >= 10 sq. km.
  • 25% if area < 10 sq. km.
@pritamd47 pritamd47 self-assigned this Feb 9, 2023
@pritamd47 pritamd47 added the bug Something isn't working label Feb 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working edge case
Projects
None yet
Development

No branches or pull requests

1 participant