release/4.0.5 (2024-12-19)
Requirements:
- As a user, I don't want to insert a product which has been already inserted with the same lidvid #195
Improvements:
- Quiet Harvest noisy logs for Invalid cookie header #223
- Optimize LDD downloads also on errors #205
- Add the date time in the log messages #203
Defects:
- Issue with ALT LDD date format error #222 [s.medium]
- harvest by default overwrites all products #213 [s.critical]
- Read time out errors occurring with big data uploads #208 [s.medium]
- OpenSearch mapping conflict issue when trying to change a type (
[illegal_argument_exception]) #204 [s.critical]
v4.0.4 (2024-11-12)
Requirements:
- As a user, I want harvest to exit with non 0 code when the arguments are not parsable #199
Improvements:
- Update harvest to support batches with data volumes larger than AOSS allowable limit #207
Defects:
- Issues identified with uncaught throttling errors #206 [s.high]
- Unknown date format used that could not be parsed by Harvest #197 [s.high]
Other closed issues:
- Double check how the schema is updated, log levels fine tune #196
v4.0.3 (2024-10-16)
release/4.0.3 (2024-10-16)
v4.0.2 (2024-10-16)
release/4.0.2 (2024-10-08)
ec2 (2024-10-08)
Requirements:
- As a user, I want to include my organization name in the harvest metadata (
ops:Harvest_Info.ops:node_name) #187
Defects:
- I want to update the OpenSearch schema whatever the number of fields to be updated #190 [s.critical]
- New records harvested in the registry don't have the expected Node value #186 [s.high]
v4.0.1 (2024-08-27)
release/4.0.0 (2024-08-27)
v4.0.0 (2024-08-27)
Improvements:
- As a data custodian, I want to load URLs / file paths without unnecessary / additional slashes #158
Defects:
Other closed issues:
- When harvest lasts more that one hour AWS credentials need to be renewed #172
- Run the synchronization of LDD as needed only #159 [s.medium]
- Add schema reference to all examples to take advantage of new schema #156
- Update to utilize new multi-tenancy approach #118
v3.8.2 (2023-12-16)
Defects:
v3.8.1 (2023-11-16)
Requirements:
- As a user, I want to ingest data products with labels having
.lblxfile extension #130
Defects:
Other closed issues:
v3.8.0 (2023-09-28)
Requirements:
- As a developer, I want to know what version of Harvest was used to load a product #119
Improvements:
- Improve skipped product INFO message #106
Defects:
- Access forbidden during nominal pipeline execution of harvest on Mars2020 archive #124 [s.critical]
v3.7.6 (2023-03-30)
v3.7.5 (2023-03-30)
Defects:
v3.7.4 (2022-12-12)
Defects:
v3.7.3 (2022-11-09)
v3.7.2 (2022-10-14)
v3.7.0 (2022-09-21)
Defects:
- Incorrect "lidvid" and "_id" fields are ingested (trailing zeros are truncated) #90 [s.critical]
Other closed issues:
- Improve CLI flag handling #85
v3.6.0 (2022-04-13)
Requirements:
- As a user, I want Harvest automatically convert date / time fields to ISO format supported by Elasticsearch #54
Improvements:
- Change the option -o help message #84
Defects:
- Error ingesting an XML boolean with values of 0/1 #78 [s.high]
- harvest stops rather than skips a file with bad permissions #75 [s.high]
Other closed issues:
- Update argument handling to use hyphenation similar to other PDS Tools #86
v3.5.2 (2022-01-11)
v3.5.1 (2021-12-10)
Defects:
v3.5.0 (2021-09-30)
Requirements:
- As a user, I want to ingest the PDS4 label as JSON in a binary blob form #60
Improvements:
- Add release datetime to version output #64
- Enable blob ingestion by default #58
- Quick fix to support date/time conversion to "ISO instant" format #55
v3.4.1 (2021-06-30)
Improvements:
- Track collection file inventory #18
Defects:
- harvest ingest is not creating all product_lidvid as an array #50 [s.low]
- Lid & vid validation, logging enhancement, fix skipped file counter #22
- Check input URIs to avoid potential security vulnerability #6 [s.low]
v3.4.0 (2021-04-16)
Requirements:
- As a user, I want to be able to ingest a directory of data that is not part of a bundle #45
- As a user, I want a default configuration for harvest included in the tool package #37
v3.3.3 (2021-01-02)
v3.3.2 (2020-12-21)
v3.3.1 (2020-12-02)
Improvements:
- Implement date conversion from PDS4 date/time strings to Solr format #24
- Update Harvest and Registry documentation to be more concise and streamlined #14
Defects:
- File system metadata not sufficiently captured per requirements #35 [s.medium]
- MD5 digest encoding is in Base64 instead of Hex #34 [s.medium]
- For multivalued fields only unique values are stored #30
- Could not parse date in yyyy-MM-ddZ format #29
- Fix bug where ingested product start_date_time is off by 12 hours #4
v3.2.2 (2020-03-28)
Defects:
- Bash script does not launch on macos, likely not on linux #20
v3.2.1 (2020-03-27)
v3.2.0 (2020-03-26)
Improvements:
- Update Harvest documentation per new scripts and upgrades #17
v3.1.0 (2020-03-26)
v3.0.0 (2020-03-26)
Improvements:
- Improve and simplify Harvest execution and configuration to only manage Registry collection #16
- Create new Harvest / Registry package to help streamline the documentation and deployment #15
Defects:
- Update Harvest to work with upgraded Solr package #13
v2.6.0 (2020-01-30)
Other closed issues:
- Open Source Harvest Tool #1
v2.5.2 (2019-10-29)
v2.5.1 (2019-10-27)
v2.5.0 (2019-10-25)
v2.4.0 (2019-10-18)
v2.3.0 (2019-10-18)
v2.2.0 (2019-10-15)
* This Changelog was automatically generated by github_changelog_generator