v3.0.0
We're excited to bring you the 3.0.0 release, it's a big one!
We three had main goals for this release:
➡️ Solve discrepancies with the official GTFS specification.
26 new rules have been added in total:
- New validation rules were added for the adopted GTFS extensions pathways.txt, levels.txt, translations.txt
- New custom rules were added for parts of the spec that were missing from this validator
- Some existing rules were improved to better reflect the specification
- The severity of some rules was updated to reflect the specification after the update to RFC 2119 has been done
Please refer to the V2.0.0 to V3.0.0 rule mapping available in NOTICE_MIGRATION.md for more details on the rule updates.
➡️ Implement automated tests that will evaluate if a new rule results in datasets from the mobility database getting additional errors. If so, the Pull Request is blocked and a report will be generated for further investigation. This ensures the stability and reliability of this validator, while still being updated with updates and extensions in the specification.
➡️ Make sure MobilityData's work has related public communication to facilitate community contribution (issues opened, public sprint board and bug tracking board, roadmap, ways for users to voice what they'd like to see next and for contributors to see how to get involved).
There are also new command line parameters that can be found in USAGE.md.
There is a lot more work included in this release, thanks to our amazing contributors. See details below!
Work ⚙️
Solve discrepancies with the spec
- chore: downgrade LeadingOrTrailingWhitespacesNotice to WARNING in #929
- chore: upgrade EmptyColumnNameNotice to ERROR in #1019
- chore: new rule - additional requirements urls (stops.txt, routes.txt, agency.txt) in #932
routes.txt
- feat: Reduce severity of SameNameAndDescriptionForRouteNotice to WARNING in #917
stops.txt
stop_times.txt
- chore: new rule - additional check on stops.location_type in #938
- chore: upgrade LocationWithUnexpectedStopTimeNotice to ERROR in #1021
pathways.txt and levels.txt
- feat: Validate how locations are referenced from stop_times.stop_id in #960
- feat: Validate that pathway endpoints have correct types in #961
- feat: verify conditional requirement of levels.txt presence in #966
- feat: additional checks on pathways.txt and stops.txt in #967
- feat: Validate that locations are reachable if a station has pathways in #997
- feat: Validate that a pathway is not a loop in #999
- feat: Validate field presence and key integrity for translations.txt in #968
- chore: upgrade PathwayUnreachableLocationNotice to ERROR in #1028
translations.txt
- chore: upgrade TranslationForeignKeyViolationNotice to ERROR in #1027
Automated test to see if a PR results in additional errors
Other work included
- We ran some analytics on all datasets from the mobility database (#1041)
Better rules - improve rule logic (#839, #944, #946, #956, #957, #958, #978, #972)
Better notice schema export
Better documentation
Bugs
- fix: Prevent NullPointerException when Throwable.getMessage returns null in #871
- fix: Catch any runtime exception when parsing a GTFS field in #1012
- fix: Swap arguments of isAssignableFrom in #872
- fix: bump debian version used in docker image in #891
- fix: Protect against start_date and end_date in wrong order in #897
- fix: Raise IllegalArgumentException for invalid date in #977
- fix: change limit on notice count during validation to 100_000 in #1018
- fix: validate route, stops and agency consistency urls in #1003
- feat: Prevent OOM in NoticeContainer and speed up hasValidationErrors in #895
- fix: Skip phone number validation when country code is unknown in #1062
- fix: False positives for StopTimeTimepointWithoutTimesNotice in #1044
Other good stuff
- ci: check code formatting on pull requests in #953
- ci: publish a new docker image on every push on master in #913
- ci: verify gradle wrapper in #964
- ci: bug triage in #982
- feat: Provide enum setters and clearFIELD methods in #843
- feat: make report names user configurable via CLI args in #852
- feat!: deprecate -f in #851
- feat: Pass severityLevel to constructor of some notices in #863
- feat: Propagate InterruptedException during feed loading in #869
- feat: Catch CSV parse errors and add validation notices in #874
- feat: print help menu in #888
- feat: Export infinity and NaN values in JSON in #889
- feat: Treat empty files as parsed unsuccessfully in #898
- feat: introduce -p CLI parameter to output beautified reports in #901
- feat: Add byPrimaryKey and getKeyColumnNames methods to GtfsTableContainer in #1000
- chore: upgrade to gradle wrapper to 7.2 in #965
- chore: Upgrade to Java 11 in #1014
Contributors 🙏
Massive shoutout to:
🎉 @aababilov
🎉 @asvechnikov2
🎉 @barbeau
🎉 @lionel-nj
🎉 @maximearmstrong
🎉 @isabelle-dr
