v4.1.0

Overview

This release includes the addition of the Web version available at https://gtfs-validator.mobilitydata.org/, new notices, support for Fare Media that was adopted in GTFS in March 2023, and initial support to programmatically generate the notice documentation.
See the notice mapping between releases in NOTICE_MIGRATION.md.

New notices

• invalid_currency_amount 🔴 ERROR
• missing_stop_name 🔴 ERROR
• stop_without_location 🔴 ERROR
• too_many_rows 🔴 ERROR
• duplicate_fare_media 🟠 WARNING
• expired_calendar 🟠 WARNING
• missing_feed_info_date 🟠 WARNING
• mixed_case_recommended_field 🟠 WARNING
• route_long_name_contains_short_name 🟠 WARNING
• trip_coverage_not_active_for_next7_days 🟠 WARNING

Impacts on existing data

You can find the impact of this release on all existing datasets we have in the Mobility database in this spreadsheet.

What's Changed

Web-based validator

• feat: Web Based Validator by @KClough in #1317
• feat: Web Validator Client Display Errors by @KClough in #1383
• feat: Add Sentry Error Monitoring to web validator client + service by @KClough in #1384
• feat: add newrelic agent to web service by @KClough in #1387
• chore: Web validator ide warnings by @KClough in #1388
• chore: add web validator information to README by @davidgamez in #1399

Bug fixes

• bug: upgrade GitHub Workflow actions to Node16 compatible versions by @bdferris-v2 in #1271
• fix: Don't ignore invalid language codes in RowParser.asLanguageCode by @asvechnikov2 in #1291
• Fix miscellaneous error-prone findings by @aababilov in #1312
• fix: byTranslationKey() for multi-key schemas without translations.txt support by @bdferris-v2 in #1302
• fix: broken link in RULES.md by @isabelle-dr in #1332
• Fix: Fix bug triage workflow by @isabelle-dr in #1336
• Fix: whitespace and anchor tag by @briandonahue in #1342
• fix: Failing Docker Image workflow by @davidgamez in #1386
• fix: Run all sub-project tests in test_pack_doc.yml workflow by @bdferris-v2 in #1382
• fix: Bump validation workflows to use large machine instances by @bdferris-v2 in #1320

Rules

• fix: Drop DuplicateFareRuleZoneIdFieldsValidator and replace with multi-column @PrimaryKey for fare_rules.txt by @bdferris-v2 in #1297
• feat: Date validity - Calendar Expiration + Trip Coverage by @KClough in #1289
• Fix 1198: separate required column from required value by @briandonahue in #1344
• fix: Add notice for missing stop name for specific location types by @briandonahue in #1316
• fix: add field value to mixed case notice by @briandonahue in #1393
• fix: generate errors when missing lat/long for stops, stations, entrances by @briandonahue in #1321
• fix 878: require agency_id when more than one agency, warn when only one by @briandonahue in #1318
• feat 267: route.long_name should not contain route.short_name by @briandonahue in #1325
• feat: Add MixedCase validation (#881) by @briandonahue in #1340
• feat: Add code for computing service coverage windows and trip counts by @bdferris-v2 in #1351

Documentation updates

• Docs: documentation updates by @isabelle-dr in #1288
• Docs: Fix RULES.md notice name for 2 notices by @isabelle-dr in #1323
• Doc: RULES.md update by @isabelle-dr in #1307
• Docs: Update RULES.md by @isabelle-dr in #1359
• Docs: Update ACCEPTANCE_TESTS.md by @isabelle-dr in #1385
• Docs: Update notice migration file by @isabelle-dr in #1357
• fix: update RULES.md with missing rule by @briandonahue in #1330

Generate the doc programmatically

• feat: Initial entry for @GtfsValidationNotice annotation by @bdferris-v2 in #1361
• feat: Update Notice source code such that field name comments match documentation in RULES.md by @bdferris-v2 in #1346
• feat: Add unit-test to check consistency of Notice field names. by @bdferris-v2 in #1345
• fix: Inconsistencies in RULES.md against underlying source by @bdferris-v2 in #1341
• feat: Add unit test to verify that every Notice has an entry in RULES.md by @bdferris-v2 in #1331
• feat: Extract notice docs from source-code for in-app use and schema export by @bdferris-v2 in #1365

Acceptance tests improvements

• feat: Update acceptance test report to include changes in WARNINGS. by @bdferris-v2 in #1354
• feat: Fail acceptance tests on significant change in warnings by @bdferris-v2 in #1377

General maintenance + other good stuff

• Use smaller integer field types for GTFS entity classes by @aababilov in #1273
• Introduce GtfsEntityBuilder interface by @aababilov in #1286
• Use int instead of long to keep CSV row number by @aababilov in #1287
• Generate registries of tables and validators by @aababilov in #1290
• Avoid unchecked conversion in table and validator registries by @aababilov in #1293
• build(deps): bump certifi from 2021.5.30 to 2022.12.7 in /scripts/mobility-database-harvester by @dependabot in #1299
• Separate table loading logic from table structure by @aababilov in #1284
• Do not invoke single file validators if table indices were not generated by @aababilov in #1300
• Always invoke single file validators (revert #1300) by @aababilov in #1303
• Discover tables and validators in runtime using io.github.classgraph by @aababilov in #1311
• Initialize one field per line in the generated code by @aababilov in #1314
• feat: Refactor JSON report comparison for better diffing in unit-tests. by @bdferris-v2 in #1352
• feat: Move key annotation and entity interfaces into a new model module by @bdferris-v2 in #1362
• feat: Initial support for GTFS Fares v2 - fare media by @bdferris-v2 in #1305
• feat: Add a more descriptive test failure message to NoticeFieldsTest. by @bdferris-v2 in #1369
• chore: Add unit tests to AnyTableLoader by @davidgamez in #1380

New Contributors

• @asvechnikov2 made their first contribution in #1291
• @briandonahue made their first contribution in #1321
• @davidgamez made their first contribution in #1386

Full Changelog: v4.0.0...v4.1.0