Merge branch 'trapi-1.5-revisions'; bump up release semver to 4.1.4 i…

…nstead
NCATSTranslator · May 9, 2024 · dfc1319 · dfc1319
2 parents e075968 + 59d6ad7
commit dfc1319
Show file tree

Hide file tree

Showing 11 changed files with 199 additions and 193 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -2,6 +2,11 @@
 
 The Reasoner Validator package is evolving along with progress in TRAPI and Biolink Model standards within the NCATS Biomedical Knowledge Translator.
 
+## 4.1.4
+
+- Added more fine-grained validation reporting inside case_input_found_in_response() TRAPI Response validation method, also injecting externalized "missing edge" validation message into method(); added/refactored validation codes accordingly
+- Full TRAPI 1.5.0 unit tests still skipped (need rework)
+
 ## 4.1.3
 
 - Generalize trapi_1.4 testing predicate to trapi_1.4 or later

diff --git a/DEVELOPER_NOTES.md b/DEVELOPER_NOTES.md
@@ -28,12 +28,12 @@ Steps to properly issue a new project release:
 2. If the **codes.yaml** were revised, regenerated the associate code documentation by running the reasoner_validator/validation_codes.py module as a script from the CLI.
 3. Run the unit test suite to ensure that nothing fails. Iterate to fix failures (in the code or in terms of revised unit tests to reflect fresh code designs)
 4. Document release changes in the **CHANGELOG.md**
-5. Update the **`[Tool Poetry]version =`** field in the **pyprojects.yaml**, e.g. "4.1.3"
+5. Update the **`[Tool Poetry]version =`** field in the **pyprojects.yaml**, e.g. "4.1.4"
 6. Run **`poetry update`** (preferably within  **`poetry shell`**)
 7. The project pip **requirements.txt** file snapshot of dependencies should also be updated at this point (type **`$ poetry export --output requirements.txt`**, assuming that the [proper poetry export plugin is installed](https://python-poetry.org/docs/pre-commit-hooks#poetry-export)). This may facilitate module deployment within environments that prefer to use pip rather than poetry to manage their deployments. 
 8. Commit or pull request merge all files (including the **poetry.lock** file) to **master**
-9. Add the equivalent Git **tag** to **master**. This should be the Semantic Version string from step 4 with an added 'v' prefix, i.e. "v4.1.3". 
+9. Add the equivalent Git **tag** to **master**. This should be the Semantic Version string from step 4 with an added 'v' prefix, i.e. "v4.1.4". 
 10. Push **master** to remote.
 11. Check if Git Actions for testing and documentation complete successfully.
-12. Create the release using the same release tag, i.e. "v4.1.3". 
-13. Check if Git Action for package deployment is successful and check if the new version (i.e. "4.1.3") is now visible on **[pypy.org](https://pypi.org/search/?q=reasoner-validator)**
+12. Create the release using the same release tag, i.e. "v4.1.4". 
+13. Check if Git Action for package deployment is successful and check if the new version (i.e. "4.1.4") is now visible on **[pypy.org](https://pypi.org/search/?q=reasoner-validator)**
diff --git a/docs/validation_codes_dictionary.md b/docs/validation_codes_dictionary.md
@@ -66,14 +66,6 @@
 
 **Description:** TRAPI Response should generally have a TRAPI request message Knowledge Graph key value in its reply.
 
-### error.trapi.response.knowledge_graph.missing_expected_edge
-
-**Message:** Knowledge Graph of TRAPI Response Message is missing expected Edge
-
-**Context:** identifier
-
-**Description:** The given TRAPI Response is expected to return specific edge(s) relating to the original (test edge?) data used to prepare the TRAPI Request!
-
 ### error.trapi.response.knowledge_graph.empty
 
 **Message:** Response returned an empty Message Knowledge Graph, which is an error in this context!
@@ -100,6 +92,30 @@
 
 **Description:** TRAPI Message.Results cannot resolve its reported identifier mappings to the original query.
 
+### error.trapi.response.missing_expected.knowledge_graph.node
+
+**Message:** Knowledge Graph of TRAPI Response Message Knowledge Graph is missing expected Node
+
+**Context:** identifier, context
+
+**Description:** The given TRAPI Response is expected to return specific nodes(s) relating to the original input data used to prepare the TRAPI Request!
+
+### error.trapi.response.missing_expected.knowledge_graph.edge
+
+**Message:** Knowledge Graph of TRAPI Response Message Knowledge Graph is missing expected Edge
+
+**Context:** identifier
+
+**Description:** The given TRAPI Response is expected to return specific edge(s) relating to the original input data used to prepare the TRAPI Request!
+
+### error.trapi.response.missing_expected.result
+
+**Message:** Knowledge Graph of TRAPI Response Message is missing expected Result
+
+**Context:** identifier
+
+**Description:** The given TRAPI Response is expected to return specific result(s) relating to the original input data used to prepare the TRAPI Request!
+
 ### error.input_edge.node.category.missing
 
 **Message:** Category missing for node

diff --git a/poetry.lock b/poetry.lock
diff --git a/pyproject.toml b/pyproject.toml
@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "reasoner-validator"
-version = "4.1.3"
+version = "4.1.4"
 description = "Validation tools for Reasoner API"
 authors = [
     "Richard Bruskiewich <richard.bruskiewich@delphinai.com>",

diff --git a/reasoner_validator/codes.yaml b/reasoner_validator/codes.yaml
@@ -60,11 +60,6 @@ error:
         missing:
           $message: "TRAPI Message is missing its Knowledge Graph component!"
           $description: "TRAPI Response should generally have a TRAPI request message Knowledge Graph key value in its reply."
-        missing_expected_edge:
-          $message: "Knowledge Graph of TRAPI Response Message is missing expected Edge"
-          $context:
-            - identifier
-          $description: "The given TRAPI Response is expected to return specific edge(s) relating to the original (test edge?) data used to prepare the TRAPI Request!"
         empty:
           $message: "Response returned an empty Message Knowledge Graph, which is an error in this context!"
           $description: "An empty Knowledge Graph is considered an error in this validation context!"
@@ -81,6 +76,24 @@ error:
             - identifier
             - output_node_binding
           $description: "TRAPI Message.Results cannot resolve its reported identifier mappings to the original query."
+      missing_expected:
+        knowledge_graph:
+          node:
+            $message: "Knowledge Graph of TRAPI Response Message Knowledge Graph is missing expected Node"
+            $context:
+              - identifier
+              - context   # 'subject' or 'object'
+            $description: "The given TRAPI Response is expected to return specific nodes(s) relating to the original input data used to prepare the TRAPI Request!"
+          edge:
+            $message: "Knowledge Graph of TRAPI Response Message Knowledge Graph is missing expected Edge"
+            $context:
+              - identifier
+            $description: "The given TRAPI Response is expected to return specific edge(s) relating to the original input data used to prepare the TRAPI Request!"
+        result:
+          $message: "Knowledge Graph of TRAPI Response Message is missing expected Result"
+          $context:
+            - identifier
+          $description: "The given TRAPI Response is expected to return specific result(s) relating to the original input data used to prepare the TRAPI Request!"
   input_edge:
     node:
       category:

diff --git a/reasoner_validator/validator.py b/reasoner_validator/validator.py
@@ -444,6 +444,9 @@ def case_node_found(target: str, identifiers: List[str], case: Dict, nodes: Dict
                 node_details = nodes[identifier]
                 if "categories" in node_details:
                     category = case[f"{target}_category"]
+                    # TODO: it may be helpful here to also try to match
+                    #       any parent classes of the specified 'category'
+                    #       to the node_details["categories"], using BMT
                     if category in node_details["categories"]:
                         return True
 
@@ -644,11 +647,6 @@ def case_input_found_in_response(
             # this, input test data edge is automatically deemed missing
             return False
 
-        # TODO: We need to check **here*** whether or not the
-        #       TRAPI response returned the original test case edge!!?!!
-        #       Not totally sure if we should first search the Results then
-        #       the Knowledge Graph, or go directly to the Knowledge Graph...
-
         # The Message Query Graph could be something like:
         # "query_graph": {
         #     "nodes": {
@@ -686,13 +684,21 @@ def case_input_found_in_response(
         subject_id = case["subject_id"] if "subject_id" in case else case["subject"]
         subject_aliases = get_aliases(subject_id)
         if not self.case_node_found("subject", subject_aliases, case, nodes):
-            # 'subject' node identifier not found?
+            self.report(
+                code="error.trapi.response.missing_expected.knowledge_graph.node",
+                identifier=subject_id,
+                context="subject"
+            )
             return False
 
         object_id = case["object_id"] if "object_id" in case else case["object"]
         object_aliases = get_aliases(object_id)
         if not self.case_node_found("object", object_aliases, case, nodes):
-            # 'object' node identifier not found?
+            self.report(
+                code="error.trapi.response.missing_expected.knowledge_graph.node",
+                identifier=object_id,
+                context="object"
+            )
             return False
 
         # In the Knowledge Graph:
@@ -746,14 +752,25 @@ def case_input_found_in_response(
                 edge_id_match = edge_id
                 break
 
+        edge_id: str = \
+            f"{case['idx']}|" +\
+            f"({case['subject_id']}#{case['subject_category']})" + \
+            f"-[{case['predicate_id']}]->" + \
+            f"({case['object_id']}#{case['object_category']})"
+
         if edge_id_match is None:
-            # Test case S--P->O edge not found?
+            self.report(
+                code="error.trapi.response.missing_expected.knowledge_graph.edge",
+                identifier=edge_id
+            )
             return False
 
         results: List = message["results"]
         if not self.case_result_found(subject_match, object_match, edge_id_match, results, trapi_version):
-            # Some components of test case S--P->O edge
-            # NOT bound within any Results?
+            self.report(
+                code="error.trapi.response.missing_expected.result",
+                identifier=edge_id
+            )
             return False
 
         # By this point, the case data assumed to be

diff --git a/requirements.txt b/requirements.txt
@@ -4,9 +4,9 @@ annotated-types==0.6.0 ; python_version >= "3.9" and python_version < "3.12" \
 attrs==23.2.0 ; python_version >= "3.9" and python_version < "3.12" \
     --hash=sha256:935dc3b529c262f6cf76e50877d35a4bd3c1de194fd41f47a2b7ae8f19971f30 \
     --hash=sha256:99b87a485a5820b23b879f04c2305b44b951b502fd64be915879d77a7e8fc6f1
-bioregistry==0.11.2 ; python_version >= "3.9" and python_version < "3.12" \
-    --hash=sha256:8ecb782505d729e6338946b3af6a2b297dfe281114381961d5acc4fdf1af9033 \
-    --hash=sha256:e2a33c6afdac19bcd76230a87750cac84eb99bebea1fd55ef9d595c033269b86
+bioregistry==0.11.3 ; python_version >= "3.9" and python_version < "3.12" \
+    --hash=sha256:7e04e926ee0068b7623b2ec1984e1872e0b152c203b71542a4edfe26985044f1 \
+    --hash=sha256:ea23736b41f2e7e7f26f544d19f5a3ab4ba41343a0a10d3162507922adc3ebfd
 bmt==1.4.0 ; python_version >= "3.9" and python_version < "3.12" \
     --hash=sha256:26a831eda3d64405d849f49ee579b44412e3e6b9c2e6587e2240a7605b28fc55 \
     --hash=sha256:ca81f3e91b3a0312bed01692a93d343d828e556ae14fc00fb016f5a3f38f44c3
@@ -294,33 +294,25 @@ pytest==7.4.4 ; python_version >= "3.9" and python_version < "3.12" \
 pytrie==0.4.0 ; python_version >= "3.9" and python_version < "3.12" \
     --hash=sha256:8f4488f402d3465993fb6b6efa09866849ed8cda7903b50647b7d0342b805379
 pyyaml==6.0.1 ; python_version >= "3.9" and python_version < "3.12" \
-    --hash=sha256:04ac92ad1925b2cff1db0cfebffb6ffc43457495c9b3c39d3fcae417d7125dc5 \
     --hash=sha256:062582fca9fabdd2c8b54a3ef1c978d786e0f6b3a1510e0ac93ef59e0ddae2bc \
-    --hash=sha256:0d3304d8c0adc42be59c5f8a4d9e3d7379e6955ad754aa9d6ab7a398b59dd1df \
     --hash=sha256:1635fd110e8d85d55237ab316b5b011de701ea0f29d07611174a1b42f1444741 \
     --hash=sha256:184c5108a2aca3c5b3d3bf9395d50893a7ab82a38004c8f61c258d4428e80206 \
     --hash=sha256:18aeb1bf9a78867dc38b259769503436b7c72f7a1f1f4c93ff9a17de54319b27 \
     --hash=sha256:1d4c7e777c441b20e32f52bd377e0c409713e8bb1386e1099c2415f26e479595 \
     --hash=sha256:1e2722cc9fbb45d9b87631ac70924c11d3a401b2d7f410cc0e3bbf249f2dca62 \
     --hash=sha256:1fe35611261b29bd1de0070f0b2f47cb6ff71fa6595c077e42bd0c419fa27b98 \
     --hash=sha256:28c119d996beec18c05208a8bd78cbe4007878c6dd15091efb73a30e90539696 \
-    --hash=sha256:326c013efe8048858a6d312ddd31d56e468118ad4cdeda36c719bf5bb6192290 \
-    --hash=sha256:40df9b996c2b73138957fe23a16a4f0ba614f4c0efce1e9406a184b6d07fa3a9 \
     --hash=sha256:42f8152b8dbc4fe7d96729ec2b99c7097d656dc1213a3229ca5383f973a5ed6d \
-    --hash=sha256:49a183be227561de579b4a36efbb21b3eab9651dd81b1858589f796549873dd6 \
     --hash=sha256:4fb147e7a67ef577a588a0e2c17b6db51dda102c71de36f8549b6816a96e1867 \
     --hash=sha256:50550eb667afee136e9a77d6dc71ae76a44df8b3e51e41b77f6de2932bfe0f47 \
     --hash=sha256:510c9deebc5c0225e8c96813043e62b680ba2f9c50a08d3724c7f28a747d1486 \
     --hash=sha256:5773183b6446b2c99bb77e77595dd486303b4faab2b086e7b17bc6bef28865f6 \
     --hash=sha256:596106435fa6ad000c2991a98fa58eeb8656ef2325d7e158344fb33864ed87e3 \
     --hash=sha256:6965a7bc3cf88e5a1c3bd2e0b5c22f8d677dc88a455344035f03399034eb3007 \
     --hash=sha256:69b023b2b4daa7548bcfbd4aa3da05b3a74b772db9e23b982788168117739938 \
-    --hash=sha256:6c22bec3fbe2524cde73d7ada88f6566758a8f7227bfbf93a408a9d86bcc12a0 \
     --hash=sha256:704219a11b772aea0d8ecd7058d0082713c3562b4e271b849ad7dc4a5c90c13c \
     --hash=sha256:7e07cbde391ba96ab58e532ff4803f79c4129397514e1413a7dc761ccd755735 \
     --hash=sha256:81e0b275a9ecc9c0c0c07b4b90ba548307583c125f54d5b6946cfee6360c733d \
-    --hash=sha256:855fb52b0dc35af121542a76b9a84f8d1cd886ea97c84703eaa6d88e37a2ad28 \
-    --hash=sha256:8d4e9c88387b0f5c7d5f281e55304de64cf7f9c0021a3525bd3b1c542da3b0e4 \
     --hash=sha256:9046c58c4395dff28dd494285c82ba00b546adfc7ef001486fbf0324bc174fba \
     --hash=sha256:9eb6caa9a297fc2c2fb8862bc5370d0303ddba53ba97e71f08023b6cd73d16a8 \
     --hash=sha256:a0cd17c15d3bb3fa06978b4e8958dcdc6e0174ccea823003a106c7d4d7899ac5 \
@@ -335,9 +327,7 @@ pyyaml==6.0.1 ; python_version >= "3.9" and python_version < "3.12" \
     --hash=sha256:bfdf460b1736c775f2ba9f6a92bca30bc2095067b8a9d77876d1fad6cc3b4a43 \
     --hash=sha256:c8098ddcc2a85b61647b2590f825f3db38891662cfc2fc776415143f599bb859 \
     --hash=sha256:d2b04aac4d386b172d5b9692e2d2da8de7bfb6c387fa4f801fbf6fb2e6ba4673 \
-    --hash=sha256:d483d2cdf104e7c9fa60c544d92981f12ad66a457afae824d146093b8c294c54 \
     --hash=sha256:d858aa552c999bc8a8d57426ed01e40bef403cd8ccdd0fc5f6f04a00414cac2a \
-    --hash=sha256:e7d73685e87afe9f3b36c799222440d6cf362062f78be1013661b00c5c6f678b \
     --hash=sha256:f003ed9ad21d6a4713f0a9b5a7a0a79e08dd0f221aff4525a2be4c346ee60aab \
     --hash=sha256:f22ac1c3cac4dbc50079e965eba2c1058622631e526bd9afd45fedd49ba781fa \
     --hash=sha256:faca3bdcf85b2fc05d06ff3fbc1f83e1391b3e724afa3feba7d13eeab355484c \

diff --git a/tests/test_response_validator.py b/tests/test_response_validator.py
@@ -204,6 +204,35 @@
     "edges": _TEST_EDGES_4
 }
 
+_TEST_KG_EDGE_SOURCES = [
+    {
+        "resource_id": "infores:chebi",
+        "resource_role": "primary_knowledge_source",
+        "upstream_resource_ids": []
+    },
+    {
+        "resource_id": "infores:biothings-explorer",
+        "resource_role": "aggregator_knowledge_source",
+        "upstream_resource_ids": [
+            "infores:chebi"
+        ]
+    },
+    {
+        "resource_id": "infores:molepro",
+        "resource_role": "aggregator_knowledge_source",
+        "upstream_resource_ids": [
+            "infores:biothings-explorer"
+        ]
+    },
+    {
+        "resource_id": "infores:arax",
+        "resource_role": "aggregator_knowledge_source",
+        "upstream_resource_ids": [
+            "infores:molepro"
+        ]
+    }
+]
+
 # From Implementation Guidelines circa June 2023
 _TEST_TRAPI_1_4_2_FULL_SAMPLE = {
     "message": {
@@ -227,34 +256,7 @@
                     "subject": "ncats.drug:9100L32L2N",
                     "predicate": "biolink:ameliorates_condition",
                     "object": "MONDO:0005148",
-                    "sources": [
-                        {
-                            "resource_id": "infores:chebi",
-                            "resource_role": "primary_knowledge_source",
-                            "upstream_resource_ids": []
-                        },
-                        {
-                            "resource_id": "infores:biothings-explorer",
-                            "resource_role": "aggregator_knowledge_source",
-                            "upstream_resource_ids": [
-                                "infores:chebi"
-                            ]
-                        },
-                        {
-                            "resource_id": "infores:molepro",
-                            "resource_role": "aggregator_knowledge_source",
-                            "upstream_resource_ids": [
-                                "infores:biothings-explorer"
-                            ]
-                        },
-                        {
-                            "resource_id": "infores:arax",
-                            "resource_role": "aggregator_knowledge_source",
-                            "upstream_resource_ids": [
-                                "infores:molepro"
-                            ]
-                        }
-                    ]
+                    "sources": _TEST_KG_EDGE_SOURCES
                 }
             }
         },
@@ -327,34 +329,7 @@
                     "subject": "MONDO:0005148",
                     "predicate": "biolink:treated_by",
                     "object": "ncats.drug:9100L32L2N",
-                    "sources": [
-                        {
-                            "resource_id": "infores:chebi",
-                            "resource_role": "primary_knowledge_source",
-                            "upstream_resource_ids": []
-                        },
-                        {
-                            "resource_id": "infores:biothings-explorer",
-                            "resource_role": "aggregator_knowledge_source",
-                            "upstream_resource_ids": [
-                                "infores:chebi"
-                            ]
-                        },
-                        {
-                            "resource_id": "infores:molepro",
-                            "resource_role": "aggregator_knowledge_source",
-                            "upstream_resource_ids": [
-                                "infores:biothings-explorer"
-                            ]
-                        },
-                        {
-                            "resource_id": "infores:arax",
-                            "resource_role": "aggregator_knowledge_source",
-                            "upstream_resource_ids": [
-                                "infores:molepro"
-                            ]
-                        }
-                    ]
+                    "sources": _TEST_KG_EDGE_SOURCES
                 }
             }
         },