Grouping issue: Automated testing #318
Assignees
Labels
Comments
|
@matentzn We discussed testing today at our meeting. Rather than making a new issue, I updated this existing one with our thoughts. The reason it's called a "grouping issue" is because I don't expect it to be something where we specifically pick a time to tackle the whole issue at once. |
|
@matentzn Btw, I ran Log
I tried this flag even though I didn't expect it to work for |
|
You need I think ca 18 GB memory in Java and 21 GB memory in docker to run this locally I think.. Do you have that configured? Else, cant if you don't need to update sources, you could use |
|
Yeah I have 28GB allocated in Docker but I didn't pass I will try |
|
Darn, looks like the |
|
Why not? If you look at for example: https://github.com/monarch-initiative/mondo-ingest/blob/main/src/ontology/mondo-ingest.Makefile#L39 COMP=false should prevent components to be rebuild. Or are you saying a non-component goals still needs too much memory? |
|
Hmm, maybe I was running it the wrong way. I tried: I was thinking that the order of the flag shouldn't matter, but I'll try the below one now. Here's all that remains from the log from yesterday. You can see it's doing something with Log
I believe I also tried |
|
COMP=false sh run.sh make test If you do this, you are passing the COMP parameter to the shell script, not make. The second solution is the right one. "Killed" means it ran out of memory, or rather, not often than not, that you gave more memory to the robot process (check run.sh) then you allocated to docker. Always make sure you are 15-20% below docker Max. |
|
Yeah I should have tried that earlier. It passed: I know about |
This was referenced Nov 16, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Overview
I felt it would be a good idea to create a general issue around automated testing (unit tests, integration tests, etc). Right now we don't have any tests at all, I believe.
Modules & specific cases to test
Details
These are some of the workflows that I think could most use some tests.
For "whole pipeline itself", we would just download sources and run; dynamic inputs. For each of these other "modules", we probably should create some static input files.
Integration tests (checking that outputs match expectations) are higher priority than unit tests (e.g. at the function level).
E.g. the bimaps, i.e.
mondo.sssom.config.yaml'scurie_map(moot if fully replaced by theextended_prefix_mapin there) andconfig/prefixes.csv(moot once it is dynamically generated frommondo.sssom.config.yaml). Basically we want to ensure it is bijective; i.e. no duplciate URI prefixes.1. Whole pipeline
Details
Nico wrote:
2. Migration pipeline (slurp)
Details
Possible tests:
I. Mock terms
Can create some static input files where we can create some mock terms that meet various conditions, and we can assert what we expect after running the workflow (e.g. whether or not the term appears in
slurp/%.tsv).II. Output file should never have more than 1k rows
3. "Unmapped" artefact pipelines
Details
Possible tests:
I. Every ingest should have no more than 50% unmapped classes
The easiest way to check this is to use
scripts/unmapped_docs.py. There is a function that creates the table inunmapped.mdin thedocs/which has these %'s calculated.Creating test input files
GitHub action
Details
Should run:
MakefiletestgoalShould not run:
When:
schedule?: Doesn't make sense unless dynamic inputspush: Tomainpull_request: TomainMakefile set up
Details
We have an existing Makefile test goal, but this is for common cases for ODK projects. To incorporate integration tests and unit tests, we probably want something like:
Nico thought about integration-test but we might have unit tests as well. I don't know if it makes sense to create a 2nd goal for unit-test; I don't see that we'll really want to run them separately.
Discussion
Module tests in isolation with static inputs
Details
Possible future changes to static test files
Adding ontology files: Although, there is some rationale to maybe creating a shortened, static version of ontology files, e.g.
ordo.owlwhich will allow SPARQL queries, etc, to run faster. Will also prevent failures in cases where the ontology changes.Caveats / more possible future changes
Details
Makefile test setup: I didn't make any dependencies for the tests goal, so if some of those files don't exist, it'll error out. Maybe this is a design flaw. I up the tests goal to conveniently automatically find and run all Python unit tests in the project. Perhaps a different way would be to split things up, like a
test-slurpthat has goal dependencies, and then a tests goal withtest-slurpas its dependency.The text was updated successfully, but these errors were encountered: