Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[WFRunner] Estimate resources dynamically
* introduce functionality to update resource estimates based on values which are collected during monitoring (the same that go into pipeline_metric_<pid>.log) * disentangle resource management from WorkflowExecutor * manage resources via ResourceManager object * memory and cpu * semaphores * number of parallel tasks to be launched * niceness * main interfaces of ResourceManager * ok_to_submit * book * unbook * add_monitored_resources * group corresponding tasks together to be able to update resource estimates for upcoming tasks * TaskResources as container of a task's resource estimate holding * resource estimates (initial and new estimates) * references to Semaphore object (optional * reference to list of related tasks (optional) * make sure, final niceness assigned to a launched process is the same that the TaskResources hold for that process * enable functionality with --dynamic-resources * add more comments to classes and functions * update --cgroup option to require entire path --> handle both cgroups v1 and cgroups v2 (cherry picked from commit ef0e84d)
- Loading branch information