Massive re-org to allow per-executor job specification formats and executor-specific task validation and expansion.

A few different renames to try and keep things more consistent.
This commit is contained in:
Ian Roddis
2021-09-03 09:10:38 -03:00
parent e746f8c163
commit d15580f47f
22 changed files with 509 additions and 300 deletions

51
TODO.md
View File

@@ -1,27 +1,30 @@
Tasks
==
- Open
- REST Server
- [ ] Add in authorization scheme (maybe PAM auth endpoint with JWT?)
- Core Functionality
- Allow `{{,!DATES}}` style interpolation, where values from `{{DATES}}` are preserved as a single, comma
separated list
- Executors
- [ ] Slurm Executor
- Loggers
- [ ] FileSystemLogger
- [ ] Add unit tests
- [ ] Add more error checking
- [ ] General logger
- [ ] Redis DAGRunLogger
- Server
- [ ] Multiple execution pools
- [ ] per-Executor parameters
- Utilities
- daggyd
- [ ] Add config file support
- [ ] Support for all the different executors / state loggers
- daggyc
- [ ] Submission
- [ ] Querying
- REST Server
- [ ] Add in authorization scheme (maybe PAM auth endpoint with JWT?)
- Core Functionality
- rename TaskList to TaskSet
- Add support for Task.runSpec
- Executors can validate runspecs
- Executors can expand runspecs
- Allow `{{,!DATES}}` style interpolation, where values from `{{DATES}}` are preserved as a single, comma
separated list
- Executors
- [ ] Slurm Executor
- Loggers
- [ ] FileSystemLogger
- [ ] Add unit tests
- [ ] Add more error checking
- [ ] General logger
- [ ] Redis DAGRunLogger
- Server
- [ ] Multiple execution pools
- [ ] per-Executor parameters
- Utilities
- daggyd
- [ ] Add config file support
- [ ] Support for all the different executors / state loggers
- daggyc
- [ ] Submission
- [ ] Querying