Kinesin/daggy
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Update TODO with current goals.

Browse Source
This commit is contained in:
Ian Roddis
2021-09-22 10:43:03 -03:00
parent e7b15b3847
commit 33942e2c63
1 changed files with 31 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files

27
TODO.md
View File

@@ -4,21 +4,30 @@ Tasks
- REST Server - REST Server
- [ ] Add in authorization scheme (maybe PAM auth endpoint with JWT?) - [ ] Add in authorization scheme (maybe PAM auth endpoint with JWT?)
- Core Functionality - Core Functionality
- rename TaskList to TaskSet
- Add support for Task.runSpec
- Executors can validate runspecs
- Executors can expand runspecs
- Allow `{{,!DATES}}` style interpolation, where values from `{{DATES}}` are preserved as a single, comma separated - Allow `{{,!DATES}}` style interpolation, where values from `{{DATES}}` are preserved as a single, comma separated
list list
- Improve DAG implementation
- Need to support millions of vertices
- Much faster cycle detection
- Executors
- Forking
- Add support for environment variables with `execvpe`
- Slurm
- Add support for environment variables
- Loggers - Loggers
- [ ] FileSystemLogger - Add in ability for loggers to be asynchronous
- [ ] Add unit tests - Additional Methods
- [ ] Add more error checking - Get DAG State
- Get Task Details
- Stream updates?
- Implementations
- [ ] General logger - [ ] General logger
- [ ] SQL logger (sqlite, postgres)
- [ ] Redis DAGRunLogger - [ ] Redis DAGRunLogger
- Server - Server
- [ ] Multiple execution pools - [ ] Multiple executors
- [ ] per-Executor parameters - [ ] Log to general logger
- [ ] Recover on startup
- Utilities - Utilities
- daggyd - daggyd
- [ ] Add config file support - [ ] Add config file support