Files
daggy/TODO.md
Ian Roddis 212bd80df2 Squashed commit of the following:
commit 73994327de890590eede353c8131f3f7c1e8aaa3
Author: Ian Roddis <gitlab@ie2r.com>
Date:   Wed Aug 25 13:38:29 2021 -0300

    - Fixing up checks for individual dag runs

commit f20e3a3dec8c063111cf60f2bec2b8f84c8a4100
Author: Ian Roddis <gitlab@ie2r.com>
Date:   Wed Aug 25 10:49:43 2021 -0300

    - Finishing serialization of DAGRun
    - Checkpointing work.

commit b490abadf93e3085e4204003de7eaa8183b4e1d5
Author: Ian Roddis <gitlab@ie2r.com>
Date:   Wed Aug 25 10:34:08 2021 -0300

    - Consolidating struct definitions into Defines.hpp
    - Renaming DAGRunRecord member runStates to taskRunStates

commit 050346ec1fd10d1091f261905c6175ffe0bcf001
Author: Ian Roddis <gitlab@ie2r.com>
Date:   Wed Aug 25 09:27:05 2021 -0300

    - Adding additional tests for server endpoints
2021-08-25 13:40:05 -03:00

1019 B

Tasks

  • Open

    • REST Server
      • Add in authorization scheme (maybe PAM auth endpoint with JWT?)
      • Flesh out server and interface
    • Core Functionality
      • Allow {{,!DATES}} style interpolation, where values from {{DATES}} are preserved as a single, comma separated list
      • Allow for tasks to define next tasks
      • Add execution gates
      • Executors
        • Slurm Executor
      • Loggers
        • FileSystemLogger
          • Add unit tests
          • Add more error checking
        • General logger
        • Redis DAGRunLogger
  • Completed

    • Core Functionality
      • Add ability to define child -> parent relationships
      • Rip apart scheduler and re-implement runDag as a utility function, taking an execution pool, task list, and logger
      • Resume a failed DAG
      • Handle return on errored DAG / Task
      • Clearing a DAG Task