This paves the way for implementing daggys and other utilities.
Squashed commit of the following:
commit 1f77239ab3c9e44d190eef94531a39501c8c4dfe
Author: Ian Roddis <gitlab@ie2r.com>
Date: Mon Oct 18 16:25:02 2021 -0300
Adding README, stdout support for daggyd logging
commit c2c237224e84a3be68aaa597ce98af1365e74a13
Author: Ian Roddis <gitlab@ie2r.com>
Date: Mon Oct 18 16:10:29 2021 -0300
removing old daggyd
commit cfea2baf61ca10c535801c5a391d2d525a1a2d04
Author: Ian Roddis <gitlab@ie2r.com>
Date: Mon Oct 18 16:10:09 2021 -0300
Moving tests into their sub-project folders
commit e41ca42069bea1db16dd76b6684a3f692fef6b15
Author: Ian Roddis <gitlab@ie2r.com>
Date: Mon Oct 18 15:57:40 2021 -0300
Splitting out daggyd from libdaggy
commit be97b146c1d2446f5c03cb78707e921f18c60bd8
Author: Ian Roddis <gitlab@ie2r.com>
Date: Mon Oct 18 15:56:55 2021 -0300
Splitting out daggyd from libdaggy
commit cb61e140e9d6d8832d61fb7037fd4c0ff6edad00
Author: Ian Roddis <gitlab@ie2r.com>
Date: Mon Oct 18 15:49:47 2021 -0300
moving daggy to libdaggy
Squashed commit of the following:
commit dc3a1bf07b5e7afdfd45e56f34596300dab6fd70
Author: Ian Roddis <gitlab@ie2r.com>
Date: Wed Oct 13 15:15:28 2021 -0300
Updating documentation a bit
commit 8ec9c8c74f587368b32d034d3240a5537a69d4b1
Author: Ian Roddis <gitlab@ie2r.com>
Date: Wed Oct 13 15:11:23 2021 -0300
Completing tests for redis
commit a6308dfa35b40b5a147394af8e3322ada871eb92
Author: Ian Roddis <gitlab@ie2r.com>
Date: Wed Oct 13 14:56:22 2021 -0300
Resolving some errors with forking environment
commit 34691b6f85abae67001f4a4c234a4f7314407331
Author: Ian Roddis <gitlab@ie2r.com>
Date: Wed Oct 13 10:53:55 2021 -0300
Checkpointing work on unit tests
commit 44c2b50fde30348938d901703ead9e279c3cd237
Author: Ian Roddis <gitlab@ie2r.com>
Date: Wed Oct 13 09:09:58 2021 -0300
Checkpointing work on redis
commit a8051b725257087e25bc452673633ba6b40e3985
Author: Ian Roddis <gitlab@ie2r.com>
Date: Fri Oct 8 13:31:41 2021 -0300
Checkpointing progress, changing state updates to a single record type
commit 456b84ad8c7dee0ff0dd39d5a7caead1ccd1126c
Author: Ian Roddis <gitlab@ie2r.com>
Date: Thu Oct 7 16:43:48 2021 -0300
Checkpointing progress
commit f19dcaa4e417c3f2f6e527c288fe51401c9fe1d7
Author: Ian Roddis <gitlab@ie2r.com>
Date: Thu Oct 7 11:53:35 2021 -0300
Moving back to hiredis to avoid boost dependency
commit e4bea6c589e82c82fd41476f164d946d77677193
Author: Ian Roddis <gitlab@ie2r.com>
Date: Wed Oct 6 10:41:16 2021 -0300
fixing comments
commit 807a73c2a406817001eec048483938545a60194c
Author: Ian Roddis <gitlab@ie2r.com>
Date: Wed Oct 6 10:40:38 2021 -0300
Switching to redis-cpp
commit d060c008d4d96bf3a81a19d35067f95f3638b8ca
Author: Ian Roddis <gitlab@ie2r.com>
Date: Tue Oct 5 17:54:06 2021 -0300
Adding hiredis dep
commit b06b11cbb5d09c6d091551e39767cd3316f88376
Author: Ian Roddis <gitlab@ie2r.com>
Date: Tue Oct 5 11:57:37 2021 -0300
Fixing failing unit test
commit fe2a43a19b2a16a9aedd9e95e71e672935ecaeb1
Author: Ian Roddis <gitlab@ie2r.com>
Date: Tue Oct 5 11:54:01 2021 -0300
Adding endpoints and updating documentation
commit 46e0deeefb8b06291ae5e2d6b8ec4749c5b0ea6f
Author: Ian Roddis <gitlab@ie2r.com>
Date: Tue Oct 5 11:49:43 2021 -0300
Completing unit tests and relevant fixes
commit e0569f370624844feee6aae4708bfe683f4156cf
Author: Ian Roddis <gitlab@ie2r.com>
Date: Mon Oct 4 17:30:59 2021 -0300
Adding in gcc tsan for debug builds to help with race conditions, fixing many of those, and fixing really crummy assumption about how futures worked that will speed up task execution by a ton.
commit c748a4f592e1ada5546908be5281d04f4749539d
Author: Ian Roddis <gitlab@ie2r.com>
Date: Mon Oct 4 10:14:43 2021 -0300
Checkpointing work that seems to have resolved the race condition
commit 7a79f2943e0d50545d976a28b4b379340a90dded
Author: Ian Roddis <gitlab@ie2r.com>
Date: Wed Sep 29 09:27:07 2021 -0300
Completing the rough-in for DAG killing / pausing / resuming
commit 4cf8d81d5f6fcf4a7dd83d8fca3e23f153aa8acb
Author: Ian Roddis <gitlab@ie2r.com>
Date: Tue Sep 28 14:53:50 2021 -0300
Adding dagrunner unit tests, adding a resetRunning method to resume
commit 54e2c1f9f5e7d5b339d71be024e0e390c4d2bf61
Author: Ian Roddis <gitlab@ie2r.com>
Date: Tue Sep 28 14:45:57 2021 -0300
Refactoring runDAG into DAGRunner
commit 682be7a11e2fae850e1bc3e207628d2335768c2b
Author: Ian Roddis <gitlab@ie2r.com>
Date: Tue Sep 28 14:34:43 2021 -0300
Adding DAGRunner class to replace Utilities::runDAG, making Slurm cancellation rc agree with SIGKILL
commit 4171b3a6998791abfc71b04f8de1ae93c4f90a78
Author: Ian Roddis <gitlab@ie2r.com>
Date: Tue Sep 28 14:14:17 2021 -0300
Adding unit tests for stopping jobs to slurm
commit dc0b1ff26a5d98471164132d35bb8a552cc75ff8
Author: Ian Roddis <gitlab@ie2r.com>
Date: Tue Sep 28 14:04:15 2021 -0300
Adding in stop method for task executors
commit e752b44f55113be54392bcbb5c3d6f251d673cfa
Author: Ian Roddis <gitlab@ie2r.com>
Date: Tue Sep 28 12:32:06 2021 -0300
Adding additional tests for loggers
commit f0773d5a84a422738fc17c9277a2b735a21a3d04
Author: Ian Roddis <gitlab@ie2r.com>
Date: Tue Sep 28 12:29:21 2021 -0300
Unit tests pass
commit 993ff2810de2d53dc6a59ab53d620fecf152d4a0
Author: Ian Roddis <gitlab@ie2r.com>
Date: Tue Sep 28 12:24:34 2021 -0300
Adding handling for new routes, still need to add tests for new routes
commit 676623b14e45759872a2dbcbc98f6a744e022a71
Author: Ian Roddis <gitlab@ie2r.com>
Date: Tue Sep 28 12:12:43 2021 -0300
Adding handling for new routes, still need to add tests for new routes
commit b9edb6ba291eb064f4c459a308ea6912fba9fa02
Author: Ian Roddis <gitlab@ie2r.com>
Date: Mon Sep 27 11:59:14 2021 -0300
Defining new endpoints, fixing dag resumption code, adding PAUSED state, refactoring DAGSpec and adding deserializer
Fixing DFS implementation of DAG validation to be much faster
Adding in additional tests to ensure the run order of expanded tasks is preserved
Adding additional compile-time checks, resolving issues that came up as a result
- Adding support for SlurmTaskExecutor in `daggyd` if DAGGY_ENABLE_SLURM is defined.
- Renaming some test cases
- Enabling compile-time slurm support
- Adding slurm documentation
- Roughing in FileSystemLogger
- Deleting Scheduler code and associated unit tests as being too complicated for maintenance.
- Refactoring namespaces for loggers and executors.
