- Missed closing of file descriptor made ForkingTaskExecutor
silently die after running out of FDs
- Tightened up scope for locks to prevent http timeout
- Simplified threadpool
This paves the way for implementing daggys and other utilities.
Squashed commit of the following:
commit 1f77239ab3c9e44d190eef94531a39501c8c4dfe
Author: Ian Roddis <gitlab@ie2r.com>
Date: Mon Oct 18 16:25:02 2021 -0300
Adding README, stdout support for daggyd logging
commit c2c237224e84a3be68aaa597ce98af1365e74a13
Author: Ian Roddis <gitlab@ie2r.com>
Date: Mon Oct 18 16:10:29 2021 -0300
removing old daggyd
commit cfea2baf61ca10c535801c5a391d2d525a1a2d04
Author: Ian Roddis <gitlab@ie2r.com>
Date: Mon Oct 18 16:10:09 2021 -0300
Moving tests into their sub-project folders
commit e41ca42069bea1db16dd76b6684a3f692fef6b15
Author: Ian Roddis <gitlab@ie2r.com>
Date: Mon Oct 18 15:57:40 2021 -0300
Splitting out daggyd from libdaggy
commit be97b146c1d2446f5c03cb78707e921f18c60bd8
Author: Ian Roddis <gitlab@ie2r.com>
Date: Mon Oct 18 15:56:55 2021 -0300
Splitting out daggyd from libdaggy
commit cb61e140e9d6d8832d61fb7037fd4c0ff6edad00
Author: Ian Roddis <gitlab@ie2r.com>
Date: Mon Oct 18 15:49:47 2021 -0300
moving daggy to libdaggy
Squashed commit of the following:
commit f3549005c0192fd77bf47d208b74a11dd21380fa
Author: Ian Roddis <gitlab@ie2r.com>
Date: Sun Oct 17 10:52:12 2021 -0300
Fixing issues with serialization of attempt records that included newlines
commit ab387d62850428e320a05cdf54fd2026369d0bb6
Author: Ian Roddis <gitlab@ie2r.com>
Date: Fri Oct 15 12:57:14 2021 -0300
Migrating to vanilla redis from rejson
commit f648cf065dea2d0a7e30aaec17441e9a37531ba0
Author: Ian Roddis <gitlab@ie2r.com>
Date: Fri Oct 15 10:23:34 2021 -0300
Making rediscontext threadsafe
commit ca51d95c5c4a5f4aaa13c5abe32da161e919d66c
Author: Ian Roddis <gitlab@ie2r.com>
Date: Fri Oct 15 10:01:33 2021 -0300
updating todo
Squashed commit of the following:
commit dc3a1bf07b5e7afdfd45e56f34596300dab6fd70
Author: Ian Roddis <gitlab@ie2r.com>
Date: Wed Oct 13 15:15:28 2021 -0300
Updating documentation a bit
commit 8ec9c8c74f587368b32d034d3240a5537a69d4b1
Author: Ian Roddis <gitlab@ie2r.com>
Date: Wed Oct 13 15:11:23 2021 -0300
Completing tests for redis
commit a6308dfa35b40b5a147394af8e3322ada871eb92
Author: Ian Roddis <gitlab@ie2r.com>
Date: Wed Oct 13 14:56:22 2021 -0300
Resolving some errors with forking environment
commit 34691b6f85abae67001f4a4c234a4f7314407331
Author: Ian Roddis <gitlab@ie2r.com>
Date: Wed Oct 13 10:53:55 2021 -0300
Checkpointing work on unit tests
commit 44c2b50fde30348938d901703ead9e279c3cd237
Author: Ian Roddis <gitlab@ie2r.com>
Date: Wed Oct 13 09:09:58 2021 -0300
Checkpointing work on redis
commit a8051b725257087e25bc452673633ba6b40e3985
Author: Ian Roddis <gitlab@ie2r.com>
Date: Fri Oct 8 13:31:41 2021 -0300
Checkpointing progress, changing state updates to a single record type
commit 456b84ad8c7dee0ff0dd39d5a7caead1ccd1126c
Author: Ian Roddis <gitlab@ie2r.com>
Date: Thu Oct 7 16:43:48 2021 -0300
Checkpointing progress
commit f19dcaa4e417c3f2f6e527c288fe51401c9fe1d7
Author: Ian Roddis <gitlab@ie2r.com>
Date: Thu Oct 7 11:53:35 2021 -0300
Moving back to hiredis to avoid boost dependency
commit e4bea6c589e82c82fd41476f164d946d77677193
Author: Ian Roddis <gitlab@ie2r.com>
Date: Wed Oct 6 10:41:16 2021 -0300
fixing comments
commit 807a73c2a406817001eec048483938545a60194c
Author: Ian Roddis <gitlab@ie2r.com>
Date: Wed Oct 6 10:40:38 2021 -0300
Switching to redis-cpp
commit d060c008d4d96bf3a81a19d35067f95f3638b8ca
Author: Ian Roddis <gitlab@ie2r.com>
Date: Tue Oct 5 17:54:06 2021 -0300
Adding hiredis dep
commit 8a4e0fb24d191bf1c1009bd4c8800b4adab21f81
Author: Ian Roddis <gitlab@ie2r.com>
Date: Tue Oct 5 17:23:21 2021 -0300
Adding support for commandString
commit 9055cbde34d2489065b03c25c02a8bea56e42d54
Author: Ian Roddis <gitlab@ie2r.com>
Date: Tue Oct 5 17:10:01 2021 -0300
Completing support for environment variables
commit 989adef378724bbc9451c5048ea9d1285eebe2f9
Author: Ian Roddis <gitlab@ie2r.com>
Date: Tue Oct 5 12:29:31 2021 -0300
Adding environment support to ForkingTaskExecutor
commit b06b11cbb5d09c6d091551e39767cd3316f88376
Author: Ian Roddis <gitlab@ie2r.com>
Date: Tue Oct 5 11:57:37 2021 -0300
Fixing failing unit test
commit fe2a43a19b2a16a9aedd9e95e71e672935ecaeb1
Author: Ian Roddis <gitlab@ie2r.com>
Date: Tue Oct 5 11:54:01 2021 -0300
Adding endpoints and updating documentation
commit 46e0deeefb8b06291ae5e2d6b8ec4749c5b0ea6f
Author: Ian Roddis <gitlab@ie2r.com>
Date: Tue Oct 5 11:49:43 2021 -0300
Completing unit tests and relevant fixes
commit e0569f370624844feee6aae4708bfe683f4156cf
Author: Ian Roddis <gitlab@ie2r.com>
Date: Mon Oct 4 17:30:59 2021 -0300
Adding in gcc tsan for debug builds to help with race conditions, fixing many of those, and fixing really crummy assumption about how futures worked that will speed up task execution by a ton.
commit c748a4f592e1ada5546908be5281d04f4749539d
Author: Ian Roddis <gitlab@ie2r.com>
Date: Mon Oct 4 10:14:43 2021 -0300
Checkpointing work that seems to have resolved the race condition
commit 7a79f2943e0d50545d976a28b4b379340a90dded
Author: Ian Roddis <gitlab@ie2r.com>
Date: Wed Sep 29 09:27:07 2021 -0300
Completing the rough-in for DAG killing / pausing / resuming
commit 4cf8d81d5f6fcf4a7dd83d8fca3e23f153aa8acb
Author: Ian Roddis <gitlab@ie2r.com>
Date: Tue Sep 28 14:53:50 2021 -0300
Adding dagrunner unit tests, adding a resetRunning method to resume
commit 54e2c1f9f5e7d5b339d71be024e0e390c4d2bf61
Author: Ian Roddis <gitlab@ie2r.com>
Date: Tue Sep 28 14:45:57 2021 -0300
Refactoring runDAG into DAGRunner
commit 682be7a11e2fae850e1bc3e207628d2335768c2b
Author: Ian Roddis <gitlab@ie2r.com>
Date: Tue Sep 28 14:34:43 2021 -0300
Adding DAGRunner class to replace Utilities::runDAG, making Slurm cancellation rc agree with SIGKILL
commit 4171b3a6998791abfc71b04f8de1ae93c4f90a78
Author: Ian Roddis <gitlab@ie2r.com>
Date: Tue Sep 28 14:14:17 2021 -0300
Adding unit tests for stopping jobs to slurm
commit dc0b1ff26a5d98471164132d35bb8a552cc75ff8
Author: Ian Roddis <gitlab@ie2r.com>
Date: Tue Sep 28 14:04:15 2021 -0300
Adding in stop method for task executors
commit e752b44f55113be54392bcbb5c3d6f251d673cfa
Author: Ian Roddis <gitlab@ie2r.com>
Date: Tue Sep 28 12:32:06 2021 -0300
Adding additional tests for loggers
commit f0773d5a84a422738fc17c9277a2b735a21a3d04
Author: Ian Roddis <gitlab@ie2r.com>
Date: Tue Sep 28 12:29:21 2021 -0300
Unit tests pass
commit 993ff2810de2d53dc6a59ab53d620fecf152d4a0
Author: Ian Roddis <gitlab@ie2r.com>
Date: Tue Sep 28 12:24:34 2021 -0300
Adding handling for new routes, still need to add tests for new routes
commit 676623b14e45759872a2dbcbc98f6a744e022a71
Author: Ian Roddis <gitlab@ie2r.com>
Date: Tue Sep 28 12:12:43 2021 -0300
Adding handling for new routes, still need to add tests for new routes
commit b9edb6ba291eb064f4c459a308ea6912fba9fa02
Author: Ian Roddis <gitlab@ie2r.com>
Date: Mon Sep 27 11:59:14 2021 -0300
Defining new endpoints, fixing dag resumption code, adding PAUSED state, refactoring DAGSpec and adding deserializer
