Kinesin/daggy
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
207 Commits 1 Branch 0 Tags
57e93b5045d0764daa70d5bcbc75a7f9a7da40c8
Commit Graph

19 Commits

Author SHA1 Message Date
Ian Roddis
57e93b5045 Simplifying daggyr server, and returning to a 
task submit / task poll model.

Squashed commit of the following:

commit 0ef57f095d15f0402915de54f83c1671120bd228
Author: Ian Roddis <tech@kinesin.ca>
Date:   Wed Feb 2 08:18:03 2022 -0400

    Simplifying task polling and reducing lock scopes

commit d77ef02021cc728849c7d1fb0185dd1a861b4a3d
Author: Ian Roddis <tech@kinesin.ca>
Date:   Wed Feb 2 08:02:47 2022 -0400

    Simplifying check

commit c1acf34440162abb890a959f3685c2d184242ed5
Author: Ian Roddis <tech@kinesin.ca>
Date:   Wed Feb 2 08:01:13 2022 -0400

    Removing capacity tracking from runner, since it is maintained in daggyd

commit 9401246f92113ab140143c1895978b9de8bd9972
Author: Ian Roddis <tech@kinesin.ca>
Date:   Wed Feb 2 07:47:28 2022 -0400

    Adding retry for submission

commit 398aa04a320347bb35f23f3f101d91ab4df25652
Author: Ian Roddis <tech@kinesin.ca>
Date:   Tue Feb 1 14:54:20 2022 -0400

    Adding in execution note, as well as requeuing the result if the peer disconnects

commit 637b14af6d5b53f25b9c38d4c8a7ed8532af5599
Author: Ian Roddis <tech@kinesin.ca>
Date:   Tue Feb 1 14:13:59 2022 -0400

    Fixing locking issues

commit 4d6716dfda8aa7f51e0abbdab833aff618915ba0
Author: Ian Roddis <tech@kinesin.ca>
Date:   Tue Feb 1 13:33:14 2022 -0400

    Single task daggyr working

commit bd48a5452a92817faf25ee44a6115aaa2f6c30d1
Author: Ian Roddis <tech@kinesin.ca>
Date:   Tue Feb 1 12:22:04 2022 -0400

    Checkpointing work
 2022-02-02 21:12:05 -04:00
Ian Roddis
5af8deabec Making DaggyTaskRunner block until a runner is available 2022-01-28 14:18:01 -04:00
Ian Roddis
cbd493ec38 Fixing issue with daggyd config parsing 2022-01-28 13:28:29 -04:00
Ian Roddis
c09d5b7d8b Adding in queuing in the daggyd process for more even distribution 2022-01-28 10:09:55 -04:00
Ian Roddis
3f5d31e036 Removing needless second map lookup 2022-01-12 19:59:10 -04:00
Ian Roddis
868e74fbeb Seems to work now 2022-01-12 13:11:55 -04:00
Ian Roddis
9a5a247f15 Checkpointing work 2022-01-12 12:50:46 -04:00
Ian Roddis
7312776e39 Fixing issue with logic for daggyr submission 2022-01-10 13:09:11 -04:00
Ian Roddis
53308c063d Fixing a number of scaling issues: 
- Missed closing of file descriptor made ForkingTaskExecutor
  silently die after running out of FDs
- Tightened up scope for locks to prevent http timeout
- Simplified threadpool
 2022-01-10 13:02:10 -04:00
Ian Roddis
f9076be081 Adding fix for race condition in task resource management 2022-01-07 16:00:04 -04:00
Ian Roddis
4da78c9dd6 Adding executor description for verbose output, adding output directory for build artifacts 2022-01-07 14:37:06 -04:00
Ian Roddis
3c6966a9ac Adding discovery of capacities after the fact. 2021-12-24 10:52:06 -04:00
Ian Roddis
8ca5cdafe2 Fixing issue with resolving 2021-12-24 10:44:19 -04:00
Ian Roddis
d832b935d7 Moving to a poll method for workers, and daggyd-preserved capacities 2021-12-24 10:22:58 -04:00
Ian Roddis
779d6adaea Moving to a poll method for workers, and daggyd-preserved capacities 2021-12-24 10:21:19 -04:00
Ian Roddis
dd473ab8f0 Adjusting capacity impact calculation to yield a more even distribution of jobs 2021-12-23 11:47:55 -04:00
Ian Roddis
85d252f43c Small bug fixes 2021-12-21 17:23:58 -04:00
Ian Roddis
d90f49b2bb Lots of fixes to poor daggyr implementation, added debugging messages 2021-12-21 16:49:45 -04:00
Ian Roddis
8d00621908 Adding support for remote execution daemons. 
Squashed commit of the following:

commit 69d5ef7a256b86a86d46e5ae374c00fded1497ea
Author: Ian Roddis <tech@kinesin.ca>
Date:   Thu Dec 16 12:15:55 2021 -0400

    Updating readme

commit 94a9f676d0f9cc0b55cdc18c4927eaea40d82c77
Author: Ian Roddis <tech@kinesin.ca>
Date:   Thu Dec 16 12:05:36 2021 -0400

    Fixing serialization of attempt records when querying entire dag

commit 945e5f90b24abf07c9af1bc4c6bbcb33e93b8069
Author: Ian Roddis <tech@kinesin.ca>
Date:   Thu Dec 16 11:37:59 2021 -0400

    Compiles cleanly...

commit 8b23e46081d47fb80dc1a2d998fc6dc4bbf301a8
Author: Ian Roddis <tech@kinesin.ca>
Date:   Thu Dec 16 10:43:03 2021 -0400

    Adding in missing source file to cmake build list

commit 6d10d9791206e2bc15788beadeea580b8e43a853
Author: Ian Roddis <tech@kinesin.ca>
Date:   Thu Dec 16 10:41:43 2021 -0400

    Adding new executors

commit 42a2c67f4d6ae99df95d917c8621d78cd99837a1
Author: Ian Roddis <tech@kinesin.ca>
Date:   Thu Dec 16 10:27:14 2021 -0400

    Fixing missing curl cmake dependency

commit 394bc4c5d51ecee7bf14712f719c8bf7e97fb0fa
Author: Ian Roddis <tech@kinesin.ca>
Date:   Thu Dec 16 10:21:58 2021 -0400

    Fixing missing curl cmake dependency

commit dd9efc8e7e7770ea1bcbccb70a1af9cfcff0414c
Author: Ian Roddis <tech@kinesin.ca>
Date:   Wed Dec 15 17:15:38 2021 -0400

    Checkpointing progress

commit 3b3b55d6037bb96e46de6763f486f4ecb92fe6a0
Author: Ian Roddis <tech@kinesin.ca>
Date:   Wed Dec 15 14:21:18 2021 -0400

    updating readme

commit 303027c11452941b2a0c0d1b04ac5942e79efd74
Author: Ian Roddis <tech@kinesin.ca>
Date:   Wed Dec 15 14:17:16 2021 -0400

    Namespacing daggyd
    Adding more error checking around deserialization of parameters
    Adding tests for runner agent

commit c592eaeba12e2a449bae401e8c1d9ed236416d52
Author: Ian Roddis <tech@kinesin.ca>
Date:   Wed Dec 15 11:20:21 2021 -0400

    Checkpointing work

commit fb1862d1cefe2b53a98659cce3c8c73d88bf5d84
Author: Ian Roddis <tech@kinesin.ca>
Date:   Wed Dec 15 09:52:29 2021 -0400

    Copying daggyd for daggyr template, adding in basic routes
 2021-12-16 12:16:12 -04:00