Ian Roddis
c9bfce31e6
I cannot type today
2022-02-01 13:38:08 -04:00
Ian Roddis
b3397381dc
Fixing logic error in submission batches
2022-02-01 13:34:19 -04:00
Ian Roddis
df108dfd28
Increasing curl timeout
2022-02-01 11:02:33 -04:00
Ian Roddis
cc7646f386
Adding delete method
2022-02-01 10:33:58 -04:00
Ian Roddis
3fdd568b00
Adding samples
2022-01-29 13:25:40 -04:00
Ian Roddis
e3a6c9086a
Removing verbose logging
2022-01-28 14:36:33 -04:00
Ian Roddis
782e6a8b97
Changing behaviour of runner to submit at most MAX_SUBMITS for execution before checking for completed tasks
2022-01-28 14:28:46 -04:00
Ian Roddis
5af8deabec
Making DaggyTaskRunner block until a runner is available
2022-01-28 14:18:01 -04:00
Ian Roddis
cbd493ec38
Fixing issue with daggyd config parsing
2022-01-28 13:28:29 -04:00
Ian Roddis
c9820c19a0
Small fixes to test cases
2022-01-28 10:23:21 -04:00
Ian Roddis
c09d5b7d8b
Adding in queuing in the daggyd process for more even distribution
2022-01-28 10:09:55 -04:00
Ian Roddis
070d2ceddf
Adding support for environment variables
2022-01-18 10:54:25 -04:00
Ian Roddis
2784aeee28
Adding in check for cores and memoryMB
2022-01-14 15:48:06 -04:00
Ian Roddis
07646de2cd
Adding SSH executor
2022-01-14 15:39:57 -04:00
Ian Roddis
9e7d78788b
Fixing issue in dagrunner where tasks were never removed from running list
2022-01-13 12:52:00 -04:00
Ian Roddis
854ca5be22
Reducing the scope of a lock
2022-01-13 10:49:09 -04:00
Ian Roddis
3f5d31e036
Removing needless second map lookup
2022-01-12 19:59:10 -04:00
Ian Roddis
c143034c00
Implementing a redirect for a GET on / to list of dagruns
2022-01-12 15:17:00 -04:00
Ian Roddis
868e74fbeb
Seems to work now
2022-01-12 13:11:55 -04:00
Ian Roddis
9a5a247f15
Checkpointing work
2022-01-12 12:50:46 -04:00
Ian Roddis
04e95cfcf3
Making fork process more descriptive in the case of failure
2022-01-10 13:12:19 -04:00
Ian Roddis
7312776e39
Fixing issue with logic for daggyr submission
2022-01-10 13:09:11 -04:00
Ian Roddis
53308c063d
Fixing a number of scaling issues:
...
- Missed closing of file descriptor made ForkingTaskExecutor
silently die after running out of FDs
- Tightened up scope for locks to prevent http timeout
- Simplified threadpool
2022-01-10 13:02:10 -04:00
Ian Roddis
efd4078f70
Adding executor log to output
2022-01-07 17:40:07 -04:00
Ian Roddis
db6fd64601
Fixing dagrun counts
2022-01-07 17:38:29 -04:00
Ian Roddis
d58dc2fc35
Removing ability for multiple servers to listen to the same address
2022-01-07 16:31:07 -04:00
Ian Roddis
58604d98cd
Removing extra prints
2022-01-07 16:14:39 -04:00
Ian Roddis
6ed15b17b2
Fixing issue with querying dag as json
2022-01-07 16:13:12 -04:00
Ian Roddis
f9076be081
Adding fix for race condition in task resource management
2022-01-07 16:00:04 -04:00
Ian Roddis
4da78c9dd6
Adding executor description for verbose output, adding output directory for build artifacts
2022-01-07 14:37:06 -04:00
Ian Roddis
856e5bd2f4
Adding in task attempts drilldown
2022-01-06 15:20:06 -04:00
Ian Roddis
1786b53f7b
Fixing sample dag
2022-01-04 17:05:45 -04:00
Ian Roddis
64a85cd996
Updating example dag
2022-01-04 17:03:42 -04:00
Ian Roddis
f1479a72d9
Fixing test cases on daggyr for new polling
2022-01-04 17:03:11 -04:00
Ian Roddis
5a4c6d0756
Fixing invalid URL
2021-12-30 14:32:43 -04:00
Ian Roddis
a4b26bce04
Handling http errors a bit more gracefully
2021-12-30 14:29:56 -04:00
Ian Roddis
d9479f14e0
Adding in check for no such file
2021-12-30 14:06:28 -04:00
Ian Roddis
ac490b6f3e
Adding html output
2021-12-30 13:44:17 -04:00
Ian Roddis
509a1fcd23
Fixing argument parsing for integers
2021-12-27 22:15:16 -04:00
Ian Roddis
3c6966a9ac
Adding discovery of capacities after the fact.
2021-12-24 10:52:06 -04:00
Ian Roddis
8ca5cdafe2
Fixing issue with resolving
2021-12-24 10:44:19 -04:00
Ian Roddis
d832b935d7
Moving to a poll method for workers, and daggyd-preserved capacities
2021-12-24 10:22:58 -04:00
Ian Roddis
779d6adaea
Moving to a poll method for workers, and daggyd-preserved capacities
2021-12-24 10:21:19 -04:00
Ian Roddis
0914ede8fb
Fixing test, since executor now autovivifies certain job attributes
2021-12-23 11:50:16 -04:00
Ian Roddis
dd473ab8f0
Adjusting capacity impact calculation to yield a more even distribution of jobs
2021-12-23 11:47:55 -04:00
Ian Roddis
85d252f43c
Small bug fixes
2021-12-21 17:23:58 -04:00
Ian Roddis
d90f49b2bb
Lots of fixes to poor daggyr implementation, added debugging messages
2021-12-21 16:49:45 -04:00
Ian Roddis
505ffb02bd
Incorporating ifdefs for features in daggyd
2021-12-16 15:18:29 -04:00
Ian Roddis
2d35d6ed4a
Making features opt-in
2021-12-16 14:17:44 -04:00
Ian Roddis
a035010669
Fixing feature checking
2021-12-16 14:17:03 -04:00
