From d4b79c48b068308755f4af84870d2f8c0686ea29 Mon Sep 17 00:00:00 2001 From: Ian Roddis <31021769+iroddis@users.noreply.github.com> Date: Sat, 31 May 2025 10:13:54 -0300 Subject: [PATCH] Adding claude directive --- CLAUDE.md | 94 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 94 insertions(+) create mode 100644 CLAUDE.md diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..56af760 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,94 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## Project Overview + +Daggy is a work orchestration framework for running workflows modeled as directed acyclic graphs (DAGs). It's designed for data processing pipelines where tasks have dependencies on each other. + +The system consists of several components: +- **libdaggy**: Core library with DAG implementation, thread pool, task executors, and loggers +- **daggyd**: REST server that receives and runs DAG specifications +- **daggyr**: Remote worker service that runs as a task executor +- **utils/daggyc**: Command-line client +- **webui**: Web interface for monitoring DAG execution + +## Build Commands + +### Prerequisites +- cmake >= 3.14 +- gcc >= 8 +- npm (for the webui) +- libslurm (optional) + +### Basic Build +```sh +# Create build directory +mkdir -p build +cd build + +# Configure with CMake +cmake .. + +# Build +make -j$(nproc) +``` + +### Build with Optional Features +```sh +# Enable Slurm support +cmake -DDAGGY_ENABLE_SLURM=ON .. + +# Enable Redis support +cmake -DDAGGY_ENABLE_REDIS=ON .. + +# Enable both features +cmake -DDAGGY_ENABLE_SLURM=ON -DDAGGY_ENABLE_REDIS=ON .. +``` + +### Web UI +```sh +cd webui +npm install +npm run build +``` + +### Running Tests +```sh +# Run all tests +cd build +tests/tests + +# Run specific test case +cd build +tests/tests "[specific_test_tag]" +``` + +## Running the Application + +### Running daggyd with Web UI +```sh +build/bin/daggyd -v --assets-dir webui/dist +``` + +## Architecture + +Daggy uses a modular architecture consisting of: + +1. **DAG Core**: Implementation of directed acyclic graph data structure with task dependencies +2. **Task Executors**: Different executors for running tasks: + - ForkingTaskExecutor (local execution) + - SlurmTaskExecutor (for HPC environments) + - DaggyRunnerTaskExecutor (for remote execution) + - SSHTaskExecutor (for SSH-based remote execution) + - NoopTaskExecutor (for testing) + +3. **State Loggers**: Track execution state + - OStreamLogger (in-memory state management) + - RedisLogger (persisting state to Redis) + +4. **Dynamic Task Features**: + - Task parameterization + - Runtime task generation + +Tasks are defined in JSON with dependencies specified via "parents" or "children" attributes, allowing complex workflow patterns. \ No newline at end of file