Adding claude directive
This commit is contained in:
94
CLAUDE.md
Normal file
94
CLAUDE.md
Normal file
@@ -0,0 +1,94 @@
|
|||||||
|
# CLAUDE.md
|
||||||
|
|
||||||
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||||
|
|
||||||
|
## Project Overview
|
||||||
|
|
||||||
|
Daggy is a work orchestration framework for running workflows modeled as directed acyclic graphs (DAGs). It's designed for data processing pipelines where tasks have dependencies on each other.
|
||||||
|
|
||||||
|
The system consists of several components:
|
||||||
|
- **libdaggy**: Core library with DAG implementation, thread pool, task executors, and loggers
|
||||||
|
- **daggyd**: REST server that receives and runs DAG specifications
|
||||||
|
- **daggyr**: Remote worker service that runs as a task executor
|
||||||
|
- **utils/daggyc**: Command-line client
|
||||||
|
- **webui**: Web interface for monitoring DAG execution
|
||||||
|
|
||||||
|
## Build Commands
|
||||||
|
|
||||||
|
### Prerequisites
|
||||||
|
- cmake >= 3.14
|
||||||
|
- gcc >= 8
|
||||||
|
- npm (for the webui)
|
||||||
|
- libslurm (optional)
|
||||||
|
|
||||||
|
### Basic Build
|
||||||
|
```sh
|
||||||
|
# Create build directory
|
||||||
|
mkdir -p build
|
||||||
|
cd build
|
||||||
|
|
||||||
|
# Configure with CMake
|
||||||
|
cmake ..
|
||||||
|
|
||||||
|
# Build
|
||||||
|
make -j$(nproc)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Build with Optional Features
|
||||||
|
```sh
|
||||||
|
# Enable Slurm support
|
||||||
|
cmake -DDAGGY_ENABLE_SLURM=ON ..
|
||||||
|
|
||||||
|
# Enable Redis support
|
||||||
|
cmake -DDAGGY_ENABLE_REDIS=ON ..
|
||||||
|
|
||||||
|
# Enable both features
|
||||||
|
cmake -DDAGGY_ENABLE_SLURM=ON -DDAGGY_ENABLE_REDIS=ON ..
|
||||||
|
```
|
||||||
|
|
||||||
|
### Web UI
|
||||||
|
```sh
|
||||||
|
cd webui
|
||||||
|
npm install
|
||||||
|
npm run build
|
||||||
|
```
|
||||||
|
|
||||||
|
### Running Tests
|
||||||
|
```sh
|
||||||
|
# Run all tests
|
||||||
|
cd build
|
||||||
|
tests/tests
|
||||||
|
|
||||||
|
# Run specific test case
|
||||||
|
cd build
|
||||||
|
tests/tests "[specific_test_tag]"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Running the Application
|
||||||
|
|
||||||
|
### Running daggyd with Web UI
|
||||||
|
```sh
|
||||||
|
build/bin/daggyd -v --assets-dir webui/dist
|
||||||
|
```
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
Daggy uses a modular architecture consisting of:
|
||||||
|
|
||||||
|
1. **DAG Core**: Implementation of directed acyclic graph data structure with task dependencies
|
||||||
|
2. **Task Executors**: Different executors for running tasks:
|
||||||
|
- ForkingTaskExecutor (local execution)
|
||||||
|
- SlurmTaskExecutor (for HPC environments)
|
||||||
|
- DaggyRunnerTaskExecutor (for remote execution)
|
||||||
|
- SSHTaskExecutor (for SSH-based remote execution)
|
||||||
|
- NoopTaskExecutor (for testing)
|
||||||
|
|
||||||
|
3. **State Loggers**: Track execution state
|
||||||
|
- OStreamLogger (in-memory state management)
|
||||||
|
- RedisLogger (persisting state to Redis)
|
||||||
|
|
||||||
|
4. **Dynamic Task Features**:
|
||||||
|
- Task parameterization
|
||||||
|
- Runtime task generation
|
||||||
|
|
||||||
|
Tasks are defined in JSON with dependencies specified via "parents" or "children" attributes, allowing complex workflow patterns.
|
||||||
Reference in New Issue
Block a user