Adding claude directive
This commit is contained in:
94
CLAUDE.md
Normal file
94
CLAUDE.md
Normal file
@@ -0,0 +1,94 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## Project Overview
|
||||
|
||||
Daggy is a work orchestration framework for running workflows modeled as directed acyclic graphs (DAGs). It's designed for data processing pipelines where tasks have dependencies on each other.
|
||||
|
||||
The system consists of several components:
|
||||
- **libdaggy**: Core library with DAG implementation, thread pool, task executors, and loggers
|
||||
- **daggyd**: REST server that receives and runs DAG specifications
|
||||
- **daggyr**: Remote worker service that runs as a task executor
|
||||
- **utils/daggyc**: Command-line client
|
||||
- **webui**: Web interface for monitoring DAG execution
|
||||
|
||||
## Build Commands
|
||||
|
||||
### Prerequisites
|
||||
- cmake >= 3.14
|
||||
- gcc >= 8
|
||||
- npm (for the webui)
|
||||
- libslurm (optional)
|
||||
|
||||
### Basic Build
|
||||
```sh
|
||||
# Create build directory
|
||||
mkdir -p build
|
||||
cd build
|
||||
|
||||
# Configure with CMake
|
||||
cmake ..
|
||||
|
||||
# Build
|
||||
make -j$(nproc)
|
||||
```
|
||||
|
||||
### Build with Optional Features
|
||||
```sh
|
||||
# Enable Slurm support
|
||||
cmake -DDAGGY_ENABLE_SLURM=ON ..
|
||||
|
||||
# Enable Redis support
|
||||
cmake -DDAGGY_ENABLE_REDIS=ON ..
|
||||
|
||||
# Enable both features
|
||||
cmake -DDAGGY_ENABLE_SLURM=ON -DDAGGY_ENABLE_REDIS=ON ..
|
||||
```
|
||||
|
||||
### Web UI
|
||||
```sh
|
||||
cd webui
|
||||
npm install
|
||||
npm run build
|
||||
```
|
||||
|
||||
### Running Tests
|
||||
```sh
|
||||
# Run all tests
|
||||
cd build
|
||||
tests/tests
|
||||
|
||||
# Run specific test case
|
||||
cd build
|
||||
tests/tests "[specific_test_tag]"
|
||||
```
|
||||
|
||||
## Running the Application
|
||||
|
||||
### Running daggyd with Web UI
|
||||
```sh
|
||||
build/bin/daggyd -v --assets-dir webui/dist
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
Daggy uses a modular architecture consisting of:
|
||||
|
||||
1. **DAG Core**: Implementation of directed acyclic graph data structure with task dependencies
|
||||
2. **Task Executors**: Different executors for running tasks:
|
||||
- ForkingTaskExecutor (local execution)
|
||||
- SlurmTaskExecutor (for HPC environments)
|
||||
- DaggyRunnerTaskExecutor (for remote execution)
|
||||
- SSHTaskExecutor (for SSH-based remote execution)
|
||||
- NoopTaskExecutor (for testing)
|
||||
|
||||
3. **State Loggers**: Track execution state
|
||||
- OStreamLogger (in-memory state management)
|
||||
- RedisLogger (persisting state to Redis)
|
||||
|
||||
4. **Dynamic Task Features**:
|
||||
- Task parameterization
|
||||
- Runtime task generation
|
||||
|
||||
Tasks are defined in JSON with dependencies specified via "parents" or "children" attributes, allowing complex workflow patterns.
|
||||
Reference in New Issue
Block a user