Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 20 additions & 10 deletions .claude/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,10 +93,10 @@ uv run poe test
uv run poe test:e2e

# Run single test file
pytest tests/it/test_resources.py
pytest tests/it/test_process.py

# Run single test
pytest tests/it/test_resources.py::test_load_manifest
pytest tests/it/test_process.py::TestCommand::test_empty

# Run tests matching a pattern
pytest -k "test_load" -v
Expand Down Expand Up @@ -197,23 +197,25 @@ Use the `Log` class from utils:
```python
from terranova.utils import Log

Log.info("message") # Blue info
Log.success("message") # Green success
Log.warning("message") # Yellow warning
Log.error("message") # Red error
Log.fatal("action", err) # Red fatal, prints error context
Log.action("message") # Yellow action marker
Log.success("message") # Green success ("Succeeded to {message}")
Log.failure("message", err) # Red failure, prints ExplainedError cause/resolution if present
Log.fatal("message", err) # Red failure + raises Exit(1)
```

### Error Handling

Custom exceptions in `src/terranova/exceptions.py`:

- `InvalidManifestError`: Manifest validation failed
- `ExplainedError`: Base for all errors that carry a `cause` and optional `resolution` shown to the user
- `ManifestError` / `InvalidManifestError`: Manifest validation failed
- `InvalidResourcesError`: Resource configuration invalid
- `MissingManifestError`: Manifest file not found
- `UnreadableManifestError`: Cannot read manifest
- `MissingRunbookEnvError`: Required env var for runbook missing
- `VersionManifestError`: Unsupported manifest version
- `RunbookError` / `AmbiguousRunbookError`: Multiple runbooks match a name
- `MissingRunbookError`: Named runbook not found
- `MissingRunbookEnvError`: Required env var for runbook missing

Wrap jsonschema `ValidationError` in `InvalidManifestError` when catching.

Expand Down Expand Up @@ -276,4 +278,12 @@ This sets `SharedContext.debug = True` which can be checked in code.
- Apache-2.0 licensed
- Original source: [elastic/terranova](https://github.com/elastic/terranova)
- Keep copyright notices for both Elasticsearch and current maintainer
- Follow existing commit message format: `type: description` (e.g., `fix:`, `refactor:`, `chore:`, `docs:`)
- Follow existing commit message format: `type: description` (e.g., `fix:`, `refactor:`, `chore:`, `docs:`)

## GitHub PR/Issue Creation

Before running `gh pr create`, use the `pr-template-validation` skill to check the PR body against `.github/PULL_REQUEST_TEMPLATE.md`.

Before running `gh issue create`, use the `issue-template-validation` skill to check the issue body against the matching `.github/ISSUE_TEMPLATE/*` template.

Both skills reject placeholder text (TBD/TODO/FIXME/WIP), empty sections, and duplicated content across sections — do not bypass them by creating the PR/issue directly without invoking the skill first.
192 changes: 192 additions & 0 deletions .claude/skills/issue-template-validation/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,192 @@
---
name: issue-template-validation
description: Validate GitHub issue template sections before creating an issue. Invoke this skill immediately before any `gh issue create` command. Do not proceed with issue creation if any PRESENT required section is empty, contains placeholders (TBD/TDB/TODO/FIXME/WIP), or duplicates another section. Detect the template type from the issue title prefix or user context and apply the matching template rules.
---

# Issue Template Validation

## When to Use This Skill

Invoke this skill **before creating any GitHub issue** to validate that all required sections contain meaningful, unique content.

## Template Types

Detect the template from the issue title prefix or user context:

| Prefix | Template |
|---|---|
| `[Bug]:` | Bug Report |
| `[Feature]:` | Feature Request |
| `[Task]:` | Task |
| `[Docs]:` | Documentation |
| `[Question]:` | Question |

If no prefix is present, infer from the content or ask the user which template they intend to use.

## Required Sections per Template

### 🐛 Bug Report

**Required (must have meaningful content):**
1. `:bug: Bug Description`
2. `✅ Expected Behavior`
3. `❌ Actual Behavior`
4. `🔁 Steps to Reproduce`

**Optional (skip validation if header absent):**
- `🖥️ Environment`
- `📸 Screenshots / Logs`
- `🧠 Additional Context`

---

### ✨ Feature Request

**Required:**
1. `🚀 Feature Summary`
2. `🤔 Problem Statement`
3. `💡 Proposed Solution`

**Optional:**
- `🔄 Alternatives Considered`
- `📈 Impact`
- `📝 Additional Context`

---

### 📋 Task

**Required:**
1. `📌 Summary`
2. `🎯 Objectives`
3. `🧩 Scope`

**Optional:**
- `🧠 Design / Decisions`
- `📅 Timeline`
- `✅ Definition of Done`
- `📣 Notes / Updates`

---

### 📚 Documentation

**Required:**
1. `📍 Location`
2. `❓ What's Wrong`
3. `✅ Suggested Improvement`

**Optional:**
- `🧠 Additional Context`

---

### ❓ Question

**Required:**
1. `❓ Question`

**Optional:**
- `🔍 What I've Tried`
- `🧾 Context`
- `📎 Additional Info`

---

## Validation Rules

**Important distinction:**
- **Missing optional section** (header not in body) = ALLOWED
- **Missing required section** (header not in body) = FAIL — ask user to add it
- **Empty section** (header present but no content) = FAIL — user must fill it or remove it

For each **required** section that IS present, and for each **optional** section that IS present, validation **FAILS** if:

### 1. Empty Content
Section header exists but contains only whitespace or placeholder list items.

**Action:** Reject and ask user to either add substantive content or remove the section header entirely.

### 2. Placeholder Text
Case-insensitive detection of: `TBD`, `TDB`, `TODO`, `FIXME`, `WIP`

This includes placeholders inside list items (e.g. `- TBD`, `- [ ] TDB`).

**Action:** Reject and ask user to replace with actual content.

### 3. Content Duplication
Section content is an exact or near-exact duplicate of another section.

**Action:** Reject and explain each section must articulate a distinct aspect.

### 4. Missing Required Section
A required section for the detected template type is entirely absent from the body.

**Action:** Reject and ask user to add the missing section with real content.

---

## What Counts as Meaningful Content

**Valid examples:**

- **Bug Description:** "When running `ensemble run --parallel`, jobs are silently dropped after the 10th concurrent execution. No error is logged."
- **Expected Behavior:** "All submitted jobs should be executed. The system should queue excess jobs if concurrency limit is hit."
- **Objectives:** "- [ ] Identify the root cause of the queue overflow\n- [ ] Add backpressure mechanism\n- [ ] Add test coverage for high-concurrency scenarios"
- **Proposed Solution:** "Add a configurable `max_queue_size` parameter to the worker executor with exponential backoff when the queue is full."

**Invalid examples:**

- `TBD`, `TDB`, `TODO`, `FIXME`, `WIP`
- `- TBD` or `- [ ] TDB`
- Empty section with just the header
- Copy of another section's content

---

## Validation Workflow

1. **Detect template type** from title prefix or context (ask if unclear)
2. **Check required sections** are all present in the body → fail if any are missing
3. **For each present section** (required or optional), validate:
- Trim whitespace
- Check if empty → FAIL
- Check for placeholder patterns (case-insensitive) → FAIL
- Check for duplicate content → FAIL
4. **If ANY validation fails:**
- Do **not** proceed with issue creation
- Report all failures with specific reasons
- Ask user to revise and retry
5. **If ALL validations pass:**
- Proceed with issue creation using `gh issue create`

---

## Error Reporting Format

```
❌ Issue validation failed. Will not create issue until these sections are fixed:

1. "🎯 Objectives" [required]
❌ Contains placeholder text: TDB
→ Replace with actual objectives as a checklist

2. "🧩 Scope" [required]
❌ Section header exists but is empty
→ Describe what is in and out of scope, or remove the header

3. "🚀 Feature Summary" [required]
❌ Section is missing from the issue body
→ Add this section with a short description of the feature

Please revise these sections and try creating the issue again.
```

---

## Implementation Notes

- Placeholder detection is case-insensitive and covers bare words as well as list items (`- TBD`, `- [ ] TDB`)
- Trim whitespace before checking for empty content
- Identify sections by emoji/heading markers from the templates
- This validation does NOT apply to issue updates after initial creation or issues created manually outside Claude
125 changes: 125 additions & 0 deletions .claude/skills/pr-template-validation/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
---
name: pr-template-validation
description: Validate PR body sections before creating a GitHub PR. Invoke this skill immediately before any `gh pr create` command. Do not proceed with PR creation if any PRESENT section is empty, contains placeholders (TBD/TODO/FIXME/WIP), or duplicates another section. Missing sections are allowed if not relevant.
---

# PR Template Validation

## When to Use This Skill

Invoke this skill **before creating any GitHub PR** to validate that all PR template sections contain meaningful, unique content.

## Template Sections to Validate

Validate these sections from `.github/PULL_REQUEST_TEMPLATE.md`:

1. **🧑‍💻 What is the change being made?**
2. **:question: Why is the change being made?**
3. **:white_check_mark: How has this been tested?**
4. **:books: How has this been documented?**
5. **:link: Related Issues**

## Validation Rules

**Important distinction:**
- **Missing section** (header not included in PR body) = ALLOWED if not relevant to the change
- **Empty section** (header exists but no content) = FAIL (user must either fill it or remove it)

For each section that IS present in the PR body, validation **FAILS** if:

### 1. Empty Content
- Section header exists but contains only whitespace or is completely empty
- **Action:** Reject and ask user to either:
- Add substantive content, OR
- Remove the section header entirely if not relevant

### 2. Placeholder Text Detection
Case-insensitive detection of:
- `TBD` (To Be Determined)
- `TODO` (To Do)
- `FIXME` (Fix Me)
- `WIP` (Work In Progress)

**Example failure:** "How has this been tested?" contains "TBD - will test later"

**Action:** Reject and ask user to replace placeholder with actual content

### 3. Content Duplication
If a section's content is an exact or near-exact duplicate of another section.

**Example failure:**
- "What is the change?" = "Add user authentication"
- "Why is the change?" = "Add user authentication"

**Action:** Reject and explain that each section must articulate a unique aspect (What vs Why vs How)

## What Counts as Meaningful Content

**Valid examples:**

- **What:** "Add JWT token refresh mechanism to extend session lifetime without requiring re-login. Implements automatic token rotation with 5-minute intervals."

- **Why:** "Users were experiencing session timeouts during long workflows. This allows sessions to stay active without interrupting user work."

- **Testing:** "Added unit tests for token rotation logic. Verified refresh tokens are generated and validated correctly. Manual testing in staging confirms token expiry and refresh behavior."

- **Documentation:** "Updated API docs with new `/auth/refresh` endpoint. Added troubleshooting section for token expiration in runbook."

- **Related Issues:** "Closes #1234. Related to #1235 for session management improvements."

**Invalid examples:**

- **Placeholder:** "TBD" or "Will add tests later"
- **Dismissive:** "N/A" or "Not applicable"
- **Duplicate:** Copy of another section's content
- **Too vague:** "Added stuff" or "Fixed bugs"

## Validation Workflow

When invoked during PR creation:

1. **Extract** the PR body content
2. **Parse** to identify which template sections are present (by emoji/heading markers)
3. **For each PRESENT section**, validate:
- Trim whitespace
- Check if empty (header exists but no content) → FAIL
- Check for placeholder patterns (case-insensitive) → FAIL
- Check for duplicate content across other sections → FAIL
- **Note:** If section header is missing entirely, skip validation (allowed if not relevant)
4. **If ANY validation fails:**
- **Do not proceed** with PR creation
- Report failures with specific reasons
- Provide examples of valid content
- Ask user to revise and retry
5. **If ALL validations pass:**
- Proceed with PR creation

## Error Reporting Format

When validation fails, report like this:

```
❌ PR validation failed. Will not create PR until these sections are fixed:

1. "Why is the change being made?"
❌ Contains placeholder text: TBD
→ Replace with actual reasoning for this change

2. "How has this been tested?"
❌ Section header exists but is empty
→ Either add testing details, or remove the section header if no testing was needed

3. "How has this been documented?"
❌ Content duplicates "What is the change being made?"
→ Explain what docs were updated, not just repeat the change

Please revise these sections and try creating the PR again.
```

## Implementation Notes

- Validation is case-insensitive for placeholder detection
- Trim all whitespace before checking for empty content
- Identify sections by emoji headers from the template
- Consider section separators (blank lines) as structural markers
- This validation does NOT apply to PR updates after initial creation or manual PRs created outside Claude