/compress

Reduce status.json file size by removing verbose story fields while preserving all critical tracking metadata and keeping full story details in markdown files.

Quick Start

Copy/agileflow:compress

Purpose

This command solves a common problem: when status.json grows large (especially on big projects with 100+ stories), it can exceed token limits and slow down agent processing. The compress command:

• Removes verbose fields (descriptions, AC, architecture context, etc.)
• Keeps only essential tracking metadata (story ID, status, owner, timestamps)
• Reduces file size by 80-90% typically
• Preserves full story content in markdown files
• Maintains complete project history
• Ensures agents can read status.json efficiently

Parameters

This command has no parameters - it operates directly on docs/09-agents/status.json.

Usage

Simply run:

Copy/agileflow:compress

The command will:

1. Validate status.json exists and is valid JSON
2. Show before stats (size, lines, story count)
3. Create automatic backup
4. Remove verbose fields
5. Write compressed version
6. Show after stats and savings

What Gets Removed

These verbose fields are stripped from status.json:

Copy- description          # Story description and context
- acceptanceCriteria   # AC bullet points
- architectureContext  # Architecture details
- technicalNotes       # Implementation hints
- testingStrategy      # Test approach
- devAgentRecord       # Implementation notes
- previousStoryInsights # Lessons from previous story
- (all other large text fields)

These are safely preserved in docs/06-stories/ markdown files instead.

What Gets Kept

Only essential tracking metadata remains in status.json:

Copy- story_id            # Story identifier
- epic                # Parent epic
- title               # Story title
- owner               # Assigned agent
- status              # Current status (ready/in-progress/blocked/done)
- estimate            # Time estimate
- created             # Creation timestamp
- updated             # Last update timestamp
- completed_at        # Completion timestamp
- dependencies        # Dependent story IDs
- branch              # Git branch name
- summary             # Short summary
- last_update         # Last modification message
- assigned_at         # Assignment timestamp

Example Output

Copy🗜️  AgileFlow Status Compression
   Purpose: Strip verbose fields from status.json
   Target: Keep only essential tracking metadata
 
💾 Backup created: docs/09-agents/status.json.backup
 
📊 Before Compression:
   Stories: 145
   Size: 384KB
   Lines: 12,847
 
✅ Compression complete!
 
📊 After Compression:
   Stories: 145 (unchanged)
   Size: 384KB → 42KB
   Lines: 12,847 → 1,203
   Saved: 89% (342KB)
 
✅ Estimated tokens: ~10,500 (safely under 25000 limit)
 
📋 Status Summary:
   ready: 23 stories
   in-progress: 8 stories
   blocked: 2 stories
   done: 112 stories
 
💾 To restore original: cp docs/09-agents/status.json.backup docs/09-agents/status.json

When to Use

Use compression when:

1. status.json exceeds 25,000 tokens

   • Agents fail with "file content exceeds maximum allowed tokens"

2. File is slow to process

   • Too large for context windows
   • Causes timeout issues

3. After major epic completion

   • Many completed stories with verbose records
   • Project has grown significantly

4. Quarterly maintenance

   • Regular cleanup to keep tracking lean

Alternative: Combine with Archival

For even better results, combine compression with archival:

Copy# Step 1: Archive completed stories older than 3 days
bash .agileflow/scripts/archive-completed-stories.sh 3
 
# Step 2: Compress remaining stories
/agileflow:compress

This two-step process achieves maximum reduction while maintaining history.

Safety & Backups

Automatic Backup

Before compression, the command creates a backup:

Copydocs/09-agents/status.json.backup

No Data Loss

• Full story content remains in docs/06-stories/ markdown files
• Story markdown files are NOT modified
• Only status.json index is compressed
• Can always restore from backup

Restore if Needed

Copycp docs/09-agents/status.json.backup docs/09-agents/status.json

The backup contains the complete original file.

Why This Architecture?

Separation of Concerns

status.json = lightweight tracking index

• "What stories exist?"
• "Who owns each story?"
• "What's the current status?"
• "When was it updated?"

docs/06-stories/ = full story content

• Complete descriptions
• Acceptance criteria
• Architecture decisions
• Implementation notes
• Test strategies

docs/09-agents/archive/ = historical data

• Completed stories older than threshold
• Monthly organization
• Full retrieval capability

Benefits

1. Fast Indexing - Agents read status.json quickly
2. Complete Content - Full details in story markdown
3. Searchable - Can grep story files for content
4. Versionable - All files are git-tracked
5. Archivable - Old stories don't bloat tracking index

Token Estimation

The command estimates token usage:

CopyToken estimation formula: tokens ≈ bytes / 4
 
Before: 384KB × 1024 bytes/KB ÷ 4 = 98,304 tokens
After: 42KB × 1024 bytes/KB ÷ 4 = 10,752 tokens
 
Saved: 87,552 tokens!

Target: Keep status.json under 25,000 tokens (100KB compressed).

Workflow Integration

With Agent Bus

The status.json compression doesn't affect:

• Agent message bus (docs/09-agents/bus/log.jsonl)
• Agent communication and coordination
• Story status tracking

With CI/CD

Compression can be added to CI automation:

Copyname: Maintenance
on:
  schedule:
    - cron: '0 2 * * 0'  # Sunday 2am UTC
 
jobs:
  compress:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Archive and compress
        run: |
          bash .agileflow/scripts/archive-completed-stories.sh 3
          /agileflow:compress
      - name: Commit if changed
        run: |
          git config user.name "github-actions"
          git config user.email "actions@github.com"
          git add docs/09-agents/
          git diff --quiet && git diff --staged --quiet || \
          git commit -m "chore(maintenance): archive and compress status"
          git push

Common Scenarios

Scenario 1: Project Growing Large

Problem: status.json is 250KB with 150 stories Solution:

Copy/agileflow:compress

Result: Reduces to 30KB, easily under token limits

Scenario 2: Many Completed Stories

Problem: 120 completed, 30 active stories; status.json bloated Solution:

Copy# Archive stories completed >7 days ago
bash .agileflow/scripts/archive-completed-stories.sh 7
 
# Then compress
/agileflow:compress

Result: Moves old stories to archive, compresses remaining

Scenario 3: Regular Maintenance

Problem: Want to keep status.json lean automatically Solution: Add to CI schedule (monthly compression) Result: Always under token limits, fast agent processing

Troubleshooting

"status.json still too large after compression"

Try more aggressive archival:

Copy# Archive stories >3 days old instead of default 7
bash .agileflow/scripts/archive-completed-stories.sh 3
 
/agileflow:compress

"Lost critical information after compression"

Restore from backup:

Copycp docs/09-agents/status.json.backup docs/09-agents/status.json

Check if story markdown files have the details in docs/06-stories/.

"Can't find story details"

Story content is in markdown files:

Copyfind docs/06-stories/ -name "US-*.md" | xargs grep "your search term"

Best Practices

1. Regular Compression - Run monthly or quarterly
2. Archive Aggressively - Move old completed stories first
3. Back Up Before - Compression creates backup automatically
4. Verify After - Check token estimate in output
5. Commit Changes - Add compressed status to git
6. Document Policy - Record when/why compression runs
7. Test Restoration - Verify backup works if needed

Related Commands

• /agileflow:status - View current story statuses
• /agileflow:validate - Check AgileFlow system health
• Archive script: bash .agileflow/scripts/archive-completed-stories.sh - Move completed stories to archive

Technical Details

Files Involved

Implementation

The compression script uses jq for JSON processing:

Copy# Extract only essential fields for each story
jq '[.[] | {story_id, epic, title, owner, status, estimate, created, updated, completed_at, dependencies, branch, summary, last_update, assigned_at}]' status.json > status-compressed.json

Token Calculation

Rough estimation used in output:

• 1 token ≈ 4 characters/bytes (approximate)
• 100KB file ≈ 25,000 tokens
• Target: < 100KB to stay well under limits

FAQ