NAS Backup Checker

A high-performance Python tool to compare directories between your SSD/local storage and NAS backup, with advanced caching and optimization features.

Features

🚀 Performance Optimizations

• Parallel Processing: Scans source and backup simultaneously using ThreadPoolExecutor
• Intelligent Caching: Stores scan results to avoid re-scanning unchanged directories
• Incremental Scanning: Only scans directories modified since last scan
• Depth Limiting: Control scan depth for faster overview comparisons
• Quick Mode: Optimized preset for fast directory structure comparison

📊 Comparison Capabilities

• Missing Backups: Files/directories on source but not on backup
• Outdated Backups: Files that exist but are newer/different on source
• Extra Files: Files only on backup (archived content)
• Matching Files: Properly backed up and up-to-date content

🎛️ Scan Modes

• Full File Scan: Compare individual files (size + modification time)
• Directory-Only Scan: Fast comparison of folder structure only
• Extension Filtering: Focus on specific file types (.mp4, .prproj, etc.)

Installation

No installation required - just Python 3.6+ with standard libraries.

Usage

Basic Usage

python3 check_nas_backup.py "/path/to/source" "/path/to/backup"

Quick Directory Overview (Recommended)

python3 check_nas_backup.py "/path/to/source" "/path/to/backup" --quick

Full Examples

# Directory-only comparison with progress info
python3 check_nas_backup.py "/Volumes/T9" "/Volumes/NAS/Backup" --dirs-only -v

# Incremental scan (only check changed directories)
python3 check_nas_backup.py "/Volumes/T9" "/Volumes/NAS/Backup" --incremental -v

# Focus on specific file types
python3 check_nas_backup.py "/Volumes/T9" "/Volumes/NAS/Backup" --extensions ".mp4,.mov,.prproj"

# Limit scan depth for faster overview
python3 check_nas_backup.py "/Volumes/T9" "/Volumes/NAS/Backup" --max-depth 2 --dirs-only

# Clear cache and do fresh scan
python3 check_nas_backup.py "/Volumes/T9" "/Volumes/NAS/Backup" --clear-cache

Command Line Options

Basic Options

Option	Description
source_dir	Path to source directory (SSD/local storage)
backup_dir	Path to backup directory (NAS/external)
-v, --verbose	Show detailed progress and scanning information

Performance Options

Option	Description	Default
--quick	Quick mode: depth-2, dirs-only, cached	Off
--parallel	Enable parallel processing	On
--cache	Use intelligent caching	On
--no-cache	Disable caching completely	Off
--incremental	Only scan changed directories	Off
--clear-cache	Clear existing cache before scan	Off

Scan Control

Option	Description	Default
--dirs-only	Compare directory structure only	Off
--max-depth N	Limit directory scan depth	Unlimited
--extensions LIST	Focus on specific file extensions	All files

Performance Guide

For Different Use Cases

📋 Quick Status Check (recommended for regular use):

python3 check_nas_backup.py "/Volumes/T9" "/Volumes/NAS/Backup" --quick -v

⚡ ~5-30 seconds for most directory structures

🔍 Detailed Project Check:

python3 check_nas_backup.py "/Volumes/T9/ProjectFolder" "/Volumes/NAS/ProjectFolder" --dirs-only -v

⚡ ~10-60 seconds depending on project size

📈 Full Deep Scan (for comprehensive analysis):

python3 check_nas_backup.py "/Volumes/T9" "/Volumes/NAS/Backup" --incremental -v

⚡ First run: several minutes, subsequent runs: ~30 seconds

🎯 Media File Focus:

python3 check_nas_backup.py "/Volumes/T9" "/Volumes/NAS/Backup" --extensions ".mp4,.mov,.avi,.mkv" -v

Cache Management

The tool automatically creates cache files in ~/.nas_backup_cache/:

• First scan: Takes longer, builds cache
• Subsequent scans: Much faster using cached data
• Incremental scans: Only check directories modified since last scan
• Clear cache: Use --clear-cache to reset and rebuild

Output Interpretation

Results Sections

🔴 Missing Backups: Critical - These files/directories exist on your SSD but are NOT backed up

Heizkraftwerk/FX6 Day 1/
Project_Files/Important_Document.pdf

🟡 Outdated Backups: Important - Backed up but backup is older than source

Festival_Video/Final_Edit/
Client_Project/Assets/

🔵 Extra Files on NAS: Informational - Files only on backup (archived/deleted from source)

Old_Project_2023/
Archived_Footage/

🟢 Matching Files: Good - Properly backed up and current

Current_Project/Source_Files/
Documentation/

Optimization Strategies

1. Caching System

• Stores directory scan results with timestamps
• Automatically skips unchanged directories on subsequent runs
• Reduces scan time by 70-90% for repeated operations

2. Parallel Processing

• Scans source and backup directories simultaneously
• Uses ThreadPoolExecutor for concurrent file system operations
• Reduces total scan time by ~40-50%

3. Incremental Mode

• Only rescans directories with modification time newer than last scan
• Ideal for regular backup status checks
• Can reduce scan time by 80-95% for large, stable directory structures

4. Depth Limiting

• --max-depth 1: Only top-level directories
• --max-depth 2: Include one level of subdirectories
• --max-depth 3: Good balance of detail vs. speed

5. Smart Defaults

• Parallel processing enabled by default
• Caching enabled by default
• Quick mode for fastest overview

Troubleshooting

Common Issues

Slow Performance on Network Drives:

# Use directory-only mode for faster network scans
python3 check_nas_backup.py "/local/path" "/network/path" --dirs-only --max-depth 3

Permission Errors:

• Ensure read access to both source and backup directories
• On macOS, may need to grant Terminal full disk access

Out of Memory on Large Directories:

# Use depth limiting and directory-only mode
python3 check_nas_backup.py "/path/source" "/path/backup" --max-depth 2 --dirs-only

Cache Issues:

# Clear cache and rebuild
python3 check_nas_backup.py "/path/source" "/path/backup" --clear-cache

Technical Details

Caching Implementation

• Cache files stored in ~/.nas_backup_cache/
• JSON format with directory timestamps
• MD5 hash-based cache file naming
• Automatic cache invalidation based on directory modification times

Parallel Processing

• ThreadPoolExecutor with dynamic worker count
• Concurrent source and backup scanning
• Thread-safe progress reporting

Incremental Logic

• Compares directory st_mtime with last scan timestamp
• Preserves unchanged directory data from cache
• Only rescans modified directory trees

Examples for Different Workflows

Daily Backup Check

# Create alias in ~/.zshrc or ~/.bashrc
alias backup-check="python3 ~/path/to/check_nas_backup.py '/Volumes/T9' '/Volumes/NAS/Data 2025' --quick -v"

# Then just run:
backup-check

Project-Specific Check

# Check specific project folder
python3 check_nas_backup.py "/Volumes/T9/Stanglwirt" "/Volumes/NAS/Backup/Stanglwirt" --dirs-only -v

Media Files Only

# Focus on video and image files
python3 check_nas_backup.py "/Volumes/T9" "/Volumes/NAS/Backup" \
  --extensions ".mp4,.mov,.avi,.mkv,.jpg,.png,.raw,.dng" -v

Archive Maintenance

# Find old files that can be removed from SSD
python3 check_nas_backup.py "/Volumes/T9" "/Volumes/NAS/Archive" --dirs-only -v
# Look at "Extra Files on NAS" section for archived content

Performance Benchmarks

Typical performance on modern systems:

Scenario	Directories	Time (Cold)	Time (Cached)	Time (Incremental)
Small project (~100 dirs)	100	2-5s	0.5-1s	0.2-0.5s
Medium project (~500 dirs)	500	10-30s	2-5s	1-2s
Large archive (~2000+ dirs)	2000+	2-10min	10-30s	5-15s

Benchmarks assume SSD source and network NAS backup

---

License

MIT License - Feel free to modify and distribute.

Contributing

This is a utility script - feel free to adapt for your specific needs!