Advanced Filesystem Patterns with Toolpack SDK

Toolpack SDK's 18 filesystem tools are powerful on their own, but their real potential emerges when you combine them into patterns that solve real-world problems. This guide explores advanced techniques for building robust, efficient, and production-ready AI agents that work with files.

We'll cover atomic operations, intelligent file discovery, batch processing strategies, error handling patterns, and complete examples of production-grade filesystem workflows.

Why Patterns Matter

Basic file operations are straightforward—read a file, write a file, done. But production systems need more:

• Atomicity — Multiple file operations that succeed or fail as a unit
• Efficiency — Minimizing I/O and context window usage
• Safety — Preventing data loss and handling errors gracefully
• Intelligence — Smart file discovery and selective reading
• Scalability — Handling large codebases without overwhelming the LLM

Patterns give you these guarantees. Let's explore them.

Pattern 1: Atomic Multi-File Operations

When scaffolding projects or refactoring code, you often need to create or modify multiple files as a single logical operation. If any file fails, you want the entire operation to roll back.

The Problem

// ❌ Non-atomic: If step 3 fails, steps 1-2 are already written
await toolpack.generate({
  messages: [{
    role: 'user',
    content: `Create a new Express API with:
    1. src/index.ts (entry point)
    2. src/routes/api.ts (routes)
    3. src/middleware/auth.ts (auth middleware)
    4. package.json (dependencies)`
  }],
  model: 'gpt-4o',
});

If auth.ts fails to write (disk full, permissions issue), you're left with a partial project.

The Solution: Batch Write with Atomic Flag

import { Toolpack } from 'toolpack-sdk';

const toolpack = await Toolpack.init({
  provider: 'openai',
  tools: true,
  defaultMode: 'agent',
});

const response = await toolpack.generate({
  messages: [{
    role: 'user',
    content: `Create a new Express API project in ./my-api with the following structure:
    - src/index.ts (Express server setup)
    - src/routes/api.ts (API routes)
    - src/middleware/auth.ts (JWT authentication)
    - package.json (with express, typescript, @types/node, @types/express)
    - tsconfig.json (strict TypeScript config)
    
    Use fs.batch_write with atomic: true to ensure all files are created together or none at all.`
  }],
  model: 'gpt-4o',
});

How It Works

The AI will call fs.batch_write with this structure:

{
  files: [
    { path: './my-api/src/index.ts', content: '...' },
    { path: './my-api/src/routes/api.ts', content: '...' },
    { path: './my-api/src/middleware/auth.ts', content: '...' },
    { path: './my-api/package.json', content: '...' },
    { path: './my-api/tsconfig.json', content: '...' },
  ],
  atomic: true,
  createDirs: true,
}

Guarantees:

• All files are written to temporary locations first
• Only after all succeed are they moved to final destinations
• If any write fails, all temporary files are cleaned up
• Your filesystem stays in a consistent state

When to Use Atomic Writes

Use Case	Atomic?	Reason
Project scaffolding	✅ Yes	Partial projects are broken
Configuration updates	✅ Yes	Configs must be consistent
Code refactoring	✅ Yes	Partial refactors break builds
Log file writes	❌ No	Partial logs are acceptable
Documentation generation	⚠️ Maybe	Depends on interdependencies

Pattern 2: Smart File Discovery

Large codebases have thousands of files. Loading them all into context is wasteful. Smart discovery finds exactly what you need.

The Three-Phase Discovery Pattern

const toolpack = await Toolpack.init({
  provider: 'anthropic',
  tools: true,
});

const response = await toolpack.generate({
  messages: [{
    role: 'user',
    content: `Find and fix all TypeScript files that use deprecated React lifecycle methods.
    
    Phase 1: Use fs.tree to understand the project structure
    Phase 2: Use fs.glob to find all .tsx and .ts files in src/
    Phase 3: Use fs.search to find files containing 'componentWillMount' or 'componentWillReceiveProps'
    Phase 4: Read only the matching files and suggest fixes`
  }],
  model: 'claude-sonnet-4',
});

Why This Works

Phase 1: Structure Understanding

• fs.tree gives a 10,000-foot view
• AI learns where source files live vs. build artifacts
• Prevents searching in node_modules or dist/

Phase 2: Pattern-Based Filtering

• fs.glob narrows to relevant file types
• Glob patterns are fast—no file content is read yet
• Example: src/**/*.{ts,tsx} finds all TypeScript files

Phase 3: Content-Based Filtering

• fs.search scans file contents for specific text
• Only matching files are identified
• Much faster than reading every file

Phase 4: Targeted Reading

• Only files that match both pattern and content are read
• Minimal context window usage
• Maximum relevance

Discovery Pattern Template

// Template for efficient file discovery
const discoveryPrompt = `
1. Use fs.tree with depth: 2 to understand the project layout
2. Use fs.glob with pattern: "{pattern}" to find candidate files
3. Use fs.search with query: "{search_term}" to filter by content
4. Use fs.batch_read to read only the matching files
5. Perform the requested operation on those files
`;

Pattern 3: Selective Line Range Reading

Reading entire files wastes tokens. For large files, read only what you need.

The Problem

// ❌ Wasteful: Loads 5,000 lines when you need 50
await toolpack.generate({
  messages: [{
    role: 'user',
    content: 'Explain the authentication logic in src/server.ts'
  }],
});

If server.ts is 5,000 lines, you're burning tokens on unrelated code.

The Solution: Two-Pass Reading

const response = await toolpack.generate({
  messages: [{
    role: 'user',
    content: `Analyze the authentication logic in src/server.ts:
    
    Pass 1: Use fs.search to find which lines contain "auth" or "authenticate"
    Pass 2: Use fs.read_file_range to read only those line ranges (±10 lines for context)
    Pass 3: Explain the authentication flow based on those sections`
  }],
  model: 'gpt-4o',
});

How It Works

Pass 1: Locate Relevant Sections

// AI calls fs.search on the src/ directory
{
  path: 'src/',
  query: 'auth|authenticate|jwt|token',
  recursive: false
}
// Returns: src/server.ts lines 45, 67, 123, 456 contain matches

Pass 2: Read Only Those Ranges

// AI calls fs.read_file_range multiple times
[
  { path: 'src/server.ts', start_line: 35, end_line: 55 },   // Around line 45
  { path: 'src/server.ts', start_line: 57, end_line: 77 },   // Around line 67
  { path: 'src/server.ts', start_line: 113, end_line: 133 }, // Around line 123
  { path: 'src/server.ts', start_line: 446, end_line: 466 }, // Around line 456
]

Result: Read ~80 lines instead of 5,000 lines—98% reduction in tokens.

When to Use Line Range Reading

File Size	Strategy	Tool
< 200 lines	Read entire file	fs.read_file
200-1000 lines	Read if relevant	fs.search first, then decide
> 1000 lines	Always use ranges	fs.search + fs.read_file_range

Pattern 4: Batch Operations for Efficiency

When working with multiple files, batch operations reduce tool calls and improve performance.

Single vs. Batch Operations

// ❌ Inefficient: 5 separate tool calls
const response = await toolpack.generate({
  messages: [{
    role: 'user',
    content: 'Read package.json, tsconfig.json, .eslintrc.json, .prettierrc, and README.md'
  }],
});
// AI makes 5 separate fs.read_file calls

// ✅ Efficient: 1 batch tool call
const response = await toolpack.generate({
  messages: [{
    role: 'user',
    content: `Read all configuration files using fs.batch_read:
    - package.json
    - tsconfig.json
    - .eslintrc.json
    - .prettierrc
    - README.md`
  }],
});
// AI makes 1 fs.batch_read call with all 5 paths

Batch Read with Error Handling

const response = await toolpack.generate({
  messages: [{
    role: 'user',
    content: `Read all config files in the project root using fs.batch_read with continueOnError: true.
    
    If any file is missing, note it but continue reading the others.
    Then summarize which configs are present and which are missing.`
  }],
  model: 'gpt-4o',
});

The AI calls:

{
  paths: [
    'package.json',
    'tsconfig.json',
    '.eslintrc.json',
    '.prettierrc',
    'jest.config.js',
  ],
  continueOnError: true
}

Result: Even if .prettierrc doesn't exist, the other 4 files are read successfully.

Batch Write Patterns

// Pattern: Generate multiple related files
const response = await toolpack.generate({
  messages: [{
    role: 'user',
    content: `Create a complete React component with:
    - components/UserProfile/index.tsx (component)
    - components/UserProfile/UserProfile.test.tsx (tests)
    - components/UserProfile/UserProfile.module.css (styles)
    - components/UserProfile/types.ts (TypeScript types)
    
    Use fs.batch_write with atomic: true and createDirs: true`
  }],
});

Pattern 5: Safe Destructive Operations

Deleting or moving files is risky. Use verification patterns to prevent accidents.

The Verify-Then-Act Pattern

const response = await toolpack.generate({
  messages: [{
    role: 'user',
    content: `Clean up old log files in ./logs directory:
    
    Step 1: Use fs.list_dir to see all files in ./logs
    Step 2: Use fs.stat on each file to check modification date
    Step 3: Show me the list of files older than 30 days
    Step 4: Wait for my confirmation before deleting anything`
  }],
  model: 'gpt-4o',
});

// After reviewing the list, confirm:
const confirmResponse = await toolpack.generate({
  messages: [
    ...previousMessages,
    { role: 'user', content: 'Confirmed. Delete those files.' }
  ],
});

Safe Move Pattern

const response = await toolpack.generate({
  messages: [{
    role: 'user',
    content: `Reorganize the src/ directory:
    
    Step 1: Use fs.tree to show current structure
    Step 2: Propose a new structure with file moves
    Step 3: For each move, use fs.exists to verify source exists and destination doesn't
    Step 4: Use fs.copy (not fs.move) to copy files to new locations first
    Step 5: Verify all copies succeeded
    Step 6: Only then delete the originals`
  }],
});

Why copy-then-delete is safer than move:

• If copy fails, original is intact
• You can verify copies before deleting
• Rollback is possible

Pattern 6: Incremental File Updates

When modifying existing files, read-modify-write patterns prevent data loss.

The Read-Modify-Write Pattern

const response = await toolpack.generate({
  messages: [{
    role: 'user',
    content: `Add a new script to package.json:
    
    Step 1: Use fs.read_file to read current package.json
    Step 2: Parse the JSON and add the new script: "test:watch": "jest --watch"
    Step 3: Use fs.write_file to write the updated content
    
    Preserve all existing formatting and field order.`
  }],
  model: 'gpt-4o',
});

In-Place Text Replacement

const response = await toolpack.generate({
  messages: [{
    role: 'user',
    content: `Update all import statements in src/ to use absolute paths:
    
    Step 1: Use fs.glob to find all .ts and .tsx files in src/
    Step 2: For each file, use fs.replace_in_file to replace:
       - "import { X } from '../../../components/X'" 
       - with "import { X } from '@/components/X'"
    
    Use fs.replace_in_file for in-place updates without reading entire files.`
  }],
});

Why fs.replace_in_file is better than read-modify-write for text replacement:

• More efficient—no need to load entire file into memory
• Preserves file encoding and line endings
• Atomic operation at the filesystem level
• Handles large files gracefully

Pattern 7: Mode-Specific Filesystem Configurations

Different tasks need different filesystem tool access. Use modes to enforce boundaries.

Read-Only Analysis Mode

import { createMode, Toolpack } from 'toolpack-sdk';

const analysisMode = createMode({
  name: 'analysis',
  displayName: 'Code Analysis',
  description: 'Read-only mode for analyzing codebases',
  systemPrompt: 'You are a code analyzer. You can read and search files but never modify them.',
  
  allowedToolCategories: ['filesystem'],
  blockedTools: [
    'fs.write_file',
    'fs.append_file',
    'fs.delete_file',
    'fs.delete_dir',
    'fs.move',
    'fs.copy',
    'fs.replace_in_file',
    'fs.batch_write',
  ],
  
  toolSearch: {
    alwaysLoadedTools: [
      'fs.read_file',
      'fs.read_file_range',
      'fs.search',
      'fs.glob',
      'fs.tree',
      'fs.list_dir',
    ],
  },
});

const toolpack = await Toolpack.init({
  provider: 'openai',
  tools: true,
  customModes: [analysisMode],
  defaultMode: 'analysis',
});

Guarantees:

• AI cannot accidentally modify files
• Only read operations are available
• Tool search is optimized for read-heavy workflows

Scaffolding Mode

const scaffoldMode = createMode({
  name: 'scaffold',
  displayName: 'Project Scaffolder',
  description: 'Creates new projects and file structures',
  systemPrompt: 'You are a project scaffolder. Create well-structured projects with best practices.',
  
  allowedToolCategories: ['filesystem'],
  blockedTools: [
    'fs.delete_file',
    'fs.delete_dir',
  ],
  
  toolSearch: {
    alwaysLoadedTools: [
      'fs.batch_write',
      'fs.create_dir',
      'fs.exists',
      'fs.tree',
    ],
  },
  
  workflow: {
    planning: {
      enabled: true,
      maxSteps: 8,
      planningPrompt: `Plan the project structure:
      1. Determine directory layout
      2. List all files to create
      3. Use fs.batch_write with atomic: true for all files
      4. Verify creation with fs.tree`,
    },
    steps: {
      enabled: true,
      retryOnFailure: true,
      maxRetries: 2,
    },
  },
});

Optimizations:

• Blocks destructive operations (delete)
• Pre-loads batch write tools
• Workflow ensures atomic scaffolding

Refactoring Mode

const refactorMode = createMode({
  name: 'refactor',
  displayName: 'Code Refactorer',
  description: 'Safely refactors code with verification',
  systemPrompt: 'You are a code refactorer. Always verify before modifying files.',
  
  allowedToolCategories: ['filesystem', 'coding'],
  
  toolSearch: {
    alwaysLoadedTools: [
      'fs.read_file',
      'fs.write_file',
      'fs.search',
      'fs.replace_in_file',
      'fs.batch_read',
      'fs.batch_write',
    ],
  },
  
  workflow: {
    planning: {
      enabled: true,
      requireApproval: true,  // Always require approval for refactoring
      planningPrompt: `Plan the refactoring:
      1. Identify all files that need changes
      2. For each file, specify exact changes
      3. Use fs.batch_write with atomic: true
      4. Verify changes don't break imports`,
    },
    steps: {
      enabled: true,
      allowDynamicSteps: false,  // Stick to approved plan
    },
    onFailure: {
      strategy: 'abort',  // Don't continue if any step fails
    },
  },
});

Pattern 8: Error Handling and Recovery

Production systems need robust error handling. Here are patterns for common failure scenarios.

Graceful Degradation

const response = await toolpack.generate({
  messages: [{
    role: 'user',
    content: `Generate a project report by reading:
    1. README.md (required)
    2. CHANGELOG.md (optional)
    3. package.json (required)
    4. LICENSE (optional)
    
    Use fs.batch_read with continueOnError: true.
    For missing optional files, note their absence in the report.
    For missing required files, report an error and stop.`
  }],
  model: 'gpt-4o',
});

Retry with Backoff

const response = await toolpack.generate({
  messages: [{
    role: 'user',
    content: `Write a large log file to disk:
    
    Step 1: Try fs.write_file
    Step 2: If it fails with "disk full" or "no space", wait 5 seconds and retry
    Step 3: If it fails again, try writing to an alternate location: /tmp/logs/
    Step 4: If that fails, report the error and suggest manual intervention`
  }],
});

Validation Before Write

const response = await toolpack.generate({
  messages: [{
    role: 'user',
    content: `Update tsconfig.json with new compiler options:
    
    Step 1: Use fs.read_file to read current tsconfig.json
    Step 2: Parse JSON to validate it's well-formed
    Step 3: Merge new options
    Step 4: Validate the merged JSON is still valid
    Step 5: Only if valid, use fs.write_file to save
    Step 6: If validation fails, report the error without writing`
  }],
});

Complete Example: Production-Grade Code Migrator

Let's combine all patterns into a complete, production-ready example.

Scenario: Migrate from CommonJS to ES Modules

import { createMode, Toolpack } from 'toolpack-sdk';

// 1. Create a specialized migration mode
const migrationMode = createMode({
  name: 'esm-migration',
  displayName: 'ESM Migration',
  description: 'Migrates CommonJS projects to ES modules',
  systemPrompt: `You are an ES module migration specialist.
  
  Always follow this workflow:
  1. Analyze project structure
  2. Identify all files needing changes
  3. Create a migration plan
  4. Get user approval
  5. Execute atomically
  6. Verify the migration`,
  
  allowedToolCategories: ['filesystem', 'coding'],
  
  workflow: {
    planning: {
      enabled: true,
      requireApproval: true,
      maxSteps: 12,
      planningPrompt: `Create a detailed migration plan:
      
      Phase 1: Discovery
      - Use fs.tree to understand project structure
      - Use fs.glob to find all .js files
      - Use fs.search to identify CommonJS patterns (require, module.exports)
      
      Phase 2: Analysis
      - Use fs.batch_read to read all affected files
      - Identify dependencies between files
      - Determine migration order
      
      Phase 3: Migration
      - Convert require() to import
      - Convert module.exports to export
      - Update package.json with "type": "module"
      - Use fs.batch_write with atomic: true
      
      Phase 4: Verification
      - Use fs.tree to confirm structure
      - Report summary of changes`,
    },
    steps: {
      enabled: true,
      retryOnFailure: true,
      maxRetries: 1,
      allowDynamicSteps: false,
    },
    onFailure: {
      strategy: 'abort',
    },
  },
});

// 2. Initialize with the migration mode
const toolpack = await Toolpack.init({
  provider: 'anthropic',
  tools: true,
  customModes: [migrationMode],
  defaultMode: 'esm-migration',
});

// 3. Execute the migration
const executor = toolpack.getWorkflowExecutor();

// Listen to workflow events for progress tracking
executor.on('workflow:plan_created', (plan) => {
  console.log('\n📋 Migration Plan Created:');
  plan.steps.forEach((step, i) => {
    console.log(`  ${i + 1}. ${step.description}`);
  });
});

executor.on('workflow:progress', async (progress) => {
  if (progress.status === 'awaiting_approval') {
    console.log('\n⏸️  Waiting for approval...');
    console.log(`This will modify ${progress.totalSteps} files.`);
    
    // In a real app, you'd prompt the user here
    const approved = true; // await getUserApproval();
    
    if (approved) {
      executor.approvePlan(progress.planId);
      console.log('✅ Migration approved. Executing...\n');
    } else {
      executor.rejectPlan(progress.planId);
      console.log('❌ Migration cancelled.\n');
    }
  } else {
    console.log(`[${progress.percentage}%] ${progress.currentStepDescription}`);
  }
});

executor.on('workflow:completed', (plan, result) => {
  console.log('\n✅ Migration completed successfully!');
  console.log(`   Files modified: ${result.metrics.stepsCompleted}`);
  console.log(`   Duration: ${result.metrics.totalDuration}ms`);
});

executor.on('workflow:failed', (plan, error) => {
  console.error('\n❌ Migration failed:', error.message);
  console.error('   No files were modified (atomic rollback)');
});

// 4. Start the migration
const response = await toolpack.generate({
  messages: [{
    role: 'user',
    content: `Migrate this CommonJS project to ES modules:
    
    Requirements:
    - Convert all require() to import statements
    - Convert all module.exports to export statements
    - Update package.json with "type": "module"
    - Preserve all comments and formatting
    - Use atomic batch write to ensure all-or-nothing migration
    
    Project root: ./my-project`
  }],
  model: 'claude-sonnet-4',
});

console.log(response.content);

What Makes This Production-Ready?

1. Specialized Mode — Tool access and workflow tailored for migration
2. Approval Gate — Human reviews plan before execution
3. Atomic Operations — All files migrate together or none at all
4. Progress Tracking — Real-time feedback via workflow events
5. Error Recovery — Automatic rollback on failure
6. Verification — Post-migration checks confirm success
7. Structured Workflow — Fixed plan prevents unexpected changes

Best Practices Summary

1. Always Use Atomic Writes for Related Files

// ✅ Good: Atomic scaffolding
content: 'Create a React component with index.tsx, styles.css, and test file using fs.batch_write with atomic: true'

// ❌ Bad: Non-atomic
content: 'Create index.tsx, then styles.css, then test file'

2. Discover Before Reading

// ✅ Good: Smart discovery
content: 'Use fs.tree, then fs.glob, then fs.search to find relevant files, then read only those'

// ❌ Bad: Read everything
content: 'Read all files in src/'

3. Use Line Ranges for Large Files

// ✅ Good: Targeted reading
content: 'Use fs.search to find authentication code, then fs.read_file_range to read those sections'

// ❌ Bad: Full file read
content: 'Read the entire 5000-line server.ts file'

4. Batch When Possible

// ✅ Good: Single batch operation
content: 'Use fs.batch_read to read all config files at once'

// ❌ Bad: Multiple individual reads
content: 'Read package.json, then tsconfig.json, then .eslintrc'

5. Verify Before Destructive Operations

// ✅ Good: Verify first
content: 'List files to delete, show me the list, wait for confirmation, then delete'

// ❌ Bad: Delete immediately
content: 'Delete all log files older than 30 days'

6. Use Modes to Enforce Boundaries

// ✅ Good: Read-only mode for analysis
const mode = createMode({
  blockedTools: ['fs.write_file', 'fs.delete_file'],
});

// ❌ Bad: Full access when you only need to read
const mode = createMode({
  allowedToolCategories: ['filesystem'],
});

7. Handle Errors Gracefully

// ✅ Good: Graceful degradation
content: 'Use fs.batch_read with continueOnError: true, note which files are missing'

// ❌ Bad: Fail on first error
content: 'Read these 10 files'

Performance Optimization Guide

Token Usage Optimization

Operation	Token Cost	Optimization
Read 1000-line file	~1500 tokens	Use fs.read_file_range
Read 10 small files	~200 tokens	Use fs.batch_read
Search 100 files	~50 tokens	Use fs.search before reading
List directory	~10 tokens	Use depth limit with fs.tree

Tool Call Optimization

Pattern	Tool Calls	Optimization
Read 5 files individually	5 calls	Use fs.batch_read → 1 call
Write 10 files individually	10 calls	Use fs.batch_write → 1 call
Search then read 3 files	4 calls	Use fs.batch_read after search → 2 calls

Context Window Management

// For large codebases, use progressive disclosure
const response = await toolpack.generate({
  messages: [{
    role: 'user',
    content: `Analyze this codebase efficiently:
    
    Level 1: Use fs.tree with depth: 2 (overview)
    Level 2: Use fs.glob to count files by type
    Level 3: Use fs.search to find files matching criteria
    Level 4: Use fs.read_file_range to read only relevant sections
    
    Never read more than 5000 tokens of file content at once.`
  }],
});

Key Takeaways

• Atomic operations — Use fs.batch_write with atomic: true for related files
• Smart discovery — Combine fs.tree, fs.glob, and fs.search before reading
• Selective reading — Use fs.read_file_range for large files
• Batch processing — Use fs.batch_read and fs.batch_write for efficiency
• Safe destructive ops — Always verify before delete/move operations
• Mode-based access — Use custom modes to enforce filesystem boundaries
• Error handling — Use continueOnError and approval gates for robustness
• Progressive disclosure — Read incrementally to manage context window

Advanced filesystem patterns transform basic file operations into production-grade workflows. Whether you're building a code migrator, project scaffolder, or analysis tool, these patterns give you the safety, efficiency, and intelligence you need for real-world applications.