Human-in-the-Loop (HITL) Confirmation

The HITL (Human-in-the-Loop) system provides user confirmation for high-risk tool operations before they execute. This adds a safety layer to prevent unintended destructive actions like deleting files, running shell commands, or modifying databases.

Overview

HITL confirmation allows you to:

• Require approval before executing high-risk tools
• Configure bypass rules to skip confirmation for trusted operations
• Filter by risk level (high-only, all, or off)
• Dynamically reload configuration without restarting

Configuration

HITL is configured by passing a hitl object (and an onToolConfirm callback) to Toolpack.init().

Basic Configuration

const sdk = await Toolpack.init({
  provider: 'openai',
  hitl: {
    enabled: true,
    confirmationMode: 'all',
    bypass: {
      tools: [],
      categories: [],
      levels: [],
    },
  },
  onToolConfirm: async (tool, args) => {
    // Return 'allow', 'deny', or 'cancel'
    const answer = await askUser(`Run ${tool.displayName}?`);
    return answer;
  },
});

Configuration Options

enabled (boolean)

• Default: false (auto-enabled when onToolConfirm is provided)
• Master switch for HITL confirmation
• When false, all tools bypass confirmation

hitl: { enabled: false }  // Disable all confirmations explicitly

confirmationMode (string)

• Default: "all"
• Options: "off" | "high-only" | "all"

Mode	Description
"off"	No confirmations (same as enabled: false)
"high-only"	Only confirm high-risk operations
"all"	Confirm both high and medium risk operations

hitl: { confirmationMode: 'high-only' }  // Only confirm high-risk tools

bypass (object)

Configure which tools, categories, or risk levels should skip confirmation.

Bypass by Tool Name:

hitl: {
  bypass: { tools: ['fs.write_file', 'fs.delete_file'] }
}

Bypass by Category:

hitl: {
  bypass: { categories: ['filesystem', 'execution'] }
}

Bypass by Risk Level:

hitl: {
  bypass: { levels: ['medium'] }  // Only bypass medium-risk tools
}

Tool Names vs Display Names

Important

Bypass rules use the tool name (e.g., "fs.write_file"), not the display name (e.g., "Write File").

Tool names use underscores and dots, while display names use spaces and are human-readable.

Finding Tool Names

Tool names follow the pattern: <category>.<action>

Common examples:

Tool Name	Display Name	Risk Level
fs.write_file	Write File	high
fs.delete_file	Delete File	high
fs.delete_dir	Delete Directory	high
exec.run	Run Command	high
exec.run_shell	Run Shell Command	high
db.insert	Insert Record	medium
db.update	Update Record	medium
db.delete	Delete Record	high
git.commit	Git Commit	medium

Risk Levels

Tools are classified into two risk levels:

High Risk ("high")

Operations that can cause permanent data loss or system changes:

• File deletion (fs.delete_file, fs.delete_dir)
• File overwrites (fs.write_file, fs.batch_write)
• File moves (fs.move)
• Command execution (exec.run, exec.run_shell, exec.run_background)
• Database deletion (db.delete)
• Code refactoring (coding.refactor_rename, coding.multi_file_edit)
• HTTP mutations (http.delete, http.post, http.put)

Medium Risk ("medium")

Operations that modify state but are generally reversible:

• Database inserts/updates (db.insert, db.update)
• Git commits (git.commit)
• File copies (fs.copy)

API Reference

addBypassRule()

Programmatically add a bypass rule to the configuration.

import { addBypassRule } from 'toolpack-sdk';

// Bypass a specific tool
await addBypassRule({
  type: 'tool',
  value: 'fs.write_file'
});

// Bypass an entire category
await addBypassRule({
  type: 'category',
  value: 'filesystem'
});

// Bypass a risk level
await addBypassRule({
  type: 'level',
  value: 'medium'
});

Parameters:

• type: 'tool' | 'category' | 'level'
• value: The tool name, category, or risk level to bypass

Returns: Promise<void>

Throws: SDKError if config write fails

removeBypassRule()

Remove a bypass rule from the configuration.

import { removeBypassRule } from 'toolpack-sdk';

// Remove a tool bypass
await removeBypassRule({
  type: 'tool',
  value: 'fs.write_file'
});

// Remove a category bypass
await removeBypassRule({
  type: 'category',
  value: 'filesystem'
});

Parameters: Same as addBypassRule()

Returns: Promise<void>

reloadConfig()

Reload the configuration to apply changes without restarting.

import { Toolpack } from 'toolpack-sdk';

const toolpack = await Toolpack.init({
  provider: 'openai',
  apiKey: process.env.OPENAI_API_KEY
});

// After modifying config
await addBypassRule({ type: 'tool', value: 'fs.write_file' });

// Reload to apply immediately
toolpack.reloadConfig();

Returns: void

Best Practices

1. Start Strict, Relax Gradually

Begin with all confirmations enabled and add bypass rules only for operations you trust:

hitl: { enabled: true, confirmationMode: 'all' }

2. Use Category Bypasses Carefully

Bypassing entire categories can skip many tools. Prefer specific tool bypasses:

hitl: {
  bypass: {
    tools: ['fs.write_file'],  // ✅ Specific
    // categories: ['filesystem'],  // ⚠️ Too broad
  }
}

3. Never Bypass High-Risk in Production

For production environments, avoid bypassing high-risk operations:

hitl: {
  confirmationMode: 'high-only',  // Always confirm high-risk
  bypass: { levels: ['medium'] }, // Only bypass medium-risk
}

4. Use high-only for Development

During development, you may want to skip medium-risk confirmations:

hitl: { confirmationMode: 'high-only' }

Examples

Example 1: Development Environment

Allow all filesystem operations but confirm commands:

const sdk = await Toolpack.init({
  provider: 'openai',
  hitl: {
    enabled: true,
    confirmationMode: 'all',
    bypass: { categories: ['filesystem'] },
  },
  onToolConfirm: myConfirmHandler,
});

Example 2: Production Environment

Confirm all high-risk operations, bypass medium-risk:

const sdk = await Toolpack.init({
  provider: 'openai',
  hitl: { enabled: true, confirmationMode: 'high-only' },
  onToolConfirm: myConfirmHandler,
});

Example 3: Trusted Automation

Bypass specific tools for automated workflows:

const sdk = await Toolpack.init({
  provider: 'openai',
  hitl: {
    enabled: true,
    bypass: { tools: ['fs.write_file', 'git.commit', 'db.insert'] },
  },
  onToolConfirm: myConfirmHandler,
});

Example 4: Programmatic Bypass at Runtime

import { Toolpack, addBypassRule } from 'toolpack-sdk';

const toolpack = await Toolpack.init({
  provider: 'openai',
  hitl: { enabled: true, confirmationMode: 'all' },
  onToolConfirm: async (tool, args) => {
    console.log(`Confirm ${tool.displayName}?`);
    const response = await getUserInput(); // Your custom prompt

    if (response === 'always') {
      await addBypassRule({ type: 'tool', value: tool.name });
      toolpack.reloadConfig();
      return 'allow';
    }

    return response; // 'allow', 'deny', or 'cancel'
  },
});

Troubleshooting

Bypass Rules Not Working

1. Check tool name format: Use tool.name (e.g., "fs.write_file"), not tool.displayName (e.g., "Write File")
2. Reload config: Call toolpack.reloadConfig() after modifying bypass rules programmatically
3. Verify enabled state: Ensure hitl.enabled is not explicitly set to false

All Tools Bypassing

Check if HITL is disabled:

hitl: { enabled: false }  // ❌ This disables all confirmations

Or if confirmation mode is off:

hitl: { confirmationMode: 'off' }  // ❌ This also disables all confirmations

Related

• Custom Tools - Learn how to add confirmation metadata to your custom tools
• Configuration Reference - Complete config file documentation