Use Deferred Tools

Use this when your agent has many tools and you want to reduce context window usage by hiding tool schemas from the LLM until they are needed. The deferred-tools plugin classifies tools as Eager (always sent) or Deferred (hidden until requested). A ToolSearch tool lets the LLM discover deferred tools on demand.

Prerequisites

• A working awaken agent runtime (see First Agent)
• The awaken-ext-deferred-tools crate

[dependencies]
awaken-ext-deferred-tools = { version = "0.1" }
awaken = { package = "awaken-agent", version = "0.1" }
tokio = { version = "1", features = ["full"] }
serde_json = "1"

Steps

1. Create the plugin and register it.

Collect all tool descriptors your agent exposes, then pass them to DeferredToolsPlugin::new. The plugin uses these to classify tools and populate the deferred registry at activation time.

use std::sync::Arc;
use awaken::engine::GenaiExecutor;
use awaken::registry_spec::{AgentSpec, ModelSpec};
use awaken::{AgentRuntimeBuilder, Plugin};
use awaken_ext_deferred_tools::DeferredToolsPlugin;

// Collect descriptors from all tools registered on the agent.
let seed_tools = vec![
    weather_tool.descriptor(),
    search_tool.descriptor(),
    debug_tool.descriptor(),
    // ... all tool descriptors
];
let agent_spec = AgentSpec::new("deferred-agent")
    .with_model("gpt-4o-mini")
    .with_system_prompt("Search for tools only when needed.")
    .with_hook_filter("ext-deferred-tools");

let runtime = AgentRuntimeBuilder::new()
    .with_provider("openai", Arc::new(GenaiExecutor::new()))
    .with_model(
        "gpt-4o-mini",
        ModelSpec {
            id: "gpt-4o-mini".into(),
            provider: "openai".into(),
            model: "gpt-4o-mini".into(),
        },
    )
    .with_agent_spec(agent_spec)
    .with_plugin(
        "ext-deferred-tools",
        Arc::new(DeferredToolsPlugin::new(seed_tools)) as Arc<dyn Plugin>,
    )
    .build()
    .expect("failed to build runtime");

2. Configure tool loading rules.

Set rules on the agent spec via the deferred_tools config key. Rules are evaluated in order and first match wins. Tools that match no rule fall back to default_mode.

use awaken_ext_deferred_tools::{
    DeferredToolsConfig, DeferralRule, ToolLoadMode,
};

let config = DeferredToolsConfig {
    rules: vec![
        DeferralRule { tool: "get_weather".into(), mode: ToolLoadMode::Eager },
        DeferralRule { tool: "debug_*".into(), mode: ToolLoadMode::Deferred },
    ],
    default_mode: ToolLoadMode::Deferred,
    ..Default::default()
};

The tool field supports exact names and glob patterns (via wildcard_match). Common patterns:

3. Understand auto-enable.

The enabled field on DeferredToolsConfig controls activation:

With auto-enable, the plugin estimates per-tool schema cost as len(parameters_json) / 4 tokens and sums the savings for all deferrable tools. If the total exceeds the overhead of maintaining the ToolSearch tool and deferred tool list in context, deferral activates automatically.

4. Understand how ToolSearch works.

The plugin automatically registers a ToolSearch tool. The LLM calls it with a query string to find deferred tools:

When ToolSearch returns results, matched tools are promoted to Eager for the remainder of the session. The tool returns up to max_results (default 5) matching tool schemas in a <functions> block.

LLM: I need to check the weather. Let me search for relevant tools.
     -> calls ToolSearch { query: "weather forecast" }

ToolSearch returns matching schemas, promotes get_weather to Eager.
Next inference: get_weather schema is included in the tool list.

5. Configure automatic re-deferral (DiscBeta).

The plugin tracks per-tool usage statistics using a discounted Beta distribution. Tools that were promoted via ToolSearch but stop being used get automatically re-deferred. This is adaptive and requires no manual tuning.

Re-deferral triggers when all of the following are true:

• The tool is currently Eager (was promoted from Deferred)
• The tool is not configured as always-Eager in rules
• The tool has been idle for at least defer_after turns
• The posterior upper credible interval (90%) falls below breakeven_p * thresh_mult

The breakeven frequency is (c - c_bar) / gamma, where c is the full schema cost and c_bar is the name-only cost. A tool is worth keeping eager only if it is used often enough that the savings from avoiding ToolSearch calls exceed the per-turn overhead.

Key parameters in DiscBetaParams:

These live under DeferredToolsConfig.disc_beta:

use awaken_ext_deferred_tools::{DeferredToolsConfig, DiscBetaParams};

let config = DeferredToolsConfig {
    disc_beta: DiscBetaParams {
        omega: 0.95,
        n0: 5.0,
        defer_after: 5,
        thresh_mult: 0.5,
        gamma: 2000.0,
    },
    beta_overhead: 1136.0,
    ..Default::default()
};

6. Enable cross-session learning.

Via AgentToolPriors (a ProfileKey), usage frequencies persist across sessions using EWMA (exponentially weighted moving average). At session end, the PersistPriorsHook writes per-tool presence frequencies to the profile store. At next session start, the LoadPriorsHook reads them back and initializes the Beta distribution with learned priors instead of the default 0.01.

This requires a ProfileStore to be configured on the runtime. No additional code is needed beyond the plugin registration — the hooks are wired automatically.

The EWMA smoothing factor is lambda = max(0.1, 1/(n+1)), where n is the session count. Early sessions contribute equally; after 10 sessions the factor stabilizes at 0.1, giving 90% weight to historical data.

Verify

1. Run the agent and trigger an inference. Check logs for the deferred_tools.list context message, which lists all deferred tool names.

2. Read DeferralState from the runtime snapshot to see the current mode of each tool:

use awaken_ext_deferred_tools::state::{DeferralState, DeferralStateValue};

let state: &DeferralStateValue = snapshot.state::<DeferralState>()
    .expect("DeferralState not found");

for (tool_id, mode) in &state.modes {
    println!("{tool_id}: {mode:?}");
}

3. Ask the LLM a question that requires a deferred tool. Confirm ToolSearch is called and the tool is promoted to Eager in subsequent turns.

4. After several turns of inactivity, verify re-deferral by checking that the tool reverts to Deferred mode in the snapshot.

Common Errors

Key Files

Related

• Add a Plugin
• Add a Tool
• Optimize Context Window