feat: add suspend/resume support for RPA invocations in evaluations

[Chibionos]

Summary

Adds support for suspend/resume pattern in evaluation runtime, enabling agents to suspend execution when invoking RPA processes and resume after completion.

Key Changes

Suspend Detection (_runtime.py)

• Runtime detects UiPathRuntimeStatus.SUSPENDED after agent execution
• Extracts trigger information from suspended executions
• Skips evaluators during suspension (evaluators only run on SUCCESSFUL completion)
• Returns triggers in partial results for resume context

Resume Support (cli_eval.py)

• Added --resume flag to evaluation CLI
• Loads checkpoint state and continues from interruption point
• Passes resume context to runtime

Trigger Pass-Through

• Suspended executions return triggers with structure:
  • interruptId: Unique interrupt identifier
  • triggerType: Type of trigger (e.g., "Api")
  • payload: InvokeProcess details (process name, input arguments, folder path)

Comprehensive Logging

• Suspension detection events (🔴 DETECTED SUSPENSION)
• Trigger extraction details (📋 Extracted N trigger(s))
• Evaluator skip reasons (⏭️ Skipping evaluators)
• Trigger pass-through confirmation (✅ Passing through triggers)
• Resume mode indication (🟢 RESUMED execution)

Testing

Testing is done via the sample agent in the companion PR: UiPath/uipath-langchain-python#397

Related

Companion PR for sample agent: UiPath/uipath-langchain-python#397

[Chibionos]

Quick Testing Guide

Install the Dev Version

pip install git+https://github.com/UiPath/uipath-python@v2.4.9-dev.suspend-resume

Run Quick Test

cd samples/event-trigger

# Test 1: Direct runtime test
python -c "
import asyncio
from uipath.functions.runtime import UiPathFunctionsRuntime

async def test():
    runtime = UiPathFunctionsRuntime(
        'test_suspend_resume_agent.py',
        'main', 
        'test_suspend_resume_agent'
    )
    result = await runtime.execute({'query': 'Test'})
    print(f'✓ Status: {result.status}')
    print(f'✓ Has Trigger: {result.trigger is not None}')
    if result.trigger:
        print(f'✓ Process: {result.trigger.payload[\"process_name\"]}')

asyncio.run(test())
"

# Test 2: Evaluation test
uipath eval test_suspend_resume_agent evaluations/test_suspend_resume.json --output-file test_output.json
cat test_output.json | python -m json.tool | grep -A 5 "status"

Expected Results

✅ Status: SUSPENDED
✅ Trigger exists with process name and arguments
✅ Evaluation output shows suspended state in JSON

Key Files Changed

• Functions Runtime: src/uipath/functions/runtime.py (+70 lines)

  • New _detect_langgraph_interrupt() method
  • Trigger extraction from __interrupt__ field

• Evaluation Runtime: src/uipath/_cli/_evals/_runtime.py (+58 lines)

  • SUSPENDED status detection
  • Skip evaluators when suspended
  • Publish suspended event

• Test Agent: samples/event-trigger/test_suspend_resume_agent.py (new file)

  • Demonstrates correct interrupt pattern
  • Uses checkpointer and returns raw dict

---

Tag: v2.4.9-dev.suspend-resume
Branch: investigate-rpa-samples

[mjnovice]

Just to understand, so the suspend and resume is actually being done by the agent and not the orchestrator, right ?