mirrors/claude-code-sdk-python
1
0
Fork 0
mirror of https://github.com/anthropics/claude-code-sdk-python.git synced 2025-12-23 09:19:52 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
claude-code-sdk-python/e2e-tests
History
Dickson Tsai 62289d2dce
feat: add dynamic permission mode and model switching to ClaudeSDKClient (#171) 
## Summary
- Adds `set_permission_mode()` method to dynamically change permission
modes during streaming sessions
- Adds `set_model()` method to switch AI models mid-conversation
- Implements control protocol support matching the TypeScript SDK's
capabilities

## Motivation
The TypeScript SDK supports dynamic control through
`setPermissionMode()` and `setModel()` methods on the Query interface.
This PR brings the same functionality to the Python SDK, allowing users
to:

1. Start with restrictive permissions for code review, then switch to
`acceptEdits` for implementation
2. Use different models for different parts of a task (e.g., Sonnet for
complex analysis, Haiku for simple tasks)
3. Adjust permissions based on workflow needs without restarting
sessions

## Changes
- **ClaudeSDKClient**: Added `set_permission_mode(mode)` and
`set_model(model)` methods
- **Internal Query class**: Added `set_model(model)` method to send
control requests
- **E2E tests**: Added comprehensive tests verifying the functionality
works with real API calls

## Test Plan
- [x] All existing unit tests pass (102 tests)
- [x] New E2E tests added and passing:
- `test_set_permission_mode`: Verifies permission mode changes take
effect
  - `test_set_model`: Confirms model switching works mid-conversation
  - `test_interrupt`: Validates interrupt capability
- [x] Type checking passes (`mypy`)
- [x] Linting passes (`ruff`)

## Usage Example
```python
async with ClaudeSDKClient() as client:
    # Start with default permissions for review
    await client.query("Analyze this code for issues")
    
    # Switch to auto-accept edits for implementation
    await client.set_permission_mode('acceptEdits')
    await client.query("Now fix the issues we found")
    
    # Use a different model for simpler tasks
    await client.set_model('claude-3-5-haiku-20241022')
    await client.query("Add a simple docstring")
```

🤖 Generated with [Claude Code](https://claude.ai/code)

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Ashwin Bhat <ashwin@anthropic.com>
2025-09-28 14:09:29 -07:00
..
conftest.py Add custom tool callbacks and e2e tests (#157) 2025-09-08 08:51:40 -07:00
README.md Add custom tool callbacks and e2e tests (#157) 2025-09-08 08:51:40 -07:00
test_agents_and_settings.py Rename ClaudeCodeOptions to ClaudeAgentOptions (#185) 2025-09-26 21:38:05 -07:00
test_dynamic_control.py feat: add dynamic permission mode and model switching to ClaudeSDKClient (#171) 2025-09-28 14:09:29 -07:00
test_include_partial_messages.py Rename ClaudeCodeOptions to ClaudeAgentOptions (#185) 2025-09-26 21:38:05 -07:00
test_sdk_mcp_tools.py Rename ClaudeCodeOptions to ClaudeAgentOptions (#185) 2025-09-26 21:38:05 -07:00
test_tool_permissions.py Rename ClaudeCodeOptions to ClaudeAgentOptions (#185) 2025-09-26 21:38:05 -07:00

README.md

End-to-End Tests for Claude Code SDK

This directory contains end-to-end tests that run against the actual Claude API to verify real-world functionality.

Requirements

API Key (REQUIRED)

These tests require a valid Anthropic API key. The tests will fail if ANTHROPIC_API_KEY is not set.

Set your API key before running tests:

export ANTHROPIC_API_KEY="your-api-key-here"

Dependencies

Install the development dependencies:

pip install -e ".[dev]"

Running the Tests

Run all e2e tests:

python -m pytest e2e-tests/ -v

Run with e2e marker only:

python -m pytest e2e-tests/ -v -m e2e

Run a specific test:

python -m pytest e2e-tests/test_mcp_calculator.py::test_basic_addition -v

Cost Considerations

⚠️ Important: These tests make actual API calls to Claude, which incur costs based on your Anthropic pricing plan.

  • Each test typically uses 1-3 API calls
  • Tests use simple prompts to minimize token usage
  • The complete test suite should cost less than $0.10 to run

Test Coverage

MCP Calculator Tests (test_mcp_calculator.py)

Tests the MCP (Model Context Protocol) integration with calculator tools:

  • test_basic_addition: Verifies the add tool executes correctly
  • test_division: Tests division with decimal results
  • test_square_root: Validates square root calculations
  • test_power: Tests exponentiation
  • test_multi_step_calculation: Verifies multiple tools can be used in sequence
  • test_tool_permissions_enforced: Ensures permission system works correctly

Each test validates:

  1. Tools are actually called (ToolUseBlock present in response)
  2. Correct tool inputs are provided
  3. Expected results are returned
  4. Permission system is enforced

CI/CD Integration

These tests run automatically on:

  • Pushes to main branch (via GitHub Actions)
  • Manual workflow dispatch

The workflow uses ANTHROPIC_API_KEY from GitHub Secrets.

Troubleshooting

"ANTHROPIC_API_KEY environment variable is required" error

  • Set your API key: export ANTHROPIC_API_KEY=sk-ant-...
  • The tests will not skip - they require the key to run

Tests timing out

  • Check your API key is valid and has quota available
  • Ensure network connectivity to api.anthropic.com

Permission denied errors

  • Verify the allowed_tools parameter includes the necessary MCP tools
  • Check that tool names match the expected format (e.g., mcp__calc__add)

Adding New E2E Tests

When adding new e2e tests:

  1. Mark tests with @pytest.mark.e2e decorator
  2. Use the api_key fixture to ensure API key is available
  3. Keep prompts simple to minimize costs
  4. Verify actual tool execution, not just mocked responses
  5. Document any special setup requirements in this README