AgentToolHub
Skill

frontend-design

frontend-design — a Claude Code skill from vercel-labs/agent-eval

Features

  • ✓Evaluate AI coding agents on your framework
  • ✓Measure agent success rates with controlled experiments
  • ✓Compare MCP servers and AI models for performance
  • ✓Assess API changes for agent compatibility
  • ✓Run experiments with CLI and manage results efficiently
  • ✓Visit GitHub for more tools and instructions

Installation Guide

The init command generates a .env.example file. Copy it and fill in your keys:

bash
cp .env.example .env

The framework loads .env.local first, then .env as a fallback, via dotenv.

Vercel AI Gateway (recommended)

One key for all models:

bash
AI_GATEWAY_API_KEY=your-ai-gateway-api-key
VERCEL_TOKEN=your-vercel-token

Direct API keys (no Vercel account required)

If you don't have a Vercel account, use provider API keys directly:

bash
ANTHROPIC_API_KEY=sk-ant-...      # For Claude Code
OPENAI_API_KEY=sk-proj-...        # For Codex

And choose ONE sandbox option (no Vercel key needed):

bash
# Option 1: Use Docker (free, no account needed)
# Just set sandbox: 'docker' in your experiment config, that's it!

# Option 2: Use Vercel (requires free account)
VERCEL_TOKEN=your-vercel-token

#### Minimal setup example

Claude Code via direct API with Docker sandbox:

typescript
// experiments/my-eval.ts
import type { ExperimentConfig } from '@vercel/agent-eval';

const config: ExperimentConfig = {
  agent: 'claude-code',  // Direct API (not vercel-ai-gateway/...)
  model: 'opus',
  runs: 1,
  sandbox: 'docker',     // No VERCEL_TOKEN needed
};

export default config;

Then just set:

bash
ANTHROPIC_API_KEY=sk-ant-...

That's it! The classifier will be disabled (since you don't have AI_GATEWAY_API_KEY), but all features work fine — you'll just see a warning that non-model failure classification is skipped.

Reviews