Consolidated Hands-on¶
Canonical: ai-agent-development-practical-implementation-deep-dive-august2025.md
This file is a shortened hands-on alias for compatibility. Content has been fully integrated into the Deep Dive.
This article is the practical edition of "AI Agent Development Revolution: Claude Code & GitHub Copilot Latest Updates Complete Guide". Going beyond understanding the overview, we provide detailed explanations of specific procedures and code examples for actually configuring and operating AI agents.
Key Points¶
Custom Subagent Creation
Complete implementation from specialized domain agent configuration to production operation
MCP Integration Implementation
MCP server integration with GitHub, Slack, Jira, and custom development
Automated Workflow Construction
Building fully automated development pipelines using Agent Mode
Production Monitoring System
Agent operation log collection, performance monitoring, and troubleshooting
Phase 1: Claude Code Subagent Implementation¶
1.1 Custom Agent Configuration File Creation¶
First, create configuration files for specialized subagents.
# Create Claude Code agent directory
mkdir -p ~/.claude/agents
mkdir -p ~/.claude/commands
mkdir -p ~/.claude/hooks
Database Specialist Agent Configuration
<!-- ~/.claude/agents/database-architect.md -->
# Database Architect Agent
## Specialization Areas
- Database design & optimization
- ERD design and relationship optimization
- Index strategies and query tuning
- Migration strategies and data migration
## Model Configuration
- Primary: Claude 3.5 Sonnet (high-precision design)
- Secondary: Claude 3.7 Sonnet (complex query optimization)
## Responsibilities
### Design Phase
- Entity Relationship Diagram (ERD) creation
- Normalization/denormalization decisions
- Index design strategy
### Implementation Phase
- DDL statement generation and review
- Migration script creation
- Performance test design
### Optimization Phase
- Query execution plan analysis
- Index effectiveness measurement
- Bottleneck identification and improvement
## Context Configuration
```yaml
context:
technologies: [PostgreSQL, MySQL, MongoDB, Redis]
patterns: [CQRS, Event Sourcing, Sharding]
tools: [pgAdmin, MySQL Workbench, Studio 3T]
Output Format Templates¶
Design Documents¶
- ERD diagrams (Mermaid format)
- Table definition documents
- Index design documents
Implementation Code¶
- DDL (CREATE TABLE statements)
- DML (INSERT/UPDATE/DELETE statements)
- Optimization query examples
**Frontend Specialist Agent Configuration** ```markdown <!-- ~/.claude/agents/frontend-specialist.md --> # Frontend Development Specialist Agent ## Specialization Areas - React/Next.js application development - TypeScript type-safe implementation - State management (Redux Toolkit, Zustand) - UI/UX component design ## Model Configuration - Primary: GPT-4o (strong in React ecosystem) - Secondary: Claude 3.5 Sonnet (TypeScript type design) ## Implementation Patterns ### Component Design ```typescript // Standard component structure interface ComponentProps { // type definitions } export const Component: React.FC<ComponentProps> = ({ // props destructuring }) => { // hooks usage // JSX return }
State Management Patterns¶
- Zustand usage (lightweight state management)
- React Query (server state management)
- React Hook Form (form state)
Directory Structure Template¶
src/
components/
ui/ # Reusable UI components
forms/ # Form-specific components
layouts/ # Layout components
hooks/ # Custom hooks
stores/ # State management
utils/ # Utility functions
types/ # TypeScript type definitions
### 1.2 Custom Slash Command Implementation
Create specialized commands for use with Claude Code.
```markdown
<!-- ~/.claude/commands/review-database.md -->
# Database Review Command
Please comprehensively review the database design from the following perspectives:
## Design Review Points
1. **Normalization Level Check**
- Compliance verification for 1st-5th normal forms
- Necessity assessment of denormalization
2. **Index Strategy Verification**
- Primary key & foreign key indexes
- Composite index effectiveness
- Unnecessary index identification
3. **Performance Prediction**
- Query execution plan simulation
- Bottleneck prediction
- Scalability evaluation
## Output Format
- Issue list (with priorities)
- Improvement suggestions (with code examples)
- Performance improvement estimates
<!-- ~/.claude/commands/frontend-scaffold.md -->
# Frontend Scaffolding
Please create the basic structure for a React + TypeScript project with the following requirements:
## Generation Targets
1. **Basic directory structure**
2. **TypeScript configuration files**
3. **ESLint/Prettier configuration**
4. **Basic component templates**
5. **State management setup**
## Tech Stack
- React 18+ with TypeScript
- Vite (build tool)
- Zustand (state management)
- React Query (data fetching)
- Tailwind CSS (styling)
## Generated File Examples
- `src/components/ui/Button.tsx`
- `src/hooks/useApi.ts`
- `src/stores/appStore.ts`
- `vite.config.ts`
- `tsconfig.json`
1.3 Agent Execution Command Examples¶
Actual usage of configured agents:
# Create ERD with database specialist agent
claude --agent database-architect "Please design a product and order management database for an e-commerce site"
# Create components with frontend specialist agent
claude --agent frontend-specialist "Please create a user profile editing form"
# Initialize project with custom command
claude /frontend-scaffold "Admin panel project"
Phase 2: GitHub Copilot Agent Mode Implementation¶
2.1 Agent Mode Project Configuration¶
Implement detailed configuration for GitHub Copilot Agent Mode.
.vscode/settings.json Configuration
{
"github.copilot.enable": {
"*": true,
"plaintext": false,
"markdown": true
},
"github.copilot.editor.enableAutoCompletions": true,
"github.copilot.chat.localeOverride": "en",
"github.copilot.chat.welcomeMessage": "always",
// Agent Mode specific settings
"github.copilot.agent.enabled": true,
"github.copilot.agent.autoTrigger": "onPush",
"github.copilot.agent.reviewEnabled": true,
// MCP integration settings
"github.copilot.mcp.enabled": true,
"github.copilot.mcp.servers": {
"github": {
"command": "github-mcp-server",
"env": {
"GITHUB_TOKEN": "${{ secrets.GITHUB_TOKEN }}"
}
},
"filesystem": {
"command": "filesystem-mcp-server",
"args": ["--root", "./"]
}
}
}
GitHub Actions Workflow Integration
# .github/workflows/copilot-agent-review.yml
name: Copilot Agent Code Review
on:
pull_request:
types: [opened, synchronize, reopened]
jobs:
copilot-review:
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
issues: write
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
- name: Install GitHub CLI
run: |
type -p curl >/dev/null || (sudo apt update && sudo apt install curl -y)
curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg | sudo dd of=/usr/share/keyrings/githubcli-archive-keyring.gpg
sudo chmod go+r /usr/share/keyrings/githubcli-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" | sudo tee /etc/apt/sources.list.d/github-cli.list > /dev/null
sudo apt update && sudo apt install gh -y
- name: Setup GitHub Copilot CLI
run: |
gh extension install github/gh-copilot
- name: Run Copilot Agent Review
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
# Get pull request diff
gh pr diff ${{ github.event.pull_request.number }} > changes.diff
# Execute code review with Agent Mode
gh copilot suggest \
--agent-mode \
"Please review the following changes and output issues and improvement suggestions in Markdown format. Evaluate from security, performance, and readability perspectives." \
< changes.diff > review.md
# Add review results to PR comments
gh pr comment ${{ github.event.pull_request.number }} --body-file review.md
- name: Quality Gate Check
run: |
# Quality check with Agent Mode
gh copilot suggest \
--agent-mode \
"Determine if this pull request is safe to merge and answer YES/NO with reasoning." \
< changes.diff > quality_gate.txt
# Save result as artifact
echo "QUALITY_GATE_RESULT=$(cat quality_gate.txt)" >> $GITHUB_ENV
- name: Update PR Status
if: contains(env.QUALITY_GATE_RESULT, 'NO')
run: |
gh pr ready ${{ github.event.pull_request.number }} --undo
gh pr comment ${{ github.event.pull_request.number }} \
--body "🚫 **Quality Gate Failed**: Agent review detected issues. Please address before merging."
2.2 Issue Auto-Processing System Construction¶
Fully automated development system using GitHub Copilot Coding Agent:
Issue Template Configuration
# .github/ISSUE_TEMPLATE/feature-request-agent.yml
name: 🤖 Feature Request (Agent Auto-Implementation)
description: AI agents will automatically implement feature requests
title: "[AGENT] "
labels: ["agent-auto", "feature"]
body:
- type: textarea
id: description
attributes:
label: Feature Details
description: Please describe the feature you want to implement in detail
placeholder: |
Example:
- User authentication system implementation
- JWT token-based
- Login, logout, password reset functionality
- Admin and regular user permission management
value: ""
validations:
required: true
- type: dropdown
id: priority
attributes:
label: Implementation Priority
options:
- "🔴 Urgent (within 24 hours)"
- "🟡 High (within 72 hours)"
- "🟢 Medium (within 1 week)"
- "⚪ Low (flexible timing)"
validations:
required: true
- type: checkboxes
id: requirements
attributes:
label: Requirements Confirmation
options:
- label: "Include test code implementation"
required: false
- label: "Automatic documentation updates"
required: false
- label: "Include security review"
required: false
Agent Auto-Implementation Workflow
# .github/workflows/agent-auto-implementation.yml
name: Agent Auto Implementation
on:
issues:
types: [opened, labeled]
jobs:
auto-implement:
if: contains(github.event.issue.labels.*.name, 'agent-auto')
runs-on: ubuntu-latest
permissions:
contents: write
issues: write
pull-requests: write
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Parse Issue Requirements
id: parse
run: |
# Parse issue body
ISSUE_BODY='${{ github.event.issue.body }}'
ISSUE_TITLE='${{ github.event.issue.title }}'
# Extract feature name (remove [AGENT] from title)
FEATURE_NAME=$(echo "$ISSUE_TITLE" | sed 's/\[AGENT\]//g' | xargs)
echo "feature_name=$FEATURE_NAME" >> $GITHUB_OUTPUT
echo "issue_number=${{ github.event.issue.number }}" >> $GITHUB_OUTPUT
- name: Create Feature Branch
run: |
BRANCH_NAME="agent/issue-${{ steps.parse.outputs.issue_number }}-$(echo '${{ steps.parse.outputs.feature_name }}' | tr ' ' '-' | tr '[:upper:]' '[:lower:]')"
git checkout -b "$BRANCH_NAME"
echo "BRANCH_NAME=$BRANCH_NAME" >> $GITHUB_ENV
- name: Setup Development Environment
run: |
# Node.js environment setup
npm install
# Install dependencies for Agent Mode
gh extension install github/gh-copilot
- name: Agent Implementation Phase
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
# Pass issue requirements to Agent Mode for implementation instructions
IMPLEMENTATION_PROMPT="Please implement a full-stack application with the following requirements:
Requirements:
${{ github.event.issue.body }}
Implementation Guidelines:
1. Follow security best practices
2. Always include test code
3. Comply with TypeScript/ESLint rules
4. Auto-generate API documentation
5. Implement proper error handling
File Structure:
- src/components/ (React components)
- src/hooks/ (custom hooks)
- src/services/ (API client)
- tests/ (test files)
- docs/ (documentation)"
# Execute implementation with Agent Mode
gh copilot suggest --agent-mode "$IMPLEMENTATION_PROMPT" > implementation_log.md
# Verify and adjust implemented files
if [[ -n $(git status --porcelain) ]]; then
git add .
git commit -m "feat: implement ${{ steps.parse.outputs.feature_name }} via Agent Mode
Auto-generated implementation based on Issue #${{ steps.parse.outputs.issue_number }}
Implemented files:
$(git diff --name-only HEAD~1)
Co-authored-by: GitHub Copilot Agent <noreply@github.com>"
fi
- name: Quality Assurance Phase
run: |
# Comprehensive code quality check by agent
QUALITY_CHECK="Please comprehensively review the implemented code from the following perspectives:
1. Presence of security vulnerabilities
2. Performance optimization opportunities
3. Sufficiency of test coverage
4. Code quality and maintainability
5. Accessibility compliance"
gh copilot suggest --agent-mode "$QUALITY_CHECK" > quality_review.md
# Run tests
npm run test 2>&1 | tee test_results.txt
# TypeScript type check
npm run type-check 2>&1 | tee type_check_results.txt
# Run ESLint
npm run lint 2>&1 | tee lint_results.txt
- name: Create Pull Request
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
# Create PR
gh pr create \
--title "🤖 [Auto-Implementation] ${{ steps.parse.outputs.feature_name }}" \
--body "## Auto-Implementation Completion Report
### Resolved Issue
Closes #${{ steps.parse.outputs.issue_number }}
### Implementation Details
$(cat implementation_log.md)
### Quality Review Results
$(cat quality_review.md)
### Test Results
\`\`\`
$(cat test_results.txt)
\`\`\`
### Type Check Results
\`\`\`
$(cat type_check_results.txt)
\`\`\`
### Lint Results
\`\`\`
$(cat lint_results.txt)
\`\`\`
---
🤖 Generated with GitHub Copilot Agent Mode
" \
--label "agent-generated,ready-for-review"
# Add PR link to issue comment
PR_URL=$(gh pr view --json url -q .url)
gh issue comment ${{ steps.parse.outputs.issue_number }} \
--body "🤖 **Auto-Implementation Complete**
Pull Request: $PR_URL
Please review if the implementation meets the requirements.
If modifications are needed, please comment on the PR and we will automatically address them."
2.3 MCP Integration Custom Development¶
Develop custom MCP servers to extend functionality.
Custom MCP Server Implementation
// mcp-servers/project-manager-server/src/index.ts
import { Server } from "@modelcontextprotocol/sdk/server/index.js";
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
import {
ListResourcesRequestSchema,
ReadResourceRequestSchema,
ListToolsRequestSchema,
CallToolRequestSchema,
} from "@modelcontextprotocol/sdk/types.js";
interface ProjectState {
name: string;
status: 'planning' | 'development' | 'testing' | 'deployment' | 'completed';
tasks: Task[];
resources: Resource[];
}
interface Task {
id: string;
title: string;
description: string;
assignee: string;
status: 'todo' | 'in_progress' | 'review' | 'done';
priority: 'low' | 'medium' | 'high' | 'urgent';
dueDate?: Date;
dependencies: string[];
}
interface Resource {
id: string;
name: string;
type: 'file' | 'url' | 'database' | 'api';
location: string;
description: string;
}
class ProjectManagerServer {
private server: Server;
private projects: Map<string, ProjectState> = new Map();
constructor() {
this.server = new Server(
{
name: "project-manager-mcp-server",
version: "1.0.0",
},
{
capabilities: {
resources: {},
tools: {},
},
}
);
this.setupHandlers();
}
private setupHandlers() {
// Resource list retrieval
this.server.setRequestHandler(
ListResourcesRequestSchema,
async () => {
const resources = [];
for (const [projectId, project] of this.projects) {
resources.push({
uri: `project://${projectId}`,
mimeType: "application/json",
name: project.name,
description: `Project: ${project.name} (${project.status})`,
});
}
return { resources };
}
);
// Resource reading
this.server.setRequestHandler(
ReadResourceRequestSchema,
async (request) => {
const uri = request.params.uri;
const match = uri.match(/^project:\/\/(.+)$/);
if (!match) {
throw new Error(`Invalid resource URI: ${uri}`);
}
const projectId = match[1];
const project = this.projects.get(projectId);
if (!project) {
throw new Error(`Project not found: ${projectId}`);
}
return {
contents: [
{
uri,
mimeType: "application/json",
text: JSON.stringify(project, null, 2),
},
],
};
}
);
// Tool list
this.server.setRequestHandler(
ListToolsRequestSchema,
async () => {
return {
tools: [
{
name: "create_project",
description: "Create new project",
inputSchema: {
type: "object",
properties: {
name: { type: "string" },
description: { type: "string" },
},
required: ["name"],
},
},
{
name: "add_task",
description: "Add task to project",
inputSchema: {
type: "object",
properties: {
projectId: { type: "string" },
title: { type: "string" },
description: { type: "string" },
assignee: { type: "string" },
priority: {
type: "string",
enum: ["low", "medium", "high", "urgent"]
},
dueDate: { type: "string", format: "date" },
},
required: ["projectId", "title"],
},
},
{
name: "update_task_status",
description: "Update task status",
inputSchema: {
type: "object",
properties: {
projectId: { type: "string" },
taskId: { type: "string" },
status: {
type: "string",
enum: ["todo", "in_progress", "review", "done"]
},
},
required: ["projectId", "taskId", "status"],
},
},
{
name: "generate_project_report",
description: "Generate project progress report",
inputSchema: {
type: "object",
properties: {
projectId: { type: "string" },
format: {
type: "string",
enum: ["markdown", "json", "html"]
},
},
required: ["projectId"],
},
},
],
};
}
);
// Tool execution
this.server.setRequestHandler(
CallToolRequestSchema,
async (request) => {
const { name, arguments: args } = request.params;
switch (name) {
case "create_project":
return this.createProject(args as { name: string; description?: string });
case "add_task":
return this.addTask(args as {
projectId: string;
title: string;
description?: string;
assignee?: string;
priority?: string;
dueDate?: string;
});
case "update_task_status":
return this.updateTaskStatus(args as {
projectId: string;
taskId: string;
status: string;
});
case "generate_project_report":
return this.generateProjectReport(args as {
projectId: string;
format?: string;
});
default:
throw new Error(`Unknown tool: ${name}`);
}
}
);
}
private async createProject(args: { name: string; description?: string }) {
const projectId = `project_${Date.now()}`;
const project: ProjectState = {
name: args.name,
status: 'planning',
tasks: [],
resources: [],
};
this.projects.set(projectId, project);
return {
content: [
{
type: "text",
text: `Created project "${args.name}". ID: ${projectId}`,
},
],
};
}
private async addTask(args: {
projectId: string;
title: string;
description?: string;
assignee?: string;
priority?: string;
dueDate?: string;
}) {
const project = this.projects.get(args.projectId);
if (!project) {
throw new Error(`Project not found: ${args.projectId}`);
}
const taskId = `task_${Date.now()}`;
const task: Task = {
id: taskId,
title: args.title,
description: args.description || '',
assignee: args.assignee || '',
status: 'todo',
priority: (args.priority as any) || 'medium',
dueDate: args.dueDate ? new Date(args.dueDate) : undefined,
dependencies: [],
};
project.tasks.push(task);
return {
content: [
{
type: "text",
text: `Added task "${args.title}" to project ${args.projectId}.`,
},
],
};
}
private async updateTaskStatus(args: {
projectId: string;
taskId: string;
status: string;
}) {
const project = this.projects.get(args.projectId);
if (!project) {
throw new Error(`Project not found: ${args.projectId}`);
}
const task = project.tasks.find(t => t.id === args.taskId);
if (!task) {
throw new Error(`Task not found: ${args.taskId}`);
}
task.status = args.status as any;
return {
content: [
{
type: "text",
text: `Updated task "${task.title}" status to "${args.status}".`,
},
],
};
}
private async generateProjectReport(args: {
projectId: string;
format?: string;
}) {
const project = this.projects.get(args.projectId);
if (!project) {
throw new Error(`Project not found: ${args.projectId}`);
}
const format = args.format || 'markdown';
if (format === 'markdown') {
const report = this.generateMarkdownReport(project);
return {
content: [
{
type: "text",
text: report,
},
],
};
}
return {
content: [
{
type: "text",
text: JSON.stringify(project, null, 2),
},
],
};
}
private generateMarkdownReport(project: ProjectState): string {
const totalTasks = project.tasks.length;
const completedTasks = project.tasks.filter(t => t.status === 'done').length;
const inProgressTasks = project.tasks.filter(t => t.status === 'in_progress').length;
const todoTasks = project.tasks.filter(t => t.status === 'todo').length;
const progress = totalTasks > 0 ? Math.round((completedTasks / totalTasks) * 100) : 0;
return `# ${project.name} Project Report
## Overview
- **Status**: ${project.status}
- **Progress**: ${progress}% (${completedTasks}/${totalTasks})
## Task Status
- ✅ Completed: ${completedTasks}
- 🔄 In Progress: ${inProgressTasks}
- ⏳ Todo: ${todoTasks}
## Task Details
${project.tasks.map(task => `
### ${task.title}
- **Status**: ${task.status}
- **Priority**: ${task.priority}
- **Assignee**: ${task.assignee || 'Unassigned'}
- **Description**: ${task.description}
${task.dueDate ? `- **Due Date**: ${task.dueDate.toLocaleDateString()}` : ''}
`).join('\n')}
---
_Generated by Project Manager MCP Server_
`;
}
async run() {
const transport = new StdioServerTransport();
await this.server.connect(transport);
}
}
const server = new ProjectManagerServer();
server.run().catch((error) => {
console.error("Server error:", error);
process.exit(1);
});
MCP Server Configuration File
{
"mcpServers": {
"project-manager": {
"command": "node",
"args": ["./mcp-servers/project-manager-server/dist/index.js"],
"env": {}
},
"github": {
"command": "github-mcp-server",
"args": ["--token", "${{ secrets.GITHUB_TOKEN }}"]
},
"database": {
"command": "database-mcp-server",
"args": ["--connection-string", "${{ secrets.DATABASE_URL }}"]
},
"slack": {
"command": "slack-mcp-server",
"env": {
"SLACK_BOT_TOKEN": "${{ secrets.SLACK_BOT_TOKEN }}"
}
}
}
}
Phase 3: Monitoring & Operations System Construction¶
3.1 Agent Operation Log System¶
Build a system to monitor and analyze AI agent operations.
// monitoring/agent-logger.ts
import fs from 'fs';
import path from 'path';
interface AgentLogEntry {
timestamp: Date;
agentType: 'claude-code' | 'copilot-agent' | 'custom-mcp';
agentId: string;
action: string;
input: string;
output: string;
executionTime: number;
success: boolean;
error?: string;
metadata?: Record<string, any>;
}
class AgentLogger {
private logDir: string;
private currentLogFile: string;
constructor(baseDir: string = './logs') {
this.logDir = baseDir;
this.ensureLogDirectory();
this.currentLogFile = this.getLogFilePath();
}
private ensureLogDirectory() {
if (!fs.existsSync(this.logDir)) {
fs.mkdirSync(this.logDir, { recursive: true });
}
}
private getLogFilePath(): string {
const today = new Date().toISOString().split('T')[0];
return path.join(this.logDir, `agent-logs-${today}.jsonl`);
}
log(entry: Omit<AgentLogEntry, 'timestamp'>) {
const logEntry: AgentLogEntry = {
timestamp: new Date(),
...entry,
};
const logLine = JSON.stringify(logEntry) + '\n';
fs.appendFileSync(this.currentLogFile, logLine);
// Send Slack alerts for critical errors
if (!entry.success && entry.agentType === 'copilot-agent') {
this.sendSlackAlert(logEntry);
}
}
private async sendSlackAlert(entry: AgentLogEntry) {
const webhook = process.env.SLACK_WEBHOOK_URL;
if (!webhook) return;
const message = {
text: `🚨 Agent Error Alert`,
blocks: [
{
type: "section",
text: {
type: "mrkdwn",
text: `*Agent Error Detected*\n\n*Agent:* ${entry.agentType} (${entry.agentId})\n*Action:* ${entry.action}\n*Error:* ${entry.error}\n*Time:* ${entry.timestamp.toISOString()}`
}
}
]
};
try {
await fetch(webhook, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(message),
});
} catch (error) {
console.error('Failed to send Slack alert:', error);
}
}
// Generate statistics
generateDailyStats(date?: string): AgentStats {
const targetDate = date || new Date().toISOString().split('T')[0];
const logFile = path.join(this.logDir, `agent-logs-${targetDate}.jsonl`);
if (!fs.existsSync(logFile)) {
return this.emptyStats();
}
const logs = fs.readFileSync(logFile, 'utf-8')
.split('\n')
.filter(line => line.trim())
.map(line => JSON.parse(line) as AgentLogEntry);
return this.calculateStats(logs);
}
private calculateStats(logs: AgentLogEntry[]): AgentStats {
const stats: AgentStats = {
totalExecutions: logs.length,
successRate: 0,
averageExecutionTime: 0,
agentBreakdown: {},
actionBreakdown: {},
errorBreakdown: {},
performanceMetrics: {
fastExecutions: 0, // < 1s
normalExecutions: 0, // 1-5s
slowExecutions: 0, // > 5s
}
};
if (logs.length === 0) return stats;
let successCount = 0;
let totalExecutionTime = 0;
logs.forEach(log => {
// Calculate success rate
if (log.success) successCount++;
// Calculate execution time
totalExecutionTime += log.executionTime;
// Performance classification
if (log.executionTime < 1000) {
stats.performanceMetrics.fastExecutions++;
} else if (log.executionTime < 5000) {
stats.performanceMetrics.normalExecutions++;
} else {
stats.performanceMetrics.slowExecutions++;
}
// Agent-specific statistics
if (!stats.agentBreakdown[log.agentType]) {
stats.agentBreakdown[log.agentType] = {
count: 0,
successCount: 0,
avgExecutionTime: 0,
};
}
stats.agentBreakdown[log.agentType].count++;
if (log.success) {
stats.agentBreakdown[log.agentType].successCount++;
}
// Action-specific statistics
stats.actionBreakdown[log.action] =
(stats.actionBreakdown[log.action] || 0) + 1;
// Error-specific statistics
if (!log.success && log.error) {
stats.errorBreakdown[log.error] =
(stats.errorBreakdown[log.error] || 0) + 1;
}
});
stats.successRate = (successCount / logs.length) * 100;
stats.averageExecutionTime = totalExecutionTime / logs.length;
// Calculate average execution time per agent
Object.keys(stats.agentBreakdown).forEach(agentType => {
const agentLogs = logs.filter(log => log.agentType === agentType);
const totalTime = agentLogs.reduce((sum, log) => sum + log.executionTime, 0);
stats.agentBreakdown[agentType].avgExecutionTime = totalTime / agentLogs.length;
});
return stats;
}
private emptyStats(): AgentStats {
return {
totalExecutions: 0,
successRate: 0,
averageExecutionTime: 0,
agentBreakdown: {},
actionBreakdown: {},
errorBreakdown: {},
performanceMetrics: {
fastExecutions: 0,
normalExecutions: 0,
slowExecutions: 0,
}
};
}
}
interface AgentStats {
totalExecutions: number;
successRate: number;
averageExecutionTime: number;
agentBreakdown: Record<string, {
count: number;
successCount: number;
avgExecutionTime: number;
}>;
actionBreakdown: Record<string, number>;
errorBreakdown: Record<string, number>;
performanceMetrics: {
fastExecutions: number;
normalExecutions: number;
slowExecutions: number;
};
}
export { AgentLogger, AgentLogEntry, AgentStats };
3.2 Performance Monitoring Dashboard¶
// monitoring/dashboard-generator.ts
import { AgentLogger, AgentStats } from './agent-logger';
import fs from 'fs';
import path from 'path';
class DashboardGenerator {
private logger: AgentLogger;
constructor(logger: AgentLogger) {
this.logger = logger;
}
generateDashboard(days: number = 7): string {
const dashboardData = this.collectDashboardData(days);
return this.generateHTML(dashboardData);
}
private collectDashboardData(days: number) {
const data = [];
const today = new Date();
for (let i = 0; i < days; i++) {
const date = new Date(today);
date.setDate(today.getDate() - i);
const dateStr = date.toISOString().split('T')[0];
const stats = this.logger.generateDailyStats(dateStr);
data.push({
date: dateStr,
stats
});
}
return data.reverse(); // Sort in chronological order
}
private generateHTML(data: Array<{date: string, stats: AgentStats}>): string {
const chartData = this.prepareChartData(data);
return `
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>AI Agent Performance Dashboard</title>
<script src="https://cdn.jsdelivr.net/npm/chart.js"></script>
<style>
body {
font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;
margin: 0; padding: 20px; background-color: #f5f5f5;
}
.container { max-width: 1200px; margin: 0 auto; }
.header {
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
color: white; padding: 30px; border-radius: 10px; margin-bottom: 30px;
text-align: center;
}
.metrics-grid {
display: grid; grid-template-columns: repeat(auto-fit, minmax(250px, 1fr));
gap: 20px; margin-bottom: 30px;
}
.metric-card {
background: white; padding: 25px; border-radius: 10px;
box-shadow: 0 4px 6px rgba(0, 0, 0, 0.1); text-align: center;
}
.metric-value {
font-size: 2.5em; font-weight: bold; color: #667eea;
margin: 10px 0;
}
.chart-container {
background: white; padding: 30px; border-radius: 10px;
box-shadow: 0 4px 6px rgba(0, 0, 0, 0.1); margin-bottom: 30px;
}
.chart-canvas { max-height: 400px; }
.error-list {
background: white; padding: 25px; border-radius: 10px;
box-shadow: 0 4px 6px rgba(0, 0, 0, 0.1);
}
.error-item {
padding: 15px; margin: 10px 0; background: #fff5f5;
border-left: 4px solid #fc8181; border-radius: 5px;
}
.agent-breakdown {
display: grid; grid-template-columns: repeat(auto-fit, minmax(300px, 1fr));
gap: 20px; margin-bottom: 30px;
}
.last-updated {
text-align: center; color: #666; font-size: 0.9em; margin-top: 30px;
}
</style>
</head>
<body>
<div class="container">
<div class="header">
<h1>🤖 AI Agent Performance Dashboard</h1>
<p>AI Agent Performance Monitoring and Analysis</p>
</div>
${this.generateMetricsSection(data)}
${this.generateChartsSection(chartData)}
${this.generateAgentBreakdownSection(data)}
${this.generateErrorAnalysisSection(data)}
<div class="last-updated">
Last updated: ${new Date().toLocaleString()}
</div>
</div>
<script>
${this.generateChartScripts(chartData)}
</script>
</body>
</html>`;
}
private generateMetricsSection(data: Array<{date: string, stats: AgentStats}>): string {
const totalStats = this.aggregateStats(data.map(d => d.stats));
return `
<div class="metrics-grid">
<div class="metric-card">
<h3>Total Executions</h3>
<div class="metric-value">${totalStats.totalExecutions.toLocaleString()}</div>
<p>Past ${data.length} days</p>
</div>
<div class="metric-card">
<h3>Success Rate</h3>
<div class="metric-value">${totalStats.successRate.toFixed(1)}%</div>
<p>Error Rate: ${(100 - totalStats.successRate).toFixed(1)}%</p>
</div>
<div class="metric-card">
<h3>Average Execution Time</h3>
<div class="metric-value">${(totalStats.averageExecutionTime / 1000).toFixed(2)}s</div>
<p>Milliseconds: ${totalStats.averageExecutionTime.toFixed(0)}ms</p>
</div>
<div class="metric-card">
<h3>Performance</h3>
<div class="metric-value">${this.getPerformanceGrade(totalStats)}</div>
<p>Fast Executions: ${totalStats.performanceMetrics.fastExecutions}</p>
</div>
</div>`;
}
private generateChartsSection(chartData: any): string {
return `
<div class="chart-container">
<h2>📊 Daily Execution Statistics</h2>
<canvas id="dailyStatsChart" class="chart-canvas"></canvas>
</div>
<div class="chart-container">
<h2>⚡ Performance Distribution</h2>
<canvas id="performanceChart" class="chart-canvas"></canvas>
</div>
<div class="chart-container">
<h2>🎯 Agent Success Rate</h2>
<canvas id="agentSuccessChart" class="chart-canvas"></canvas>
</div>`;
}
private generateAgentBreakdownSection(data: Array<{date: string, stats: AgentStats}>): string {
const totalStats = this.aggregateStats(data.map(d => d.stats));
return `
<div class="agent-breakdown">
${Object.entries(totalStats.agentBreakdown).map(([agentType, breakdown]) => `
<div class="metric-card">
<h3>${agentType}</h3>
<div class="metric-value">${breakdown.count}</div>
<p>Executions</p>
<hr>
<p><strong>Success Rate:</strong> ${((breakdown.successCount / breakdown.count) * 100).toFixed(1)}%</p>
<p><strong>Average Time:</strong> ${(breakdown.avgExecutionTime / 1000).toFixed(2)}s</p>
</div>
`).join('')}
</div>`;
}
private generateErrorAnalysisSection(data: Array<{date: string, stats: AgentStats}>): string {
const totalStats = this.aggregateStats(data.map(d => d.stats));
const topErrors = Object.entries(totalStats.errorBreakdown)
.sort(([,a], [,b]) => b - a)
.slice(0, 5);
if (topErrors.length === 0) {
return `
<div class="error-list">
<h2>🎉 Error Analysis</h2>
<p style="text-align: center; color: #48bb78; font-size: 1.2em;">
Excellent! No errors occurred in the past ${data.length} days.
</p>
</div>`;
}
return `
<div class="error-list">
<h2>⚠️ Most Frequent Errors (Top 5)</h2>
${topErrors.map(([error, count]) => `
<div class="error-item">
<strong>${error}</strong>
<span style="float: right; color: #e53e3e;">${count} times</span>
<br>
<small>Consider implementing countermeasures</small>
</div>
`).join('')}
</div>`;
}
private prepareChartData(data: Array<{date: string, stats: AgentStats}>) {
return {
labels: data.map(d => d.date),
dailyExecutions: data.map(d => d.stats.totalExecutions),
dailySuccessRate: data.map(d => d.stats.successRate),
dailyAvgTime: data.map(d => d.stats.averageExecutionTime / 1000),
};
}
private generateChartScripts(chartData: any): string {
return `
// Daily execution statistics chart
new Chart(document.getElementById('dailyStatsChart'), {
type: 'line',
data: {
labels: ${JSON.stringify(chartData.labels)},
datasets: [{
label: 'Executions',
data: ${JSON.stringify(chartData.dailyExecutions)},
borderColor: '#667eea',
backgroundColor: 'rgba(102, 126, 234, 0.1)',
yAxisID: 'y'
}, {
label: 'Success Rate (%)',
data: ${JSON.stringify(chartData.dailySuccessRate)},
borderColor: '#48bb78',
backgroundColor: 'rgba(72, 187, 120, 0.1)',
yAxisID: 'y1'
}]
},
options: {
responsive: true,
scales: {
y: {
type: 'linear',
display: true,
position: 'left',
},
y1: {
type: 'linear',
display: true,
position: 'right',
grid: {
drawOnChartArea: false,
},
}
}
}
});
// Performance distribution chart
const totalStats = ${JSON.stringify(this.aggregateStats(data.map(d => d.stats)))};
new Chart(document.getElementById('performanceChart'), {
type: 'doughnut',
data: {
labels: ['Fast (<1s)', 'Normal (1-5s)', 'Slow (>5s)'],
datasets: [{
data: [
totalStats.performanceMetrics.fastExecutions,
totalStats.performanceMetrics.normalExecutions,
totalStats.performanceMetrics.slowExecutions
],
backgroundColor: ['#48bb78', '#ed8936', '#e53e3e']
}]
},
options: {
responsive: true,
plugins: {
legend: {
position: 'bottom'
}
}
}
});
// Agent success rate chart
const agentLabels = Object.keys(totalStats.agentBreakdown);
const agentSuccessRates = agentLabels.map(agent =>
(totalStats.agentBreakdown[agent].successCount / totalStats.agentBreakdown[agent].count) * 100
);
new Chart(document.getElementById('agentSuccessChart'), {
type: 'bar',
data: {
labels: agentLabels,
datasets: [{
label: 'Success Rate (%)',
data: agentSuccessRates,
backgroundColor: '#667eea',
borderColor: '#4c51bf',
borderWidth: 1
}]
},
options: {
responsive: true,
scales: {
y: {
beginAtZero: true,
max: 100
}
}
}
});`;
}
private aggregateStats(statsArray: AgentStats[]): AgentStats {
// Implementation to aggregate multi-day statistics
const aggregated: AgentStats = {
totalExecutions: 0,
successRate: 0,
averageExecutionTime: 0,
agentBreakdown: {},
actionBreakdown: {},
errorBreakdown: {},
performanceMetrics: {
fastExecutions: 0,
normalExecutions: 0,
slowExecutions: 0,
}
};
// Aggregation logic implementation
// ... (implementation omitted)
return aggregated;
}
private getPerformanceGrade(stats: AgentStats): string {
const fastRatio = stats.performanceMetrics.fastExecutions / stats.totalExecutions;
if (fastRatio > 0.8) return 'A+';
if (fastRatio > 0.6) return 'A';
if (fastRatio > 0.4) return 'B';
if (fastRatio > 0.2) return 'C';
return 'D';
}
saveDashboard(outputPath: string = './monitoring/dashboard.html') {
const html = this.generateDashboard();
fs.writeFileSync(outputPath, html);
console.log(`Dashboard saved to: ${outputPath}`);
}
}
export { DashboardGenerator };
3.3 Operations Automation Script¶
#!/bin/bash
# monitoring/deploy-monitoring.sh
set -e
echo "🚀 Starting AI Agent Monitoring System deployment"
# 1. Create necessary directories
echo "📁 Creating directory structure..."
mkdir -p logs monitoring/static monitoring/reports
# 2. Install Node.js dependencies
echo "📦 Installing dependencies..."
npm install --production
# 3. Compile TypeScript
echo "🔧 Compiling TypeScript..."
npx tsc --project monitoring/tsconfig.json
# 4. Configure monitoring service
echo "⚙️ Configuring monitoring service..."
# Create systemd service file
sudo tee /etc/systemd/system/agent-monitor.service > /dev/null <<EOF
[Unit]
Description=AI Agent Monitoring Service
After=network.target
[Service]
Type=simple
User=$(whoami)
WorkingDirectory=$(pwd)
ExecStart=/usr/bin/node monitoring/dist/monitor-service.js
Restart=always
RestartSec=10
Environment=NODE_ENV=production
Environment=LOG_LEVEL=info
[Install]
WantedBy=multi-user.target
EOF
# 5. Configure cron (daily report generation)
echo "⏰ Setting up cron jobs..."
(crontab -l 2>/dev/null; echo "0 1 * * * cd $(pwd) && node monitoring/dist/generate-daily-report.js") | crontab -
# 6. Configure Nginx (dashboard publishing)
if command -v nginx >/dev/null 2>&1; then
echo "🌐 Configuring Nginx..."
sudo tee /etc/nginx/sites-available/agent-dashboard > /dev/null <<EOF
server {
listen 8080;
server_name localhost;
location / {
root $(pwd)/monitoring/static;
index dashboard.html;
try_files \$uri \$uri/ =404;
}
location /api/ {
proxy_pass http://localhost:3001;
proxy_http_version 1.1;
proxy_set_header Upgrade \$http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host \$host;
proxy_cache_bypass \$http_upgrade;
}
}
EOF
sudo ln -sf /etc/nginx/sites-available/agent-dashboard /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx
fi
# 7. Start services
echo "▶️ Starting services..."
sudo systemctl daemon-reload
sudo systemctl enable agent-monitor
sudo systemctl start agent-monitor
# 8. Generate initial dashboard
echo "📊 Generating initial dashboard..."
node monitoring/dist/generate-dashboard.js
# 9. Health check
echo "🏥 Running health check..."
sleep 5
if systemctl is-active --quiet agent-monitor; then
echo "✅ Monitoring service is running normally"
else
echo "❌ Failed to start monitoring service"
exit 1
fi
# 10. Setup completion message
echo ""
echo "🎉 AI Agent Monitoring System deployment complete!"
echo ""
echo "📊 Dashboard: http://localhost:8080"
echo "📝 Log files: $(pwd)/logs/"
echo "⚙️ Service status: systemctl status agent-monitor"
echo ""
echo "Example commands:"
echo " - Update dashboard: node monitoring/dist/generate-dashboard.js"
echo " - Generate daily report: node monitoring/dist/generate-daily-report.js"
echo " - Monitor logs: tail -f logs/agent-logs-$(date +%Y-%m-%d).jsonl"
Summary¶
This article provides detailed explanations of specific implementation methods for actually operating the AI agent technologies introduced in the morning overview article:
- Claude Code Subagents: Configuration and operation methods for specialized agents
- GitHub Copilot Agent Mode: Building fully automated development pipelines
- MCP Integration: Custom protocol development and system integration
- Monitoring & Operations: Performance monitoring and troubleshooting
Through these implementations, you can evolve AI agents from mere tools into strategic development partners.