Automating Secret Detection with GitHub Secret Scanning API

This is a follow-up to the morning article

Morning article: Complete Guide for Handling Accidental Secret Exposure on GitHub

Goals

• Implement periodic monitoring scripts using Secret Scanning API
• Build automatic notification system (Slack/Discord) for detection events
• Complete automation with GitHub Actions and retry strategies for failures

Architecture Overview

The monitoring system consists of three components:

1. API Poller: Periodically fetches alerts via GitHub API
2. Notification Handler: Sends new detections to Slack/Discord
3. State Manager: Manages state to prevent duplicate notifications

Implementation Steps

Step 1: Python Monitoring Script Implementation

scripts/secret_scanner.py:

import os
import json
import requests
from datetime import datetime
from typing import List, Dict

class SecretScanner:
    def __init__(self, token: str, repo: str):
        self.token = token
        self.repo = repo
        self.api_base = f"https://api.github.com/repos/{repo}"
        self.headers = {
            "Authorization": f"token {token}",
            "Accept": "application/vnd.github.v3+json"
        }
        self.state_file = ".scanner_state.json"

    def get_alerts(self) -> List[Dict]:
        """Fetch unresolved secret alerts"""
        url = f"{self.api_base}/secret-scanning/alerts"
        params = {"state": "open", "per_page": 100}

        response = requests.get(url, headers=self.headers, params=params)
        if response.status_code == 404:
            return []  # Secret scanning not enabled
        response.raise_for_status()
        return response.json()

    def filter_new_alerts(self, alerts: List[Dict]) -> List[Dict]:
        """Filter only new alerts"""
        known_ids = self.load_state()
        new_alerts = [a for a in alerts if a["number"] not in known_ids]

        # Update state
        if new_alerts:
            for alert in new_alerts:
                known_ids.add(alert["number"])
            self.save_state(known_ids)

        return new_alerts

Step 2: Notification Handler Implementation

class NotificationHandler:
    def __init__(self, webhook_url: str, platform: str = "slack"):
        self.webhook_url = webhook_url
        self.platform = platform

    def send_alert(self, alert: Dict) -> bool:
        """Send alert to Slack/Discord"""
        if self.platform == "slack":
            payload = self._format_slack_message(alert)
        else:
            payload = self._format_discord_message(alert)

        response = requests.post(self.webhook_url, json=payload)
        return response.status_code == 200

    def _format_slack_message(self, alert: Dict) -> Dict:
        return {
            "text": "🚨 Secret detected in repository",
            "attachments": [{
                "color": "danger",
                "fields": [
                    {"title": "Secret Type", "value": alert["secret_type"], "short": True},
                    {"title": "File", "value": alert["locations"][0]["path"], "short": True},
                    {"title": "Detected", "value": alert["created_at"], "short": False},
                    {"title": "URL", "value": alert["html_url"], "short": False}
                ]
            }]
        }

Step 3: GitHub Actions Workflow Setup

.github/workflows/secret-monitoring.yml:

name: Secret Scanning Monitor
on:
  schedule:
    - cron: '*/30 * * * *'  # Every 30 minutes
  workflow_dispatch:  # Manual execution enabled

jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Setup Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.11'

      - name: Install dependencies
        run: pip install requests

      - name: Run scanner with retry
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}
        run: |
          for i in 1 2 3; do
            python scripts/secret_scanner.py && break
            echo "Attempt $i failed, retrying..."
            sleep 10
          done

Benchmark / Comparison

Monitoring Method	Detection Speed	False Positive Rate	Operational Cost
Manual Check	24+ hours	High (many misses)	High (human effort)
Git hooks only	Immediate	Medium (local only)	Low
API Periodic Monitoring	Within 30 min	Low (GitHub verified)	Very Low (automated)
API + Webhooks	Real-time	Lowest	Low (setup only)

Failure Patterns and Mitigations

Symptom	Cause	Mitigation
404 Error	Secret scanning not enabled	Enable in repository settings or graceful handling
Rate limit exceeded	Too many API calls	Adjust interval to 30+ min, conditional polling
Duplicate notifications	Poor state management	Persist .scanner_state.json with Actions artifacts
Webhook failure	URL expired/permissions	Configure fallback notification destination

Extension Ideas for Automation

• Priority Assessment: Auto-classify urgency based on secret_type
• Auto-fix PR: Create PR to update .gitignore upon detection
• Issue Creation: Auto-create GitHub Issues for each alert
• Metrics Collection: Send detection frequency to Datadog/CloudWatch
• Compliance Reports: Auto-generate monthly summaries

Next Steps

Build upon this guide to implement further security enhancements and production optimization.