Skip to main content
Alerts fire after all evals and enrichments finish for a session. Register them with app.alert() in your evals file. 
app.alert('name', (context) => { /* ... */ });

​
What alerts receive

FieldTypeDescription
projectNamestringProject name
sessionIdstringSession ID
evalSummaryEvalRunSummary | undefinedAggregated eval results (present when evals are registered)
enrichSummaryEnrichRunSummary | undefinedAggregated enrichment results (present when enrichments are registered)
evalSummary and enrichSummary contain all results for the session - both cached and freshly computed. The alert always sees the complete picture.

​
evalSummary fields

evalSummary.passCount      // Number of passing evals
evalSummary.failCount      // Number of failing evals
evalSummary.errorCount     // Number of evals that errored
evalSummary.skippedCount   // Number of skipped evals
evalSummary.results        // Array of individual eval results
evalSummary.totalDurationMs

​
When alerts fire

Alerts fire once per session content version - after all eval/enrichment work for that session completes.
TriggerBehavior
Initial page loadAlerts fire when the last queued item completes
Background processingAlerts fire when the last item for a session completes
Page reload (all cached)No items enter the queue - no alerts fire
Re-run AllClears dedup marker first → alerts fire exactly once
Re-run single itemDedup marker still valid → alerts do not re-fire
Session content changesNew content hash → alerts fire again
Each alert callback is individually try/caught. A throwing alert never blocks other alerts.

​
Options

OptionTypeDefaultDescription
suppressOnRecomputebooleantrueWhen true, alert is suppressed during recomputes triggered by code changes. Set to false for audit logging that should always fire.
// Always fire, even on code-change recomputes
app.alert('audit-log', ({ projectName, sessionId }) => {
  console.log(`[AUDIT] Processed ${projectName}/${sessionId}`);
}, { suppressOnRecompute: false });

​
Examples

​
Log every session result

app.alert('log-results', ({ projectName, sessionId, evalSummary, enrichSummary }) => {
  const evals = evalSummary
    ? `${evalSummary.passCount} pass, ${evalSummary.failCount} fail`
    : 'no evals';
  const enrichments = enrichSummary
    ? `${enrichSummary.results.length} enrichments`
    : 'no enrichments';
  console.log(`[ALERT] ${projectName}/${sessionId}: ${evals} | ${enrichments}`);
});

​
Warn on failures

app.alert('warn-on-failure', ({ projectName, sessionId, evalSummary }) => {
  if (evalSummary && evalSummary.failCount > 0) {
    const failed = evalSummary.results
      .filter(r => !r.error && !r.skipped && !r.pass)
      .map(r => r.name);
    console.warn(`[FAILURE] ${projectName}/${sessionId}: ${failed.join(', ')} failed`);
  }
});

​
Slack webhook on failure

app.alert('slack-on-failure', async ({ projectName, sessionId, evalSummary }) => {
  if (evalSummary && evalSummary.failCount > 0) {
    await fetch('https://hooks.slack.com/services/YOUR/WEBHOOK/URL', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({
        text: `${evalSummary.failCount} evals failed for ${projectName}/${sessionId}`,
      }),
    });
  }
});

​
Write results to a file for CI

app.alert('ci-report', async ({ projectName, sessionId, evalSummary }) => {
  if (!evalSummary) return;
  const fs = await import('fs/promises');
  await fs.appendFile('eval-results.jsonl', JSON.stringify({
    projectName,
    sessionId,
    passCount: evalSummary.passCount,
    failCount: evalSummary.failCount,
    results: evalSummary.results.map(r => ({ name: r.name, pass: r.pass, score: r.score })),
    timestamp: new Date().toISOString(),
  }) + '\n');
});

​
Usage

Add alerts to your evals file alongside evals and enrichments:
my-evals.js
import { createApp } from 'claudeye';

const app = createApp();

app.eval('under-50-turns', ({ stats }) => ({
  pass: stats.turnCount <= 50,
  score: Math.max(0, 1 - stats.turnCount / 100),
}));

app.alert('warn-on-failure', ({ projectName, sessionId, evalSummary }) => {
  if (evalSummary?.failCount > 0) {
    console.warn(`[FAILURE] ${projectName}/${sessionId}`);
  }
});

app.listen();
Alerts work with both UI-triggered runs and background processing: 
claudeye --evals ./my-evals.js --queue-interval 30