Replaying Failed Paystack Webhooks Safely

Webhook replay is a recovery tool, not a routine operation. Here are the scenarios that require it:

Deployment broke the handler. You deployed a code change that introduced a bug in your webhook processing logic. Events were received and stored, but processing failed or produced incorrect results. After fixing the bug, you need to replay the events that arrived during the broken window.

Infrastructure outage. Your database went down for 30 minutes. Webhooks arrived (and were stored in the queue or a separate store), but processing failed because the database was unreachable. After the database recovers, the events in your queue get processed, but any that exhausted their retries during the outage need manual replay.

Missing processing for a specific event type. You discover that your handler never had a case for "transfer.reversed" events. Transfers have been reversed over the past two weeks, but your database does not reflect it. You need to find all stored transfer.reversed events and replay them through your newly-added handler.

Data corruption recovery. A bug corrupted the processed state of certain transactions. After fixing the data, you replay the relevant webhooks to re-derive the correct state from the original Paystack events.

In all these cases, the prerequisite is the same: you must have stored the raw webhook payloads. If you did not, your options are limited to the Paystack dashboard (manual reconciliation) or the Transaction Verify API (polling each transaction individually). See Storing Raw Webhook Payloads for Audit for the storage pattern.

Before you replay a single event, your handler must be idempotent. This means processing the same event twice produces the same result as processing it once. No double charges, no duplicate emails, no repeated credits.

The standard pattern is to check the processing state before doing any work:

async function handleChargeSuccess(data) {
  const reference = data.reference;

  // Check: was this already processed?
  const order = await db.query(
    'SELECT status FROM orders WHERE reference = $1',
    [reference]
  );

  if (order.rows.length > 0 && order.rows[0].status === 'paid') {
    console.log('Already processed, skipping: ' + reference);
    return; // Idempotent exit
  }

  // Process the payment
  await db.query(
    'UPDATE orders SET status = $1, paid_at = NOW() WHERE reference = $2',
    ['paid', reference]
  );

  // Send confirmation (also needs idempotency)
  await sendReceiptIfNotSent(reference);
}

Every side effect must be idempotent. If processing a charge.success event sends a receipt email, the email-sending function must check whether a receipt was already sent for this reference. If processing credits a wallet, the wallet credit function must check whether the credit was already applied.

If your handlers are not idempotent and you cannot make them idempotent quickly, do not replay. Instead, manually inspect each failed event and process them one by one with manual verification. This is slower but prevents damage.

For the full idempotency pattern, see Idempotent Webhook Handlers: The Complete Pattern.

The simplest replay is processing one event at a time. Read the stored payload from your database and feed it through your processing function:

async function replayWebhookEvent(eventId) {
  // 1. Fetch the stored event
  const result = await db.query(
    'SELECT id, event_type, raw_body, signature, processed FROM webhook_events WHERE id = $1',
    [eventId]
  );

  if (result.rows.length === 0) {
    throw new Error('Event not found: ' + eventId);
  }

  const event = result.rows[0];

  // 2. Safety check: warn if already processed
  if (event.processed) {
    console.warn('Event ' + eventId + ' was already processed. Re-processing due to replay.');
  }

  // 3. Verify signature integrity
  const secret = process.env.PAYSTACK_SECRET_KEY;
  const rawBody = JSON.stringify(event.raw_body);
  const expectedHash = require('crypto')
    .createHmac('sha512', secret)
    .update(rawBody)
    .digest('hex');

  if (expectedHash !== event.signature) {
    throw new Error('Signature mismatch for event ' + eventId + '. Payload may be tampered.');
  }

  // 4. Process the event
  await processWebhookEvent(event.raw_body);

  // 5. Mark as processed
  await db.query(
    'UPDATE webhook_events SET processed = TRUE, processed_at = NOW() WHERE id = $1',
    [eventId]
  );

  console.log('Replayed event ' + eventId + ': ' + event.event_type);
}

The signature verification in step 3 is important. It confirms that the payload in your database is the same one Paystack originally sent. If someone modified the payload in the database (accidentally or maliciously), the signature will not match, and you should not process it.

Note: the signature re-verification only works if you stored the raw body in the exact format Paystack sent it. If you parsed it to JSON and stored the JSON object, re-serializing it might produce a different string (different key ordering, different whitespace). This is one reason to store the raw body as a string alongside the parsed JSONB version.

After an outage, you may need to replay hundreds or thousands of events. The key constraint is ordering: events should be replayed in the order they were received. Processing a refund before the original charge, or a subscription cancellation before the subscription creation, can leave your data in an inconsistent state.

async function replayFailedEvents(startTime, endTime) {
  // Fetch all unprocessed events in chronological order
  const result = await db.query(
    'SELECT id, event_type, raw_body ' +
    'FROM webhook_events ' +
    'WHERE processed = FALSE ' +
    '  AND received_at BETWEEN $1 AND $2 ' +
    'ORDER BY received_at ASC',
    [startTime, endTime]
  );

  console.log('Found ' + result.rows.length + ' events to replay');

  let success = 0;
  let failed = 0;

  for (const event of result.rows) {
    try {
      await processWebhookEvent(event.raw_body);

      await db.query(
        'UPDATE webhook_events SET processed = TRUE, processed_at = NOW() WHERE id = $1',
        [event.id]
      );

      success++;
    } catch (err) {
      await db.query(
        'UPDATE webhook_events SET error_message = $1 WHERE id = $2',
        ['Replay failed: ' + err.message, event.id]
      );

      failed++;
      console.error('Replay failed for event ' + event.id + ': ' + err.message);
    }
  }

  console.log('Replay complete. Success: ' + success + ', Failed: ' + failed);
  return { success, failed };
}

Process events sequentially (one at a time), not in parallel. Parallel processing can cause race conditions when two related events for the same customer are processed simultaneously. Sequential processing is slower but safer.

If you have thousands of events to replay and sequential processing is too slow, you can parallelize by customer or by reference. Group events by customer ID or transaction reference, process each group sequentially, but process different groups in parallel. This maintains ordering within a customer's events while speeding up the overall replay.

Build the replay tool before you need it. When an outage happens at 2 AM, you want to run a command, not write one. A good replay tool supports three modes:

1. Replay a single event by ID. For targeted recovery of specific events identified during investigation.

2. Replay all unprocessed events in a time range. For outage recovery where you know when the outage started and ended.

3. Replay all events of a specific type in a time range. For cases where you added a new handler and need to replay historical events.

Here is a CLI tool that supports all three modes:

// scripts/replay-webhooks.js
const { program } = require('commander');
const db = require('../config/database');
const { processWebhookEvent } = require('../handlers/webhook');

program
  .command('single <eventId>')
  .description('Replay a single webhook event')
  .action(async (eventId) => {
    await replayWebhookEvent(parseInt(eventId));
    process.exit(0);
  });

program
  .command('range')
  .description('Replay unprocessed events in a time range')
  .requiredOption('--start <datetime>', 'Start time (ISO 8601)')
  .requiredOption('--end <datetime>', 'End time (ISO 8601)')
  .option('--event-type <type>', 'Filter by event type')
  .option('--dry-run', 'Show what would be replayed without processing')
  .action(async (options) => {
    if (options.dryRun) {
      const count = await countEventsToReplay(options.start, options.end, options.eventType);
      console.log('Would replay ' + count + ' events');
    } else {
      await replayFailedEvents(options.start, options.end, options.eventType);
    }
    process.exit(0);
  });

program.parse();

The --dry-run flag is essential. Before replaying 500 events, you want to know how many events match your criteria. A dry run shows the count without processing anything. Always dry run first.

Log every replay action with the operator's identity (who ran the command) and a timestamp. This creates an audit trail showing that the replay was intentional and authorized, not accidental or malicious.

Replaying is powerful, and power needs guardrails. Build these checks into your replay tool:

1. Signature re-verification. Before processing a stored payload, re-compute the HMAC signature and compare it to the stored signature. If they do not match, the payload may have been modified in the database. Do not process it. Investigate first.

2. Time window limits. Do not allow replaying events older than a configurable threshold (for example, 30 days) without explicit confirmation. Old events are more likely to conflict with the current state of your data. A charge.success from two weeks ago might reference an order that has already been refunded, cancelled, or fulfilled through other means.

3. Maximum batch size. Cap the number of events that can be replayed in a single batch (for example, 1,000 events). If your query returns 50,000 unprocessed events, something is seriously wrong and needs investigation before bulk replaying.

4. Idempotency verification. Before starting a batch replay, run a quick test: take the first event in the batch and check whether your handler correctly skips it if it has already been processed. If your handler does not have idempotency logic, abort the replay.

5. Rate limiting. Do not replay events as fast as your database can handle. Add a small delay between events (100ms to 500ms) to avoid overwhelming your database or downstream services. Payment processing services often have rate limits, and hitting them during a replay will cause cascading failures.

async function replayWithSafetyChecks(eventId) {
  const event = await fetchEvent(eventId);

  // Check 1: Age limit
  const ageInDays = (Date.now() - new Date(event.received_at).getTime()) / (1000 * 60 * 60 * 24);
  if (ageInDays > 30) {
    throw new Error(
      'Event is ' + Math.round(ageInDays) + ' days old. Use --force to replay old events.'
    );
  }

  // Check 2: Signature integrity
  const isValid = verifySignature(event.raw_body, event.signature);
  if (!isValid) {
    throw new Error('Signature verification failed. Payload may be corrupted.');
  }

  // Check 3: Already processed warning
  if (event.processed) {
    console.warn('WARNING: Event was already processed. Re-processing.');
  }

  // Process
  await processWebhookEvent(event.raw_body);
}

Paystack's dashboard lets you resend individual webhook events. This is useful for quick tests but has significant limitations for production recovery:

• One at a time. You cannot batch-resend events from the dashboard. If 200 events failed, you need to click through each one individually.
• No filtering. You cannot filter by event type, time range, or processing status from Paystack's side. You have to find each event manually.
• No ordering control. If you resend events out of order, you can create data inconsistencies.
• Rate concerns. Resending from Paystack triggers the full webhook delivery flow, including signature computation and HTTP delivery. Your own replay skips the network round-trip and goes straight to processing.

Your own replay tool is better in every dimension except one: if you did not store the raw payloads, Paystack's dashboard is your only option. This is another reason why storing raw payloads is not optional for a production integration.

Use Paystack's dashboard resend for one-off debugging: "Did my handler process this specific event correctly?" Use your own replay tool for recovery operations: "Replay everything that failed between 3 PM and 5 PM yesterday."

After replaying events, verify that the replay produced the expected results. Do not assume success just because the replay script completed without errors.

1. Check the processed count. The replay script should report how many events were successfully processed and how many failed. If any failed, investigate those individually.

2. Spot-check specific transactions. Pick 5 to 10 transactions from the replay batch and manually verify them. Check that the order status, wallet balance, or subscription state in your database matches what you would expect based on the Paystack event.

3. Run reconciliation. After a large replay, run your reconciliation job to compare your database against Paystack's transaction records. Any remaining discrepancies after the replay need individual investigation.

4. Check for side effects. If your webhook handler sends emails, SMS messages, or push notifications, verify that the replay did not send duplicate notifications to customers. Idempotent handlers should prevent this, but verify anyway. A customer getting 50 "payment confirmed" emails because you bulk-replayed events is a bad experience.

-- Post-replay verification query
SELECT
  COUNT(*) as total_in_range,
  SUM(CASE WHEN processed = TRUE THEN 1 ELSE 0 END) as processed,
  SUM(CASE WHEN processed = FALSE THEN 1 ELSE 0 END) as still_unprocessed
FROM webhook_events
WHERE received_at BETWEEN '2026-07-15 14:00:00' AND '2026-07-15 17:00:00';

If the "still_unprocessed" count is greater than zero after the replay, those events need individual attention. They likely hit permanent errors (missing records, invalid data) that cannot be resolved by simply re-running the handler.