Implement Data Validation Layers Between Integrations

When data flows between systems, assume it can arrive malformed -- missing fields, wrong types, unexpected nulls. Add a validation layer between source and destination that checks each record and splits output into valid and invalid streams. Valid records continue to the destination; invalid ones are logged with clear error messages for review and reprocessing.

Why validate data between integrations?

When data flows from one system to another, assume it can arrive malformed: missing fields, wrong types, unexpected nulls, strings where you expected numbers, or arrays where you expected objects. Adding a validation layer between the source and the destination catches these issues before they cause cryptic errors in the destination system or corrupt its data.

Real-world example: Your workflow receives customer data from a webhook and writes it to a CRM. The validation layer ensures every record has valid email, name, and phone before the CRM write.

How do you build a validation layer?

The Code node checks each record and returns two outputs -- valid records and invalid records:

const items = $input.all();
const valid = [];
const invalid = [];

for (const item of items) {
  const data = item.json;
  const errors = [];

  // Required field checks
  if (!data.email || typeof data.email !== 'string') {
    errors.push('Missing or invalid email');
  } else if (!/^[^\s@]+@[^\s@]+\.[^\s@]+$/.test(data.email)) {
    errors.push(`Invalid email format: ${data.email}`);
  }

  if (!data.first_name || typeof data.first_name !== 'string') {
    errors.push('Missing first_name');
  }

  if (!data.last_name || typeof data.last_name !== 'string') {
    errors.push('Missing last_name');
  }

  // Type coercion and normalization
  if (data.phone) {
    // Strip non-numeric characters
    data.phone = String(data.phone).replace(/[^0-9+]/g, '');
    if (data.phone.length < 10) {
      errors.push(`Phone number too short: ${data.phone}`);
    }
  }

  // Numeric field validation
  if (data.revenue !== undefined && data.revenue !== null) {
    const revenue = Number(data.revenue);
    if (isNaN(revenue) || revenue < 0) {
      errors.push(`Invalid revenue value: ${data.revenue}`);
    } else {
      data.revenue = revenue;  // Ensure it is a number, not string
    }
  }

  // Enum validation
  const validSources = [
    'website', 'referral', 'advertising', 'organic', 'partner'
  ];
  if (data.source && !validSources.includes(data.source)) {
    errors.push(
      `Invalid source "${data.source}". Must be one of: ${validSources.join(', ')}`
    );
  }

  if (errors.length > 0) {
    invalid.push({
      json: {
        original_data: data,
        validation_errors: errors,
        rejected_at: new Date().toISOString()
      }
    });
  } else {
    valid.push({ json: data });
  }
}

// Output 1: valid records -> CRM write
// Output 2: invalid records -> error handling
return [valid, invalid];

Wire the two outputs to their destinations:

Webhook Trigger
  |
  Code Node: Validate Data
  |
  +-> Output 1 (Valid):    CRM - Create/Update Contact
  |
  +-> Output 2 (Invalid):  Google Sheets - Log Rejections
                            + Slack Alert (if > 10% rejection rate)

Optionally alert when the rejection rate is high:

const validCount = $('Validate Data').first().json.validCount;
const invalidCount = $input.all().length;
const total = validCount + invalidCount;
const rejectionRate = ((invalidCount / total) * 100).toFixed(1);

if (parseFloat(rejectionRate) > 10) {
  return [{
    json: {
      alert: `High rejection rate: ${rejectionRate}% (${invalidCount}/${total} records rejected)`,
      sample_errors: $input.all().slice(0, 3).map(
        i => i.json.validation_errors
      )
    }
  }];
}

return [];  // No alert needed

Should you validate in both directions?

Warning: Validate in Both Directions. If you run a two-way sync (Tip 3), validate data in both directions. System A's data format assumptions may differ from System B's. Each direction needs its own validation layer tailored to the destination system's requirements.

Validation layers prevent garbage-in-garbage-out scenarios, provide clear error messages for debugging, and create an audit trail of rejected data that you can review and re-process.

Related: Use Path Parameters in Webhook URLs for Dynamic Routing · Use Edit Fields in "Map Each" Mode for Simple Renames