Sendgrid Integration

🎯 Overview

Sendgrid is a cloud-based email delivery platform that provides transactional and marketing email services. The DashClicks integration uses Sendgrid subusers to isolate email sending for each account, with comprehensive webhook-based email tracking, inbound email handling, and real-time event processing via Socket.IO.

Provider: Twilio Sendgrid (https://sendgrid.com) API Version: v3 Integration Type: API Key Authentication with Subuser Architecture

� Documentation Structure

This integration is documented across multiple focused guides:

1. Authentication & API Keys - API key management, credential setup, middleware
2. Email Sending - Transactional email API, variable replacement, attachments
3. Webhooks & Events - Event processing, inbound email parsing, real-time tracking
4. Subuser & Domain Management - Subuser lifecycle, domain configuration, DNS validation

�📁 Directory Structure

Documentation Structure:

sendgrid/
├── 📄 index.md - Overview and architecture (this file)
├── 📄 authentication.md - API key management
├── 📄 email-sending.md - Email delivery API
├── 📄 webhooks.md - Event processing
└── 📄 domain-management.md - Subuser and domain setup

Source Code Location:

• Base Path: external/Integrations/Sendgrid/
• Controllers: Controllers/ApiKeyController.js, Controllers/MailController.js, Controllers/SubUserController.js, Controllers/WebHookController.js
• Models: Models/ApiModel.js, Models/SubUserModel.js, Models/WebHookModel.js, Models/SendgridCollection.js, Models/SubUserCollection.js, Models/ContactCollection.js, Models/ConvoCollection.js, Models/WasabiConfig.js
• Providers: Providers/wasabi.js, Providers/socket.js
• Middleware: Middleware/getSendgridApiKey.js
• Routes: Routes/mail.js, Routes/subusers.js, Routes/webhook.js, Routes/apikey.js
• Entry: index.js

Shared Models Used:

• shared/models/account.js - Account with sendgrid subuser details
• shared/models/communication.js - Email tracking and history
• shared/models/conversation-prospect.js - Prospect conversations
• shared/models/contact.js - Contact management
• shared/models/user.js - User with sendgrid_emails array
• shared/models/sendgrid-key.js - API key storage
• shared/models/dnd.js - Do Not Disturb list
• shared/models/support-*.js - Support room/message/conversation models
• shared/models/onebalance-queue.js - Credit usage tracking
• shared/models/config.js - Wasabi configuration

🗄️ MongoDB Collections

📚 Detailed Schema: See Database Collections Documentation

integrations-sendgrid-keys

• Purpose: Store API keys for each DashClicks account
• Primary Use: Authenticate Sendgrid API requests on behalf of subusers

Schema:

{
  _id: ObjectId,
  account_id: ObjectId,      // DashClicks account ID
  api_key: String            // Sendgrid API key for the subuser
}

integrations-sendgrid-subusers

• Purpose: Store Sendgrid subuser details and domain configuration
• Primary Use: Manage subuser lifecycle and domain assignments

Schema:

{
  _id: ObjectId,
  account_id: ObjectId,
  sendgrid: {
    subuser: {
      username: String,      // Sendgrid subuser username
      user_id: Number,       // Sendgrid subuser ID
      email: String,         // Subuser email address
      disabled: Boolean      // Enabled/disabled status
    },
    domain_assigned: Boolean,
    domains: [{
      id: Number,            // Domain ID
      domain: String,        // Domain name (e.g., "dashclicks.com")
      subdomain: String,     // Subdomain (e.g., "mail")
      valid: Boolean,        // DNS validation status
      dns: {                 // DNS records to configure
        mail_cname: {...},
        dkim1: {...},
        dkim2: {...}
      }
    }]
  }
}

sendgrid_tracker

• Purpose: Track email status and webhook events
• Primary Use: Monitor email delivery, opens, clicks, bounces

Schema:

{
  _id: ObjectId,
  account_id: ObjectId,
  uid: ObjectId,             // User who sent the email
  msg_id: String,            // Sendgrid message ID
  status: String,            // Email status
  type: String,              // "OUTGOING" or "INCOMING"
  conv_message_id: ObjectId, // Associated conversation message
  conv_room_id: ObjectId,    // Associated conversation room
  events: [{
    event: String,           // "delivered", "open", "click", "bounce", etc.
    email: String,
    timestamp: Number,
    sg_event_id: String,
    sg_message_id: String,
    account_id: ObjectId,
    uid: ObjectId
  }]
}

communication (communications)

• Purpose: Store email communication records (inbound and outbound)
• Primary Use: Email history, threading, event tracking, attachments

Schema:

{
  _id: ObjectId,
  account_id: ObjectId,      // Required - DashClicks account
  contact_id: [ObjectId],    // Array of associated contacts
  conversation_id: [ObjectId], // Array of conversation IDs
  sent_by: ObjectId,         // User who sent the email
  sender_id: ObjectId,       // Sender ID
  module: String,            // "SENDGRID" (required)
  message_type: String,      // "EMAIL" (required)
  type: String,              // "INCOMING" or "OUTGOING"
  from: String,              // Sender email
  to: [String],              // Recipient emails
  cc: [String],              // CC recipients
  bcc: [String],             // BCC recipients
  subject: String,           // Email subject
  html: String,              // HTML content
  text: String,              // Plain text content
  body: Object,              // Complete Sendgrid request body
  headers: Object,           // Response headers from Sendgrid
  msgID: String,             // Sendgrid Message-ID header (x-message-id)
  referenceID: String,       // In-Reply-To header (for email threading)
  reply_to: Object,          // Reply-to address
  origin: String,            // Origin context (e.g., "conversations", "campaigns")
  success: Boolean,          // Delivery success status
  is_note: Boolean,          // Is internal note (default: false)
  task_id: ObjectId,         // Associated task
  mentions: [ObjectId],      // Mentioned users
  read_by: [ObjectId],       // Users who read the message
  attachments: [{            // Email attachments stored in Wasabi
    file_name: String,
    type: String,            // MIME type
    bucket: String,          // Wasabi bucket
    size: Number,            // File size in bytes
    key: String              // Wasabi storage key
  }],
  events: [{                 // Webhook events from Sendgrid
    event: String,           // "delivered", "open", "click", "bounce", etc.
    email: String,           // Recipient email
    timestamp: Number,       // Unix timestamp
    sg_event_id: String,     // Sendgrid event ID
    sg_message_id: String,   // Sendgrid message ID
    account_id: ObjectId,
    uid: ObjectId,
    useragent: String,       // User agent (for open/click events)
    ip: String,              // IP address (for open/click events)
    url: String              // Clicked URL (for click events)
  }],
  use_credits: Boolean,      // Charge account credits (default: true)
  conversation_communication_in_progress: Boolean, // Processing flag
  createdAt: Date,
  updatedAt: Date
}

Indexes:

• { reference_id: 1 } - For email threading lookups
• { reference_id: 1, message_type: 1, success: 1 } - Filter by status
• { reference_id: 1, 'events.event': 1 } - Event queries
• { _id: 1, message_type: 1, events: 1 } - Message event aggregation

conversation-prospect (conversation_prospects)

• Purpose: Prospect conversation threads
• Primary Use: Link emails to conversation UI

Schema:

{
  _id: ObjectId,
  channel_id: String,        // "prospect" (auto-set)
  title: String,             // Conversation title
  image: String,             // Avatar/image
  created_by: ObjectId,      // User who created (required)
  contact_id: ObjectId,      // Associated contact (required)
  account_id: ObjectId,      // Account ID
  users: [ObjectId],         // Users in conversation (required)
  unread_count: Object,      // { [user_id]: count }
  snooze: [ObjectId],        // Snoozed by users
  snooze_till: Object,       // { [user_id]: timestamp }
  first_seen: Object,        // { [user_id]: timestamp }
  last_contacted: Object,    // { [user_id]: timestamp }
  is_archive: Boolean,       // Archived status (default: false)
  is_open: Boolean,          // Open/closed status (default: true)
  removed_users: [ObjectId], // Users removed from conversation
  show_on_top: [ObjectId],   // Pinned for users
  last_activity: ObjectId,   // Last communication document (ref: Communication)
  createdAt: Date,
  updatedAt: Date
}

Additional Collections Used

contact (contacts)

• Purpose: Contact/prospect database
• Fields: email, parent_account, name, phone, etc.

dnd (do_not_disturb)

• Purpose: Do Not Disturb list (unsubscribed emails)
• Schema:

{
  _id: ObjectId,
  account_id: ObjectId,
  value: String,             // Email address
  type: String,              // "permanent" or "temporary"
  reason: String,            // Reason for DND
  createdAt: Date
}

onebalance-queue (onebalance_queues)

• Purpose: Track credit usage for billing
• Schema:

{
  _id: ObjectId,
  account_id: ObjectId,
  event: String,             // "email"
  additional_info: {
    communication_id: ObjectId,
    reference: ObjectId
  },
  createdAt: Date
}

config (configs)

• Purpose: Store Wasabi S3 configuration
• Type: "wasabi-download"
• Fields: EndPoint, Region, AccessKey, SecretKey, Bucket

Architecture Pattern

Sendgrid integration uses a subuser-per-account model, isolating email sending reputation and quotas for each DashClicks client.

🔐 Authentication & Configuration

Authentication Method: API Key (Master + Subuser API Keys)

Required Environment Variables: