Skip to main content

📬 Notifications Service

📖 Overview​

The Notifications Service is the centralized notification delivery system for the entire DashClicks platform. It handles all outbound communications across multiple channels:

  • Email Notifications via SendGrid
  • SMS Notifications via Twilio
  • Firebase Cloud Messaging (FCM) for web push notifications
  • Expo Push Notifications for mobile apps

Architecture: MongoDB Change Streams + Bull Queue + Scheduled Cron Jobs
Port: 5008 (HTTP endpoint for manual triggers)
Technology Stack: Node.js, Bull, Redis, MongoDB, SendGrid, Twilio, Firebase, Expo

🏗️ Architecture​

Core Components​

graph TB
    subgraph "Trigger Layer"
        CS[Change Streams<br/>MongoDB Watch]
        CRON[Cron Jobs<br/>Scheduled Tasks]
        HTTP[HTTP Endpoints<br/>Manual Triggers]
    end

    subgraph "Processing Layer"
        SVC[Service Modules<br/>19+ Modules]
        LOGIC[Business Logic<br/>Filtering & Validation]
    end

    subgraph "Queue Layer"
        NQ[NotificationQueue<br/>MongoDB Collection]
        BULL[Bull Queue<br/>Redis-backed]
    end

    subgraph "Delivery Layer"
        SG[SendGrid<br/>Email]
        TW[Twilio<br/>SMS]
        FCM_D[Firebase<br/>Web Push]
        EXPO[Expo<br/>Mobile Push]
    end

    CS --> SVC
    CRON --> SVC
    HTTP --> SVC
    SVC --> LOGIC
    LOGIC --> NQ
    NQ --> BULL
    BULL --> SG
    BULL --> TW
    BULL --> FCM_D
    BULL --> EXPO

Notification Flow Pattern​

3-Stage Process:

  1. Trigger Detection:

    • Change Streams detect MongoDB document changes in real-time
    • Cron jobs run on schedule (every 5-30 seconds for different modules)
    • HTTP endpoints receive manual triggers from other services

  2. Queue Creation:

    • Service modules validate and prepare notification data
    • Insert notification records into NotificationQueue collection
    • Queue processor polls collection every 5 seconds

  3. Delivery Execution:

    • Bull queue picks up pending notifications
    • Validates recipients (DND checks, token validation, credit checks)
    • Calls external APIs (SendGrid, Twilio, FCM, Expo)
    • Tracks delivery status and creates communication logs

🔧 Environment-Controlled Architecture​

The service uses 30+ environment flags to enable/disable modules independently, allowing fine-grained control over which notification types are active:

# Core Queue Processor (REQUIRED - processes all queued notifications)
QUEUE_ENABLED=true

# Store Notifications
STORE_ENABLED=true                    # New subscriptions, renewals, cancellations
ORDERS_REMINDER_ENABLED=true          # Order setup reminders

# CRM Notifications
QUEUE_CRM_REMINDERS_ENABLED=true      # CRM task reminders
CONTACTS_IMPORT_ENABLED=true          # Contact import completion
CONTACTS_CHANGE_ASSINGEE_ENABLED=true # Contact reassignment
DEALS_CHANGE_ASSINGEE_ENABLED=true    # Deal reassignment

# Product Notifications
INSTASITES_VIEWED_ENABLED=true        # InstaSite view tracking
INSTASITES_FAILED_ENABLED=true        # InstaSite failure alerts
INSTAREPORTS_FAILED_ENABLED=true      # InstaReport failure alerts

# Communication Notifications
FORMS_ENABLED=true                    # Form submissions
FORMS_NO_RESPONSE_ENABLED=true        # 7-day no-response follow-ups
INBOUND_ENABLED=true                  # Inbound lead tracking
INBOUND_CAMPAIGN_DISCONNECTED_ENABLED=true  # Campaign disconnections
INBOUND_LEAD_ASSIGNED_ENABLED=true    # Lead assignments
SUPPORT_DELAYED_EMAILS_ENABLED=true   # Support email delays
SUPPORT_NOTIFICATION_STREAM_ENABLED=true    # Support ticket notifications

# Marketing Notifications
FUNNEL_ENABLED=true                   # Funnel-related notifications
AUTO_REVIEW_REQUEST_ENABLED=true      # Automatic review requests

# Account Notifications
USERS_ACCEPT_INVITE=true              # User invite acceptance
ACCOUNT_SIGNUP_REMINDER=true          # Incomplete signup reminders
AFFILIATE_PAYOUT_ENABLED=true         # Affiliate payout notifications

# Project Notifications
PROJECT_ENABLED=true                  # Project tasks, approvals, completions

# HTTP Endpoints
HOOKS_ENABLED=true                    # Enable HTTP endpoints
PORT=5008                             # HTTP server port

# External Service Configuration
REDIS_HOST=localhost                  # Redis for Bull queue
REDIS_PORT=6379
SENDGRID_API_KEY=your_key             # SendGrid API key
TWILIO_ACCOUNT_SID=your_sid           # Twilio credentials
TWILIO_AUTH_TOKEN=your_token
FIREBASE_PROJECT_ID=your_project      # Firebase FCM

Pattern: Only enabled modules load on startup. This allows:

  • Running subset of modules for development/testing
  • Disabling specific notification types without code changes
  • Resource optimization by loading only necessary modules
  • Independent scaling of different notification types

📦 Service Modules​

The Notifications Service is organized into 19+ specialized modules, each handling a specific domain of notifications:

Core Services​

Queue Processor (services/queue/)​

Purpose: Bull queue processor - core delivery engine for all notifications
Trigger: Cron (every 5 seconds)
Environment: QUEUE_ENABLED=true
Flow: Poll NotificationQueue → Add to Bull → Deliver via external APIs
Supported Types: email, sms, fcm, expo-push
Critical: This module MUST be enabled for any notifications to be delivered

Store & Commerce Services​

Store Notifications (services/store/)​

Purpose: E-commerce notifications for subscriptions, invoices, and orders
Triggers:

  • Change Stream on StoreInvoice (past due invoices, renewals)
  • Change Stream on StoreSubscription (new, canceled, upgraded/downgraded)
  • Cron jobs for onboarding reminders and order setup status
    Environment: STORE_ENABLED=true, ORDERS_REMINDER_ENABLED=true
    Events:
  • Past due invoices (billing_reason: subscription_cycle, status: open)
  • Subscription renewals (status: paid)
  • New subscriptions (insert, status: active)
  • Cancellation requests (cancel_at_period_end: true)
  • Subscription upgrades/downgrades (previousPlan.id exists)
  • Order setup reminders (14-day follow-ups)

CRM Services​

Contact Notifications (services/contacts/)​

Purpose: Contact management notifications
Triggers:

  • Change Stream on contact assignee changes
  • Change Stream on contact imports
    Environment:
  • CONTACTS_CHANGE_ASSINGEE_ENABLED=true
  • CONTACTS_IMPORT_ENABLED=true
    Notifications:
  • Contact reassignment alerts to new assignee
  • Contact import completion summaries

Deal Notifications (services/deals/)​

Purpose: Deal pipeline notifications
Trigger: Change Stream on deal assignee changes
Environment: DEALS_CHANGE_ASSINGEE_ENABLED=true
Notifications: Deal reassignment alerts

CRM Reminders (services/reminders/)​

Purpose: Scheduled CRM task reminders
Trigger: Cron job (checks reminder schedules)
Environment: QUEUE_CRM_REMINDERS_ENABLED=true
Notifications: Task due date reminders, follow-up reminders

Product Services​

InstaSites Notifications (services/instasites/)​

Purpose: InstaSites product notifications
Triggers:

  • Change Stream on instasites.viewed field updates
  • Change Stream on instasites.status: failed
    Environment:
  • INSTASITES_VIEWED_ENABLED=true
  • INSTASITES_FAILED_ENABLED=true
    Notifications:
  • View tracking notifications
  • Failure alerts to account owners

InstaReports Notifications (services/instareports/)​

Purpose: InstaReports failure alerts
Trigger: Change Stream on instareports.status: failed
Environment: INSTAREPORTS_FAILED_ENABLED=true
Notifications: Failure alerts with error details

Communication Services​

Forms Notifications (services/forms/)​

Purpose: Form submission tracking and follow-ups
Triggers:

  • Change Stream on form submissions
  • Cron job for 7-day no-response follow-ups
    Environment:
  • FORMS_ENABLED=true
  • FORMS_NO_RESPONSE_ENABLED=true
    Notifications:
  • New form submission alerts
  • 7-day no-response reminders

Inbound Leads (services/inbound/)​

Purpose: Inbound lead tracking and campaign monitoring
Triggers:

  • Change Stream on new inbound leads
  • Change Stream on campaign disconnections
  • Change Stream on lead assignments
    Environment:
  • INBOUND_ENABLED=true
  • INBOUND_CAMPAIGN_DISCONNECTED_ENABLED=true
  • INBOUND_LEAD_ASSIGNED_ENABLED=true
    Notifications:
  • New lead alerts
  • Campaign disconnection warnings
  • Lead assignment notifications

Support Conversations (services/conversations/)​

Purpose: Support ticket and conversation notifications
Triggers:

  • Change Stream on support conversations
  • Cron job for delayed email sending (batches messages)
    Environment:
  • SUPPORT_NOTIFICATION_STREAM_ENABLED=true
  • SUPPORT_DELAYED_EMAILS_ENABLED=true
    Notifications:
  • New support message alerts
  • Batched email summaries (reduces email frequency)

Marketing Services​

Funnels (services/funnels/)​

Purpose: Marketing funnel notifications
Trigger: Change Stream on funnel events
Environment: FUNNEL_ENABLED=true
Notifications: Funnel-related event notifications

Review Requests (services/reviews/)​

Purpose: Automated review request system
Trigger: Cron job (every 30 seconds, checks review schedules)
Environment: AUTO_REVIEW_REQUEST_ENABLED=true
Features:

  • Configurable delay before sending (e.g., 1 day after purchase)
  • Optional reminders (repeating notifications)
  • Send-once or repeat options (configurable retry count)
  • Custom templates for email/SMS
  • Integration with Stripe, Square, Shopify, QuickBooks
    Notifications:
  • Initial review requests
  • Reminder emails/SMS for non-responders

Account & User Services​

User Invites (services/users/)​

Purpose: User invitation notifications
Trigger: Change Stream on user invite acceptance
Environment: USERS_ACCEPT_INVITE=true
Notifications: Welcome emails on invite acceptance

Account Signup Reminders (services/account/)​

Purpose: Incomplete signup follow-ups
Trigger: Cron job checking signup completion
Environment: ACCOUNT_SIGNUP_REMINDER=true
Notifications: Reminders to complete account setup

Affiliate Payouts (services/affiliates/)​

Purpose: Affiliate program notifications
Trigger: Change Stream on affiliate payout records
Environment: AFFILIATE_PAYOUT_ENABLED=true
Notifications: Payout confirmation emails

Project Management Services​

Project Notifications (services/projects/)​

Purpose: Project management notifications
Triggers:

  • Change Stream on project events
  • Cron jobs for auto-approval, auto-completion, approval reminders
    Environment: PROJECT_ENABLED=true
    Notifications:
  • Task assignment alerts
  • Approval request notifications
  • Task completion confirmations
  • Auto-approval reminders

Utility Services​

A2P Compliance (services/a2p/)​

Purpose: A2P (Application-to-Person) SMS compliance tracking
Details: Handles SMS compliance requirements

Communication Tracking (services/communication/)​

Purpose: Communication log management
Details: Tracks all sent communications for audit and reporting

Domain Management (services/domains/)​

Purpose: Domain-related notifications
Details: Lightning domain notifications

🔄 Queue Processing System​

Queue Flow​

sequenceDiagram
    participant SM as Service Module
    participant NQ as NotificationQueue<br/>(MongoDB)
    participant CRON as Queue Processor<br/>(Cron: every 5s)
    participant BULL as Bull Queue<br/>(Redis)
    participant API as External API<br/>(SendGrid/Twilio/etc)

    SM->>NQ: Insert notification record
    Note over NQ: Document stored with:<br/>type, recipient, content, etc.

    CRON->>NQ: Poll for pending notifications
    NQ-->>CRON: Return pending items

    CRON->>BULL: Add job(s) to Bull queue
    Note over BULL: Redis-backed queue<br/>with retry logic

    BULL->>BULL: Process job
    BULL->>API: Send notification
    API-->>BULL: Response

    alt Success
        BULL->>NQ: Delete from queue
        BULL->>Communication: Save communication log
    else Failure
        BULL->>NQ: Increment tries counter
        BULL->>BULL: Retry with exponential backoff
    end

NotificationQueue Schema​

The central queue collection that stores all pending notifications:

{
  _id: ObjectId,              // Used as Bull job ID
  type: String,               // 'email', 'sms', 'fcm', 'expo-push'
  origin: String,             // Module that created notification
                              // (e.g., 'reviews', 'store', 'contacts')

  sender_account: ObjectId,   // Sending account
  sender_user: ObjectId,      // Sending user (optional)

  recipient: {
    name: String,             // Full name
    first_name: String,
    last_name: String,
    email: String,            // For email notifications
    phone: String,            // For SMS notifications
    user_id: ObjectId         // For FCM/Expo notifications
  },

  content: {
    // For email:
    template_id: String,      // SendGrid template ID
    subject: String,
    body: String,             // HTML content
    additional_data: Object,  // Template variables
    additional_recipient_data: Object,
    additional_message_data: {
      from: String,           // Custom from email
      reply_to: Object,       // Reply-to address
      cc: Object,             // CC recipients
      bcc: Object,            // BCC recipients
      files: Array,           // Attachments
      in_reply_to: String,    // For email threads
      message_id: ObjectId,   // Reference to conversation message
      review_request_id: ObjectId
    },

    // For SMS:
    content: String,          // SMS text
    from: String,             // Twilio number

    // For FCM:
    title: String,
    body: String,
    click_action: String,     // URL to open on click
    data: Object,             // Custom data payload
    module: String,
    type: String,

    // For Expo:
    // Similar to FCM
  },

  check_credits: Boolean,     // Whether to check/deduct account credits
  internal_sender: Boolean,   // DashClicks internal notification (no credits)

  tries: Number,              // Retry attempt counter
  last_error: String,         // Last error message

  created_at: Date,
  updated_at: Date
}

Origin Values: contacts, deals, templates, conversations, instasites, inbound, instareports, dashclicks, analytics, projects, forms, reviews, users, sites, communication, reputation, lightning_domains, a2p_renewal, store, accounts, funnels, general

Queue Processor Configuration​

The queue processor manages job execution with retry logic:

// From services/queue/queue.js
const MAX_RETRIES = 10;
const BACKOFF_DELAY = 10000; // 10 seconds

// Poll interval
cron.schedule('*/5 * * * * *', async () => {
  // Every 5 seconds
  await processQueuedList();
});

// Job configuration
{
  jobId: notification._id.toString(),
  attempts: MAX_RETRIES,
  backoff: {
    type: 'exponential',
    delay: BACKOFF_DELAY
  },
  delay: 0  // Or custom delay for scheduled notifications
}

// Special handling for review requests
// Supports repeatable notifications with custom intervals
{
  jobId: notification._id.toString(),
  attempts: MAX_RETRIES,
  backoff: { type: 'exponential', delay: BACKOFF_DELAY },
  repeat: {
    every: reminder * 24 * 60 * 60 * 1000,  // days in milliseconds
    limit: retry || 1  // Number of repeats
  }
}

Job Processing Logic​

The queue processor handles four notification types:

Email Processing:

  1. Create sender payload from account business info
  2. Generate SendGrid payload with template/content
  3. Check DND (Do Not Disturb) list
  4. Send via SendGrid API
  5. Save communication log
  6. Delete from queue on success

SMS Processing:

  1. Fetch Twilio account credentials
  2. Generate Twilio payload
  3. Check DND list
  4. Send via Twilio API
  5. Save communication log
  6. Delete from queue on success

FCM Processing:

  1. Build notification with title, body, click action
  2. Send to Firebase Cloud Messaging
  3. Delete from queue on success

Expo Push Processing:

  1. Fetch user's Expo push tokens
  2. Generate Expo payload
  3. Validate tokens (remove invalid)
  4. Send via Expo Push API
  5. Delete from queue on success

📧 Delivery Mechanisms​

Email (SendGrid)​

Flow: NotificationQueue → Bull → SendGrid API → Communication Log

Key Components:

  • utils/senderCreate.js::generateSendgridPayload() - Payload generation
  • utilities2/src/notification.js::sendEmail() - API call
  • utils/communications.js::saveEmailCommunication() - Logging

Features:

  • Template-based emails (SendGrid dynamic templates)
  • Dynamic data injection (sender info, recipient info, custom variables)
  • DND (Do Not Disturb) list checking
  • Credit usage tracking (can use account's SendGrid or DashClicks')
  • Communication history logging
  • Reply-to, CC, BCC support
  • File attachments
  • Email threading (in-reply-to headers)

Payload Structure:

{
  from: {
    name: "User from Business Name",  // Or just business name
    email: "info@dashboardnotifications.com"  // Or custom
  },
  personalizations: [{
    to: [{ name: "Recipient Name", email: "recipient@example.com" }],
    cc: [{ email: "cc@example.com" }],  // Optional
    bcc: [{ email: "bcc@example.com" }],  // Optional
    dynamic_template_data: {
      sender: {
        name: "Owner Name",
        first_name: "John",
        last_name: "Doe",
        email: "business@example.com",
        account: "Business Name",
        address: { street, unit, city, state, zip, country },
        website: "https://example.com",
        logo: "https://cdn.example.com/logo.png",
        phone: "(123) 456-7890",
        social: { facebook, linkedin, youtube, twitter, instagram }
      },
      recipient: {
        name: "Recipient Name",
        first_name: "Jane",
        last_name: "Smith",
        email: "recipient@example.com"
      },
      // Module-specific variables
      ...additionalData
    }
  }],
  template_id: "d-1234567890abcdef",  // SendGrid template
  reply_to: { email: "reply@example.com" },  // Optional
  headers: {
    "In-Reply-To": "<message-id>",  // For threading
    "References": "<message-id>"
  },
  attachments: [{  // Optional
    filename: "document.pdf",
    content: "base64_encoded_content",
    type: "application/pdf"
  }]
}

SendGrid API Key Management:

  • Each account can have its own SendGrid API key
  • Falls back to DashClicks master key if account key invalid
  • Automatically regenerates invalid keys using master key
  • Supports SendGrid subusers for account isolation

SMS (Twilio)​

Flow: NotificationQueue → Bull → Twilio API → Communication Log

Key Components:

  • utils/senderCreate.js::generateTwilioPayload() - Payload generation
  • utilities2/src/notification.js::sendSMS() - API call
  • utils/communications.js::saveSMSCommunication() - Logging

Features:

  • DND list checking
  • Credit usage tracking
  • A2P (Application-to-Person) compliance
  • Communication history logging
  • Status callbacks for delivery tracking
  • Messaging service support (for number pools)

Payload Structure:

{
  accountSID: "AC1234567890abcdef",  // Twilio account SID
  authToken: "your_auth_token",

  to: "+12345678901",  // Recipient phone (E.164 format)

  // Option 1: Specific number
  from: "+19876543210",

  // Option 2: Messaging service (for DashClicks internal)
  messagingServiceSid: "MG1234567890abcdef",

  body: "Your SMS message content",

  statusCallback: "https://api.dashclicks.com/webhook?status=true&tracking=JWT_TOKEN"
}

Credit Handling:

  • If check_credits: true → Uses account's Twilio credentials
  • If check_credits: false → Uses DashClicks master Twilio account
  • Status callbacks include JWT token for tracking

FCM (Firebase Cloud Messaging)​

Flow: NotificationQueue → Bull → FCM API

Key Components:

  • utilities2/src/notification.js::sendFCM() - API call

Features:

  • Web push notifications (browser)
  • Bell notifications (in-app notification center)
  • Account and user targeting
  • Custom click actions
  • Data payloads for custom handling
  • Token management and validation
  • Socket emission for real-time bell notifications

Payload Structure:

{
  message: {
    token: "fcm_device_token",  // Or array of tokens
    webpush: {
      headers: {
        Urgency: "high",
        TTL: "300"  // Time to live in seconds
      },
      notification: {
        title: "Notification Title",
        body: "Notification body text",
        icon: "https://cdn.example.com/icon.png",
        click_action: "https://app.dashclicks.com/path"
      },
      data: {  // All values must be strings
        module: "contacts",
        type: "assignment",
        account_id: "507f1f77bcf86cd799439011",
        // Additional custom data
      },
      fcm_options: {
        link: "https://app.dashclicks.com/path"
      }
    }
  }
}

Notification Types:

  • Browser Push: Shows OS-level notification (requires subType: 'browser')
  • Bell Notification: Stored in database for in-app notification center (requires subType: 'bell')

Bell Notification Flow:

  1. Save to FcmNotification collection
  2. Emit via socket to bell_notification event
  3. Real-time delivery to connected users
  4. Persisted for retrieval when user comes online

Expo Push (Mobile)​

Flow: NotificationQueue → Bull → Expo Push API

Key Components:

  • utils/senderCreate.js::generateExpoPayload() - Payload generation
  • utilities2/src/notification.js::sendExpoPushNotification() - API call

Features:

  • Mobile app push notifications (iOS & Android)
  • Token validation (checks Expo.isExpoPushToken())
  • Batch delivery (chunks notifications for API limits)
  • Automatic token cleanup (removes invalid tokens)

Payload Structure:

[
  {
    to: 'ExponentPushToken[xxxxxxxxxxxxxxxxxxxxxx]',
    title: 'Notification Title',
    body: 'Notification body text',
    sound: 'default',
    data: {
      // Custom data payload
      module: 'projects',
      type: 'task_assigned',
      project_id: '507f1f77bcf86cd799439011',
    },
  },
  // ... more notifications
];

Token Management:

  • Tokens stored in ExpoPushToken collection
  • Each user can have multiple tokens (multiple devices)
  • Invalid tokens are logged and can be cleaned up
  • Supports chunked delivery for large batches

🚨 Error Handling & Retry Logic​

Retry Strategy​

The queue processor implements exponential backoff for failed deliveries:

attempts: 10  // Maximum retry attempts
backoff: {
  type: 'exponential',
  delay: 10000  // Base delay: 10 seconds
}

// Retry delays:
Attempt 1: 10 seconds
Attempt 2: 20 seconds
Attempt 3: 40 seconds
Attempt 4: 80 seconds
Attempt 5: 160 seconds (~2.7 minutes)
Attempt 6: 320 seconds (~5.3 minutes)
Attempt 7: 640 seconds (~10.7 minutes)
Attempt 8: 1280 seconds (~21.3 minutes)
Attempt 9: 2560 seconds (~42.7 minutes)
Attempt 10: 5120 seconds (~85.3 minutes)

Total time: ~2.7 hours before final failure

Failure Tracking​

On each failed attempt:

  1. Increment try counter: tries field in NotificationQueue incremented
  2. Store error message: last_error field updated with error details
  3. Log error: Error logged with full context (job ID, data, error)
  4. Retry with backoff: Job automatically retried by Bull

After MAX_RETRIES (10 attempts):

  1. Create failed communication log: Save record of failure for audit
  2. Delete from queue: Remove notification to prevent further attempts
  3. Final error log: Log final failure with all attempt details

DND (Do Not Disturb) Checks​

Before sending email or SMS, the system checks the DND list:

const dnd = await DND.find({
  account_id: sender_account,
  value: { $in: [recipient_email_or_phone] },
});

if (dnd.length) {
  // Delete from queue, don't send
  // For repeatable jobs (review reminders), only skip this attempt
  await QueueModel.deleteOne({ _id: job.id });
  return done();
}

DND List Sources:

  • User-initiated unsubscribes
  • Bounce/complaint handling
  • Admin-added blocks
  • Regulatory compliance (e.g., TCPA for SMS)

Communication Logging​

All attempts (success or failure) are logged in the Communication collection:

Successful Communication Log:

{
  user_id: ObjectId,          // Sender user
  account_id: ObjectId,       // Sender account
  module: String,             // Origin module (e.g., 'reviews')

  // Email-specific
  to: String,                 // Recipient email
  from: String,               // Sender email
  subject: String,
  body: String,               // HTML content
  sendgrid_message_id: String,

  // SMS-specific
  to_number: String,          // Recipient phone
  from_number: String,        // Twilio number
  message: String,            // SMS text
  twilio_message_id: String,

  // Tracking
  message_id: ObjectId,       // Related conversation message
  review_request_id: ObjectId,
  contact_id: ObjectId,       // Related contact

  // Status
  success: true,
  use_credits: Boolean,       // Whether credits were deducted

  created_at: Date
}

Failed Communication Log (after max retries):

{
  // Same fields as above, plus:
  success: false,
  error: String,              // Final error message
  tries: 10                   // Number of attempts
}

User Preference Verification​

Before creating notifications, modules can check user preferences:

const canNotify = await NotificationUtil.verify({
  userID: user_id,
  accountID: account_id,
  module: 'reviews', // Module name
  type: 'requests', // Notification type
  subType: 'email', // Channel (email/sms/fcm/etc)
});

if (!canNotify) {
  // User opted out, skip notification
  return;
}

Preference Hierarchy:

  1. Check if user has required scope/permissions for module
  2. Check user's notification preferences in UserConfig
  3. Default to enabled if no preference set (opt-out model)
  4. Respect explicit false settings

🔗 Integration with Other Services​

Internal API​

Relationship: Creates notification triggers and manages user preferences

Integration Points:

  • Review Requests: Internal API creates ReviewRequest documents → Notification Service sends emails/SMS
  • Deal Assignments: Internal API updates deal assignee → Change Stream triggers notification
  • Contact Imports: Internal API completes import → Change Stream triggers summary email
  • User Preferences: Internal API manages UserConfig for notification opt-out

Data Flow:

Internal API → MongoDB → Change Stream → Notification Service
Internal API → Create ReviewRequest → Cron Job → Notification Service

Queue Manager​

Relationship: Creates scheduled notifications for background jobs

Integration Points:

  • InstaReports: Queue Manager processes reports → Updates status → Change Stream triggers notification
  • InstaSites: Queue Manager builds sites → Updates status → Change Stream triggers notification
  • Analytics: Queue Manager processes data → May trigger summary notifications

Data Flow:

Queue Manager → MongoDB Status Update → Change Stream → Notification Service

Conversation Socket​

Relationship: Real-time chat triggers notification batching

Integration Points:

  • Support Messages: New support messages → Notification Service batches for email summaries
  • Message Tracking: Conversation Socket tracks unread → Notification Service sends reminders

Data Flow:

Conversation Socket → Support Message → Change Stream → Notification Service
Notification Service → Delayed Email Batch → SendGrid → Email Delivery

General Socket​

Relationship: Real-time event distribution for bell notifications

Integration Points:

  • Bell Notifications: Notification Service → Socket emission → Connected clients
  • Real-time Updates: FCM bell notifications delivered via socket

Data Flow:

Notification Service → Save Bell Notification → Emit Socket Event → General Socket → Clients

External API​

Relationship: Third-party integrations trigger notifications

Integration Points:

  • Stripe Webhooks: Subscription events → Store notifications
  • Twilio Status Callbacks: SMS delivery status → Update communication logs
  • SendGrid Webhooks: Email events (opens, clicks, bounces) → Update logs

📊 Key Models​

NotificationQueue​

Collection: notifications.queue
Purpose: Pending notification storage
Used By: All service modules (write), queue processor (read/delete)
Indexes:

  • type - For filtering by notification type
  • tries - For retry logic
  • origin - For tracking source module

Document Lifecycle:

  1. Created by service module
  2. Polled by queue processor (every 5 seconds)
  3. Added to Bull queue
  4. Deleted on successful delivery
  5. Updated with tries and last_error on failure
  6. Deleted after MAX_RETRIES

DND (Do Not Disturb)​

Collection: dnds
Purpose: Block list for emails/phones
Used By: Queue processor (email/SMS validation)
Fields:

{
  account_id: ObjectId,       // Account that owns this DND entry
  value: String,              // Email or phone number
  type: String,               // 'email' or 'phone'
  reason: String,             // 'user_request', 'bounce', 'complaint', etc.
  created_at: Date
}

Sources:

  • User unsubscribe requests
  • Bounce handling (hard bounces, spam complaints)
  • Admin-added blocks
  • Regulatory compliance

Communication​

Collection: communications
Purpose: Notification delivery logs (audit trail)
Used By: Queue processor (logging), reporting
Retention: Permanent (for compliance and reporting)

Uses:

  • Audit trail for all sent communications
  • Credit usage tracking
  • Delivery status monitoring
  • Contact last_contacted updates
  • Compliance reporting

Config​

Collection: configs
Purpose: Active notification module configuration
Document:

{
  _id: ObjectId,
  type: "notification-modules",
  active: ["email", "sms", "fcm", "expo-push"]
}

Used By: Queue processor (filtering notification types)
Purpose: Allows enabling/disabling notification channels without code changes

ExpoPushToken​

Collection: expo_push_tokens
Purpose: User device tokens for Expo push notifications
Used By: Expo notification payload generation

Fields:

{
  user_id: ObjectId,
  account_id: ObjectId,
  token: String,              // ExponentPushToken[...]
  device_info: Object,        // Optional device metadata
  created_at: Date
}

FcmToken​

Collection: fcm_tokens
Purpose: User device tokens for Firebase Cloud Messaging
Used By: FCM notification payload generation

Fields:

{
  user_id: ObjectId,
  account_id: ObjectId,
  web_token: [String],        // Array of FCM tokens for web
  created_at: Date
}

FcmNotification​

Collection: fcm_notifications
Purpose: Bell notification storage (in-app notification center)
Used By: FCM bell notification type

Fields:

{
  module: String,             // Origin module
  type: String,               // Notification type
  users: [ObjectId],          // Recipient users
  account: ObjectId,          // Account
  message: {
    title: String,
    body: String,
    data: Object,
    click_action: String
  },
  metadata: Object,           // Module-specific data
  read_by: [ObjectId],        // Users who read this
  created_at: Date
}

UserConfig​

Collection: user_configs
Purpose: User notification preferences
Used By: Notification verification (opt-out checking)

Fields:

{
  user_id: ObjectId,
  account_id: ObjectId,
  type: String,               // Module type
  preferences: {
    [module]: {
      notifications: {
        [type]: {
          email: Boolean,     // true/undefined = enabled, false = disabled
          sms: Boolean,
          browser: Boolean,
          bell: Boolean
        }
      }
    }
  }
}

🎯 Common Patterns​

Change Stream Pattern​

All change stream-based modules follow this pattern:

// From services/store/changeStream.js
const stream = Model.watch(
  [
    {
      $match: {
        // Filter for specific operations and conditions
        operationType: 'update',
        'fullDocument.status': 'active',
        'updateDescription.updatedFields.field': { $exists: true },
      },
    },
    {
      $addFields: {
        // Add metadata flags for notification types
        'metadata.notification_type': {
          $cond: [
            {
              $and: [
                /* conditions */
              ],
            },
            true,
            false,
          ],
        },
      },
    },
  ],
  {
    fullDocument: 'updateLookup', // Include full document
    startAtOperationTime: RESUMETIME || undefined, // Resume from last run
  },
);

stream.on('change', async data => {
  await processNotifications(data);
});

Key Elements:

  • $match stage: Filter relevant operations
  • $addFields stage: Add metadata flags for notification routing
  • fullDocument: 'updateLookup': Always include full document state
  • startAtOperationTime: Resume from last timestamp (prevents duplicates on restart)
  • Error handling: Automatic reconnection on disconnect

Cron Job Pattern​

All cron-based modules follow this pattern:

// From index.js
let inProgress = false;
cron.schedule('*/30 * * * * *', async () => {
  if (!inProgress) {
    inProgress = true;
    try {
      await processNotifications();
    } catch (err) {
      logger.error({ error: err });
    } finally {
      inProgress = false;
    }
  }
});

Key Elements:

  • inProgress flag: Prevents overlapping executions
  • Try-catch: Ensures errors don't crash the service
  • Finally block: Always resets inProgress flag

Queue Creation Pattern​

All modules create notifications using this pattern:

// Service module creates notification
await NotificationQueue.create({
  type: 'email', // or 'sms', 'fcm', 'expo-push'
  origin: 'reviews', // Source module

  sender_account: account_id,
  sender_user: user_id, // Optional

  recipient: {
    name: contact.name,
    first_name: contact.first_name,
    last_name: contact.last_name,
    email: contact.email, // For email
    phone: contact.phone, // For SMS
    user_id: user._id, // For FCM/Expo
  },

  content: {
    template_id: 'd-1234567890abcdef', // For email
    subject: 'Your Subject',
    additional_data: {
      // Template variables
    },
    additional_message_data: {
      // Tracking data
      message_id: conversation_message_id,
      review_request_id: review_request_id,
    },
  },

  check_credits: true, // Deduct credits
  internal_sender: false, // Not DashClicks internal
});

Notification Helper Functions​

Modules use helper functions for consistency:

// From utils/notification.js
await processEmailv2({
  verification: {
    module: 'reviews',
    type: 'requests',
    subType: 'email',
  },
  recipient: {
    accountID: account_id,
    users: [contact1, contact2], // Array of contacts
  },
  content: {
    sender_user: user_id,
    subject: 'Subject',
    body: '<html>...</html>', // Or template_id
  },
  user_check: false, // Skip user preference check
  check_credits: true,
  additional_message_data: {
    /* tracking */
  },
});

await processSMSv2({
  verification: { module, type, subType },
  recipient: { accountID, users },
  data: {
    content: 'SMS text',
    from: '+19876543210',
  },
  sender_user: user_id,
  user_check: false,
  check_credits: true,
  additional_message_data: {
    /* tracking */
  },
});

📈 Statistics​

Service Modules: 19+ specialized modules
Environment Flags: 30+ configuration options
Notification Types: 4 (email, SMS, FCM, Expo)
Max Retries: 10 attempts per notification
Queue Poll Interval: 5 seconds
Base Backoff Delay: 10 seconds
Total Retry Time: ~2.7 hours before final failure
Supported External APIs: SendGrid, Twilio, Firebase, Expo
Change Streams: 15+ active MongoDB watch streams
Cron Jobs: 20+ scheduled tasks

🐛 Troubleshooting​

Issue: Notifications Not Being Sent​

Symptoms: Users report not receiving emails/SMS

Diagnosis Steps:

  1. Check environment flags:

    # Ensure required flags are enabled
    QUEUE_ENABLED=true  # REQUIRED for all notifications
    [MODULE]_ENABLED=true  # Module-specific flag

  2. Check NotificationQueue:

    // Check if notifications are being created
    db.getCollection('notifications.queue').find({
      sender_account: ObjectId('account_id'),
      created_at: { $gte: new Date(Date.now() - 3600000) }, // Last hour
    });

  3. Check service logs:

    # Look for "RUNNING" messages
    grep "SERVICE RUNNING" notifications.log

    # Look for errors
    grep "error" notifications.log

  4. Check Redis connection:

    redis-cli -h $REDIS_HOST -p $REDIS_PORT ping
    # Should return: PONG

  5. Check external API credentials:

    // For SendGrid
    const key = await SendgridKey.findOne({ account_id });

    // For Twilio
    const account = await Account.findById(account_id, { twilio_account: 1 });

Issue: Notifications Delayed​

Symptoms: Notifications arrive late

Causes:

  1. Queue backlog: Too many pending notifications

    // Check queue size
    db.getCollection('notifications.queue').count();

  2. Retry delays: Notifications in exponential backoff

    // Check retrying notifications
    db.getCollection('notifications.queue').find({ tries: { $gte: 1 } });

  3. External API rate limits: SendGrid/Twilio throttling

    • Check API dashboard for rate limit errors
    • Consider upgrading API plan

Solutions:

  • Scale up Redis for Bull queue
  • Add more notification service instances
  • Increase queue processor poll frequency (reduce from 5s to 3s)
  • Implement queue prioritization for critical notifications

Issue: Duplicate Notifications​

Symptoms: Users receive same notification multiple times

Causes:

  1. Repeatable job configuration: Review requests configured to repeat

    // Check auto review request settings
    db.getCollection('reviews.auto_requests').findOne({
      account_id: ObjectId('account_id'),
    });

  2. Multiple service instances: Same change stream event processed twice

    • Ensure startAtOperationTime is properly configured
    • Check for service restart loops

  3. Failed deletion: Notification not deleted after success

    • Check for transaction errors in logs
    • Verify MongoDB connection stability

Solutions:

  • Review repeat configuration for review requests
  • Implement idempotency checks using message IDs
  • Ensure proper transaction handling
  • Add duplicate detection based on content/recipient/timestamp

Issue: Notifications Failing with Errors​

Symptoms: High failure rate in logs

Common Errors:

SendGrid 401 Unauthorized:

// API key expired or invalid
// Solution: Regenerate key (automatically handled)
// Check: Account.sendgrid.subuser.username

Twilio 21211 Invalid 'To' Number:

// Phone number not in E.164 format
// Solution: Validate phone format before queuing
// Use: awesome-phonenumber library

FCM Invalid Token:

// Token expired or unregistered
// Solution: Remove invalid tokens from FcmToken collection

Expo DeviceNotRegistered:

// Token no longer valid
// Solution: Remove from ExpoPushToken collection

DND Check Blocking:

// Email/phone in DND list
// Solution: User must unsubscribe or admin remove
// Check: DND.find({ value: email_or_phone })

Issue: Credits Not Being Deducted​

Symptoms: Notifications sent but credits unchanged

Check:

  1. check_credits flag:

    // Ensure flag is true in NotificationQueue
    {
      check_credits: true;
    }

  2. internal_sender flag:

    // Ensure internal_sender is false
    {
      internal_sender: false;
    }

  3. Account balance:

    // Check OneBalance credits
    db.getCollection('onebalance').findOne({
      account_id: ObjectId('account_id'),
    });

📝 Best Practices​

For Service Module Development​

  1. Always use helper functions: processEmailv2(), processSMSv2()
  2. Include user verification: Check preferences before creating notifications
  3. Add comprehensive metadata: Include all tracking IDs for audit trail
  4. Handle errors gracefully: Catch and log errors, don't crash service
  5. Use transactions: Ensure atomic operations for critical updates
  6. Test DND scenarios: Verify DND list is respected
  7. Implement resume tokens: Use startAtOperationTime for change streams

For Notification Content​

  1. Use templates: SendGrid dynamic templates for consistency
  2. Personalize: Include recipient name, sender info
  3. Mobile-friendly: Ensure emails render well on mobile
  4. Clear CTAs: Make action items obvious
  5. Unsubscribe links: Include in all marketing emails
  6. Test thoroughly: Send test notifications before production

For Operations​

  1. Monitor queue size: Alert if queue grows beyond threshold
  2. Track delivery rates: Monitor success/failure ratios
  3. Review logs regularly: Check for recurring errors
  4. Update API keys: Rotate credentials periodically
  5. Scale proactively: Add instances before hitting limits
  6. Backup configuration: Document all environment flags

🔗 Related Documentation​

  • Queue Manager Documentation - Background job processing
  • Internal API Communication Endpoints (link removed - file does not exist) - Conversation API
  • Shared Models - MongoDB schema reference
  • External Integrations: SendGrid (link removed - file does not exist) - Email delivery
  • External Integrations: Twilio (link removed - file does not exist) - SMS delivery

📚 Additional Resources​

External Documentation​

Internal Resources​

  • Notification Templates (link removed - file does not exist)
  • Credit System Documentation (link removed - file does not exist)
  • User Preferences API (link removed - file does not exist)

Service Type: Background Processing + HTTP Endpoint
Port: 5008 (HTTP endpoints)
Dependencies: MongoDB, Redis, SendGrid, Twilio, Firebase, Expo
Status: Core Service - handles all platform notifications
Maintainer: Platform Team
Last Updated: October 2025

💬

Documentation Assistant

Ask me anything about the docs

Hi! I'm your documentation assistant. Ask me anything about the docs!

I can help you with:
- Code examples
- Configuration details
- Troubleshooting
- Best practices

Try asking: How do I configure the API?
09:31 AM