Email Sending API

📧 Email Sending API

Send Email Endpoint

Endpoint: POST /v1/integrations/sendgrid/mail

Status: ⚠️ Currently disabled for maintenance (returns 503)

Middleware Chain:

1. verifyAuthorization() - JWT validation
2. verifyAccessAndStatus() - Account status check
3. upload.array('attachments', 12) - File upload (max 12 files, 2MB limit per field)
4. validateRequestSchemaV2() - Request validation
5. getSendgridApiKey - API key injection

📝 Request Format

Request Body:

{
  "subject": "Test Email",           // Required - supports {{variables}}
  "sender": {                        // Required
    "email": "sender@dashclicks.com",
    "name": "John Doe"
  },
  "recipients": [{                   // Required - min 1 recipient
    "email": "recipient@example.com",
    "name": "Jane Smith",
    "recipient_id": "507f1f77bcf86cd799439011"  // Optional: contact ID for tracking
  }],
  "cc": [{                           // Optional
    "email": "cc@example.com",
    "name": "CC Recipient",
    "recipient_id": "507f..."        // Optional
  }],
  "bcc": [{                          // Optional
    "email": "bcc@example.com",
    "name": "BCC Recipient",
    "recipient_id": "507f..."        // Optional
  }],
  "reply_to": {                      // Optional
    "email": "reply@dashclicks.com",
    "name": "Support Team"
  },
  "content": [{                      // Required if no template_id
    "type": "text/html",             // "text/html" or "text/plain"
    "value": "<p>Email content with {{contact.name}} and {{business.name}}</p>"
  }],
  "origin": "conversations",         // Required - tracking context
  "template_id": "d-1234567890",     // Optional: Sendgrid dynamic template
  "dynamic_template_data": {         // Required if template_id present
    "variable": "value",
    "name": "John",
    "custom_field": "data"
  },
  "person_id": "507f...",            // Optional: For {{person.*}} variables
  "business_id": "507f...",          // Optional: For {{business.*}} variables
  "instasite_id": "507f...",         // Optional: For {{instasite.*}} variables
  "instareport_id": "507f...",       // Optional: For {{instareport.*}} variables
  "fallback_values": "{\"name\":\"Customer\"}" // Optional: JSON string of fallback values
}

File Upload:

• Field name: attachments
• Max files: 12
• Max size: 2MB per field
• Supported: All MIME types
• Attachments uploaded to Wasabi S3 and linked in Communication document

🔄 Variable Replacement System

The updateMessage() utility processes content and subject to replace variables:

Supported Variables

Contact Variables:

• {{contact.name}}
• {{contact.email}}
• {{contact.phone}}

Person Variables:

• {{person.first_name}}
• {{person.last_name}}
• {{person.email}}

Business Variables:

• {{business.name}}
• {{business.address}}
• {{business.phone}}

Instasite Variables:

• {{instasite.url}}
• {{instasite.name}}

Instareport Variables:

• {{instareport.url}}
• {{instareport.link}}

User Variables:

• {{user.name}}
• {{user.email}}

Variable Resolution Priority

1. Fetch from Database: Query using provided IDs (person_id, business_id, etc.)
2. Use Fallback Values: Apply fallback_values if database lookup fails
3. Leave Unchanged: Keep placeholder if no fallback available

Example:

// Input
const subject = 'Hello {{contact.name}}, welcome to {{business.name}}!';
const content = '<p>Dear {{contact.name}},</p><p>Visit us at {{instasite.url}}</p>';

// After replacement
const processedSubject = 'Hello Jane Smith, welcome to ABC Company!';
const processedContent = '<p>Dear Jane Smith,</p><p>Visit us at https://abc.instasite.com</p>';

✅ Pre-Send Validations

1. DND Check

Purpose: Ensure recipients are not on Do Not Disturb list

Query:

const dndList = await DND.find({
  account_id: ObjectId(account_id),
  value: { $in: recipients.map(r => r.email) },
});

if (dndList.length > 0) {
  throw {
    message: 'DND enabled for received email(s).',
    errno: 400,
  };
}

DND Collection Schema:

{
  _id: ObjectId,
  account_id: ObjectId,
  value: String,             // Email address
  type: String,              // "permanent" or "temporary"
  reason: String,            // Reason for DND
  createdAt: Date
}

2. Balance Verification

Purpose: Check if account has sufficient email credits

Call:

await verifyBalance({
  account: account_doc,
  event: 'email',
  user_id: uid,
  quantity: recipients.length,
});

Error Response:

{
  "success": false,
  "errno": 400,
  "message": "Insufficient balance",
  "additional_info": {
    "required": 5,
    "available": 2
  }
}

3. Contact Lookup

Purpose: Link email to existing contacts and conversations

Query:

const contacts = await ContactCollection.findContactsByEmail(
  recipients.map(r => ({ email: r.email, recipientID: r.recipient_id })),
  account_id,
);

Creates:

• Links to existing contacts
• Creates conversation threads automatically
• Associates emails with contact records

📤 Processing Flow

1. Variable Replacement

// Process subject
const processedSubject = await updateMessage({
  message: subject,
  person_id,
  business_id,
  instasite_id,
  instareport_id,
  fallback_values: JSON.parse(fallback_values || '{}'),
  user_id: uid,
});

// Process each content block
const processedContent = await Promise.all(
  content.map(async (block) => ({
    ...block,
    value: await updateMessage({
      message: block.value,
      person_id,
      business_id,
      instasite_id,
      instareport_id,
      fallback_values: JSON.parse(fallback_values || '{}'),
      user_id: uid,
    }),
  })),
);
```### 2. Attachment Handling

**Upload to Wasabi**:

```javascript
const wasabiConfig = await WasabiConfig.findConfig();

const uploadedAttachments = await wasabiProvider.upload(wasabiConfig, req.files);

// Result format
[
  {
    bucket: 'dashclicks-wasabi',
    type: 'application/pdf',
    file_name: 'document.pdf',
    size: 12345,
    key: 'uuid/document.pdf',
  },
];

Encode for Sendgrid:

const sendgridAttachments = req.files.map(file => ({
  filename: file.originalname,
  content: file.buffer.toString('base64'),
  type: file.mimetype,
}));

3. Sendgrid Request Construction

const mailData = {
  personalizations: [
    {
      to: recipients,
      cc: cc,
      bcc: bcc,
      subject: processedSubject,
      custom_args: {
        account_id: account_id,
        uid: uid,
        origin: origin,
      },
      headers: {
        'X-Account-ID': account_id,
        'X-User-ID': uid,
        'X-Email-Origin': origin,
      },
      dynamic_template_data: dynamic_template_data, // If template_id present
    },
  ],
  from: sender,
  reply_to: replyTo,
  content: processedContent, // Omitted if template_id present
  template_id: template_id, // Optional
  attachments: sendgridAttachments,
};

4. API Call

Sendgrid API Request:

POST https://api.sendgrid.com/v3/mail/send
Authorization: Bearer SG.abc123xyz...
Content-Type: application/json

{
  "personalizations": [{
    "to": [{"email": "recipient@example.com", "name": "Jane Smith"}],
    "subject": "Welcome to DashClicks, Jane!",
    "custom_args": {
      "account_id": "507f191e810c19729de860ea",
      "uid": "507f191e810c19729de860eb",
      "origin": "conversations"
    }
  }],
  "from": {"email": "sender@dashclicks.com", "name": "John Doe"},
  "content": [{
    "type": "text/html",
    "value": "<p>Hello Jane Smith!</p>"
  }],
  "attachments": [{
    "filename": "welcome.pdf",
    "content": "JVBERi0xLjQK...",
    "type": "application/pdf"
  }]
}

Extract Message ID:

const response = await sendgridClient.send(mailData);
const messageId = response[0].headers['x-message-id'];

5. Tracking Document Creation

const communication = await Communication.create({
  sent_by: senderID,
  origin: origin,
  account_id: account_id,
  headers: response.headers,
  body: {
    ...mailData,
    attachments: [], // Removed to reduce document size
  },
  msgID: messageId,
  type: 'OUTGOING',
  module: 'SENDGRID',
  message_type: 'EMAIL',
  contact_id: contactIDs,
  conversation_id: conversationIDs,
  attachments: uploadedAttachments, // Wasabi metadata only
  success: true,
});

6. Conversation Management

For Each Contact:

1. Find or Create Conversation:

   let conversation = await ConvoCollection.findConvo(contact._id, account_id);
   
   if (!conversation) {
     conversation = await ConvoCollection.save({
       channel_id: 'prospect',
       title: contact.name || contact.email,
       created_by: uid,
       contact_id: contact._id,
       account_id: account_id,
       users: [uid],
       is_open: true,
     });
   }

2. Update Conversation:

   await ConversationProspect.findByIdAndUpdate(conversation._id, {
     $set: {
       last_activity: communication._id,
       'last_contacted.' + uid: new Date(),
     },
     $inc: {
       ['unread_count.' + uid]: 1,
     },
     $addToSet: {
       users: uid,
     },
   });

3. Emit Socket Events:

   await socketProvider.emitConversation(conversation._id, 'new');
   await socketProvider.emitMessage(communication._id);

7. Credit Tracking

await OnebalanceQueue.create({
  account_id: account_id,
  event: 'email',
  additional_info: {
    communication_id: communication._id,
    reference: communication._id,
  },
});

📨 Response Format

Success Response:

{
  "success": true,
  "message": "SUCCESS",
  "data": {
    "_id": "507f1f77bcf86cd799439011",
    "sent_by": {
      "name": "John Doe",
      "email": "john@example.com",
      "first_name": "John",
      "last_name": "Doe",
      "phone": "+1234567890",
      "image": "https://..."
    },
    "origin": "conversations",
    "account_id": "507f191e810c19729de860ea",
    "headers": {
      "x-message-id": "abc123.filter456.789.012@sendgrid.net"
    },
    "body": {
      /* Sendgrid request body */
    },
    "msgID": "abc123.filter456.789.012@sendgrid.net",
    "type": "OUTGOING",
    "module": "SENDGRID",
    "message_type": "EMAIL",
    "contact_id": ["507f..."],
    "conversation_id": ["507f..."],
    "attachments": [
      {
        "bucket": "dashclicks-wasabi",
        "type": "application/pdf",
        "file_name": "document.pdf",
        "size": 12345,
        "key": "uuid/document.pdf"
      }
    ],
    "createdAt": "2025-10-10T10:00:00.000Z",
    "modifiedAt": "2025-10-10T10:00:00.000Z"
  }
}

🚨 Error Responses

DND Error

{
  "success": false,
  "errno": 400,
  "message": "DND enabled for received email(s)."
}

Insufficient Balance

{
  "success": false,
  "errno": 400,
  "message": "Insufficient balance",
  "additional_info": {
    "required": 5,
    "available": 2
  }
}

API Key Missing

{
  "success": false,
  "errno": 403,
  "message": "Sendgrid api key is misconfigured.",
  "error": "SENDGRID_API_MISCONFIGURED"
}

Invalid Recipients

{
  "success": false,
  "errno": 400,
  "message": "\"recipients\" must contain at least 1 items"
}

🧪 Testing

Basic Email Send

curl -X POST http://localhost:5003/v1/e/sendgrid/mail \
  -H "Authorization: Bearer {jwt_token}" \
  -H "Content-Type: application/json" \
  -d '{
    "subject": "Test Email",
    "sender": {
      "email": "test@dashclicks.com",
      "name": "Test User"
    },
    "recipients": [{
      "email": "recipient@example.com",
      "name": "Recipient"
    }],
    "content": [{
      "type": "text/html",
      "value": "<p>Test content</p>"
    }],
    "origin": "conversations"
  }'

Email with Attachments

curl -X POST http://localhost:5003/v1/e/sendgrid/mail \
  -H "Authorization: Bearer {jwt_token}" \
  -F "subject=Test Email with Attachments" \
  -F 'sender={"email":"test@dashclicks.com","name":"Test User"}' \
  -F 'recipients=[{"email":"recipient@example.com","name":"Recipient"}]' \
  -F 'content=[{"type":"text/html","value":"<p>Test content</p>"}]' \
  -F 'origin=conversations' \
  -F "attachments=@document.pdf" \
  -F "attachments=@image.jpg"

Email with Variables

curl -X POST http://localhost:5003/v1/e/sendgrid/mail \
  -H "Authorization: Bearer {jwt_token}" \
  -H "Content-Type: application/json" \
  -d '{
    "subject": "Hello {{contact.name}}!",
    "sender": {
      "email": "test@dashclicks.com",
      "name": "Test User"
    },
    "recipients": [{
      "email": "recipient@example.com",
      "name": "Jane Smith",
      "recipient_id": "507f1f77bcf86cd799439011"
    }],
    "content": [{
      "type": "text/html",
      "value": "<p>Dear {{contact.name}}, welcome to {{business.name}}!</p>"
    }],
    "origin": "conversations",
    "business_id": "507f191e810c19729de860ea",
    "fallback_values": "{\"business\":{\"name\":\"ABC Company\"}}"
  }'

⚡ Performance Considerations

Processing Times:

• Variable replacement: ~10-50ms per email
• DND check: ~5-10ms (indexed query)
• Balance verification: ~10-20ms
• Contact lookup: ~10-30ms (indexed)
• Wasabi upload: ~100-500ms per file
• Sendgrid API call: ~200-800ms
• Total processing: ~500ms - 2s per email

Optimization Tips:

1. Use Templates: Sendgrid templates skip content processing (~50ms savings)
2. Reduce Attachments: Large files significantly increase processing time
3. Batch Recipients: Send to multiple recipients in single email when possible
4. Cache Variables: For bulk emails, cache frequently used variable data
5. Async Processing: Consider queue for bulk email campaigns

⚠️ Important Notes

• 📧 Currently Disabled: Endpoint returns 503 - under maintenance
• 🔒 DND Enforcement: Always checks Do Not Disturb list before sending
• 💾 Attachment Storage: Files stored in Wasabi, only metadata in database
• 🔗 Auto-Threading: Emails automatically linked to conversation threads
• 💳 Credit Tracking: Every email creates billing record in OnebalanceQueue
• 🎯 Variable System: Supports person, business, instasite, instareport data
• 📊 Real-time Updates: Socket events for conversation and message updates
• 🔄 Conversation Creation: Automatically creates prospect conversations
• 📈 Contact Linking: Emails linked to contacts by recipient_id or email
• ⚡ Parallel Processing: Contact lookup and variable replacement parallelized

🔗 Related Documentation

• Sendgrid Mail Send API
• Dynamic Templates
• Attachments
• Authentication
• Webhooks
• Domain Management