Shared Utilities Documentation

Overview

Total Utilities: 65+ helper functions and classes

Location: shared/utilities/

Distribution: Copied to 7 services via npm run copySharedFiles

Pattern: Exported functions and classes for common operations

Usage: Imported by all services for consistent business logic

⚠️ CRITICAL: Git Behavior Warning

DO NOT EDIT utility files in service directories:

• ❌ internal/api/v1/utilities/*.js
• ❌ external/Integrations/utilities/*.js
• ❌ queue-manager/utilities/*.js
• ❌ conversation-socket/utilities/*.js
• ❌ general-socket/utilities/*.js
• ❌ dashboard-gateway/utilities/*.js
• ❌ tests/utilities/*.js

These folders are Git-ignored and regenerated on every build. Changes made here will be lost.

✅ ALWAYS EDIT: shared/utilities/ files only, then run npm run copySharedFiles

Utility Categories

Core Utilities (12 files)

Files: auth.js, db.js, logger.js, request.js, constants.js, index.js, object.js, pick.js, with-transaction.js, admin-check.js, cf-auth.js, support-origin-check.js

Purpose: Fundamental utilities used throughout the application

Used By: All services

auth.js - Authentication Utility

Exports: Password hashing, session verification, Intercom HMAC, API app retrieval

Functions:

Password Management

const { newHash, verifyHash } = require('./utilities/auth');

// Create password hash with crypto.scrypt
const hash = await newHash('password123', 'random-salt');
// Returns: 'salt:hexkey'

// Verify password against hash
const isValid = await verifyHash('password123', storedHash);
// Returns: boolean

Security Pattern: Uses crypto.scrypt for password hashing (more secure and slower than bcrypt, preventing brute-force attacks)

Session Verification

const { verifySession } = require('./utilities/auth');

// Verify session validity
const result = verifySession(session)(req, res, next);
// Returns: 'VALID_SESSION', 'SESSION_EXPIRED', or 'INVALID_SESSION'

// Usage in middleware
router.use((req, res, next) => {
  const status = verifySession(req.sessionId);
  if (status !== 'VALID_SESSION') {
    return res.status(401).json({ error: 'Unauthorized' });
  }
  next();
});

OAuth App Configuration

const { getApiApp } = require('./utilities/auth');

// Get default session-enabled app
const app = await getApiApp({ session: true });

// Get specific OAuth client
const app = await getApiApp({
  session: true,
  client_id: 'abc123',
});

// Returns app configuration:
// {
//   client_id: 'abc123',
//   client_secret: 'secret',
//   redirect_uris: ['http://localhost:3000/callback'],
//   grant_types: ['authorization_code', 'refresh_token'],
//   ...
// }

Intercom User Hashing

const { intercomHash } = require('./utilities/auth');

// Generate HMAC-SHA256 hash for Intercom identity verification
const hash = intercomHash(userId);
// Returns: HMAC hash string

// Used in Intercom integration
const intercomSettings = {
  user_id: userId,
  user_hash: intercomHash(userId),
  // ... other settings
};

db.js - Database Connection Manager

Exports: Connection management with event handling and pooling

Functions:

Connect to MongoDB

const { connect, isConnected, disconnect, connection } = require('./utilities/db');

// Connect to MongoDB with options
await connect({
  uri: process.env.MONGO_DB_URL, // Connection string
  initiator: 'internal-api', // Service name for logs
  options: { maxPoolSize: 10 }, // Mongoose options
  overrideEvents: ['connecting'], // Skip verbose events
});

// With custom options
await connect({
  uri: process.env.MONGO_DB_URL,
  initiator: 'queue-manager',
  options: {
    maxPoolSize: 20,
    minPoolSize: 5,
    serverSelectionTimeoutMS: 5000,
    socketTimeoutMS: 45000,
  },
});

Check Connection Status

// Check if connected
if (isConnected()) {
  console.log('Database is connected');
}

// Wait for connection before querying
async function ensureConnected() {
  if (!isConnected()) {
    await connect({ uri: process.env.MONGO_DB_URL });
  }
}

Graceful Disconnect

// Disconnect with service identifier
await disconnect('internal-api');

// Use in process cleanup
process.on('SIGINT', async () => {
  await disconnect('internal-api');
  process.exit(0);
});

Direct Connection Access

// Access Mongoose connection directly
const db = connection;

// Use native MongoDB driver
const accounts = await db.collection('accounts').findOne({ _id: accountId });

// Access connection state
console.log(connection.readyState);
// 0 = disconnected
// 1 = connected
// 2 = connecting
// 3 = disconnecting

Event Handling: Automatically registers MongoDB lifecycle events

• connecting, connected, open - Connection establishment
• reconnected - Connection recovery after disconnect
• disconnected, close - Connection termination
• error - Connection errors

Singleton Pattern: Prevents multiple connection attempts with in-flight check

Connection Pooling: Default pool size of 10 connections for optimal performance

catch-async.js - Async Error Handler

Purpose: Wraps async Express route handlers to catch errors automatically

const catchAsync = require('./utilities/catch-async');

// Without catch-async (manual error handling)
router.get('/accounts', async (req, res, next) => {
  try {
    const accounts = await Account.find();
    res.json(accounts);
  } catch (err) {
    next(err); // Must remember to call next(err)
  }
});

// With catch-async (automatic error handling)
router.get(
  '/accounts',
  catchAsync(async (req, res, next) => {
    const accounts = await Account.find();
    res.json(accounts);
    // Errors automatically caught and passed to error handler
  }),
);

Error Enrichment: Enhances errors with HTTP response data if present

// Catches axios errors and extracts response data
if (err.response?.data?.meta?.errors?.length) {
  error.message = err.response.data.meta.errors[0].message;
  error.additional_info = err.response.data.meta.errors[0].type;
  error.errno = 400;
}

Debug Mode: Includes stack traces in development environments

Benefits:

• Eliminates boilerplate try-catch blocks
• Consistent error handling across routes
• Enriches errors with additional context
• Works with Express error middleware

with-transaction.js - Transaction Wrapper

Purpose: Simplifies MongoDB multi-document transactions

const withTransaction = require('./utilities/with-transaction');

// Execute multiple operations atomically
await withTransaction(async session => {
  // All operations use the same session
  await Account.updateOne({ _id: accountId }, { balance: 100 }, { session });

  await User.updateOne({ _id: userId }, { credits: 50 }, { session });

  await BillingCharge.create(
    [
      {
        account_id: accountId,
        amount: 50,
      },
    ],
    { session },
  );

  // All succeed or all rollback
});

ACID Guarantees: Ensures atomicity, consistency, isolation, durability

Automatic Rollback: Rolls back all changes if any operation fails

Best Practices:

// Good: Short transaction, specific operations
await withTransaction(async session => {
  await Account.updateOne({ _id: accountId }, { balance: 100 }, { session });
  await BillingCharge.create([{ amount: 50 }], { session });
});

// Bad: Long-running operation in transaction
await withTransaction(async session => {
  await sendEmail(); // External API call - don't do this!
  await Account.updateOne({ _id: accountId }, { balance: 100 }, { session });
});

// Good: External operation outside transaction
await sendEmail();
await withTransaction(async session => {
  await Account.updateOne({ _id: accountId }, { email_sent: true }, { session });
});

When to Use Transactions:

• ✅ Financial operations (billing, credits)
• ✅ Multi-document consistency (create account + user)
• ✅ Race condition prevention (inventory updates)
• ❌ Single document updates (atomic by default)
• ❌ Read-only operations
• ❌ Operations where eventual consistency is acceptable

logger.js - Structured Logging

Purpose: Centralized logging utility for all services

const logger = require('./utilities/logger');

// Log levels
logger.info('User logged in', { userId, timestamp });
logger.warn('High memory usage', { usage: '85%' });
logger.error('Database connection failed', { error });
logger.debug('Query executed', { query, duration });

// With context
logger.info('Payment processed', {
  accountId,
  amount,
  currency: 'USD',
  transactionId,
});

Communication Utilities (7 files)

Files: mail.js, sms.js, webhooks.js, fcm.js, emit-socket.js, sendnotifications.js, verify-notification.js

Purpose: External communication - email, SMS, push notifications, webhooks

Integrations: SendGrid (email), Twilio (SMS), Firebase (push), Socket.IO (real-time)

Used By: Notification Service, Queue Manager, Internal API

mail.js - SendGrid Email Class

Class-Based Utility: Comprehensive email sending with conversation integration

Features:

1. Template Variable Replacement: Uses update-message.js utility to replace {{variable}} placeholders
2. Conversation Integration: Auto-creates conversation records for tracking
3. OneBalance Integration: Verifies and deducts credits before sending
4. DND (Do Not Disturb) Check: Validates recipients against DND list
5. Attachment Upload: Uploads files to Wasabi S3-compatible storage
6. Socket Emission: Notifies real-time services of new messages
7. Contact Linking: Associates emails with CRM contacts automatically

Send Immediate Email

const Mail = require('./utilities/mail');
const mailService = new Mail();

const result = await mailService.send({
  mailBody: {
    recipients: [{ email: 'client@example.com', name: 'John Doe' }],
    cc: [{ email: 'manager@example.com' }],
    bcc: [{ email: 'admin@example.com' }],
    subject: 'Welcome {{first_name}}!',
    sender: {
      email: 'no-reply@dashclicks.com',
      name: 'DashClicks',
    },
    content: [
      {
        type: 'text/html',
        value: '<h1>Welcome {{first_name}}!</h1><p>Your account is ready.</p>',
      },
    ],
    files: [
      {
        filename: 'invoice.pdf',
        content: base64Content,
        type: 'application/pdf',
      },
    ],
    reply_to: { email: 'support@dashclicks.com' },
    origin: 'campaigns',
  },
  accountID: '507f1f77bcf86cd799439011',
  userID: '507f1f77bcf86cd799439012',
  personID: '507f1f77bcf86cd799439013', // For template variables
  customFields: { company_name: 'Acme Corp' },
  fallbackValues: {
    subject: { first_name: 'Customer' },
    body: { first_name: 'Customer', company_name: 'Your Company' },
  },
});

// Returns: Communication document with _id
console.log(result._id); // MongoDB ObjectId of sent email

Schedule Email for Later

await mailService.schedule({
  mailBody: {
    recipients: [{ email: 'client@example.com' }],
    subject: 'Scheduled Reminder',
    content: [{ type: 'text/html', value: '<p>Your appointment is tomorrow</p>' }],
  },
  accountID,
  userID,
  check_credits: true, // Verify credits before scheduling
  scheduled_time: new Date('2025-10-20T10:00:00Z'),
});

Credit System Flow

// Verifies balance before sending
await verifyBalance({
  account: accountDoc,
  event: 'email',
  user_id: userID,
  quantity: recipients.length, // Cost per recipient
});

// After successful send, logs credit usage
await new OnebalanceQueue({
  account_id: accountID,
  event: 'email',
  additional_info: {
    communication_id: mailDoc._id,
    reference: mailDoc._id,
  },
}).save();

Conversation Auto-Creation

// For each recipient contact
let convo = await findConvo(contactID, accountID);
if (!convo) {
  // Create new conversation
  convo = await saveConvo({
    created_by: userID,
    contact_id: contactID,
    account_id: accountID,
    users: [userID],
    unread_count: { [userID]: 0 },
    last_contacted: { [contactID]: timestamp },
    first_seen: { [userID]: timestamp, [contactID]: timestamp },
  });

  // Emit to Conversation Socket for real-time update
  await emitConversation(convo._id, 'new');
}

// Update conversation activity
await convo.updateOne({
  last_activity: mailDoc._id,
  is_open: true,
  unread_count: {
    /* increment for all users except sender */
  },
});

sms.js - Twilio SMS Class

Purpose: SMS sending via Twilio with similar patterns to mail.js

const SMS = require('./utilities/sms');
const smsService = new SMS();

await smsService.send({
  recipients: [{ phone: '+15551234567', name: 'John Doe' }],
  message: 'Hello {{first_name}}, your order is ready!',
  accountID,
  userID,
  personID, // For template variables
  customFields: { order_id: '12345' },
});

Features: Credit verification, conversation linking, DND check, template variables

webhooks.js - Webhook Delivery

Purpose: Webhook delivery and retry logic

const { triggerWebhook } = require('./utilities/webhooks');

await triggerWebhook({
  url: 'https://example.com/webhook',
  event: 'deal.created',
  payload: {
    deal_id: dealId,
    account_id: accountId,
    created_at: new Date(),
  },
  accountID,
  retries: 3,
});

Features: Automatic retries, exponential backoff, delivery tracking

fcm.js - Firebase Cloud Messaging

Purpose: Push notifications via Firebase

const { sendPushNotification } = require('./utilities/fcm');

await sendPushNotification({
  tokens: ['fcm_device_token_1', 'fcm_device_token_2'],
  title: 'New Message',
  body: 'You have a new message from John',
  data: {
    conversation_id: convoId,
    type: 'message',
  },
  accountID,
});

Validation & Error Handling (5 files)

Files: validate.js, api-error.js, catch-async.js, catch-errors.js, custom.validation.js

Purpose: Request validation, error handling, and custom validators

Used By: All API services (Internal, External, Router)

validate.js - Request Validation

Purpose: Joi/Yup schema validation middleware

const validate = require('./utilities/validate');
const Joi = require('joi');

const accountSchema = {
  body: Joi.object({
    name: Joi.string().required(),
    email: Joi.string().email().required(),
    phone: Joi.string().optional(),
  }),
};

router.post('/accounts', validate(accountSchema), controller);

api-error.js - Standardized Error Class

Purpose: Custom error class with errno and additional_info

const { ApiError } = require('./utilities/api-error');

// Throw custom error
throw new ApiError(404, 'ACCOUNT_NOT_FOUND', { account_id: accountId });

// In error handler
app.use((error, req, res, next) => {
  res.status(error.errno || 500).json({
    status: false,
    errno: error.errno || 500,
    message: error.message,
    additional_info: error.additional_info,
  });
});

pick.js - Object Property Picker

Purpose: Whitelist object properties for security

const { pick } = require('./utilities/pick');

// Only allow whitelisted fields
const data = pick(req.body, ['name', 'email', 'phone']);

// Ignore other fields from request
const account = await Account.create({
  ...data,
  created_by: req.auth.user_id, // Server-side, not from request
});

Business Logic Utilities (15 files)

Files: deal.js, contact.js, pipeline.js, pipeline-stage.js, activity.js, crm-contact.js, filter.js, instareport.js, instasite.js, project.js, review-request.js, store.js, e-doc.js, reputation.js, listings.js

Purpose: Domain-specific business logic and data processing

Used By: Internal API, Queue Manager, External API

deal.js - Deal Operations

Purpose: Deal/opportunity calculations and stage changes

const dealUtils = require('./utilities/deal');

// Calculate deal value
const value = dealUtils.calculateValue(deal);

// Move deal to next stage
await dealUtils.moveToStage(dealId, nextStageId, userId);

// Calculate win probability
const probability = dealUtils.calculateWinProbability(deal);

contact.js - Contact Operations

Purpose: Contact management and merging

const contactUtils = require('./utilities/contact');

// Merge duplicate contacts
await contactUtils.mergeContacts(primaryContactId, duplicateContactIds);

// Enrich contact data
await contactUtils.enrichContactData(contactId);

filter.js - Data Filtering

Purpose: Query filtering and search

const filterUtils = require('./utilities/filter');

// Apply filters to query
const query = filterUtils.applyFilters(baseQuery, {
  status: 'active',
  created_after: '2025-01-01',
  tags: ['vip', 'enterprise'],
});

Integration Helpers (8 files)

Files: stripe.js, stripe/, semrush-auth.js, wasabi.js, short-url.js, priceUtility.js, getschedulesutility.js, onebalance.js

Purpose: Third-party integration helpers and API clients

Integrations: Stripe, SEMrush, Wasabi (S3-compatible storage)

Used By: Internal API, External API, Queue Manager

stripe.js - Stripe Wrapper

Purpose: Stripe payment processing wrapper

const stripeUtils = require('./utilities/stripe');

// Create customer
const customer = await stripeUtils.createCustomer({
  email: 'customer@example.com',
  name: 'John Doe',
});

// Create subscription
const subscription = await stripeUtils.createSubscription({
  customer: customerId,
  price: priceId,
});

// Process payment
const charge = await stripeUtils.createCharge({
  amount: 5000, // $50.00
  currency: 'usd',
  customer: customerId,
});

wasabi.js - Cloud Storage

Purpose: Wasabi S3-compatible storage operations

const wasabiUtils = require('./utilities/wasabi');

// Upload file
const url = await wasabiUtils.uploadFile({
  filename: 'document.pdf',
  content: fileBuffer,
  contentType: 'application/pdf',
  bucket: 'dashclicks-documents',
});

// Delete file
await wasabiUtils.deleteFile({
  url: fileUrl,
  bucket: 'dashclicks-documents',
});

onebalance.js - Credit System

Purpose: OneBalance credit system operations

const { verifyBalance, deductCredits } = require('./utilities/onebalance');

// Check if account has sufficient credits
await verifyBalance({
  account: accountDocument, // Full account document
  event: 'email', // Event type: email, sms, api_call, etc.
  user_id: userId,
  quantity: 10, // Number of credits needed
});

// Supported events:
// - email: Email sending
// - sms: SMS sending
// - api_call: API requests
// - lead_generation: Lead finder usage
// - report_generation: Report creation
// - site_generation: Site creation

// Throws error if insufficient balance
// Error: { message: 'INSUFFICIENT_BALANCE', balance: 5, required: 10 }

Domain & URL Utilities (5 files)

Files: domains.js, cleanupDomain.js, extract-url.js, funnelDomainStatusUtility.js, movedomainutility.js

Purpose: Domain management, URL parsing, and validation

Used By: Internal API, External API (domain integrations)

domains.js - Domain Operations

Purpose: Domain validation and DNS operations

const domainUtils = require('./utilities/domains');

// Validate domain
const isValid = await domainUtils.validateDomain('example.com');

// Get DNS records
const records = await domainUtils.getDNSRecords('example.com');

// Verify DNS setup
const verified = await domainUtils.verifyDNSSetup('example.com', expectedRecords);

cleanupDomain.js - Domain Normalization

Purpose: Domain string normalization

const { cleanupDomain } = require('./utilities/cleanupDomain');

// Remove protocol and trailing slash
const clean = cleanupDomain('https://example.com/');
// Returns: 'example.com'

// Handle subdomains
const clean = cleanupDomain('https://www.example.com/path');
// Returns: 'www.example.com'

Data Processing Utilities (8 files)

Files: filter.js, object.js, pick.js, replaceKeysInString.js, processShared.js, processFunnelSteps.js, update-message.js, lead-info-template.js

Purpose: Data transformation, filtering, and processing

Used By: All services for data transformation

update-message.js - Template Variables

Purpose: Dynamic variable replacement with fallbacks

const { updateMessage } = require('./utilities/update-message');

const content = await updateMessage({
  content: 'Hello {{first_name}}, your {{product_name}} is ready!',
  userID: userId,
  accountID: accountId,
  personID: contactId, // Contact for {{first_name}}, {{email}}, etc.
  businessID: businessId, // Business contact for company variables
  instasiteID: siteId, // InstaSite for {{site_url}}, {{site_name}}
  customFields: {
    // Custom merge fields
    product_name: 'Website',
    delivery_date: '2025-10-15',
  },
  fallbackValues: {
    // Fallbacks if variables not found
    first_name: 'Customer',
    product_name: 'Service',
  },
});

// Result: "Hello John, your Website is ready!"
// Or with fallback: "Hello Customer, your Service is ready!"

Supported Variable Sources:

1. Person (Contact): {{first_name}}, {{last_name}}, {{email}}, {{phone}}
2. Business: {{business_name}}, {{business_email}}, {{business_phone}}
3. InstaSite: {{site_url}}, {{site_name}}, {{site_id}}
4. InstaReport: {{report_url}}, {{report_date}}
5. Custom Fields: Any key from customFields object
6. Fallback Values: Used when primary source has no data

Account & User Management (6 files)

Files: accounts.js, assigntoaccountsormanager.js, account-invite.js, delete-user.js, delete-user.new.js, create-defaults.js

Purpose: Account and user operations

Used By: Internal API, Queue Manager

accounts.js - Account Operations

Purpose: Account management operations

const accountUtils = require('./utilities/accounts');

// Get account hierarchy
const hierarchy = await accountUtils.getAccountHierarchy(accountId);
// Returns: [mainAccount, ...subAccounts]

// Check if account is parent
const isParent = await accountUtils.isParentAccount(accountId);

create-defaults.js - Default Data

Purpose: Create default data for new accounts

const { createDefaults } = require('./utilities/create-defaults');

await createDefaults({
  accountID,
  userID,
  defaults: [
    'pipelines', // Default CRM pipelines
    'templates', // Default email templates
    'settings', // Default account settings
  ],
});

Admin & Access Control (3 files)

Files: admin-check.js, admin-filters.js, cf-auth.js

Purpose: Admin permissions and access control

Used By: Internal API (admin routes)

admin-check.js - Admin Verification

Purpose: Admin role verification

const { isAdmin } = require('./utilities/admin-check');

// Check if user is admin
if (isAdmin(user)) {
  // Admin-only operations
}

// Middleware
router.use('/admin', (req, res, next) => {
  if (!isAdmin(req.auth.user)) {
    return res.status(403).json({ error: 'Admin only' });
  }
  next();
});

Financial Utilities (2 files)

Files: credits.js, currency.js

Purpose: Credit management and currency conversion

Used By: Internal API, Billing services

credits.js - Credit Operations

Purpose: Account credit operations

const creditUtils = require('./utilities/credits');

// Add credits
await creditUtils.addCredits(accountId, 100, 'purchase');

// Deduct credits
await creditUtils.deductCredits(accountId, 10, 'email_campaign');

// Get balance
const balance = await creditUtils.getBalance(accountId);

currency.js - Currency Conversion

Purpose: Currency conversion and formatting

const currencyUtils = require('./utilities/currency');

// Convert currency
const converted = await currencyUtils.convert({
  amount: 100,
  from: 'USD',
  to: 'EUR',
});

// Format currency
const formatted = currencyUtils.format(100, 'USD');
// Returns: '$100.00'

Advanced Utility Patterns

Class-Based Services

Pattern: Use classes for stateful services with multiple operations

// Good example: mail.js
class Mail {
  constructor() {
    this.sendgridApiKey = null;
    this.client = null;
  }

  async getApiKey(accountID) {
    if (!this.sendgridApiKey) {
      // Load and cache API key
    }
  }

  async send(options) {
    /* ... */
  }
  async schedule(options) {
    /* ... */
  }

  // Private helpers
  async findContactsByEmail(emails, accountID) {
    /* ... */
  }
  async saveConvo(data) {
    /* ... */
  }
}

module.exports = Mail;

// Usage
const mail = new Mail();
await mail.send({...});

When to Use Classes:

• Multiple related operations
• Shared state (cached API keys, connections)
• Private helper methods
• Complex initialization

Function-Based Utilities

Pattern: Use exported functions for stateless operations

// Good example: auth.js
async function newHash(password, salt) {
  /* ... */
}
async function verifyHash(password, hash) {
  /* ... */
}
function intercomHash(id) {
  /* ... */
}

module.exports = {
  newHash,
  verifyHash,
  intercomHash,
};

// Usage
const { verifyHash } = require('./utilities/auth');
const isValid = await verifyHash(password, hash);

When to Use Functions:

• Stateless transformations
• Simple validation logic
• Pure functions with no side effects
• Single responsibility per function

Usage Patterns

Importing Utilities

// Core utilities
const { verifyToken } = require('./utilities/auth');
const logger = require('./utilities/logger');
const db = require('./utilities/db');

// Communication utilities
const Mail = require('./utilities/mail');
const SMS = require('./utilities/sms');
const { triggerWebhook } = require('./utilities/webhooks');

// Business logic utilities
const dealUtils = require('./utilities/deal');
const contactUtils = require('./utilities/contact');

// Validation and error handling
const { ApiError } = require('./utilities/api-error');
const catchAsync = require('./utilities/catch-async');

Database Transactions

const withTransaction = require('./utilities/with-transaction');

// Wrap operations in transaction
await withTransaction(async session => {
  await Account.updateOne({ _id: accountId }, { balance: 100 }, { session });
  await User.updateOne({ _id: userId }, { credits: 50 }, { session });
  // Both succeed or both rollback
});

Error Handling

// Wrap async routes with catch-async
router.get(
  '/path',
  catchAsync(async (req, res) => {
    /* ... */
  }),
);

// Throw API errors
throw new ApiError(400, 'Invalid request');

// Global error handler
app.use((error, req, res, next) => {
  logger.error({ req, error });
  res.status(error.errno || 500).json({
    status: false,
    errno: error.errno || 500,
    message: error.message,
    additional_info: error.additional_info,
  });
});

Performance Considerations

Utility Performance Guidelines

Email Sending (mail.js):

• Single email: ~500ms (SendGrid API call)
• Batch email: ~100ms per recipient (parallel processing)
• With attachments: +200ms per MB

Database Connection (db.js):

• Initial connect: ~1-2 seconds
• Subsequent queries: ~5-50ms (connection pooled)
• Reconnection: ~500ms

Authentication (auth.js):

• Password hash creation: ~100ms (scrypt intentionally slow)
• Password verification: ~100ms
• Session validation: < 10ms

Optimization Techniques

Caching:

// Cache frequently accessed data
class Mail {
  constructor() {
    this.apiKeyCache = new Map();
  }

  async getApiKey(accountID) {
    if (this.apiKeyCache.has(accountID)) {
      return this.apiKeyCache.get(accountID);
    }
    const key = await SendgridKey.findOne({ account_id: accountID });
    this.apiKeyCache.set(accountID, key);
    return key;
  }
}

Parallel Processing:

// Process multiple operations in parallel
await Promise.all([sendEmail(recipient1), sendEmail(recipient2), sendEmail(recipient3)]);

Security Best Practices

Input Sanitization

Pattern: Validate and sanitize all inputs

const validate = require('./utilities/validate');
const { pick } = require('./utilities/pick');

router.post(
  '/accounts',
  validate(accountSchema), // Joi/Yup validation
  catchAsync(async (req, res) => {
    // Only allow whitelisted fields
    const data = pick(req.body, ['name', 'email', 'phone']);

    // Sanitize
    data.email = data.email.toLowerCase().trim();

    const account = await Account.create({
      ...data,
      created_by: req.auth.user_id, // Server-side, not from request
    });

    res.json(account);
  }),
);

Never Trust Client Data

// Bad: Trust client-provided IDs
await Account.updateOne(
  { _id: req.body.account_id }, // User could change this!
  { balance: req.body.balance }, // User could set any balance!
);

// Good: Use authenticated user's context
await Account.updateOne(
  {
    _id: req.auth.account_id, // From JWT token
    _id: req.params.id, // From URL, validate ownership
  },
  pick(req.body, ['name', 'email']), // Whitelist allowed fields
);

Encrypt Sensitive Data

Pattern: Encrypt integration keys, tokens, passwords

// Store encrypted
const encryptedToken = encryptString(apiToken);
await GoogleAdsToken.create({
  account_id: accountId,
  access_token: encryptedToken,
  refresh_token: encryptedRefreshToken,
});

// Retrieve and decrypt
const token = await GoogleAdsToken.findOne({ account_id });
const decryptedToken = decryptString(token.access_token);

Testing Utilities

Unit Tests

describe('Auth Utility', () => {
  it('should hash and verify password', async () => {
    const password = 'test123';
    const hash = await newHash(password, 'salt');

    const isValid = await verifyHash(password, hash);
    expect(isValid).toBe(true);
  });
});

Integration Tests

describe('Mail Utility', () => {
  it('should send email and create conversation', async () => {
    const mail = new Mail();

    const result = await mail.send({
      mailBody: {
        recipients: [{ email: 'test@example.com' }],
        subject: 'Test',
        content: [{ type: 'text/html', value: '<p>Test</p>' }],
      },
      accountID,
      userID,
    });

    expect(result._id).toBeDefined();

    const convo = await ConversationProspect.findOne({
      account_id: accountID,
    });
    expect(convo).toBeDefined();
  });
});

Related Documentation

• Shared Models - Mongoose schemas and data models
• Shared Middlewares - Express middleware functions
• Shared Overview - Main documentation with distribution process
• Internal API - Service using all utilities
• External API - Service using integration utilities

---

Resource Type: Helper Functions & Classes
Total: 65+ utilities
Distribution: 7 services receive utility copies
Git Strategy: Only shared/utilities/ is version controlled
Last Updated: October 2025