Skip to main content

Shared Models Documentation

Overview​

Total Models: 207 Mongoose schemas (excluding node_modules/ and plugins/)

Location: shared/models/

Distribution: Copied to 7 services via npm run copySharedFiles

Pattern: All models use Mongoose with global options from _globalSchemaOptions.js

Naming Convention: [domain]-[entity].js (e.g., account-user.js, billing-charge.js)

Collections: 200+ MongoDB collections

⚠️ CRITICAL: Git Behavior Warning​

DO NOT EDIT model files in service directories:

  • ❌ internal/api/v1/models/*.js
  • ❌ external/Integrations/models/*.js
  • ❌ queue-manager/models/*.js
  • ❌ conversation-socket/models/*.js
  • ❌ general-socket/models/*.js
  • ❌ dashboard-gateway/models/*.js
  • ❌ tests/models/*.js

These folders are Git-ignored and regenerated on every build. Changes made here will be lost.

✅ ALWAYS EDIT: shared/models/ files only, then run npm run copySharedFiles

Global Schema Configuration​

File: shared/models/_globalSchemaOptions.js

Purpose: Consistent Mongoose schema configuration across all 207 models

module.exports = {
  timestamps: {
    createdAt: 'created_at', // Auto-generate created_at field
    updatedAt: 'updated_at', // Auto-generate updated_at field
  },
  toJSON: {
    virtuals: true, // Include virtual properties
    versionKey: false, // Exclude __v field
    transform: function (doc, ret, game) {
      delete ret._id; // Remove _id from JSON output
      delete ret.__v; // Remove __v from JSON output
    },
  },
  toObject: {
    virtuals: true, // Include virtual properties
    versionKey: false, // Exclude __v field
    transform: function (doc, ret, game) {
      delete ret._id; // Remove _id from object output
      delete ret.__v; // Remove __v from object output
    },
  },
};

Applied By: All models in shared/models/

Benefits:

  • Consistent Timestamps: All models use created_at and updated_at (not createdAt/updatedAt)
  • Clean JSON: Virtual id property replaces _id, no __v field
  • Virtual Support: Mongoose virtuals included in JSON/Object serialization
  • Immutable Pattern: created_at set once, updated_at auto-updates

Model Architecture Principles​

Flexible Schema Pattern​

Pattern: Most models use { strict: false } to allow dynamic fields beyond defined schema

// Example from account.js
const accountSchema = new mongoose.Schema(
  {
    business: { type: mongoose.SchemaTypes.ObjectId, ref: 'Contact' },
    parent_account: { type: mongoose.SchemaTypes.ObjectId, ref: 'Account' },
    manager: { type: mongoose.SchemaTypes.ObjectId, ref: 'User' },
    // ... defined fields
  },
  {
    strict: false, // Allows storing arbitrary fields
    timestamps: { createdAt: 'created_at', updatedAt: 'updated_at' },
  },
);

Benefits:

  • Rapid Development: Add fields without schema migrations
  • Integration Flexibility: Store varying external API responses
  • Backward Compatibility: Old code won't break if new fields added

Trade-offs:

  • Less validation at database level
  • Requires application-level validation
  • Can accumulate unused fields over time

Custom JSON Transformation​

Pattern: All models use toJSON.js plugin for consistent output

// Applied to every model
schema.methods.toJSON = require('./toJSON');

// Transforms:
// - Removes _id and __v fields
// - Includes virtual properties
// - Adds 'id' virtual field

Rich Model Methods​

Pattern: Complex models include custom instance methods

// Example from user.js
userSchema.methods.updateConversationsStatus = async function (sockets, status) {
  // Complex logic for updating user presence across socket connections
  // Manages conversation_socket_status object
  // Updates conversation_active_status (ACTIVE/IDLE/OFFLINE)
  // Tracks conversation_last_active timestamp
};

userSchema.methods.getCalendars = function (auth) {
  // Retrieves connected Google/Outlook/Internal calendars
  // Returns available vs connected calendar distinction
};

Sparse Unique Indexes​

Pattern: Used for optional unique fields

// From user.js additional_info
phones: [
  {
    number: { type: String, unique: true, sparse: true }, // Only enforces uniqueness when present
    main: { type: Boolean, default: false },
    name: { type: String, default: 'default' },
  },
];

Model Categories​

Account Management (6 models)​

Files: account*.js, workspace-*.js

Purpose: Core account, user, and workspace management

Key Models:

  • account.js - Main account entity (agencies/clients)
  • account-user.js - Account-user relationships
  • user.js - User authentication and profile
  • workspace-account.js - Workspace relationships
  • workspace-user.js - User workspace permissions
  • account-credit.js - Account credit balances

Collections: accounts, users, workspaces, accountcredits

API & OAuth (5 models)​

Files: api-*.js

Purpose: OAuth 2.0 authentication and API access control

Key Models:

  • api-session.js - Active user sessions (used by Dashboard Gateway)
  • api-refresh-token.js - OAuth refresh tokens
  • api-grant.js - Authorization grants
  • api-app.js - OAuth client applications
  • api-scope.js - API permission scopes

Collections: apisessions, apirefreshtokens, apigrants, apiapps, apiscopes

Billing & Payments (7 models)​

Files: billing-*.js, stripe-subscription.js, consumer-customer-stripe-customer.js

Purpose: Payment processing, subscriptions, and financial transactions

Key Models:

  • billing-customer.js - Customer billing profiles
  • billing-subscription.js - Active subscriptions
  • billing-charge.js - Payment charges
  • billing-refund.js - Refund transactions
  • billing-dispute.js - Payment disputes
  • stripe-subscription.js - Stripe-specific subscriptions
  • stripe-key.js - Stripe API credentials

Collections: billingcustomers, billingsubscriptions, billingcharges, etc.

CRM (9 models)​

Files: contact.js, deal.js, pipeline*.js, crm-*.js, activity*.js

Purpose: Customer relationship management - contacts, deals, pipelines, activities

Key Models:

  • contact.js - CRM contacts
  • deal.js - Sales deals/opportunities
  • pipeline.js - Sales pipelines
  • pipeline-stage.js - Pipeline stages/steps
  • activity.js - CRM activities (calls, meetings, etc.)
  • crm-log.js - Audit logs for CRM changes
  • crm-note.js - Notes attached to CRM entities
  • crm-tag.js - Tags for categorization
  • automation.js - CRM workflow automations

Collections: contacts, deals, pipelines, pipelinestages, activities, etc.

Relationships: Contacts → Deals → Pipeline Stages → Pipelines

Conversations & Messaging (11 models)​

Files: conversation*.js, message*.js, room*.js, support.*.js

Purpose: Real-time messaging, chat rooms, and support conversations

Key Models:

  • conversation.js - Main conversation entity (legacy)
  • conversation-conversation.js - V2 conversations
  • conversation-message.js - Chat messages
  • conversation-room.js - Chat rooms
  • conversation-channel.js - Communication channels
  • support.conversation.js - Support ticket conversations
  • support.message.js - Support messages
  • support.room.js - Support chat rooms
  • message.model.js - Generic message model
  • room.model.js - Generic room model

Collections: conversations, messages, rooms, supportconversations, etc.

Used By: Conversation Socket service, General Socket service, Internal API

Store & E-Commerce (14 models)​

Files: store-*.js, product.js

Purpose: Product catalog, orders, subscriptions, coupons, payouts

Key Models:

  • store-product.js - Product catalog
  • store-price.js - Product pricing tiers
  • store-order.js - Customer orders
  • store-subscription.js - Recurring subscriptions
  • store-cart.js - Shopping cart
  • store-coupon.js - Discount coupons
  • store-promo-code.js - Promotional codes
  • store-invoice.js - Invoices
  • store-payout.js - Seller payouts
  • store-refund.js - Refunds
  • store-dispute.js - Payment disputes
  • store-credit.js - Store credits
  • store-bundles.js - Product bundles

Collections: storeproducts, storeorders, storesubscriptions, etc.

Funnels (11 models)​

Files: funnel*.js, funnels.*.js

Purpose: Marketing funnel builder with templates, steps, and components

Key Models:

  • funnels.js - Main funnel entity
  • funnel.step.js - Funnel steps/pages
  • funnel.step.components.js - Step components (elements)
  • funnel.blocks.js - Reusable content blocks
  • funnel.analytics.js - Funnel analytics data
  • funnels.domains.js - Custom domains for funnels
  • funnels.global.templates.js - Global funnel templates
  • funnel.global.templates.categories.js - Template categories
  • funnel.global.templates.steps.js - Template steps
  • funnel.tags.js - Funnel tags/categories

Collections: funnels, funnelsteps, funnelcomponents, funnelanalytics, etc.

InstaReports (4 models)​

Files: instareports*.js

Purpose: Automated marketing reports for clients

Key Models:

  • instareports.js - Main report configuration
  • instareports-queue.js - Report generation queue
  • instareports-additional-information.js - Extra report data
  • instareports-industry-average.js - Industry benchmark data

Collections: instareports, instareportsqueues, etc.

InstaSites (5 models)​

Files: instasite*.js

Purpose: Instant website builder for clients

Key Models:

  • instasite.js - Main website entity
  • instasite-queue.js - Website generation queue
  • instasite-template.js - Website templates
  • instasite-template-category.js - Template categories
  • instasites-additional-information.js - Extra site data

Collections: instasites, instasitequeues, instasitetemplates, etc.

Forms (7 models)​

Files: forms*.js

Purpose: Form builder, responses, and analytics

Key Models:

  • forms.js - Form configurations
  • forms.templates.js - Form templates
  • forms.categories.js - Form categories
  • forms-userresponse.js - Form submissions
  • forms-sent-requests.js - Form distribution tracking
  • forms-user-categories.js - User-specific categories
  • forms-user-tags.js - User-specific tags

Collections: forms, formsresponses, formscategories, etc.

eDocuments (14 models)​

Files: edocs-*.js

Purpose: Document management system with templates, categories, and workflows

Key Models:

  • edocs-document.js - Main document entity
  • edocs-template.js - Document templates
  • edocs-category.js - User categories
  • edocs-default-category.js - System categories
  • edocs-file.js - Uploaded files
  • edocs-file-association.js - File relationships
  • edocs-queue.js - Document processing queue
  • edocs-contact.js - Document recipients
  • edocs-snippets.js - Reusable text snippets
  • edocs-tag.js - Document tags

Collections: edocsdocuments, edocstemplates, edocscategories, etc.

Calendar & Scheduling (16 models)​

Files: event-type*.js, google-calendar*.js, outlook-calendar*.js, availability-scheduler.js, scheduled-event.js, member-meeting.js, internal-calendar.js

Purpose: Meeting scheduling, calendar integrations, and availability management

Key Models:

  • event-type.js - Event types (meeting types)
  • event-type-team.js - Team event types
  • scheduled-event.js - Scheduled meetings
  • member-meeting.js - Meeting participants
  • google-calendar.js - Google Calendar sync
  • google-calendar-event.js - Google events
  • google-calendar-token.js - Google auth tokens
  • outlook-calendar.js - Outlook Calendar sync
  • availability-scheduler.js - Availability rules
  • internal-calendar.js - Internal calendar events

Collections: eventtypes, scheduledevents, membermeetings, etc.

Integrations: Google Calendar, Outlook Calendar

Analytics (9 models)​

Files: analytics-*.js, campaign-data.js, semrush*.js, google-analytics-token.js

Purpose: Marketing analytics, SEO tracking, campaign data

Key Models:

  • analytics-seo.js - SEO metrics and rankings
  • analytics-seo-config.js - SEO configuration
  • analytics-googleanalytics-userconfig.js - Google Analytics config
  • analytics-facebook.ads.userconfig.js - Facebook Ads config
  • analytics-callrail-userconfig.js - CallRail config
  • analytics-calltrackingmetrics-userconfig.js - Call tracking config
  • analytics-tiktokanalytics-userconfig.js - TikTok Analytics config
  • campaign-data.js - Marketing campaign data
  • semrush.js - SEMrush data

Collections: analyticsseo, campaigndata, semrush, etc.

Reviews (9 models)​

Files: review*.js, reviews*.js

Purpose: Review management, auto-responses, review requests

Key Models:

  • reviews.js - Customer reviews
  • review.configs.js - Review platform configurations
  • review.requests.js - Review request campaigns
  • review.auto-response-rules.js - Auto-response rules
  • review.auto-review-request.js - Automated review requests
  • review.response-templates.js - Response templates
  • review-request-template.js - Request templates
  • review.widget.js - Review display widgets

Collections: reviews, reviewconfigs, reviewrequests, etc.

Projects (6 models)​

Files: projects-*.js

Purpose: Project management with tasks, files, and reports

Key Models:

  • projects-tasks.js - Project tasks
  • projects-files.js - Project files
  • projects-notebooks.js - Project notes
  • projects-reports.js - Project reports
  • projects-pulse.js - Project activity feed
  • projects-dashboard-preferences.js - Dashboard settings

Collections: projectstasks, projectsfiles, projectsnotebooks, etc.

Integration Keys & Tokens (25 models)​

Files: *-key.js, *-token.js

Purpose: API credentials and OAuth tokens for third-party integrations

Key Models:

  • google-ads-token.js, google-analytics-token.js, google-business-token.js
  • facebook-ads-keys.js, tik-tok-token.js, bing-ads-token.js
  • stripe-key.js, sendgrid-key.js, twilio-number.js
  • duda-key.js, yext-publishers-logo.js
  • hubspot-key.js, salesforce-key.js, pipedrive-key.js, zoho-key.js
  • mailchimp-key.js, active-campaign-key.js, constant-contact-key.js
  • keap-key.js, callrail-key.js, calltrackingmetrics-key.js
  • semrush-auth.js, square-up-token.js, zoom-key.js

Collections: googleadstokens, facebookadskeys, stripekeys, etc.

Pattern: Store encrypted API keys and OAuth tokens for external services

Lead Finder (13 models)​

Files: lead-finder-*.js, leads-data.js

Purpose: Lead generation and scraping system

Key Models:

  • lead-finder-lead.js - Found leads
  • lead-finder-lead-group.js - Lead groups
  • lead-finder-user-lead.js - User-lead associations
  • lead-finder-scraper.js - Scraper configurations
  • lead-finder-scrape-log.js - Scraping logs
  • lead-finder-google-rank.js - Google ranking data
  • lead-finder-industry.js - Industry classifications
  • lead-finder-location.js - Geographic data
  • leads-data.js - Additional lead data

Collections: leadfinderleads, leadfindergroups, leadfinderlogs, etc.

Support & Chat (7 models)​

Files: support.*.js

Purpose: Customer support system with conversations, messages, and sessions

Key Models:

  • support.conversation.js - Support conversations
  • support.message.js - Support messages
  • support.room.js - Support rooms
  • support.inbox.js - Support inboxes
  • support.sessions.js - Support sessions
  • support.tokens.js - Support authentication
  • support.message.lock.model.js - Message locking

Collections: supportconversations, supportmessages, supportrooms, etc.

Used By: Conversation Socket service, Internal API

Queue & Background Jobs (9 models)​

Files: *-queue.js, queues.js, queue.js

Purpose: Background job processing and queue management

Key Models:

  • queue.js - Generic queue jobs
  • queues.js - Queue definitions
  • contact-queue.js - Contact import queue
  • csv-queue.js - CSV processing queue
  • webhook-queue.js - Webhook delivery queue
  • notification-queue.js - Notification queue
  • instareports-queue.js - Report generation queue
  • instasite-queue.js - Site generation queue
  • edocs-queue.js - Document processing queue

Collections: queues, contactqueues, csvqueues, webhookqueues, etc.

Used By: Queue Manager service

Webhooks (5 models)​

Files: webhook*.js, *-webhook*.js

Purpose: Webhook management and delivery

Key Models:

  • webhook.js - Webhook configurations
  • webhook-log.js - Webhook delivery logs
  • webhook-queue.js - Webhook delivery queue
  • facebook-webhooks.js - Facebook webhook data
  • duda-webhook-data.js - Duda webhook data

Collections: webhooks, webhooklogs, facebookwebhooks, etc.

Notifications (5 models)​

Files: notification-*.js, fcm-*.js, meeting-*-reminder.js, expo-push-tokens.js

Purpose: Push notifications, email/SMS reminders

Key Models:

  • notification-queue.js - Notification delivery queue
  • fcm-notification.js - Firebase Cloud Messaging notifications
  • fcm-token.js - FCM device tokens
  • meeting-emails-reminder.js - Meeting email reminders
  • meeting-sms-reminder.js - Meeting SMS reminders
  • expo-push-tokens.js - Expo push notification tokens

Collections: notificationqueues, fcmnotifications, fcmtokens, etc.

Affiliates (3 models)​

Files: affiliate*.js, affiliates-*.js

Purpose: Affiliate program management and payouts

Key Models:

  • affiliate-payout.js - Affiliate payouts
  • affiliate-payout-account.js - Payout account details
  • affiliates-leaderboard.js - Affiliate rankings

Collections: affiliatepayouts, affiliateleaderboard, etc.

OneBalance (3 models)​

Files: onebalance*.js

Purpose: Unified balance/credit system

Key Models:

  • onebalance.js - Balance accounts
  • onebalance-queue.js - Balance update queue
  • onebalance-usage_logs.js - Usage tracking logs

Collections: onebalance, onebalancequeues, onebalanceusagelogs

Miscellaneous (20+ models)​

Additional Models:

  • activity.js - Generic activity tracking
  • announcement.js - System announcements
  • automation.js - Workflow automations
  • communication.js - Communication logs
  • config.js - System configuration
  • currency.js - Currency exchange rates
  • dashboard-meta.js - Dashboard metadata
  • dnd.js - Do Not Disturb settings
  • fields-filter.js - Field filtering rules
  • filter.js - Saved filters
  • log.js, logs.js - System logs
  • offers.js - Special offers
  • product.js - Generic products
  • quotas.js - Usage quotas
  • reminder.js - Reminders
  • short-url.js - URL shortener
  • socket.js - Socket connections
  • template.js, template-default.js - Generic templates
  • tier-override.js - Plan tier overrides
  • unsubscription-list.js - Email unsubscribes
  • uploads.js - File uploads
  • user-config.js - User configuration
  • visitor.js - Anonymous visitors

Model Relationships & Data Flow​

Core Data Relationships​

erDiagram
    Account ||--o{ User : "has members"
    Account ||--o{ Contact : "manages"
    Account ||--o{ Deal : "owns"
    Account ||--o{ Funnel : "creates"
    Account ||--o{ InstaReport : "generates"
    Account ||--o{ InstaSite : "builds"

    User ||--o{ Deal : "assigned to"
    User ||--o{ Activity : "performs"
    User ||--o{ ApiSession : "has sessions"

    Contact ||--o{ Deal : "associated with"
    Contact ||--o{ Activity : "logs"
    Contact ||--o{ ConversationProspect : "chats in"

    Deal }o--|| PipelineStage : "in stage"
    PipelineStage }o--|| Pipeline : "belongs to"

    ConversationProspect ||--o{ Message : "contains"
    ConversationProspect }o--|| Contact : "with"

    Funnel ||--o{ FunnelStep : "has steps"
    FunnelStep ||--o{ FunnelComponent : "contains"

    ApiSession }o--|| ApiRefreshToken : "uses"
    ApiSession }o--|| User : "authenticates"

Account Hierarchy Pattern​

Multi-Tenant Architecture: All data is scoped to accounts

// Parent-Child Account Relationship
{
    _id: ObjectId("agency_account"),
    business: ObjectId("contact_id"),          // Business contact info
    parent_account: null,                      // Top-level agency
    manager: ObjectId("user_id"),              // Account manager
    main: true
}

{
    _id: ObjectId("client_account"),
    business: ObjectId("contact_id"),
    parent_account: ObjectId("agency_account"), // Child of agency
    manager: ObjectId("user_id"),
    main: false
}

// Referral tracking
{
    _id: ObjectId("referred_account"),
    referred_by: ObjectId("referring_account"), // Affiliate tracking
    affiliate: {
        code: "REF123",
        clicks: 45
    }
}

Query Pattern: Always filter by account hierarchy

// Find all accounts in hierarchy
const accountIds = await Account.find({
  $or: [{ _id: mainAccountId }, { parent_account: mainAccountId }],
}).distinct('_id');

// Query scoped to account
const contacts = await Contact.find({
  parent_account: { $in: accountIds },
});

CRM Data Flow​

Contact → Deal → Pipeline Flow:

// 1. Contact created
const contact = await Contact.create({
  email: 'lead@example.com',
  first_name: 'John',
  last_name: 'Doe',
  parent_account: accountId,
  created_by: userId,
});

// 2. Deal associated with contact
const deal = await Deal.create({
  contact_id: contact._id,
  account_id: accountId,
  assigned_to: userId,
  pipeline_stage_id: stageId,
  title: 'Website Design Project',
  value: 5000,
  currency: 'USD',
});

// 3. Activity logged
const activity = await Activity.create({
  account_id: accountId,
  user_id: userId,
  contact_id: contact._id,
  deal_id: deal._id,
  type: 'call',
  note: 'Initial consultation call',
  duration: 1800, // 30 minutes in seconds
});

// 4. Move deal through pipeline
await deal.updateOne({
  pipeline_stage_id: nextStageId,
  $push: {
    stage_history: {
      stage_id: nextStageId,
      entered_at: new Date(),
      moved_by: userId,
    },
  },
});

Conversation Integration Pattern​

Email/SMS → Conversation → Contact Link:

// When email is sent via mail.js utility
// 1. Find or create conversation
let convo = await ConversationProspect.findOne({
  contact_id: contactId,
  account_id: accountId,
  is_archive: false,
});

if (!convo) {
  convo = await ConversationProspect.create({
    contact_id: contactId,
    account_id: accountId,
    created_by: userId,
    users: [userId],
    unread_count: { [userId]: 0 },
    last_contacted: { [contactId]: timestamp },
    is_open: true,
  });
}

// 2. Create communication record
const communication = await Communication.create({
  conversation_id: convo._id,
  account_id: accountId,
  contact_id: contactId,
  sent_by: userId,
  type: 'OUTGOING',
  module: 'SENDGRID',
  message_type: 'EMAIL',
  msgID: sendgridMessageId,
  body: emailContent,
});

// 3. Update conversation
await convo.updateOne({
  last_activity: communication._id,
  $inc: {
    [`unread_count.${otherUserId}`]: 1, // Increment for other users
  },
});

// 4. Emit real-time event
await emitMessage(communication._id);
await emitConversation(convo._id);

Store & Billing Integration​

Product → Price → Order → Subscription Flow:

// 1. Product catalog
const product = await StoreProduct.create({
  account_id: accountId,
  name: 'Website Maintenance',
  description: 'Monthly website maintenance service',
  type: 'service',
  recurring: true,
});

// 2. Pricing tiers
const price = await StorePrice.create({
  product_id: product._id,
  account_id: accountId,
  amount: 99.0,
  currency: 'USD',
  interval: 'month',
  active: true,
});

// 3. Customer order
const order = await StoreOrder.create({
  account_id: sellerAccountId,
  customer_account: buyerAccountId,
  items: [
    {
      product_id: product._id,
      price_id: price._id,
      quantity: 1,
    },
  ],
  total: 99.0,
  status: 'completed',
});

// 4. Create subscription
const subscription = await StoreSubscription.create({
  account_id: sellerAccountId,
  customer_account: buyerAccountId,
  product_id: product._id,
  price_id: price._id,
  status: 'active',
  current_period_start: new Date(),
  current_period_end: addMonths(new Date(), 1),
});

// 5. Process billing
const charge = await BillingCharge.create({
  customer_id: billingCustomerId,
  amount: 99.0,
  currency: 'USD',
  subscription_id: subscription._id,
  status: 'succeeded',
  stripe_charge_id: 'ch_xxx',
});

Queue & Background Processing Pattern​

Trigger → Queue → Process → Complete:

// 1. User action triggers queue entry
const queueItem = await InstareportsQueue.create({
  account_id: accountId,
  report_id: reportId,
  type: 'generate',
  status: 'pending',
  priority: 1,
  created_by: userId,
});

// 2. Queue Manager picks up job (via cron)
const pendingJobs = await InstareportsQueue.find({
  status: 'pending',
  attempts: { $lt: 3 },
})
  .sort({ priority: -1, created_at: 1 })
  .limit(10);

// 3. Process job
for (const job of pendingJobs) {
  await job.updateOne({
    status: 'processing',
    started_at: new Date(),
    $inc: { attempts: 1 },
  });

  try {
    // Generate report
    const report = await generateReport(job.report_id);

    // Mark complete
    await job.updateOne({
      status: 'completed',
      completed_at: new Date(),
      result: { pdf_url: report.url },
    });
  } catch (error) {
    await job.updateOne({
      status: 'failed',
      error: error.message,
      failed_at: new Date(),
    });
  }
}

Integration Keys & OAuth Tokens​

Secure Credential Storage Pattern:

// Account-scoped API keys
const googleAdsToken = await GoogleAdsToken.create({
  account_id: accountId,
  owner: userId,
  refresh_token: encryptedRefreshToken, // Encrypted at rest
  access_token: encryptedAccessToken,
  token_expiration: expiryDate,
  customer_id: googleCustomerId,
  scopes: ['adwords'],
});

// Retrieve and refresh if needed
const token = await GoogleAdsToken.findOne({
  account_id: accountId,
  owner: userId,
});

if (new Date() > token.token_expiration) {
  // Refresh token logic
  const newToken = await refreshGoogleToken(token.refresh_token);
  await token.updateOne({
    access_token: newToken.access_token,
    token_expiration: calculateExpiry(newToken.expires_in),
  });
}

// Use in API call
const campaigns = await googleAdsClient.listCampaigns({
  customerId: token.customer_id,
  accessToken: decryptToken(token.access_token),
});

Schema Design Best Practices​

Flexible Schema with Strict: False​

Pattern: Allow dynamic fields while maintaining typed core fields

const schema = new mongoose.Schema(
  {
    // Typed core fields
    user_id: { type: mongoose.SchemaTypes.ObjectId, ref: 'User', required: true },
    account_id: { type: mongoose.SchemaTypes.ObjectId, ref: 'Account', required: true },
    status: { type: String, enum: ['active', 'inactive'], default: 'active' },

    // Defined optional fields
    metadata: Object,

    // ... other fields
  },
  {
    strict: false, // Allow arbitrary additional fields
    timestamps: { createdAt: 'created_at', updatedAt: 'updated_at' },
  },
);

Benefits:

  • Rapid Development: Add fields without schema migrations
  • Integration Flexibility: Store varying external API responses
  • Backward Compatibility: Old code won't break if new fields added

Trade-offs:

  • Less validation at database level
  • Requires application-level validation
  • Can accumulate unused fields over time

Embedded vs Referenced Documents​

Embedded Documents (Store data inline):

// Good for: Small, rarely queried independently, strong ownership
{
    _id: ObjectId("funnel"),
    name: "Landing Page",
    steps: [  // Embedded array
        {
            name: "Hero Section",
            order: 1,
            components: [/* nested components */]
        }
    ]
}

Referenced Documents (Store ObjectId pointer):

// Good for: Large, queried independently, shared between documents
{
    _id: ObjectId("deal"),
    contact_id: ObjectId("contact"),  // Reference
    assigned_to: ObjectId("user"),    // Reference
    pipeline_stage_id: ObjectId("stage")  // Reference
}

// Query with population
const deal = await Deal.findById(dealId)
    .populate('contact_id')
    .populate('assigned_to')
    .populate('pipeline_stage_id');

Index Strategy​

Critical Indexes (All models should have):

// Account scoping (multi-tenant)
schema.index({ account_id: 1 });

// User queries
schema.index({ user_id: 1 });
schema.index({ account_id: 1, user_id: 1 });

// Status filtering
schema.index({ status: 1 });
schema.index({ account_id: 1, status: 1 });

// Date range queries
schema.index({ created_at: -1 });
schema.index({ account_id: 1, created_at: -1 });

// Unique constraints
schema.index({ email: 1 }, { unique: true, sparse: true });

Compound Index Order: Most selective field first

// Good: Status has few unique values, account_id is selective
schema.index({ account_id: 1, status: 1, created_at: -1 });

// Bad: status first makes index less effective
schema.index({ status: 1, account_id: 1, created_at: -1 });

Usage Patterns​

Importing Models​

// In any service (after copySharedFiles)
const Account = require('./models/account');
const User = require('./models/user');
const Contact = require('./models/contact');
const Deal = require('./models/deal');

// Query examples
const account = await Account.findById(accountId);
const users = await User.find({ account_id: accountId });
const deal = await Deal.findOne({ _id: dealId }).populate('contact_id');

Multi-Tenant Query Pattern​

// Always scope by account
const contacts = await Contact.find({
  account_id: req.auth.account_id, // From JWT token
  status: 'active',
});

// Include account hierarchy (parent + children)
const accountIds = await Account.find({
  $or: [{ _id: mainAccountId }, { parent_account: mainAccountId }],
}).distinct('_id');

const contacts = await Contact.find({
  account_id: { $in: accountIds },
});

Performance Optimization​

Lean Queries (Read-only data):

// Without lean: Full Mongoose document (heavy)
const account = await Account.findById(accountId);
// Returns: Mongoose document with methods, getters, setters

// With lean: Plain JavaScript object (light)
const account = await Account.findById(accountId).lean();
// Returns: Plain object, ~2-5x faster

// Use lean when:
// - No need to call instance methods
// - No need to save changes
// - Read-only data for API responses

Projection (Select specific fields):

// Bad: Fetch all fields (including large ones)
const users = await User.find({ account_id: accountId });

// Good: Select only needed fields
const users = await User.find({ account_id: accountId })
  .select('first_name last_name email')
  .lean();

// Even better: Exclude heavy fields
const users = await User.find({ account_id: accountId })
  .select('-password -reset_token') // Exclude sensitive fields
  .lean();

Batch Operations:

// Bad: Multiple individual queries
for (const userId of userIds) {
  await User.updateOne({ _id: userId }, { last_seen: new Date() });
}

// Good: Single bulk operation
await User.updateMany({ _id: { $in: userIds } }, { last_seen: new Date() });

// Better: Bulk write for different operations
await User.bulkWrite([
  {
    updateOne: {
      filter: { _id: userId1 },
      update: { $set: { status: 'active' } },
    },
  },
  {
    updateOne: {
      filter: { _id: userId2 },
      update: { $set: { status: 'inactive' } },
    },
  },
]);

Performance Metrics​

Expected Query Times (with proper indexes):

OperationExpected TimeOptimization
findById()< 10msIndexed by _id automatically
find({ account_id })< 50msCompound index with account_id
find() with .populate()< 100msUse .lean() when possible
aggregate() pipelines< 200msIndex pipeline match stages
Bulk operations< 500msBatch 1000 documents max

Memory Management:

  • Model Instances: ~2-5KB per document (without lean)
  • Lean Documents: ~0.5-1KB per document
  • Best Practice: Use .lean() for large result sets
// Bad: 10,000 full documents = ~30MB memory
const contacts = await Contact.find({ account_id });

// Good: 10,000 lean documents = ~7MB memory
const contacts = await Contact.find({ account_id }).lean();

Security Considerations​

Multi-Tenant Isolation​

Account-Based Scoping: Every query must filter by account

// Security layer in middleware
router.use((req, res, next) => {
  req.account_id = req.auth.account_id; // From JWT
  next();
});

// All queries scoped
const data = await Model.find({
  account_id: req.account_id, // Required!
});

Data Encryption​

At Rest:

  • MongoDB encryption at rest (server-side)
  • Application-level encryption for API keys/tokens
  • Passwords hashed with crypto.scrypt (never stored plain)

Encrypted Fields Pattern:

// Store encrypted
const encryptedToken = encryptString(apiToken);
await GoogleAdsToken.create({
  account_id: accountId,
  access_token: encryptedToken,
  refresh_token: encryptedRefreshToken,
});

// Retrieve and decrypt
const token = await GoogleAdsToken.findOne({ account_id });
const decryptedToken = decryptString(token.access_token);

Audit Logging​

Change Tracking: CRM models include audit fields

{
    created_by: ObjectId("user"),
    created_at: ISODate("..."),
    updated_at: ISODate("..."),
    modified_by: ObjectId("user"),
    deleted_at: ISODate("..."),  // Soft delete
    deleted_by: ObjectId("user")
}

Testing Models​

Unit Tests​

describe('User Model', () => {
  it('should update conversation status', async () => {
    const user = await User.create({
      email: 'test@example.com',
      account: accountId,
    });

    await user.updateConversationsStatus('socket123', 'ACTIVE');

    const updated = await User.findById(user._id);
    expect(updated.conversation_active_status).toBe('ACTIVE');
  });
});

Integration Tests​

describe('Deal Model', () => {
  it('should track stage history', async () => {
    const deal = await Deal.create({
      account_id: accountId,
      contact_id: contactId,
      pipeline_stage_id: stage1Id,
      title: 'Test Deal',
    });

    await deal.updateOne({
      pipeline_stage_id: stage2Id,
      $push: {
        stage_history: {
          stage_id: stage2Id,
          entered_at: new Date(),
        },
      },
    });

    const updated = await Deal.findById(deal._id);
    expect(updated.stage_history).toHaveLength(1);
    expect(updated.pipeline_stage_id).toBe(stage2Id);
  });
});

Resource Type: Mongoose Schemas
Total: 207 models
Collections: 200+ MongoDB collections
Distribution: 7 services receive model copies
Git Strategy: Only shared/models/ is version controlled
Last Updated: October 2025

💬

Documentation Assistant

Ask me anything about the docs

Hi! I'm your documentation assistant. Ask me anything about the docs!

I can help you with:
- Code examples
- Configuration details
- Troubleshooting
- Best practices

Try asking: How do I configure the API?
09:31 AM