Skip to main content

Leads Service

📖 Overview

Controller Path: internal/api/v1/inbound/Controllers/leads.js
Service Path: internal/api/v1/inbound/Services/leads.js

The Leads service provides comprehensive business logic for managing inbound leads captured through various integrations.

Core Responsibilities

  • Lead Retrieval & Filtering: Advanced filtering, search, and pagination with role-based access control
  • CRM Integration: Manual and automatic contact creation from lead data with duplicate detection
  • InstaSite/InstaReport Rebuild: Retry failed builds with OneBalance verification
  • Bulk Operations: Efficient soft/hard deletion with multi-tenant isolation
  • Analytics & Reporting: Widget data generation for dashboard metrics and trend analysis
  • Phone Formatting: International phone number normalization with country code detection

🗄️ Collections Used

📚 Full Schema: See Database Collections Documentation

inbound.leads

  • Operations: Read/Write for lead storage and retrieval
  • Model: shared/models/leads-data.js
  • Usage: Main collection storing all inbound leads with campaign metadata

campaigns

  • Operations: Read for campaign configuration
  • Model: shared/models/campaign-data.js
  • Usage: Campaign settings for contact creation and integration config

crm.contacts

  • Operations: Write for contact creation from leads
  • Model: shared/models/contact.js
  • Usage: Creating business/person contacts in CRM from lead data

_accounts

  • Operations: Read for account details and OneBalance verification
  • Model: shared/models/account.js
  • Usage: Account tier and subscription validation

instasites

  • Operations: Write for generating InstaSites
  • Model: shared/models/instasites.js
  • Usage: InstaSite creation from leads

instareports

  • Operations: Write for generating InstaReports
  • Model: shared/models/instareports.js
  • Usage: InstaReport creation from leads

inbound.widgets

  • Operations: Read for pre-computed statistics
  • Model: shared/models/inbound-widgets.js
  • Usage: Dashboard analytics and widget data

🔄 Data Flow

Lead Retrieval with Filtering

sequenceDiagram
participant Client
participant LeadsService
participant MongoDB
participant Utilities

Client->>LeadsService: getFilters(account_id, filters)
LeadsService->>MongoDB: Aggregate filters with counts

alt Role-based filtering
LeadsService->>LeadsService: Add user_id filter (non-admin)
end

MongoDB-->>LeadsService: Filter counts by category
LeadsService-->>Client: Return filter options

Client->>LeadsService: listAllLeads(account_id, filters, pagination)
LeadsService->>LeadsService: Build query conditions
LeadsService->>LeadsService: Apply role-based restrictions
LeadsService->>MongoDB: Query with filters, sort, pagination
MongoDB-->>LeadsService: Lead documents with populated data
LeadsService->>Utilities: generatePagination(count, page, limit)
Utilities-->>LeadsService: Pagination metadata
LeadsService-->>Client: Leads + pagination

CRM Contact Creation from Lead

sequenceDiagram
participant Client
participant LeadsService
participant MongoDB
participant CRMContact
participant CountryCodes

Client->>LeadsService: addToCrm(contactdata)
LeadsService->>LeadsService: Extract contact fields
LeadsService->>CountryCodes: Match country code
CountryCodes-->>LeadsService: Country calling code

LeadsService->>LeadsService: Format phone (+country_code)

alt Missing email AND phone
LeadsService-->>Client: Error: NOT_ENOUGH_DATA
end

LeadsService->>MongoDB: findLeadAndCampaign(lead_id)
MongoDB-->>LeadsService: Lead + campaign data

alt Campaign: add_person_or_business = business
LeadsService->>LeadsService: Combine first_name + last_name → name
LeadsService->>CRMContact: addBusinesses(contacts)
else Campaign: person
LeadsService->>CRMContact: addPeople(contacts)
end

alt Contact created successfully
CRMContact-->>LeadsService: New contact with _id
LeadsService->>MongoDB: Update lead with contact_id
MongoDB-->>LeadsService: Update confirmed
LeadsService-->>Client: Success with new contact
else Contact already exists (duplicate)
CRMContact-->>LeadsService: Error: CONTACT_ALREADY_EXIST + contact_id
LeadsService->>MongoDB: Update lead with existing contact_id
LeadsService->>MongoDB: Find existing contact details
MongoDB-->>LeadsService: Existing contact document
LeadsService-->>Client: Success with existing contact
end

InstaSite/InstaReport Rebuild from Failed Lead

sequenceDiagram
participant Client
participant LeadsService
participant MongoDB
participant OneBalance
participant InstaSiteService
participant InstaReportService

Client->>LeadsService: rebuildFromLead(lead_id)
LeadsService->>MongoDB: findLeadAndCampaign(lead_id)
MongoDB-->>LeadsService: Lead + campaign data

alt No build_failed field
LeadsService-->>Client: Error: No failed build
end

alt integration != instasite_forms|instareport_forms
LeadsService-->>Client: Error: Wrong campaign type
end

alt add_person_or_business != business
LeadsService-->>Client: Error: Must be business type
end

alt No contact_id in lead
LeadsService-->>Client: Error: Contact ID missing
end

LeadsService->>MongoDB: Find account by account_id
MongoDB-->>LeadsService: Account document

LeadsService->>OneBalance: verifyBalance(event, account, quantity: 0)

alt Insufficient balance
OneBalance-->>LeadsService: Error: Usage exceeded
LeadsService-->>Client: Error: No usage available
end

OneBalance-->>LeadsService: Balance verified

alt integration = instasite_forms
LeadsService->>InstaSiteService: instasite(account_id, owner, [business_id], template)
InstaSiteService-->>LeadsService: [{ instasite: site_id }]
LeadsService->>MongoDB: Update lead: {instasite: site_id, $unset: build_failed}
else integration = instareport_forms
LeadsService->>InstaReportService: instareport(account_id, owner, businesses, template)
InstaReportService-->>LeadsService: [{ instareport: report_id }]
LeadsService->>MongoDB: Update lead: {instareport: report_id, $unset: build_failed}
end

MongoDB-->>LeadsService: Update confirmed
LeadsService-->>Client: 200 Success: Build completed

🎯 API Endpoints

1. GET /v1/inbound/leads/filters

Purpose: Generate dynamic filter options with counts for lead filtering UI.

Authentication: Required (JWT)

Authorization: Scopes inbound, inbound.leads

Query Parameters:

{
startdate: String, // OPTIONAL - ISO date for range
enddate: String, // OPTIONAL - ISO date for range
freshness: String, // OPTIONAL - Pre-filter by lead freshness
type: String, // OPTIONAL - Pre-filter by lead type
integration: String, // OPTIONAL - Pre-filter by integration
reps: String, // OPTIONAL - Representative IDs (comma-separated)
searchtext: String // OPTIONAL - Text search query
}

Request Example:

GET /v1/inbound/leads/filters?startdate=2025-01-01&enddate=2025-12-31&freshness=new
Authorization: Bearer {JWT_TOKEN}

Response:

// Success - 200
{
success: true,
message: "SUCCESS",
data: {
integrations: [
{ value: "facebook_ads", label: "Facebook Ads", count: 42 },
{ value: "clickfunnels", label: "ClickFunnels", count: 18 }
],
reps: [
{
id: "507f1f77bcf86cd799439011",
name: "John Doe",
count: 35
}
],
freshness: [
{ value: "new", label: "New", count: 20 },
{ value: "read", label: "Read", count: 30 },
{ value: "contacted", label: "Contacted", count: 10 }
],
total_count: 60,
today_count: 5,
this_week_count: 15,
this_month_count: 40
}
}

MongoDB Operations:

OperationCollectionQueryPurpose
getFiltersleads-dataAggregationGenerate filter counts

Role-Based Filtering:

  • Admin: Sees all account leads
  • Non-Admin: Only leads assigned to them (user_id)

2. GET /v1/inbound/leads or /v1/inbound/leads/:lead_id

Purpose: Retrieve all leads or a specific lead with pagination and filtering.

Authentication: Required (JWT)

Authorization: Scopes inbound, inbound.leads

Query Parameters:

{
page: Number, // Page number (1-based)
limit: Number, // Items per page
sort_by: String, // Sort field ("created_at", "freshness", "integration")
order: String, // "asc" | "desc" (default: "desc")
searchtext: String, // OPTIONAL - Search in lead data
startdate: String, // OPTIONAL - ISO date
enddate: String, // OPTIONAL - ISO date
freshness: String, // OPTIONAL - Lead freshness filter
type: String, // OPTIONAL - Lead type filter
integration: String, // OPTIONAL - Integration filter
reps: String // OPTIONAL - Representative filter
}

Path Parameters (Optional):

{
lead_id: ObjectId; // OPTIONAL - Get single lead by ID
}

Request Examples:

# List all leads with pagination
GET /v1/inbound/leads?page=1&limit=20&sort_by=created_at&order=desc
Authorization: Bearer {JWT_TOKEN}

# Get single lead by ID
GET /v1/inbound/leads/507f1f77bcf86cd799439011
Authorization: Bearer {JWT_TOKEN}

# Filter by integration and date range
GET /v1/inbound/leads?integration=facebook_ads&startdate=2025-01-01&enddate=2025-12-31
Authorization: Bearer {JWT_TOKEN}

Response - List:

// Success - 200
{
success: true,
message: "SUCCESS",
data: [
{
_id: ObjectId,
lead_id: String, // External lead ID
lead_data: { // Raw lead data from integration
id: String,
created_time: String,
field_data: Array,
// ... integration-specific fields
},
campaign_id: ObjectId,
campaign_name: String, // Populated from campaign
integration: String, // From campaign
account_id: ObjectId,
owner: ObjectId,
user_id: ObjectId, // Assigned representative
contact_id: ObjectId, // OPTIONAL - CRM contact
freshness: String, // "new" | "read" | "contacted"
errors: [ // OPTIONAL - Processing errors
{
type: String, // "contactCreation" | "dealCreateError" | etc.
details: Object
}
],
build_failed: { // OPTIONAL - InstaReport/InstaSite build failure
message: String,
error: Object
},
instasite: ObjectId, // OPTIONAL - Generated InstaSite
instareport: ObjectId, // OPTIONAL - Generated InstaReport
created_at: Date,
updated_at: Date
}
],
pagination: {
page: 1,
limit: 20,
total: 150,
total_pages: 8,
has_next: true,
has_prev: false
}
}

Response - Single Lead:

// Success - 200
{
success: true,
message: "SUCCESS",
data: {
// ... all lead fields
}
}

// Error - Lead not found - 404
{
success: false,
errno: 404,
message: "Lead not found",
error_code: "LEAD_NOT_FOUND"
}

MongoDB Operations:

OperationCollectionQueryPurpose
findleads-dataComplex filter with paginationGet leads with sorting

Sorting Options:

  • created_at - Sort by lead capture date (default)
  • freshness - Sort by lead status
  • integration - Sort by integration type
  • Default: created_at descending (newest first)

Search Functionality:

  • Searches across lead data fields (email, name, phone, etc.)
  • Case-insensitive partial matching
  • Searches within JSON lead_data structure

Role-Based Access:

// Non-admin users only see their assigned leads
if (user.role !== 'admin') {
query.user_id = new mongoose.Types.ObjectId(uid);
}

3. POST /v1/inbound/leads/:leadid/rebuild

Purpose: Rebuild failed InstaSite or InstaReport from lead data.

Authentication: Required (JWT)

Authorization: Scopes inbound, inbound.leads

Request Parameters:

// Path Parameters
{
leadid: ObjectId; // REQUIRED - Lead ID to rebuild from
}

Request Example:

POST /v1/inbound/leads/507f1f77bcf86cd799439011/rebuild
Authorization: Bearer {JWT_TOKEN}

Response:

// Success - 200
{
success: true,
message: "SUCCESS"
}

// Error - No build failure - 422
{
success: false,
errno: 422,
message: "There is no failed instasite/report associated with this lead."
}

// Error - Not InstaSites/InstaReport campaign - 422
{
success: false,
errno: 422,
message: "The selected lead is not of a campaign related to Instasite/InstaReport"
}

// Error - Contact setting not business - 422
{
success: false,
errno: 422,
message: "The contact create setting for the campaign is set to person"
}

// Error - Contact ID missing - 422
{
success: false,
errno: 422,
message: "Contact Id not found in lead info"
}

MongoDB Operations:

OperationCollectionQueryPurpose
findLeadAndCampaignleads-data + join{_id}Get lead with campaign details
findByIdaccounts{_id}Get account for OneBalance check
updateleads-data{_id}Update lead with build result

Business Rules:

  1. Build Failure Required: Lead must have build_failed field
  2. Campaign Type: Only instasite_forms or instareport_forms
  3. Contact Type: Campaign must have add_person_or_business: "business"
  4. Contact Exists: Lead must have valid contact_id
  5. Usage Check: Verifies OneBalance usage before building
  6. Template: Uses campaign's instasite_template_id or instareport_template_id

OneBalance Integration:

// Verify usage available (doesn't consume, just checks)
await verifyBalance({
event: 'instasite', // or 'instareport'
account: account,
user_id: uid,
quantity: 0, // 0 = check only, don't consume
});

4. DELETE /v1/inbound/leads

Purpose: Bulk delete leads (soft delete or permanent).

Authentication: Required (JWT)

Authorization: Scopes inbound, inbound.leads

Request Parameters:

// Body
{
leads: [ObjectId]; // REQUIRED - Array of lead IDs to delete
}

// Query Parameters
{
deleteall: Boolean; // OPTIONAL - "true" = hard delete, "false" = soft delete
}

Request Example:

DELETE /v1/inbound/leads?deleteall=false
Authorization: Bearer {JWT_TOKEN}
Content-Type: application/json

{
"leads": [
"507f1f77bcf86cd799439011",
"507f1f77bcf86cd799439012",
"507f1f77bcf86cd799439013"
]
}

Response:

// Success - 200
{
success: true,
message: "SUCCESS",
data: 3 // Number of leads deleted
}

Deletion Types:

TypedeleteallBehaviorUse Case
Soft Deletefalseor omittedSetsis_deleted: trueDefault - preserves historical data
Hard DeletetruePermanently removes documentsData cleanup, GDPR compliance

MongoDB Operations:

OperationCollectionQueryPurpose
deleteAllleads-data{_id: {$in: [...], account_id, owner}}Delete leads

Security:

  • Validates account_id and owner to prevent unauthorized deletion
  • Bulk operation uses $in operator for efficiency
  • Returns count of successfully deleted leads

5. POST /v1/inbound/leads/add-to-crm

Purpose: Manually add lead to CRM (creates contact from lead data).

Authentication: Required (JWT)

Authorization: Scopes inbound, inbound.leads

Request Parameters:

// Body
{
contactdata: {
lead_id: ObjectId, // REQUIRED - Lead to create contact from
first_name: String, // OPTIONAL - Override lead data
last_name: String, // OPTIONAL - Override lead data
name: String, // OPTIONAL - Full name (for business)
email: String, // OPTIONAL - Override lead data
phone: String, // OPTIONAL - Override lead data
country: String // OPTIONAL - For phone formatting
}
}

Request Example:

POST /v1/inbound/leads/add-to-crm
Authorization: Bearer {JWT_TOKEN}
Content-Type: application/json

{
"contactdata": {
"lead_id": "507f1f77bcf86cd799439011",
"first_name": "John",
"last_name": "Doe",
"email": "john.doe@example.com",
"phone": "4155551234",
"country": "United States"
}
}

Response:

// Success - 200
{
success: true,
message: "SUCCESS",
data: {
errors: [],
savedContacts: [
{
_id: ObjectId,
name: "John Doe",
email: "john.doe@example.com",
phone: "+14155551234",
address: {
country: "United States"
},
source: "inbound",
account_id: ObjectId,
owner: ObjectId,
created_at: Date,
updated_at: Date
}
]
}
}

// Error - Not enough data - 400
{
success: false,
data: "NOT_ENOUGH_DATA"
}

// Success - Duplicate contact (auto-handled) - 200
{
success: true,
message: "SUCCESS",
data: {
errors: [],
savedContacts: [
{
_id: ObjectId, // Existing contact ID
// ... existing contact data
}
]
}
}

Phone Formatting:

// Country code detection and formatting
if (country && codes) {
const match = codes.find(c => c.country.match(new RegExp('\\b' + country + '\\b', 'gi')));

if (match?.countryCodes?.[0] && phone) {
phone = '+' + match.countryCodes[0] + phone;
}
}

MongoDB Operations:

OperationCollectionQueryPurpose
findLeadAndCampaignleads-data + join{_id}Get lead with campaign
addPeople or addBusinessescontactsNew documentCreate CRM contact
updateleads-data{_id}Link lead to contact
findOnecontacts{_id}Get existing contact (on duplicate)

Duplicate Handling:

  • If contact already exists (same email/phone), retrieves existing contact
  • Updates lead with existing contact_id
  • Returns existing contact as success (not an error)
  • This allows re-linking leads to contacts without creating duplicates

Business Rules:

  1. Required Fields: Must have email OR phone (at least one)
  2. Source Attribution: All contacts created with source: "inbound"
  3. Contact Type: Determined by campaign's add_person_or_business setting
  4. Name Handling: For business contacts, first_name + last_name combined into name
  5. Country Codes: Automatically prepended to phone numbers when country provided

6. GET /v1/inbound/leads/widget

Purpose: Retrieve lead statistics for dashboard widgets.

Authentication: Required (JWT)

Authorization: Scopes inbound, inbound.leads

Request Parameters:

// Query Parameters
{
startdate: String, // OPTIONAL - ISO date for range
enddate: String // OPTIONAL - ISO date for range
}

Request Example:

GET /v1/inbound/leads/widget?startdate=2025-01-01&enddate=2025-12-31
Authorization: Bearer {JWT_TOKEN}

Response:

// Success - 200
{
success: true,
data: {
total_leads: 450,
new_leads: 25,
contacted_leads: 200,
converted_leads: 180,

// Time-based breakdowns
today: 5,
this_week: 35,
this_month: 120,
last_month: 150,

// Integration breakdown
by_integration: [
{
integration: "facebook_ads",
count: 200,
percentage: 44.4
},
{
integration: "clickfunnels",
count: 150,
percentage: 33.3
}
],

// Representative performance
by_rep: [
{
user_id: ObjectId,
name: "John Doe",
leads_assigned: 100,
leads_contacted: 75,
conversion_rate: 75.0
}
],

// Trend data (for charts)
daily_trend: [
{ date: "2025-01-01", count: 10 },
{ date: "2025-01-02", count: 15 }
],

// Top campaigns
top_campaigns: [
{
campaign_id: ObjectId,
campaign_name: "Summer Lead Gen",
lead_count: 85,
contact_rate: 90.5
}
]
}
}

MongoDB Operations:

OperationCollectionQueryPurpose
getLeadsWidgetDataleads-data + widgetsComplex aggregationGenerate statistics

Widget Metrics:

  1. Lead Counts: Total, new, contacted, converted
  2. Time Breakdowns: Today, this week, this month, last month
  3. Integration Performance: Leads by source with percentages
  4. Representative Performance: Leads assigned/contacted per rep
  5. Trends: Daily/weekly lead capture trends
  6. Top Campaigns: Best performing campaigns by lead volume

Role-Based Filtering:

  • Admin: Sees all account widget data
  • Non-Admin: Only sees data for leads assigned to them

📊 Data Models

Lead Data

{
_id: ObjectId,
lead_id: String, // External lead ID (from integration)

// Lead Source
lead_data: Object, // Raw JSON from integration webhook
campaign_id: ObjectId, // Parent campaign
integration: String, // Inherited from campaign

// Account & Assignment
account_id: ObjectId, // DashClicks account
owner: ObjectId, // Campaign owner
user_id: ObjectId, // Assigned representative

// CRM Integration
contact_id: ObjectId, // OPTIONAL - Created CRM contact

// Lead Status
freshness: String, // "new" | "read" | "contacted"

// Error Tracking
errors: [
{
type: String, // Error type
details: Object // Error details
}
],

// InstaSites/InstaReport
instasite: ObjectId, // OPTIONAL - Generated InstaSite
instareport: ObjectId, // OPTIONAL - Generated InstaReport
build_failed: { // OPTIONAL - Build failure details
message: String,
error: Object,
timestamp: Date
},

// Metadata
is_deleted: Boolean, // Soft delete flag
created_at: Date,
updated_at: Date
}

🔀 Integration Points

Internal Services

  • CRM Module: Contact creation (person/business)
  • InstaSites: Auto-build sites for business leads
  • InstaReports: Auto-build reports for business leads
  • OneBalance: Usage verification before builds
  • Campaigns Module: Campaign details and settings

Utilities

  • country-codes: Phone number formatting
  • crmContact: Contact creation utilities
  • instasite: InstaSite builder
  • instareport: InstaReport builder

🛡️ Error Handling

Common Errors

ErrorStatusMessageCause
Lead not found404"Lead not found"Invalid lead ID
Not enough data400"NOT_ENOUGH_DATA"Missing email and phone
No build failure422"There is no failed instasite/report..."No build_failed field
Wrong campaign type422"The selected lead is not of a campaign..."Not InstaSites/InstaReport
Wrong contact type422"The contact create setting..."Not set to business
Missing contact422"Contact Id not found..."No contact_id in lead

📈 Performance Considerations

  • Pagination: Always use pagination for lead lists
  • Indexing: Ensure indexes on account_id, user_id, campaign_id, created_at, is_deleted
  • Search: Consider full-text search indexes for lead_data JSON fields
  • Aggregations: Widget queries can be expensive - consider caching
  • Role Filtering: Non-admin queries more efficient (smaller result sets)

🔒 Security Considerations

  1. Multi-Tenant Isolation: All queries filtered by account_id
  2. Role-Based Access: Non-admins only see assigned leads
  3. Deletion Validation: Validates ownership before deletion
  4. CRM Integration: Source tracking prevents unauthorized contact creation
  5. Build Verification: OneBalance check prevents unlimited builds

⚠️ Important Notes

  1. Role-Based Access Control: All functions enforce account_id scoping. Non-admin users see only leads assigned to them (user_id filter). Always validate role before querying.

  2. Phone Number Formatting: Phone formatting requires valid country name matching country-codes data. If country not found or phone already has +, original phone used. Always validate phone format after processing.

  3. Contact Type Consistency: Campaign's add_person_or_business setting determines contact type (person vs business). Business contacts use single name field, person contacts use first_name + last_name. InstaSite/InstaReport builds require business type.

  4. Duplicate Contact Handling: CRM module detects duplicates by email or phone. When duplicate detected, lead associated with existing contact (not error). Check for error_type: 'CONTACT_ALREADY_EXIST' to handle gracefully.

  5. OneBalance Verification: Rebuild operations verify OneBalance usage before building. quantity: 0 means check only (no consumption). Actual consumption happens in instasite/instareport utilities. Always check verifyBalance errors.

  6. Soft vs Hard Delete: Default deletion is soft (sets is_deleted: true). Hard delete (deleteall: true) permanently removes data. Use hard delete for GDPR compliance. Soft deletes preserve historical data for reporting.

  7. Single Lead Query: When lead_id in params, function returns single lead object (not array). If lead not found or access denied, throws 404 error. Check for params.lead_id to determine response structure.

  8. Widget Data Caching: Widget queries are expensive (multiple aggregations). Consider caching for 5-15 minutes. Invalidate cache on lead creation/update/deletion. Pre-compute for large accounts using background jobs.

  9. Search Performance: Text search operates on lead_data JSON fields (email, name, phone, custom fields). Large datasets require text indexes. Consider limiting search to specific fields for better performance.

  10. Build Failure Tracking: Failed InstaSite/InstaReport builds store error in build_failed field. Check for this field before attempting rebuild. Remove build_failed on successful rebuild. Errors include timestamp and full error object.

  11. Source Attribution: All contacts created via addToCrm tagged with source: 'inbound'. This enables source tracking and reporting. Don't override source in contact creation.

  12. Pagination Best Practices: Always use pagination for lead lists. Default sort is created_at desc (newest first). Ensure compound indexes on frequently filtered fields (account_id, user_id, created_at, is_deleted).


💬

Documentation Assistant

Ask me anything about the docs

Hi! I'm your documentation assistant. Ask me anything about the docs!

I can help you with:
- Code examples
- Configuration details
- Troubleshooting
- Best practices

Try asking: How do I configure the API?
09:31 AM