Leads Service
📖 Overview
Controller Path: internal/api/v1/inbound/Controllers/leads.js
Service Path: internal/api/v1/inbound/Services/leads.js
The Leads service provides comprehensive business logic for managing inbound leads captured through various integrations.
Core Responsibilities
- Lead Retrieval & Filtering: Advanced filtering, search, and pagination with role-based access control
- CRM Integration: Manual and automatic contact creation from lead data with duplicate detection
- InstaSite/InstaReport Rebuild: Retry failed builds with OneBalance verification
- Bulk Operations: Efficient soft/hard deletion with multi-tenant isolation
- Analytics & Reporting: Widget data generation for dashboard metrics and trend analysis
- Phone Formatting: International phone number normalization with country code detection
🗄️ Collections Used
📚 Full Schema: See Database Collections Documentation
inbound.leads
- Operations: Read/Write for lead storage and retrieval
- Model:
shared/models/leads-data.js - Usage: Main collection storing all inbound leads with campaign metadata
campaigns
- Operations: Read for campaign configuration
- Model:
shared/models/campaign-data.js - Usage: Campaign settings for contact creation and integration config
crm.contacts
- Operations: Write for contact creation from leads
- Model:
shared/models/contact.js - Usage: Creating business/person contacts in CRM from lead data
_accounts
- Operations: Read for account details and OneBalance verification
- Model:
shared/models/account.js - Usage: Account tier and subscription validation
instasites
- Operations: Write for generating InstaSites
- Model:
shared/models/instasites.js - Usage: InstaSite creation from leads
instareports
- Operations: Write for generating InstaReports
- Model:
shared/models/instareports.js - Usage: InstaReport creation from leads
inbound.widgets
- Operations: Read for pre-computed statistics
- Model:
shared/models/inbound-widgets.js - Usage: Dashboard analytics and widget data
🔄 Data Flow
Lead Retrieval with Filtering
sequenceDiagram
participant Client
participant LeadsService
participant MongoDB
participant Utilities
Client->>LeadsService: getFilters(account_id, filters)
LeadsService->>MongoDB: Aggregate filters with counts
alt Role-based filtering
LeadsService->>LeadsService: Add user_id filter (non-admin)
end
MongoDB-->>LeadsService: Filter counts by category
LeadsService-->>Client: Return filter options
Client->>LeadsService: listAllLeads(account_id, filters, pagination)
LeadsService->>LeadsService: Build query conditions
LeadsService->>LeadsService: Apply role-based restrictions
LeadsService->>MongoDB: Query with filters, sort, pagination
MongoDB-->>LeadsService: Lead documents with populated data
LeadsService->>Utilities: generatePagination(count, page, limit)
Utilities-->>LeadsService: Pagination metadata
LeadsService-->>Client: Leads + pagination
CRM Contact Creation from Lead
sequenceDiagram
participant Client
participant LeadsService
participant MongoDB
participant CRMContact
participant CountryCodes
Client->>LeadsService: addToCrm(contactdata)
LeadsService->>LeadsService: Extract contact fields
LeadsService->>CountryCodes: Match country code
CountryCodes-->>LeadsService: Country calling code
LeadsService->>LeadsService: Format phone (+country_code)
alt Missing email AND phone
LeadsService-->>Client: Error: NOT_ENOUGH_DATA
end
LeadsService->>MongoDB: findLeadAndCampaign(lead_id)
MongoDB-->>LeadsService: Lead + campaign data
alt Campaign: add_person_or_business = business
LeadsService->>LeadsService: Combine first_name + last_name → name
LeadsService->>CRMContact: addBusinesses(contacts)
else Campaign: person
LeadsService->>CRMContact: addPeople(contacts)
end
alt Contact created successfully
CRMContact-->>LeadsService: New contact with _id
LeadsService->>MongoDB: Update lead with contact_id
MongoDB-->>LeadsService: Update confirmed
LeadsService-->>Client: Success with new contact
else Contact already exists (duplicate)
CRMContact-->>LeadsService: Error: CONTACT_ALREADY_EXIST + contact_id
LeadsService->>MongoDB: Update lead with existing contact_id
LeadsService->>MongoDB: Find existing contact details
MongoDB-->>LeadsService: Existing contact document
LeadsService-->>Client: Success with existing contact
end
InstaSite/InstaReport Rebuild from Failed Lead
sequenceDiagram
participant Client
participant LeadsService
participant MongoDB
participant OneBalance
participant InstaSiteService
participant InstaReportService
Client->>LeadsService: rebuildFromLead(lead_id)
LeadsService->>MongoDB: findLeadAndCampaign(lead_id)
MongoDB-->>LeadsService: Lead + campaign data
alt No build_failed field
LeadsService-->>Client: Error: No failed build
end
alt integration != instasite_forms|instareport_forms
LeadsService-->>Client: Error: Wrong campaign type
end
alt add_person_or_business != business
LeadsService-->>Client: Error: Must be business type
end
alt No contact_id in lead
LeadsService-->>Client: Error: Contact ID missing
end
LeadsService->>MongoDB: Find account by account_id
MongoDB-->>LeadsService: Account document
LeadsService->>OneBalance: verifyBalance(event, account, quantity: 0)
alt Insufficient balance
OneBalance-->>LeadsService: Error: Usage exceeded
LeadsService-->>Client: Error: No usage available
end
OneBalance-->>LeadsService: Balance verified
alt integration = instasite_forms
LeadsService->>InstaSiteService: instasite(account_id, owner, [business_id], template)
InstaSiteService-->>LeadsService: [{ instasite: site_id }]
LeadsService->>MongoDB: Update lead: {instasite: site_id, $unset: build_failed}
else integration = instareport_forms
LeadsService->>InstaReportService: instareport(account_id, owner, businesses, template)
InstaReportService-->>LeadsService: [{ instareport: report_id }]
LeadsService->>MongoDB: Update lead: {instareport: report_id, $unset: build_failed}
end
MongoDB-->>LeadsService: Update confirmed
LeadsService-->>Client: 200 Success: Build completed
🎯 API Endpoints
1. GET /v1/inbound/leads/filters
Purpose: Generate dynamic filter options with counts for lead filtering UI.
Authentication: Required (JWT)
Authorization: Scopes inbound, inbound.leads
Query Parameters:
{
startdate: String, // OPTIONAL - ISO date for range
enddate: String, // OPTIONAL - ISO date for range
freshness: String, // OPTIONAL - Pre-filter by lead freshness
type: String, // OPTIONAL - Pre-filter by lead type
integration: String, // OPTIONAL - Pre-filter by integration
reps: String, // OPTIONAL - Representative IDs (comma-separated)
searchtext: String // OPTIONAL - Text search query
}
Request Example:
GET /v1/inbound/leads/filters?startdate=2025-01-01&enddate=2025-12-31&freshness=new
Authorization: Bearer {JWT_TOKEN}
Response:
// Success - 200
{
success: true,
message: "SUCCESS",
data: {
integrations: [
{ value: "facebook_ads", label: "Facebook Ads", count: 42 },
{ value: "clickfunnels", label: "ClickFunnels", count: 18 }
],
reps: [
{
id: "507f1f77bcf86cd799439011",
name: "John Doe",
count: 35
}
],
freshness: [
{ value: "new", label: "New", count: 20 },
{ value: "read", label: "Read", count: 30 },
{ value: "contacted", label: "Contacted", count: 10 }
],
total_count: 60,
today_count: 5,
this_week_count: 15,
this_month_count: 40
}
}
MongoDB Operations:
| Operation | Collection | Query | Purpose |
|---|---|---|---|
getFilters | leads-data | Aggregation | Generate filter counts |
Role-Based Filtering:
- Admin: Sees all account leads
- Non-Admin: Only leads assigned to them (
user_id)
2. GET /v1/inbound/leads or /v1/inbound/leads/:lead_id
Purpose: Retrieve all leads or a specific lead with pagination and filtering.
Authentication: Required (JWT)
Authorization: Scopes inbound, inbound.leads
Query Parameters:
{
page: Number, // Page number (1-based)
limit: Number, // Items per page
sort_by: String, // Sort field ("created_at", "freshness", "integration")
order: String, // "asc" | "desc" (default: "desc")
searchtext: String, // OPTIONAL - Search in lead data
startdate: String, // OPTIONAL - ISO date
enddate: String, // OPTIONAL - ISO date
freshness: String, // OPTIONAL - Lead freshness filter
type: String, // OPTIONAL - Lead type filter
integration: String, // OPTIONAL - Integration filter
reps: String // OPTIONAL - Representative filter
}
Path Parameters (Optional):
{
lead_id: ObjectId; // OPTIONAL - Get single lead by ID
}
Request Examples:
# List all leads with pagination
GET /v1/inbound/leads?page=1&limit=20&sort_by=created_at&order=desc
Authorization: Bearer {JWT_TOKEN}
# Get single lead by ID
GET /v1/inbound/leads/507f1f77bcf86cd799439011
Authorization: Bearer {JWT_TOKEN}
# Filter by integration and date range
GET /v1/inbound/leads?integration=facebook_ads&startdate=2025-01-01&enddate=2025-12-31
Authorization: Bearer {JWT_TOKEN}
Response - List:
// Success - 200
{
success: true,
message: "SUCCESS",
data: [
{
_id: ObjectId,
lead_id: String, // External lead ID
lead_data: { // Raw lead data from integration
id: String,
created_time: String,
field_data: Array,
// ... integration-specific fields
},
campaign_id: ObjectId,
campaign_name: String, // Populated from campaign
integration: String, // From campaign
account_id: ObjectId,
owner: ObjectId,
user_id: ObjectId, // Assigned representative
contact_id: ObjectId, // OPTIONAL - CRM contact
freshness: String, // "new" | "read" | "contacted"
errors: [ // OPTIONAL - Processing errors
{
type: String, // "contactCreation" | "dealCreateError" | etc.
details: Object
}
],
build_failed: { // OPTIONAL - InstaReport/InstaSite build failure
message: String,
error: Object
},
instasite: ObjectId, // OPTIONAL - Generated InstaSite
instareport: ObjectId, // OPTIONAL - Generated InstaReport
created_at: Date,
updated_at: Date
}
],
pagination: {
page: 1,
limit: 20,
total: 150,
total_pages: 8,
has_next: true,
has_prev: false
}
}
Response - Single Lead:
// Success - 200
{
success: true,
message: "SUCCESS",
data: {
// ... all lead fields
}
}
// Error - Lead not found - 404
{
success: false,
errno: 404,
message: "Lead not found",
error_code: "LEAD_NOT_FOUND"
}
MongoDB Operations:
| Operation | Collection | Query | Purpose |
|---|---|---|---|
find | leads-data | Complex filter with pagination | Get leads with sorting |
Sorting Options:
created_at- Sort by lead capture date (default)freshness- Sort by lead statusintegration- Sort by integration type- Default:
created_atdescending (newest first)
Search Functionality:
- Searches across lead data fields (email, name, phone, etc.)
- Case-insensitive partial matching
- Searches within JSON lead_data structure
Role-Based Access:
// Non-admin users only see their assigned leads
if (user.role !== 'admin') {
query.user_id = new mongoose.Types.ObjectId(uid);
}
3. POST /v1/inbound/leads/:leadid/rebuild
Purpose: Rebuild failed InstaSite or InstaReport from lead data.
Authentication: Required (JWT)
Authorization: Scopes inbound, inbound.leads
Request Parameters:
// Path Parameters
{
leadid: ObjectId; // REQUIRED - Lead ID to rebuild from
}
Request Example:
POST /v1/inbound/leads/507f1f77bcf86cd799439011/rebuild
Authorization: Bearer {JWT_TOKEN}
Response:
// Success - 200
{
success: true,
message: "SUCCESS"
}
// Error - No build failure - 422
{
success: false,
errno: 422,
message: "There is no failed instasite/report associated with this lead."
}
// Error - Not InstaSites/InstaReport campaign - 422
{
success: false,
errno: 422,
message: "The selected lead is not of a campaign related to Instasite/InstaReport"
}
// Error - Contact setting not business - 422
{
success: false,
errno: 422,
message: "The contact create setting for the campaign is set to person"
}
// Error - Contact ID missing - 422
{
success: false,
errno: 422,
message: "Contact Id not found in lead info"
}
MongoDB Operations:
| Operation | Collection | Query | Purpose |
|---|---|---|---|
findLeadAndCampaign | leads-data + join | {_id} | Get lead with campaign details |
findById | accounts | {_id} | Get account for OneBalance check |
update | leads-data | {_id} | Update lead with build result |
Business Rules:
- Build Failure Required: Lead must have
build_failedfield - Campaign Type: Only
instasite_formsorinstareport_forms - Contact Type: Campaign must have
add_person_or_business: "business" - Contact Exists: Lead must have valid
contact_id - Usage Check: Verifies OneBalance usage before building
- Template: Uses campaign's
instasite_template_idorinstareport_template_id
OneBalance Integration:
// Verify usage available (doesn't consume, just checks)
await verifyBalance({
event: 'instasite', // or 'instareport'
account: account,
user_id: uid,
quantity: 0, // 0 = check only, don't consume
});
4. DELETE /v1/inbound/leads
Purpose: Bulk delete leads (soft delete or permanent).
Authentication: Required (JWT)
Authorization: Scopes inbound, inbound.leads
Request Parameters:
// Body
{
leads: [ObjectId]; // REQUIRED - Array of lead IDs to delete
}
// Query Parameters
{
deleteall: Boolean; // OPTIONAL - "true" = hard delete, "false" = soft delete
}
Request Example:
DELETE /v1/inbound/leads?deleteall=false
Authorization: Bearer {JWT_TOKEN}
Content-Type: application/json
{
"leads": [
"507f1f77bcf86cd799439011",
"507f1f77bcf86cd799439012",
"507f1f77bcf86cd799439013"
]
}
Response:
// Success - 200
{
success: true,
message: "SUCCESS",
data: 3 // Number of leads deleted
}
Deletion Types:
| Type | deleteall | Behavior | Use Case |
|---|---|---|---|
| Soft Delete | falseor omitted | Setsis_deleted: true | Default - preserves historical data |
| Hard Delete | true | Permanently removes documents | Data cleanup, GDPR compliance |
MongoDB Operations:
| Operation | Collection | Query | Purpose |
|---|---|---|---|
deleteAll | leads-data | {_id: {$in: [...], account_id, owner}} | Delete leads |
Security:
- Validates
account_idandownerto prevent unauthorized deletion - Bulk operation uses
$inoperator for efficiency - Returns count of successfully deleted leads
5. POST /v1/inbound/leads/add-to-crm
Purpose: Manually add lead to CRM (creates contact from lead data).
Authentication: Required (JWT)
Authorization: Scopes inbound, inbound.leads
Request Parameters:
// Body
{
contactdata: {
lead_id: ObjectId, // REQUIRED - Lead to create contact from
first_name: String, // OPTIONAL - Override lead data
last_name: String, // OPTIONAL - Override lead data
name: String, // OPTIONAL - Full name (for business)
email: String, // OPTIONAL - Override lead data
phone: String, // OPTIONAL - Override lead data
country: String // OPTIONAL - For phone formatting
}
}
Request Example:
POST /v1/inbound/leads/add-to-crm
Authorization: Bearer {JWT_TOKEN}
Content-Type: application/json
{
"contactdata": {
"lead_id": "507f1f77bcf86cd799439011",
"first_name": "John",
"last_name": "Doe",
"email": "john.doe@example.com",
"phone": "4155551234",
"country": "United States"
}
}
Response:
// Success - 200
{
success: true,
message: "SUCCESS",
data: {
errors: [],
savedContacts: [
{
_id: ObjectId,
name: "John Doe",
email: "john.doe@example.com",
phone: "+14155551234",
address: {
country: "United States"
},
source: "inbound",
account_id: ObjectId,
owner: ObjectId,
created_at: Date,
updated_at: Date
}
]
}
}
// Error - Not enough data - 400
{
success: false,
data: "NOT_ENOUGH_DATA"
}
// Success - Duplicate contact (auto-handled) - 200
{
success: true,
message: "SUCCESS",
data: {
errors: [],
savedContacts: [
{
_id: ObjectId, // Existing contact ID
// ... existing contact data
}
]
}
}
Phone Formatting:
// Country code detection and formatting
if (country && codes) {
const match = codes.find(c => c.country.match(new RegExp('\\b' + country + '\\b', 'gi')));
if (match?.countryCodes?.[0] && phone) {
phone = '+' + match.countryCodes[0] + phone;
}
}
MongoDB Operations:
| Operation | Collection | Query | Purpose |
|---|---|---|---|
findLeadAndCampaign | leads-data + join | {_id} | Get lead with campaign |
addPeople or addBusinesses | contacts | New document | Create CRM contact |
update | leads-data | {_id} | Link lead to contact |
findOne | contacts | {_id} | Get existing contact (on duplicate) |
Duplicate Handling:
- If contact already exists (same email/phone), retrieves existing contact
- Updates lead with existing
contact_id - Returns existing contact as success (not an error)
- This allows re-linking leads to contacts without creating duplicates
Business Rules:
- Required Fields: Must have
emailORphone(at least one) - Source Attribution: All contacts created with
source: "inbound" - Contact Type: Determined by campaign's
add_person_or_businesssetting - Name Handling: For business contacts,
first_name + last_namecombined intoname - Country Codes: Automatically prepended to phone numbers when country provided
6. GET /v1/inbound/leads/widget
Purpose: Retrieve lead statistics for dashboard widgets.
Authentication: Required (JWT)
Authorization: Scopes inbound, inbound.leads
Request Parameters:
// Query Parameters
{
startdate: String, // OPTIONAL - ISO date for range
enddate: String // OPTIONAL - ISO date for range
}
Request Example:
GET /v1/inbound/leads/widget?startdate=2025-01-01&enddate=2025-12-31
Authorization: Bearer {JWT_TOKEN}
Response:
// Success - 200
{
success: true,
data: {
total_leads: 450,
new_leads: 25,
contacted_leads: 200,
converted_leads: 180,
// Time-based breakdowns
today: 5,
this_week: 35,
this_month: 120,
last_month: 150,
// Integration breakdown
by_integration: [
{
integration: "facebook_ads",
count: 200,
percentage: 44.4
},
{
integration: "clickfunnels",
count: 150,
percentage: 33.3
}
],
// Representative performance
by_rep: [
{
user_id: ObjectId,
name: "John Doe",
leads_assigned: 100,
leads_contacted: 75,
conversion_rate: 75.0
}
],
// Trend data (for charts)
daily_trend: [
{ date: "2025-01-01", count: 10 },
{ date: "2025-01-02", count: 15 }
],
// Top campaigns
top_campaigns: [
{
campaign_id: ObjectId,
campaign_name: "Summer Lead Gen",
lead_count: 85,
contact_rate: 90.5
}
]
}
}
MongoDB Operations:
| Operation | Collection | Query | Purpose |
|---|---|---|---|
getLeadsWidgetData | leads-data + widgets | Complex aggregation | Generate statistics |
Widget Metrics:
- Lead Counts: Total, new, contacted, converted
- Time Breakdowns: Today, this week, this month, last month
- Integration Performance: Leads by source with percentages
- Representative Performance: Leads assigned/contacted per rep
- Trends: Daily/weekly lead capture trends
- Top Campaigns: Best performing campaigns by lead volume
Role-Based Filtering:
- Admin: Sees all account widget data
- Non-Admin: Only sees data for leads assigned to them
📊 Data Models
Lead Data
{
_id: ObjectId,
lead_id: String, // External lead ID (from integration)
// Lead Source
lead_data: Object, // Raw JSON from integration webhook
campaign_id: ObjectId, // Parent campaign
integration: String, // Inherited from campaign
// Account & Assignment
account_id: ObjectId, // DashClicks account
owner: ObjectId, // Campaign owner
user_id: ObjectId, // Assigned representative
// CRM Integration
contact_id: ObjectId, // OPTIONAL - Created CRM contact
// Lead Status
freshness: String, // "new" | "read" | "contacted"
// Error Tracking
errors: [
{
type: String, // Error type
details: Object // Error details
}
],
// InstaSites/InstaReport
instasite: ObjectId, // OPTIONAL - Generated InstaSite
instareport: ObjectId, // OPTIONAL - Generated InstaReport
build_failed: { // OPTIONAL - Build failure details
message: String,
error: Object,
timestamp: Date
},
// Metadata
is_deleted: Boolean, // Soft delete flag
created_at: Date,
updated_at: Date
}
🔀 Integration Points
Internal Services
- CRM Module: Contact creation (person/business)
- InstaSites: Auto-build sites for business leads
- InstaReports: Auto-build reports for business leads
- OneBalance: Usage verification before builds
- Campaigns Module: Campaign details and settings
Utilities
country-codes: Phone number formattingcrmContact: Contact creation utilitiesinstasite: InstaSite builderinstareport: InstaReport builder
🛡️ Error Handling
Common Errors
| Error | Status | Message | Cause |
|---|---|---|---|
| Lead not found | 404 | "Lead not found" | Invalid lead ID |
| Not enough data | 400 | "NOT_ENOUGH_DATA" | Missing email and phone |
| No build failure | 422 | "There is no failed instasite/report..." | No build_failed field |
| Wrong campaign type | 422 | "The selected lead is not of a campaign..." | Not InstaSites/InstaReport |
| Wrong contact type | 422 | "The contact create setting..." | Not set to business |
| Missing contact | 422 | "Contact Id not found..." | No contact_id in lead |
📈 Performance Considerations
- Pagination: Always use pagination for lead lists
- Indexing: Ensure indexes on
account_id,user_id,campaign_id,created_at,is_deleted - Search: Consider full-text search indexes for lead_data JSON fields
- Aggregations: Widget queries can be expensive - consider caching
- Role Filtering: Non-admin queries more efficient (smaller result sets)
🔒 Security Considerations
- Multi-Tenant Isolation: All queries filtered by
account_id - Role-Based Access: Non-admins only see assigned leads
- Deletion Validation: Validates ownership before deletion
- CRM Integration: Source tracking prevents unauthorized contact creation
- Build Verification: OneBalance check prevents unlimited builds
⚠️ Important Notes
-
Role-Based Access Control: All functions enforce account_id scoping. Non-admin users see only leads assigned to them (user_id filter). Always validate role before querying.
-
Phone Number Formatting: Phone formatting requires valid country name matching country-codes data. If country not found or phone already has +, original phone used. Always validate phone format after processing.
-
Contact Type Consistency: Campaign's
add_person_or_businesssetting determines contact type (person vs business). Business contacts use singlenamefield, person contacts usefirst_name+last_name. InstaSite/InstaReport builds require business type. -
Duplicate Contact Handling: CRM module detects duplicates by email or phone. When duplicate detected, lead associated with existing contact (not error). Check for
error_type: 'CONTACT_ALREADY_EXIST'to handle gracefully. -
OneBalance Verification: Rebuild operations verify OneBalance usage before building.
quantity: 0means check only (no consumption). Actual consumption happens in instasite/instareport utilities. Always check verifyBalance errors. -
Soft vs Hard Delete: Default deletion is soft (sets is_deleted: true). Hard delete (
deleteall: true) permanently removes data. Use hard delete for GDPR compliance. Soft deletes preserve historical data for reporting. -
Single Lead Query: When
lead_idin params, function returns single lead object (not array). If lead not found or access denied, throws 404 error. Check for params.lead_id to determine response structure. -
Widget Data Caching: Widget queries are expensive (multiple aggregations). Consider caching for 5-15 minutes. Invalidate cache on lead creation/update/deletion. Pre-compute for large accounts using background jobs.
-
Search Performance: Text search operates on lead_data JSON fields (email, name, phone, custom fields). Large datasets require text indexes. Consider limiting search to specific fields for better performance.
-
Build Failure Tracking: Failed InstaSite/InstaReport builds store error in
build_failedfield. Check for this field before attempting rebuild. Remove build_failed on successful rebuild. Errors include timestamp and full error object. -
Source Attribution: All contacts created via addToCrm tagged with
source: 'inbound'. This enables source tracking and reporting. Don't override source in contact creation. -
Pagination Best Practices: Always use pagination for lead lists. Default sort is created_at desc (newest first). Ensure compound indexes on frequently filtered fields (account_id, user_id, created_at, is_deleted).
🔗 Related Documentation
- Webhooks Service - Lead capture from external integrations
- Campaigns Service - Campaign configuration and management
- Reports Service - Lead analytics and reporting
- CRM Contact Service - Contact creation and management
- InstaSites Service - InstaSite builder
- InstaReports Service - InstaReport generator
- OneBalance Service - Usage verification and billing
- Country Codes Utility (link removed - file does not exist) - Phone number formatting
- Pagination Utility (link removed - file does not exist) - Pagination helper