Skip to main content

InstaReports Reporting Service

📖 Overview​

internal/api/v1/instareports/Controllers/reporting.js provides analytics and reporting capabilities for InstaReports module. Tracks report generation metrics, view rates, failure rates, and industry breakdowns with time-series analysis for dashboard visualization.

File Path: internal/api/v1/instareports/Controllers/reporting.js

🗄️ Collections Used​

📚 Full Schema: See Database Collections Documentation

instareports​

  • Operations: Read with aggregation pipelines for analytics
  • Model: shared/models/instareports.js
  • Usage Context: Primary data source for all reporting metrics - status, created dates, viewed flags, business categories

🔄 Data Flow​

sequenceDiagram
    participant Dashboard
    participant Service
    participant DB as InstaReports DB
    participant Date as Date Calculator

    Dashboard->>Service: ReportingStatus(startDate, endDate)
    Service->>DB: Aggregate total counts (all-time)
    DB-->>Service: total_built, total_viewed, total_failed

    alt Date range provided
        Service->>Date: Calculate day intervals
        Service->>DB: Fetch reports in date range
        DB-->>Service: All matching reports

        loop For each day
            Service->>Service: Filter reports by day
            Service->>Service: Count built/viewed/failed
            Service->>Service: Aggregate to result array
        end

        Service-->>Dashboard: Daily breakdown + totals
    else No date range
        Service-->>Dashboard: All-time totals only
    end

🔧 Business Logic & Functions​

Reporting & Analytics​

ReportingStatus(req, res, next)​

Purpose: Generates comprehensive report generation statistics including all-time totals and optional daily time-series breakdown. Primary endpoint for InstaReports dashboard analytics.

Parameters:

  • req.query.startDate (String, optional) - Start date ISO format (e.g., '2024-10-01')
  • req.query.endDate (String, optional) - End date ISO format (e.g., '2024-10-31')
  • req.auth.account_id (ObjectId) - Account identifier for filtering

Returns: Promise<Response> - Analytics data structure:

{
  success: true,
  data: [                          // Only if date range provided
    {
      date: '01 Oct 2024',
      data: {
        built: 15,                 // GENERATED reports
        viewed: 12,                // Viewed reports
        failed: 2                  // FAILED reports
      }
    },
    {
      date: '02 Oct 2024',
      data: { built: 18, viewed: 14, failed: 1 }
    }
    // ... one entry per day
  ],
  total_by_date: {                // Only if date range provided
    total_built: 450,
    total_viewed: 380,
    total_failed: 25
  },
  total: {                        // All-time totals
    total_built: 1234,
    total_viewed: 980,
    total_failed: 45
  }
}

Business Logic Flow:

  1. Extract Parameters

    • Get startDate, endDate from query
    • Get account_id from auth context

  2. Calculate All-Time Totals (Always)

    • Build aggregation pipeline with $facet:
      • total_built: Count where status: 'GENERATED'
      • total_viewed: Count where viewed: true
      • total_failed: Count where status: 'FAILED'
    • Use $ifNull to default counts to 0
    • Store in totalCounts

  3. If Date Range Provided (Optional detailed breakdown):

    a. Parse Dates

    • Start: moment(startDate).startOf('day') - 00:00:00
    • End: moment(endDate).endOf('day') - 23:59:59
    • Extract endDay date for comparison

    b. Fetch Reports in Range

    • Match: account_id AND created >= sdt AND created <= edt
    • Retrieve all matching reports (not aggregated)

    c. Iterate Through Days

    • Initialize: date_itr = startDate
    • While date_itr < endDate:
      • Create day object: { date, data: { viewed: 0, built: 0, failed: 0 } }
      • Filter reports created on current day
      • Count:
        • viewed: Count reports where viewed: true
        • built: Count reports where status: 'GENERATED'
        • failed: Count reports where status: 'FAILED'
      • Add counts to total_by_date accumulators
      • If date_itr <= endDay: Push day object to final_res
      • Else: Add counts to last day in final_res (overflow)
      • Increment: date_itr.setDate(date_itr.getDate() + 1)

    d. Return Detailed Response

    • data: Array of daily breakdowns
    • total_by_date: Sum of date range
    • total: All-time totals

  4. If No Date Range (Simple response):

    • Return only total (all-time counts)

Aggregation Pipeline (All-Time Totals):

[
  { $match: { account_id: ObjectId(account_id) } },
  {
    $facet: {
      total_built: [{ $match: { status: { $eq: 'GENERATED' } } }, { $count: 'total_built' }],
      total_viewed: [{ $match: { viewed: { $eq: true } } }, { $count: 'total_viewed' }],
      total_failed: [{ $match: { status: { $eq: 'FAILED' } } }, { $count: 'total_failed' }],
    },
  },
  {
    $project: {
      total_built: { $arrayElemAt: ['$total_built.total_built', 0] },
      total_viewed: { $arrayElemAt: ['$total_viewed.total_viewed', 0] },
      total_failed: { $arrayElemAt: ['$total_failed.total_failed', 0] },
    },
  },
  {
    $addFields: {
      total_built: { $ifNull: ['$total_built', 0] },
      total_viewed: { $ifNull: ['$total_viewed', 0] },
      total_failed: { $ifNull: ['$total_failed', 0] },
    },
  },
];

Date Iteration Logic:

// Example: startDate = 2024-10-01, endDate = 2024-10-03
// Iteration 1: 2024-10-01 00:00 to 2024-10-02 00:00 → day 1
// Iteration 2: 2024-10-02 00:00 to 2024-10-03 00:00 → day 2
// Iteration 3: 2024-10-03 00:00 to 2024-10-04 00:00 → day 3
// If endDate time is 23:59:59, captures full day 3

Overflow Handling:

// If endDate time extends beyond midnight:
// endDate = '2024-10-03T15:00:00'
// Reports from 2024-10-03 00:00 to 15:00 → added to day 3
// Reports from 2024-10-03 15:00 to 23:59 → added to day 3 (overflow)

Key Business Rules:

  • All-time totals always calculated (independent of date range)
  • Daily breakdown optional (requires both startDate and endDate)
  • viewed flag independent of status (GENERATED reports may not be viewed)
  • built counts only GENERATED status (not QUEUED or RETRYING)
  • failed counts only FAILED status (permanent failures)
  • Date range inclusive of start and end dates
  • Timezone handling via moment (default: server timezone)

Status Definitions:

  • built (status: 'GENERATED'): Report successfully generated with all scrapers complete
  • viewed (viewed: true): At least one recipient opened report link
  • failed (status: 'FAILED'): Report generation permanently failed (max retries exceeded)

Response Examples:

With Date Range:

{
  success: true,
  data: [
    { date: '01 Oct 2024', data: { built: 15, viewed: 12, failed: 2 } },
    { date: '02 Oct 2024', data: { built: 18, viewed: 14, failed: 1 } },
    { date: '03 Oct 2024', data: { built: 12, viewed: 10, failed: 0 } }
  ],
  total_by_date: {
    total_built: 45,
    total_viewed: 36,
    total_failed: 3
  },
  total: {
    total_built: 1234,
    total_viewed: 980,
    total_failed: 45
  }
}

Without Date Range:

{
  success: true,
  total: {
    total_built: 1234,
    total_viewed: 980,
    total_failed: 45
  }
}

Error Handling:

  • Errors passed to Express error handler via next(err)
  • No validation on date format (moment parses flexibly)
  • Invalid account_id returns zero counts (no error)

Example Usage:

// Dashboard overview (all-time)
GET /v1/instareports/reporting/status

// Time-series chart (last 30 days)
GET /v1/instareports/reporting/status?startDate=2024-10-01&endDate=2024-10-31

Side Effects:

  • ⚠️ Two aggregation queries: all-time totals + date range (if provided)
  • ⚠️ Date range query fetches ALL matching reports (not paginated)
  • ⚠️ Client-side iteration through days (CPU intensive for large ranges)
  • ⚠️ No caching - recalculates on every request

ReportingIndustryBreakdown(req, res, next)​

Purpose: Generates industry-wise breakdown of report generation and view rates for specified date range. Used for industry performance analysis and trend identification.

Parameters:

  • req.query.startDate (String, required) - Start date ISO format
  • req.query.endDate (String, required) - End date ISO format
  • req.auth.parent_account (ObjectId) - Parent account ID (main account filter)

Returns: Promise<Response> - Industry breakdown:

{
  success: true,
  data: [
    {
      industry: 'Automotive',
      built: 45,                // GENERATED reports
      viewed: 38                // Viewed reports
    },
    {
      industry: 'Restaurants',
      built: 32,
      viewed: 28
    },
    {
      industry: 'Health & Medical',
      built: 27,
      viewed: 22
    }
    // ... one entry per industry
  ]
}

Business Logic Flow:

  1. Validate Parameters

    • Check startDate and endDate presence
    • If missing: Return error { success: false, message: 'Start Date and End Date are required' }

  2. Parse Date Range

    • Start: moment(startDate).startOf('day') - 00:00:00
    • End: moment(endDate).endOf('day') - 23:59:59

  3. Fetch Reports

    • Match criteria:
      • account_id: parent_account (main account only)
      • created: { $gte: sdt, $lte: edt }
      • status: 'GENERATED' (only successfully built reports)
    • Retrieve: Full report documents (need business_info.businessCategory)

  4. Group by Industry

    • Use Array.reduce() to aggregate:
      • Key: details.business_info.businessCategory
      • Accumulate:
        • built: Count where status: 'GENERATED'
        • viewed: Count where viewed: true
    • Build result array: [{ industry, built, viewed }, ...]

  5. Return Industry Breakdown

Reduce Logic:

// Start: empty object {}
// Iteration 1: Report with category 'Automotive'
//   - Create entry: { industry: 'Automotive', built: 1, viewed: 0 }
// Iteration 2: Another 'Automotive' report (viewed)
//   - Update entry: { industry: 'Automotive', built: 2, viewed: 1 }
// Iteration 3: Report with category 'Restaurants'
//   - Create entry: { industry: 'Restaurants', built: 1, viewed: 0 }
// Result: [
//   { industry: 'Automotive', built: 2, viewed: 1 },
//   { industry: 'Restaurants', built: 1, viewed: 0 }
// ]

Key Business Rules:

  • Only GENERATED reports included (excludes QUEUED, RETRYING, FAILED)
  • Filters by parent_account (not account_id) - main account scope
  • Date range required (no default to all-time)
  • Industry categories from business contact data (24 standard categories)
  • viewed independent of recipient count (any recipient view counts)
  • Industries with zero reports not included in response

Industry Categories (from business contacts):

[
  'Active Life',
  'Arts & Entertainment',
  'Automotive',
  'Beauty & Spas',
  'Education',
  'Event Planning & Services',
  'Financial Services',
  'Food',
  'Health & Medical',
  'Home Services',
  'Hotels & Travel',
  'Industrial Goods & Manufacturing',
  'Local Services',
  'Mass Media',
  'Mining & Agriculture',
  'Nigthlife',
  'Other',
  'Pets',
  'Professional Services',
  'Public Services & Government',
  'Real Estate',
  'Religious Organizations',
  'Restaurants',
  'Shopping',
];

Parent Account Filtering:

// Why parent_account?
// - Main accounts see all reports (own + sub-accounts)
// - Sub-accounts inherit parent's data
// - Ensures consistent industry breakdown across account hierarchy

Response Example:

{
  success: true,
  data: [
    { industry: 'Automotive', built: 45, viewed: 38 },
    { industry: 'Restaurants', built: 32, viewed: 28 },
    { industry: 'Health & Medical', built: 27, viewed: 22 },
    { industry: 'Professional Services', built: 18, viewed: 15 },
    { industry: 'Beauty & Spas', built: 12, viewed: 10 },
    { industry: 'Other', built: 8, viewed: 5 }
  ]
}

Error Handling:

  • Missing dates: Return 500 with error message
  • Invalid dates: moment parses flexibly (may return unexpected results)
  • Invalid parent_account: Returns empty array
  • Other errors: Passed to Express error handler via next(err)

Example Usage:

// Industry performance last month
GET /v1/instareports/reporting/industry-breakdown?startDate=2024-10-01&endDate=2024-10-31

// Quarter analysis
GET /v1/instareports/reporting/industry-breakdown?startDate=2024-07-01&endDate=2024-09-30

Side Effects:

  • ⚠️ Fetches ALL matching reports (not paginated)
  • ⚠️ Client-side reduce operation (CPU intensive for large datasets)
  • ⚠️ No aggregation pipeline - loads full documents
  • ⚠️ No caching - recalculates on every request

🔀 Integration Points​

External Services​

None - Pure database analytics

Internal Dependencies​

  • moment-timezone - Date parsing and formatting with timezone support
  • mongoose.Types - ObjectId type handling

Shared Models​

  • InstaReport - Primary data source for all analytics queries

🧪 Edge Cases & Special Handling​

Case: No Reports in Date Range​

Condition: Date range has zero matching reports
Handling: Returns empty data array, zero total_by_date counts
Result: Valid response with zeros (not error)

Case: Missing Date Parameters​

Condition: ReportingStatus called without startDate/endDate
Handling: Returns only all-time total counts, no data or total_by_date
Result: Valid simplified response

Condition: ReportingIndustryBreakdown called without dates
Handling: Returns 500 error with message
Result: Request blocked (dates required)

Case: Date Range Crosses Months​

Condition: startDate in one month, endDate in another
Handling: Iterates day-by-day across month boundary seamlessly
Result: Accurate daily breakdown spanning months

Case: Single Day Query​

Condition: startDate = endDate
Handling: Returns one entry in data array for that day
Result: Valid single-day breakdown

Case: Future Dates​

Condition: startDate/endDate in future
Handling: Query matches zero reports (created dates in past)
Result: Empty results with zero counts

Case: Reversed Dates​

Condition: startDate > endDate
Handling: Query matches zero reports (created >= future && created <= past)
Result: Empty results (no validation/error)

Case: Viewed Report Without Views​

Condition: Report marked viewed: false but status GENERATED
Handling: Counted in built, not counted in viewed
Result: Accurate view tracking

Case: Industry Category Null/Undefined​

Condition: Business contact missing businessCategory
Handling: Grouped under null or undefined key
Result: May create unexpected industry entry

Case: Multiple Industries Same Name​

Condition: Two businesses with same category (e.g., 'Automotive')
Handling: Aggregated correctly into single industry entry
Result: Accurate industry counts

Case: Large Date Range​

Condition: Date range spans years (365+ days)
Handling: Iterates through every day, may timeout
Result: Performance degradation or timeout error

⚠️ Important Notes​

  • 📊 All-Time vs Date Range: All-time totals always included, date range optional
  • 🎯 Status Filtering: Only GENERATED reports in industry breakdown, all statuses in main reporting
  • 📈 View Tracking: viewed flag independent of status (GENERATED reports may not be viewed)
  • 🌐 Parent Account Scope: Industry breakdown uses parent_account (main + sub-accounts)
  • 📅 Date Handling: Uses moment-timezone for parsing (flexible but may accept invalid formats)
  • ⚡ Performance: No pagination - loads all matching reports into memory
  • 🔁 No Caching: Recalculates on every request (consider Redis for high traffic)
  • 🚨 No Validation: Date format not validated (moment parses flexibly)
  • 📌 Day Iteration: Client-side loop may be slow for large date ranges (consider aggregation pipeline)
  • 🏭 Industry Categories: 24 standard categories, 'Other' as catch-all

🔗 Related Documentation​

  • Parent Module: InstaReports Module
  • Related Service: InstaReport Service
  • Controller: internal/api/v1/instareports/Controllers/reporting.js
  • Routes: internal/api/v1/instareports/Routes/reporting.js
  • Models:
    • InstaReport (link removed - file does not exist)
💬

Documentation Assistant

Ask me anything about the docs

Hi! I'm your documentation assistant. Ask me anything about the docs!

I can help you with:
- Code examples
- Configuration details
- Troubleshooting
- Best practices

Try asking: How do I configure the API?
09:31 AM