Filters Service

📖 Overview

Service Path: internal/api/v1/filters/Services/filter.js
Provider Path: internal/api/v1/filters/Providers/count.js

The Filters service provides dynamic filtering capabilities for CRM entities (deals and contacts) with real-time count aggregation. Core responsibilities include:

• Filter Management: CRUD operations for custom filters
• Real-Time Counts: Aggregate counts for filtered results
• Visibility Control: User and team-level filter sharing
• Multi-Entity Support: Filters for deals, contacts, people, and businesses
• Default Filters: System-provided and user-created filters
• Permission System: Owner and admin controls for public filters

🗄️ Collections Used

🔄 Data Flow

Filter Creation with Count

sequenceDiagram
    participant User
    participant FilterService
    participant FilterModel
    participant CountProvider
    participant DealsDB
    participant ContactsDB

    User->>FilterService: Create filter (POST)
    Note over User,FilterService: { type, title, filter, isPublic }

    FilterService->>FilterService: Check permissions
    Note over FilterService: Only owner/admin can create public filters

    FilterService->>FilterModel: Save filter
    FilterModel-->>FilterService: Saved filter doc

    FilterService->>FilterService: Determine entity type

    alt Deal Filter
        FilterService->>CountProvider: dealCount()
        CountProvider->>DealsDB: Aggregate with filter conditions
        DealsDB-->>CountProvider: Count result
        CountProvider-->>FilterService: { filterID, total }
    else Contact Filter
        FilterService->>CountProvider: contactCount()
        CountProvider->>ContactsDB: Aggregate with filter conditions
        ContactsDB-->>CountProvider: Count result
        CountProvider-->>FilterService: { filterID, total }
    end

    FilterService->>FilterService: Add count to filter object
    FilterService-->>User: Filter with count

    style FilterService fill:#e3f2fd
    style CountProvider fill:#fff4e6

Filter List with Batch Counts

flowchart TD
    A[Get Filters Request] --> B[Fetch Filters from DB]
    B --> C{Filter Type?}

    C -->|Deals| D[Convert filters to aggregation pipelines]
    C -->|Contacts| E[Convert filters to aggregation pipelines]

    D --> F[Build visibility options]
    E --> G[Build visibility options]

    F --> H[Single $facet aggregation]
    G --> I[Single $facet aggregation]

    H --> J[Extract counts for each filter]
    I --> K[Extract counts for each filter]

    J --> L[Merge counts with filter objects]
    K --> L

    L --> M{Check ownership}
    M -->|User owns filter| N[Set allow_update = true]
    M -->|User doesn't own| O[Set allow_update = false]

    N --> P[Return filter list with counts]
    O --> P

    style H fill:#e8f5e9
    style I fill:#e8f5e9
    style J fill:#fff4e6
    style K fill:#fff4e6

🔧 Business Logic & Functions

FilterService.get({ accountID, userID, type, pipeline, is_owner })

Purpose: Retrieve all filters for a specific entity type with real-time counts

Parameters:

• accountID (ObjectId) - Account ID
• userID (ObjectId) - User ID
• type (String) - Filter type: 'deal', 'contacts', 'people', or 'businesses'
• pipeline (ObjectId) - Pipeline ID (required for deal filters)
• is_owner (Boolean) - Whether user is account owner

Returns:

{
  success: true,
  message: 'SUCCESS',
  data: [
    {
      id: String,
      title: String,
      type: String,            // 'deal', 'contacts', 'people', 'businesses'
      filter: Object,          // Filter conditions
      is_public: Boolean,
      is_default: Boolean,
      user: ObjectId,
      count: Number,           // Real-time count
      total: Number,           // Always 0 (legacy)
      allow_update: Boolean    // True if user owns filter
    }
  ]
}

Business Logic Flow:

1. Fetch Filters

   const filterRS = await FilterModel.list(accountID, userID, type);

2. Apply Count Provider

   let filterData;
   
   if (type === 'deal') {
     filterData = await countProvider.dealFilters({
       filtersDocs: filterRS,
       searchType: type,
       pipeline,
       account_id: accountID,
       uid: userID,
       is_owner,
     });
   } else if (type === 'contacts' || type === 'people' || type === 'businesses') {
     filterData = await countProvider.contactFilters({
       filtersDocs: filterRS,
       searchType: type,
       account_id: accountID,
       uid: userID,
       is_owner,
     });
   }

3. Set Ownership Permissions

   return {
     success: true,
     message: 'SUCCESS',
     data: filterData?.map(f => {
       if (f?.user?.toString() == userID.toString()) {
         f.allow_update = true;
       } else {
         f.allow_update = false;
       }
       return f;
     }),
   };

Key Business Rules:

• Real-Time Counts: Counts recalculated on every request
• Ownership: Users can only update their own filters
• Visibility: Respects user permissions (owner sees all)
• Type Mapping: 'people' → 'person', 'businesses' → 'business'

---

FilterService.post({ accountID, userID, type, title, filter, isPublic, is_owner, role, pipeline })

Purpose: Create a new filter with initial count

Parameters:

• accountID (ObjectId) - Account ID
• userID (ObjectId) - User ID
• type (String) - Filter type
• title (String) - Filter name
• filter (Object) - Filter conditions
• isPublic (Boolean) - Make filter visible to team
• is_owner (Boolean) - Whether user is account owner
• role (String) - User role
• pipeline (ObjectId) - Pipeline ID (for deal filters)

Returns:

{
  success: true,
  message: 'SUCCESS',
  data: {
    id: String,
    title: String,
    type: String,
    filter: Object,
    is_public: Boolean,
    count: Number,
    allow_update: true
  }
}

Business Logic Flow:

1. Validate Permissions

   if (isPublic && !is_owner && !role == 'admin') {
     throw new Error('Can not add filter for team.');
   }

2. Save Filter

   const savedData = await FilterModel.add(accountID, userID, type, filter, title, isPublic);

3. Calculate Initial Count

   if (type === 'deal') {
     const countObj = await countProvider.dealCount({
       filterObj: savedData.filter,
       filterID: savedData.id,
       pipeline,
       account_id: accountID,
       uid: userID,
       is_owner,
     });
     savedData.count = countObj.total;
   } else if (type === 'contacts' || type === 'people' || type === 'businesses') {
     const countObj = await countProvider.contactCount({
       filterObj: savedData.filter,
       filterID: savedData.id,
       searchType: type,
       account_id: accountID,
       uid: userID,
       is_owner,
     });
     savedData.count = countObj.total;
   }

4. Return with Permissions

   savedData.allow_update = true;
   return { success: true, message: 'SUCCESS', data: savedData };

Key Business Rules:

• Public Filter Restriction: Only owners and admins can create public (team) filters
• Count Calculation: Immediate count provided on creation
• Ownership: Creator automatically has update permission

---

FilterService.put({ accountID, userID, filter, title, filterID, isPublic, pipeline, is_owner })

Purpose: Update existing filter or duplicate if default filter

Parameters:

• accountID (ObjectId) - Account ID
• userID (ObjectId) - User ID
• filter (Object) - Updated filter conditions
• title (String) - Updated title
• filterID (String) - Filter ID to update
• isPublic (Boolean) - Updated visibility
• pipeline (ObjectId) - Pipeline ID (for deals)
• is_owner (Boolean) - Whether user is account owner

Returns:

{
  success: true,
  message: 'SUCCESS',
  data: {
    id: String,
    title: String,
    filter: Object,
    count: Number,
    allow_update: true
  }
}

Business Logic Flow:

1. Find Existing Filter

   const filterData = await FilterModel.findOne(filterID, accountID, userID);
   if (!filterData) {
     throw Error('Not exist!');
   }

2. Handle Default Filters

   let savedData = {};
   if (filterData.is_default) {
     // Default filters can't be edited, create new one
     const type = filter.type;
     savedData = await FilterModel.add(accountID, userID, type, filter, title, isPublic);
   } else {
     // Update existing filter
     savedData = await FilterModel.update(filterID, filter, title, isPublic);
   }

3. Recalculate Count

   if (savedData.type === 'deal') {
     const countObj = await countProvider.dealCount({
       filterObj: savedData.filter,
       filterID: savedData.id,
       pipeline,
       account_id: accountID,
       uid: userID,
       is_owner,
     });
     savedData.count = countObj.total;
   } else if (
     savedData.type === 'contacts' ||
     savedData.type === 'people' ||
     savedData.type === 'businesses'
   ) {
     const countObj = await countProvider.contactCount({
       filterObj: savedData.filter,
       filterID: savedData.id,
       searchType: savedData.type,
       account_id: accountID,
       uid: userID,
       is_owner,
     });
     savedData.count = countObj.total;
   }

Key Business Rules:

• Default Filter Protection: Default filters are duplicated instead of edited
• Count Recalculation: Count updated after every edit
• Ownership Preserved: User maintains ownership of duplicated default filters

---

FilterService.delete({ accountID, userID, filterID })

Purpose: Delete a user's filter

Parameters:

• accountID (ObjectId) - Account ID
• userID (ObjectId) - User ID
• filterID (String) - Filter ID to delete

Returns:

{
  success: true,
  message: 'SUCCESS'
}

Business Logic Flow:

1. Validate Filter Exists

   const filterData = await FilterModel.findOne(filterID, accountID, userID);
   if (!filterData) {
     throw Error('Not exist!');
   }

2. Delete Filter

   await FilterModel.deleteById(filterID);
   return { success: true, message: 'SUCCESS' };

Key Business Rules:

• Ownership Check: User can only delete their own filters
• Default Filters: Cannot be deleted (protected by model)

---

🔀 Count Provider Functions

countProvider.dealCount({ filterObj, filterID, pipeline, account_id, uid, is_owner })

Purpose: Calculate count for a single deal filter

Parameters:

• filterObj (Object) - Filter conditions
• filterID (String) - Filter ID for response
• pipeline (ObjectId) - Pipeline ID
• account_id (ObjectId) - Account ID
• uid (ObjectId) - User ID
• is_owner (Boolean) - Whether user is owner

Returns:

{
  filterID: String,
  total: Number
}

Business Logic Flow:

1. Convert Filter to MongoDB Query

   const filterJsonString = JSON.stringify(filterObj || {});
   const filterString = Buffer.from(filterJsonString).toString('base64');
   const filterConditions = await filterUtility(filterString, uid);

2. Validate Pipeline

   let foundPipeline = await Pipeline.findOne({
     _id: pipeline,
     account_id: account_id,
   });
   
   if (!foundPipeline) {
     return Promise.resolve({ filterID, total: 0 });
   }

3. Build Query with Visibility

   let options = {
     account_id: new mongoose.Types.ObjectId(account_id),
     pipeline_id: new mongoose.Types.ObjectId(pipeline),
     ...(filterConditions ? filterConditions : {}),
   };
   
   if (!is_owner) {
     options.$and = [
       ...(options.$and || []),
       {
         $or: [
           { visibility: 'all' },
           { owner: new mongoose.Types.ObjectId(uid) },
           { followers: new mongoose.Types.ObjectId(uid) },
         ],
       },
     ];
   }

4. Count Documents

   let count = await Deal.countDocuments(options);
   return Promise.resolve({ filterID, total: count });

Key Business Rules:

• Visibility: Non-owners only see their own deals, followed deals, or public deals
• Pipeline Required: Returns 0 if pipeline not found
• Error Handling: Returns 0 on error (graceful degradation)

---

countProvider.dealFilters({ filtersDocs, searchType, pipeline, account_id, uid, is_owner })

Purpose: Calculate counts for multiple deal filters in single aggregation

Parameters:

• filtersDocs (Array) - Array of filter documents
• searchType (String) - 'deal'
• pipeline (ObjectId) - Pipeline ID
• account_id (ObjectId) - Account ID
• uid (ObjectId) - User ID
• is_owner (Boolean) - Whether user is owner

Returns:

[
  {
    id: String,
    title: String,
    type: String,
    filter: Object,
    count: Number,
    total: 0,
  },
];

Business Logic Flow:

1. Convert All Filters

   let filters = await filtersDocs.reduce((n, c) => {
     n.push(
       new Promise(async (resolve, reject) => {
         try {
           let f = c.toJSON();
           const filterString = Buffer.from(JSON.stringify(f.filter || {})).toString('base64');
           resolve({
             id: f.id,
             filter: await filterUtility(filterString, uid),
             raw: f,
           });
         } catch (err) {
           reject(err);
         }
       }),
     );
     return n;
   }, []);
   
   filters = await Promise.all(filters);

2. Build $facet Pipelines

   let pipelines = {};
   
   for (f of filters) {
     pipelines[f.id] = [
       {
         $match: f.filter,
       },
       {
         $group: {
           _id: 'total',
           total: { $sum: 1 },
         },
       },
     ];
   }

3. Single Aggregation for All Counts

   let options = {
     account_id: new mongoose.Types.ObjectId(account_id),
     pipeline_id: new mongoose.Types.ObjectId(pipeline),
   };
   
   if (!is_owner) {
     options.$and = [
       ...(options.$and || []),
       {
         $or: [
           { visibility: 'all' },
           { owner: new mongoose.Types.ObjectId(uid) },
           { followers: new mongoose.Types.ObjectId(uid) },
         ],
       },
     ];
   }
   
   let count = await Deal.aggregate([
     {
       $match: options,
     },
     {
       $facet: pipelines,
     },
   ]);

4. Merge Counts with Filter Objects

   filters = Object.keys(count[0]).map(c => {
     return {
       ...filters.find(f => f.id == c).raw,
       count: count[0][c][0]?.total || 0,
       total: 0,
     };
   });
   
   return Promise.resolve(filters);

Key Business Rules:

• Performance: Single aggregation for all filters (not N queries)
• $facet: Each filter gets its own pipeline branch
• Visibility: Applied at aggregation level
• Error Handling: Returns 0 count on filter error

---

countProvider.contactCount({ filterObj, filterID, searchType, account_id, uid, is_owner })

Purpose: Calculate count for a single contact filter

Parameters:

• filterObj (Object) - Filter conditions
• filterID (String) - Filter ID
• searchType (String) - 'contacts', 'people', or 'businesses'
• account_id (ObjectId) - Account ID
• uid (ObjectId) - User ID
• is_owner (Boolean) - Whether user is owner

Returns:

{
  filterID: String,
  total: Number
}

Business Logic Flow:

1. Map Search Type to Contact Type

   let type;
   if (searchType === 'people') {
     type = 'person';
   } else if (searchType === 'businesses') {
     type = 'business';
   }

2. Build Query Options

   const filterJsonString = JSON.stringify(filterObj || {});
   const filterString = Buffer.from(filterJsonString).toString('base64');
   const filterConditions = await filterUtility(filterString, uid);
   
   let options = {
     parent_account: new mongoose.Types.ObjectId(account_id),
     type,
     ...(filterConditions ? filterConditions : {}),
   };
   
   if (!is_owner) {
     options.$and = [
       ...(options.$and || []),
       {
         $or: [
           { visibility: 'all' },
           { owner: new mongoose.Types.ObjectId(uid) },
           { followers: new mongoose.Types.ObjectId(uid) },
         ],
       },
     ];
   }

3. Count with Limit

   let count = await Contact.aggregate([
     {
       $match: options,
     },
     {
       $limit: 1000, // Performance optimization
     },
     {
       $group: {
         _id: 'total',
         total: { $sum: 1 },
       },
     },
   ]);
   
   return Promise.resolve({ filterID, total: count[0]?.total || 0 });

Key Business Rules:

• Type Mapping: people → person, businesses → business
• 1000 Limit: Caps count at 1000 for performance
• Visibility: Non-owners see only accessible contacts
• Error Handling: Returns 0 on error

---

countProvider.contactFilters({ filtersDocs, searchType, account_id, uid, is_owner })

Purpose: Calculate counts for multiple contact filters in single aggregation

Parameters:

• filtersDocs (Array) - Array of filter documents
• searchType (String) - 'contacts', 'people', or 'businesses'
• account_id (ObjectId) - Account ID
• uid (ObjectId) - User ID
• is_owner (Boolean) - Whether user is owner

Returns:

[
  {
    id: String,
    title: String,
    type: String,
    filter: Object,
    count: Number,
    total: 0,
  },
];

Business Logic Flow: Similar to dealFilters but with contact-specific type mapping and no pipeline requirement.

---

🔀 Integration Points

CRM Module

Deal Filtering:

// Get filtered deals with saved filter
const filters = await filterService.get({
  accountID,
  userID,
  type: 'deal',
  pipeline: pipelineId,
  is_owner,
});

// Apply filter to deal query
const dealsQuery = {
  ...filters[0].filter,
  pipeline_id: pipelineId,
};

Contact Filtering:

// Get people filters
const filters = await filterService.get({
  accountID,
  userID,
  type: 'people',
  is_owner,
});

Filter Utility

Filter Decoding:

const filterUtility = require('../../utilities/filter');

// Convert base64 filter string to MongoDB query
const filterString = Buffer.from(JSON.stringify(filterObj)).toString('base64');
const mongoQuery = await filterUtility(filterString, uid);

🧪 Edge Cases & Special Handling

Default Filter Protection

Cannot Edit Default Filters:

if (filterData.is_default) {
  // Create duplicate instead of editing
  savedData = await FilterModel.add(accountID, userID, type, filter, title, isPublic);
}

Public Filter Permissions

Only Owners/Admins Can Create Public Filters:

if (isPublic && !is_owner && !role == 'admin') {
  throw new Error('Can not add filter for team.');
}

Missing Pipeline for Deals

Graceful Degradation:

let foundPipeline = await Pipeline.findOne({ _id: pipeline, account_id });

if (!foundPipeline) {
  return Promise.resolve({ filterID, total: 0 });
}

Empty Filter List

No Pipelines Built:

if (pipelines && Object.keys(pipelines).length === 0) {
  throw new Error(`Filters data not found for type ${type}`);
}

Count Limit for Performance

Contact Count Capped at 1000:

let count = await Contact.aggregate([
  { $match: options },
  { $limit: 1000 }, // Prevent slow queries
  { $group: { _id: 'total', total: { $sum: 1 } } },
]);

⚠️ Important Notes

1. Real-Time Counts: Filter counts are recalculated on every request. For high-traffic applications, consider caching.

2. Visibility Rules: Non-owners only see:

   • Deals/contacts they own
   • Deals/contacts they follow
   • Deals/contacts with visibility='all'

3. Performance: dealFilters() and contactFilters() use single $facet aggregation for all filters instead of N queries.

4. Filter Utility: Filters are base64-encoded JSON that's decoded and converted to MongoDB queries by the filter utility.

5. Type Mapping: Frontend uses 'people'/'businesses', backend uses 'person'/'business'.

6. Default Filters: System-provided filters cannot be edited, only duplicated.

7. Pipeline Requirement: Deal filters require a valid pipeline ID. Contact filters do not.

8. Count Limit: Contact counts capped at 1000 for performance (large contact lists).

9. Ownership Permissions: Users can only update/delete their own filters.

10. Public Filters: Only account owners and admins can create team-wide (public) filters.

11. Error Handling: Count providers return 0 on error (graceful degradation).

12. Aggregation Optimization: Uses $facet to run multiple filter counts in parallel within single query.

🔗 Related Documentation

• CRM Deals Module - Deal filtering integration
• CRM Contacts Module - Contact filtering integration
• Pipelines - Pipeline management for deal filters