Activities Module

🎯 Overview

The Activities module orchestrates every REST endpoint that reads, filters, and mutates CRM activity timelines for contacts and deals. It keeps customer-facing feeds aligned with real-world interactions by hydrating each timeline entry with type-specific payloads, author metadata, and threaded comments.

It exists to unify several distinct feature surfaces—manual notes, automated emails and SMS, reminders, Instasites/Instareports, and general logs—under a single audit trail with consistent permissions. The router centralizes validation, visibility rules, and population logic so that any client (dashboard, public APIs, automations) receives the same normalized activity response structure.

Operationally, the module acts as a façade over multiple shared models, applying opinionated business rules (ownership, followers, account preferences) before persisting or exposing data. This allows downstream analytic jobs and UI components to consume activities without re-implementing guardrails.

📁 Directory Structure

internal/api/v1/activities/
├── 📄 index.js                    # Express router exporting all /v1/activities routes
├── 📄 README.md                   # Legacy readme (kept for historical context)
├── 📂 helper/
│   └── 📄 helper.js               # Custom Joi helpers (ObjectId, date, time validation)
├── 📂 middleware/
│   └── 📄 index.js                # Log activity validator selecting subtype schemas
└── 📂 schemas/
    ├── 📄 index.js                 # Route-to-Joi schema map consumed by validation utility
    ├── 📄 post_activity_details.js # Post payload schemas for log subtypes (call/email/meeting)
    └── 📄 put_activity_details.js  # Put payload schemas mirroring post definitions

🗄️ MongoDB Collections

Collection: _activity

• Purpose: Canonical activity documents representing every timeline event.
• Model Reference: shared/models/activity.js
• Key Fields: account, created_by, contact, deal, activity_type, logs_type, activity_details, comments, pinned, type
• Indexes: Compound indexes on (account, type), (contact, created), and (deal, created) support timeline queries.

Collection: _activity.comments

• Purpose: Stores threaded comments attached to activity records (primarily notes).
• Model Reference: shared/models/activity-comment.js
• Key Fields: account, created_by, activity_id, content, created, updated
• Indexes: Indexed on activity_id for efficient hydration and deletion.

Collection: crm.notes

• Purpose: Primary note bodies that can appear on timelines.
• Model Reference: shared/models/crm-note.js
• Key Fields: account, created_by, contact, deal, content, attachments, pinned
• Indexes: (account, contact) and (account, deal) for ownership filtering.

Collection: crm.notes.comments

• Purpose: Persist threaded comments on CRM notes surfaced inside activities.
• Model Reference: shared/models/crm-note-comment.js
• Key Fields: account, created_by, note_id, content, timestamps
• Indexes: note_id index for fast aggregation.

Collection: crm.reminders

• Purpose: Reminder payloads associated with activities and assigned users.
• Model Reference: shared/models/crm-reminder.js
• Key Fields: account, contact, deal, headline, content, assigned, due_date_time
• Indexes: (account, due_date_time) for scheduling dashboards.

Collection: communications

• Purpose: Messages (email, SMS) and delivery event logs that feed activity entries.
• Model Reference: shared/models/communication.js
• Key Fields: account_id, contact_id, sent_by, message_type, events, data
• Indexes: (account_id, contact_id) plus TTL/event indexes maintained by the communications service.

Collection: crm.contacts

• Purpose: CRM contact metadata and ownership flags used for visibility enforcement.
• Model Reference: shared/models/contact.js
• Key Fields: parent_account, account, owner, followers, visibility, last_activity_date
• Indexes: Ownership and visibility indexes support guardrail queries.

Collection: crm.deals

• Purpose: CRM deal metadata parallel to contacts.
• Model Reference: shared/models/deal.js
• Key Fields: account_id, owner, followers, visibility, last_activity_date
• Indexes: (account_id, pipeline_stage) and follower indexes for permission checks.

Collection: instasites

• Purpose: Landing page generation artifacts surfaced as activities.
• Model Reference: shared/models/instasite.js
• Key Fields: account_id, created_by, status, details
• Indexes: Status-based indexes for deployment telemetry.

Collection: instareports

• Purpose: Audit reports tied to activities.
• Model Reference: shared/models/instareport.js
• Key Fields: account_id, created_by, status, details, generated_at
• Indexes: account_id and generated_at indexes for reporting windows.

📚 Note: For full schema definitions and relationships, review the shared models under shared/models/.

🏗️ Architecture Overview

• Pattern: Layered Express router with validation middleware, service-style helpers, and shared Mongoose models.
• Key Dependencies: verifyScope, validateRequestSchemaV2, generatePagination, setActivityID, and activityExpandOptions from internal/api/v1/utilities.
• External Services: Leverages shared communications data and Instasite/Instareport services via MongoDB models (no direct HTTP calls within the module).
• Auth Context: Relies on upstream middleware populating req.auth (account id, user id, owner flag, preferences) to apply visibility rules.

🔗 Submodules

• 📘 ./helper – Joi custom validators for Mongo ObjectIds and date/time values.
• 📗 ./middleware – Conditional middleware that selects the right log subtype schema before hitting the controller logic.
• 📙 ./schemas – Route-level Joi definitions plus per-log-type payload schemas for create/update flows.

🚀 Quick Start

// Fetch the latest contact activities with expanded entity payloads
const { data, pagination } = await apiClient.get('/v1/activities', {
  params: {
    type: 'contact',
    contact_id: '64f4b0f0b2a3f7123b6c1234',
    expand: true,
    page: 1,
    limit: 25,
  },
  headers: {
    X-Session-Id: "MongoObjectId",
  },
});

console.log(pagination.total, 'activities available');

🔐 Required Environment Variables

Variable	Description	Required
—	No module-specific environment variables. Requires upstream authentication middleware to set req.auth.	❌ Optional

📊 Key Metrics & Performance

• Sorting by pinned then created keeps high-priority activities surfaced without additional queries.
• Concurrency is handled via Promise.all (count + fetch); pagination defaults should stay under 100 to avoid large populate chains.
• Email/SMS hydration reads the latest communication events to compute delivery flags; heavy historical accounts may benefit from communication event archiving.

🔧 Business Logic

🔁 Common Processing Pipeline

flowchart TD
  A[📥 Request Received] --> B[🔐 verifyScope activities scope check]
  B --> C{✅ Authenticated?}
  C -->|No| Z[❌ 403 Not Allowed]
  C -->|Yes| D[🧾 validateRequestSchemaV2]
  D --> E[🛡️ Apply Visibility Guardrails]
  E --> F[🗂️ Resolve ActivityType + populate plan]
  F --> G[📊 Query + Hydrate]
  G --> H[🧮 generatePagination]
  H --> I[🧼 setActivityID + post-process]
  I --> J[📤 Standardized Response]

  classDef start fill:#e1f5ff,stroke:#333,stroke-width:1px;
  classDef decision fill:#fff3cd,stroke:#333,stroke-width:1px;
  classDef error fill:#f8d7da,stroke:#333,stroke-width:1px;
  class A start;
  class C decision;
  class Z error;

Contract Summary

• Inputs: HTTP request with path/query/body, authenticated user context on req.auth.
• Outputs: { success, message, data, pagination? } with activities normalized via setActivityID.
• Success Criteria: Caller sees only owned/followed entities, hydrated payload aligns with activity type, pagination metadata accurate.

📥 GET /v1/activities

Purpose – GET

Returns a mixed activity feed for a contact or deal with optional type filtering and entity expansion.

Input Contract – GET

• type ('contact' | 'deal') – required; selects entity collection.
• contact_id / deal_id – required per type for scoped feeds.
• Pagination – page, limit (defaults applied by validation).
• expand (boolean) – include full entity/creator payloads instead of lean projection.
• activity_type[] – optional filter array.

Processing Flow – GET

1. Resolve entity visibility via Entity[type].findOne using owner/follower preferences from req.auth.account.preferences.
2. Construct base filter: { account: req.auth.account_id, type, [contact|deal]: entityId }.
3. Apply optional activity type filter and pinned ordering (sort: [['pinned', -1], ['created', -1]]).
4. Execute Promise.all for countDocuments and populated query with skip/limit.
5. Call ActivityType[activity.activity_type].populate for type-specific hydrate (notes, reminders, communications, Instasites/reports).
6. Invoke getComments on note activities and compute communication flags (sent, delivered, opened, clicked).
7. Append sentinel “contact_created” or “deal_created” records to last page using account/entity metadata.
8. Normalize IDs (setActivityID) and build pagination object via generatePagination.

Business Rules – GET

• Non-owners only see entities they own/follow or that are globally visible per account preferences.
• Pinned activities always float above chronological items.
• Creation sentinel inserted only when requesting the final page to avoid duplicate context.

Response Shape – GET

• Array of normalized activity objects with hydrated activity_details, entity, creator, and comments (if applicable).
• Pagination object with total, limit, page, pages.

Failure Modes – GET

• Throws notFound if entity is inaccessible or missing.
• Throws notAllowed if visibility guardrails fail for non-owner.

📬 POST /v1/activities/filter

Purpose – POST Filter

Advanced search endpoint layering filters over the base feed for analytics-style views.

Input Contract – POST Filter

• Same query params as the list route.
• Body filters: date ({ from, to }), created_by[], activity_type[], logs_type[].

Processing Flow – POST Filter

1. Validate body with schemas.activities_filter ensuring arrays and date bounds are well-formed.
2. Build dynamic $and pipeline combining date range, creators, and type filters.
3. Merge permission clauses identical to GET /v1/activities.
4. Execute count + populated query concurrently; note sentinel entries are omitted.
5. Post-process with getComments, communication event parsing, and setActivityID.

Business Rules – POST Filter

• Activity and log type filters operate on both activity_type and logs_type fields to catch log subtypes.
• Date filter is inclusive and uses account timezone preferences from req.auth.account.preferences when present.

Response Shape – POST Filter

• Hydrated activities array, pagination metadata.

Failure Modes – POST Filter

• Returns empty array if filters exclude all activities; validation errors bubble up with badRequest responses.

🎛️ GET /v1/activities/:activity_type

Purpose – GET Activity Type

Fetches timeline items constrained to a single activity type (e.g., notes).

Processing Flow – GET Activity Type

• Forces filter.activity_type = params.activity_type before executing the standard list flow.
• Reuses permission and hydration logic; sentinel creation events suppressed to keep type-specific feeds pure.

📝 POST /v1/activities/:activity_type

Purpose – POST Activity Type

Creates a new activity entry tied to a contact or deal.

Input Contract – POST Activity Type

• type ('contact' | 'deal') and matching entity id.
• activity_details object with type-specific payload (note content, reminder fields, communication metadata, etc.).
• Optional logs_type for activity_type === 'logs'.

Processing Flow – POST Activity Type

1. Run shared schema validation plus validateActivityDetailsSchema for log subtypes (CALL/EMAIL/MEETING).
2. Confirm entity accessibility (Entity[type].findOne).
3. Call Activity.create with normalized payload (account, creator, entity references, pinned default false).
4. Hydrate the resulting activity using the same pipeline as the read endpoints.
5. Update entity last_activity_date via Mongoose hooks.

Business Rules – POST Activity Type

• Only owners/followers can log activities unless account preferences allow global visibility.
• Log activities enforce subtype-specific schemas to ensure metrics parity downstream.

Response Shape – POST Activity Type

• Newly created, populated activity object (ID normalized, comments array empty for notes).

Side Effects – POST Activity Type

• Mongoose middleware updates crm.contacts / crm.deals last_activity_date.

Failure Modes – POST Activity Type

• Validation errors return badRequest describing the offending field.
• Unauthorized access throws notAllowed.

🔍 GET /v1/activities/:activity_type/:id

Purpose – GET Activity By ID

Retrieves a single activity by ID for detail views.

Processing Flow – GET Activity By ID

• Filters by { _id: params.id, account: req.auth.account_id, activity_type }.
• Non-owners must be entity owners/followers or the activity creator; otherwise notAllowed is raised.
• Response retains array structure for backward compatibility (data: [activity]).

Post-Processing – GET Activity By ID

• Hydrates entity, creator, comments, and type-specific payload prior to normalization.

♻️ PUT /v1/activities/:activity_type/:id

Purpose – PUT Activity

Updates activity metadata, pinned status, or payload details.

Input Contract – PUT Activity

• Optional pinned boolean.
• Updated activity_details object (dot-flattened internally for $set).
• For logs, logs_type and nested payload must pass put_activity_details schema.

Processing Flow – PUT Activity

1. Validate payload (route schema + conditional log schema).
2. Confirm visibility and fetch existing document.
3. Build $set object, flattening nested objects to dot paths for Mongoose updates.
4. Execute findOneAndUpdate with { new: true } to retrieve updated doc.
5. Rehydrate + normalize like read endpoints.

Business Rules – PUT Activity

• Pinned state can only be toggled by users with visibility rights.
• Reminders propagate assigned user updates to the hydrated payload.

Response Shape – PUT Activity

• Updated, normalized activity.

Failure Modes – PUT Activity

• Returns notFound when ID does not exist for caller’s account.

🗑️ DELETE /v1/activities/:activity_type/:id

Purpose – DELETE Activity

Removes an activity from the timeline.

Processing Flow – DELETE Activity

• Performs the same ownership check as read/update routes.
• Calls findByIdAndDelete; cascades to comment cleanup handled in model hooks.
• Responds with { success: true, message: 'Activity deleted successfully.' }.

Failure Modes – DELETE Activity

• notAllowed for unauthorized deletions; notFound if already removed.

💬 Comment Routes (POST/PUT/DELETE /:activity_type/:id/comments)

Purpose – Comment Routes

Manage note comment threads attached to activities.

Processing Flow – Comment Routes

1. Validate params/body with comment-specific Joi schemas.
2. Call respective Activity model helpers (addComment, updateComment, deleteComment).
3. Helpers mutate _activity.comments array and persist ActivityComment documents.
4. Rehydrate parent activity (including refreshed comments) before responding.

Business Rules – Comment Routes

• Comments are limited to note-based activities; helper short-circuits for other types.
• Updated/deleted comments log updated_by metadata for audit compatibility.

Failure Modes – Comment Routes

• Missing comment IDs return notFound.
• Permission checks mirror parent activity rules.

🛠️ Helper Functions

• activityExpandOptions(expand: boolean) – Determines lean vs. full projections when populating activities and related entities.
• getComments(noteId) – Aggregates crm.notes.comments for the supplied note, ordered by creation time.
• validateMongoID/Date/Time – Joi extensions reused across schemas to ensure consistent ID and temporal validation.
• setActivityID(activity) – Clones _id to id on the activity and nested activity_details documents for client consumption.

🔄 Data Flow

graph TB
    A[📥 Client Request] --> B[🔐 Scope + Auth Validation]
    B --> C[🧾 Joi Schema Validation]
    C --> D[🛡️ Visibility Filter]
    D --> E[🗃️ Mongo Query]
    E --> F[🧬 Populate Type-Specific Details]
    F --> G[💬 Attach Comments / Flags]
    G --> H[🧮 Pagination Metadata]
    H --> I[📤 Response Builder]

    style A fill:#e1f5ff
    style I fill:#e1ffe1

🧪 Edge Cases & Special Handling

• Empty Feeds: Returns empty data array with pagination totals of zero; sentinel entries are skipped to avoid confusion.
• Unpinned to Pinned Transitions: Clients must refetch after pinning because sort order changes; API ensures pinned flag updates atomically.
• Orphaned Comments: Model hooks remove orphaned comment IDs during delete flows so subsequent hydration will not fail.
• Legacy Single Activity Response: GET /:activity_type/:id continues to return an array for backward compatibility—client adapters should handle single-item arrays.

⚠️ Important Notes

• 🚨 Permissions: Always pass the authenticated user’s token with the activities scope. Non-owner users inherit strict visibility filters; bypassing through service tokens is discouraged unless executing system jobs.
• 💡 Best Practice: Request small page sizes (25–50) when expand=true to limit populate cost.
• 🔒 Security: The module never exposes raw communication payloads containing PII unless the caller has access to the related contact/deal.

🔗 Related Documentation

• shared/models/activity.js – Source of activity schema hooks and static helpers.
• internal/api/v1/utilities/validateRequestSchemaV2.js – Shared validation utility consumed by this module.
• internal/api/v1/utilities/verifyScope.js – Scope enforcement used across internal services.