CRM Module

🎯 Overview

internal/api/v1/crm powers DashClicks' customer relationship management stack. It combines ten focused services that orchestrate contacts, deals, pipelines, reminders, and automations while enforcing account-level permissions, owner visibility rules, and data hygiene workflows. Controllers keep HTTP concerns thin; almost all business decisions live inside the service layer.

The module exposes /v1/crm/* routes, backs the dashboard CRM experience, and feeds reporting as well as downstream automation systems (notifications, queue-manager jobs, warehouse exports).

📁 Directory Structure

internal/api/v1/crm/
├── .gitignore
├── index.js                 # Router bootstrap that mounts entity routers
├── README.md                # Developer setup guide specific to CRM module
├── controllers/             # Express controllers (automation, contact, deal, ...)
│   ├── automation.controller.js
│   ├── contact.controller.js
│   ├── deal.controller.js
│   ├── field.controller.js
│   ├── import.controller.js
│   ├── index.js
│   ├── note.controller.js
│   ├── pipeline.controller.js
│   ├── reminder.controller.js
│   └── tag.controller.js
├── routes/                  # Router definitions + middleware wiring per entity
│   ├── automation.route.js
│   ├── contact.route.js
│   ├── deal.route.js
│   ├── field.route.js
│   ├── import.route.js
│   ├── index.js
│   ├── note.route.js
│   ├── pipeline.route.js
│   ├── reminder.route.js
│   ├── stage.route.js
│   └── tag.route.js
├── services/                # Business logic layer (contact.service.js, etc.)
│   ├── automation.service.js
│   ├── contact.service.js
│   ├── deal.service.js
│   ├── field.service.js
│   ├── import.service.js
│   ├── index.js
│   ├── note.service.js
│   ├── pipeline.service.js
│   ├── reminder.service.js
│   ├── stage.service.js
│   └── tag.service.js
├── tests/                   # Jest suites (service/controller specs + fixtures)
│   ├── automation.test.js
│   ├── contact.test.js
│   ├── deal.test.js
│   ├── field.test.js
│   ├── note.test.js
│   ├── pipeline.test.js
│   ├── reminder.test.js
│   ├── stage.test.js
│   ├── tag.test.js
│   └── fixtures/
│       └── field.js
├── utils/                   # Shared helpers (contact/deal/pipeline mappers, expansions)
│   ├── automation.js
│   ├── contact.js
│   ├── deal.js
│   ├── field.js
│   ├── index.js
│   ├── pipeline.js
│   └── stage.js
└── validations/             # Joi schemas by domain (contact/, deal/, reminder/, ...)
    ├── index.js
    ├── automation/
    │   └── index.js
    ├── contact/
    │   ├── attachment.js
    │   ├── filter.js
    │   ├── getAll.js
    │   ├── index.js
    │   ├── search.js
    │   ├── business/
    │   │   ├── business-put.js
    │   │   ├── business.js
    │   │   ├── get.js
    │   │   ├── getAll.js
    │   │   └── index.js
    │   └── person/
    │       ├── get.js
    │       ├── getAll.js
    │       ├── index.js
    │       ├── person-put.js
    │       └── person.js
    ├── deal/
    │   └── index.js
    ├── field/
    │   └── index.js
    ├── import/
    │   └── index.js
    ├── note/
    │   └── index.js
    ├── pipeline/
    │   └── index.js
    ├── reminder/
    │   └── index.js
    ├── stage/
    │   └── index.js
    └── tag/
        └── index.js

Each subdirectory mirrors the service names below; there are no Git-tracked models/ or utilities/ folders here because Mongoose models and helpers are copied from shared/ at build time (see repository guide).

🗄️ MongoDB Collections

📚 Full Schema: See Database Collections Documentation

Core Entities

• crm.contacts → person/business records with embedded addresses, custom fields, attachments
• crm.deals → pipeline-bound opportunities with stage, owner, value, and automation flags
• crm.pipeline / crm.pipeline.stages → pipeline definitions, stage ordering, automation metadata
• crm.activity → audit log of notes, reminders, stage changes (read via utilities)
• crm.reminders → scheduled follow-ups and tasks
• crm.notes → entity notes + threaded comments
• crm.tags → account-scoped tagging taxonomy

Supporting Collections

• _accounts→ sub-account linkage and configuration
• crm.contact.import.results → status + diagnostics for import jobs
• queues → background jobs spawned by CRM services (imports, mass linking, automations)
• automation.logs → execution history for stage automations

🏗️ Architecture Overview

Key Responsibilities

• Normalise and secure CRUD for CRM entities under strict account/owner scoping
• Link contacts ↔ deals ↔ pipelines for reporting and automation triggers
• Provide mass operations (bulk delete, export, import) backed by queues
• Surface analytics endpoints (new contacts, pipeline leaderboards, automation logs)

Design Decisions

• Service-first pattern: Controllers defer to services; tests target services directly
• Owner visibility enforcement: Most queries branch on is_owner flag and account preferences
• Mongoose aggregations: Heavy pipelines compute filtered lists, analytics, exports
• Background work: Expensive flows push jobs into Queue or Google Pub/Sub
• Shared utilities: utils/contact|deal|pipeline supply reusable expansion, mapping, tag validation, timezone helpers

🔗 Submodules

Domain	Doc	What it covers
Contacts	👥 Contacts	People & businesses, linking, exports, mass updates
Deals	💼 Deals	Pipeline-scoped opportunities, attachments, automation flags
Pipelines	🔄 Pipelines	Pipeline CRUD, analytics, automation logs
Stages	⚙️ Stages	Per-pipeline stage CRUD and automation snapshots
Automation	🤖 Automation	Stage-triggered workflow definitions and lifecycle
Notes	📝 Notes	Notes, comments, activity fan-out
Reminders	⏰ Reminders	Task queues, categorisation, completion rules
Tags	🔖 Tags	Tag CRUD, locking, bulk removal side-effects
Fields	🧩 Fields	Custom property configuration + user layout alignment
Import	📥 Import	CSV/connector ingestion and queue orchestration

🔄 Data Flow

flowchart TD
    subgraph Contact Intake
        A[Create/Import Request] --> B{Validate + Map}
        B -->|pass| C[Persist Contact]
        C --> D[Expand/Enrich]
        D --> E[Queue Background Work]
    end
    subgraph Deal Lifecycle
        F[Deal Mutations] --> G[Pipeline Stage Logic]
        G --> H{Automation?
        Stage change?}
        H -->|yes| I[Trigger Jobs + Logs]
        G --> J[Currency Normalisation]
    end
    C -. links .-> F
    E --> K[Queue Collection]
    I --> L[automation.logs]
    K --> M[Queue Manager]
    M --> N[Notifications / Imports / Mass updates]

🔐 Configuration & Dependencies

Variable	Description	Required
BASE_CURRENCY	Fallback currency for deal/pipeline conversions	✅
GOOGLE_CLOUD_PROJECT_ID	Pub/Sub project used by import service	✅ (imports)
APP_SECRET	JWT secret for internal socket notifications (contact linking)	✅ (contacts)
CONVERSATION_SOCKET	Base URL for socket emit service when notifying support	✅ (contacts)

External Services

• Wasabi S3 via shared/utilities/wasabi for contact exports + attachment cleanup
• Google Pub/Sub for asynchronous imports (CSV + connectors)
• Queue Manager for mass association jobs, automation retries, reminder fan-out
• Currency util uses shared exchanger to display pipeline totals consistently

🚀 Quick Start

1. Install shared dependencies: pnpm install (root) followed by npm run dev:build as outlined in repo guide.
2. Start CRM service through VS Code debug configuration “Internal API – CRM”. This ensures symlinked models/utilities and proper env injection.
3. Seed required configs: confirm BASE_CURRENCY, Pub/Sub credentials, and Wasabi keys exist in environment.
4. Run targeted tests before shipping changes: pnpm test --filter crm (integration suites cover controllers + services).
5. Copy shared models if editing shared/ dependencies: npm run copySharedFiles.