Forms Module

🚀 Module summary

The internal Forms module lives under internal/api/v1/forms. It owns everything required to let agencies build forms, publish them to clients, request responses, and push the resulting data into downstream automations (CRM contacts, onboarding jobs, Instasites/Instareports builds, deal creation, and alerts). Most request validation happens in validators/, while controllers coordinate shared utilities, campaign data, notifications, and Mongoose aggregations.

📁 Directory structure

internal/api/v1/forms/
├── Controllers/
│   ├── forms.js            # Form authoring, categories/tags, send requests, widgets
│   ├── templates.js        # Re-usable template catalogue
│   └── userresponse.js     # Response ingest, updates, pending queues, analytics
├── Models/
│   ├── forms.js
│   ├── forms-sent-requests.js
│   ├── templates.js
│   ├── usercategories.js
│   ├── userresponse.js
│   └── usertags.js
├── Routes/
│   ├── forms.js            # Mounted at /forms in the Internal API
│   └── templates.js        # Mounted at /forms/templates
├── utilities/
│   ├── inbound.js          # Post-submission orchestration (CRM, Instasite, notifications)
│   └── country-codes/
│       ├── codes.js
│       └── tests/
│           ├── inbound-util.test.js
│           └── testing_constants.js
├── validators/
│   ├── index.js            # Joi schemas for /forms routes
│   └── templates.js        # Joi schemas for /forms/templates
├── README.md
└── index.js                # Router bootstrap

🧩 Key dependencies & utilities

Dependency	Purpose
shared/models/forms.js	Backing Mongoose schema for authored forms
shared/models/forms-userresponse.js	Stores submissions and request-link metadata
shared/models/forms-sent-requests.js	Tracks invite links, status, and resend metadata
internal/api/utilities/index.js	Domain resolution, pagination helper, auth/scope middlewares
internal/api/utilities/mail.js / sms.js	Email + SMS delivery wrappers (SendGrid + Twilio)
internal/api/utilities/inbound.js	Converts submissions into CRM contacts, deals, Instasites, Instareports, and queue jobs
shared/models/campaign-data.js	Determines automation recipes (create CRM contact, notify, build assets)
shared/models/contact.js, _users	Resolve recipient/contact data when sending requests
shared/models/store-order.js	Onboarding order state machine updated after form responses

Environment variables referenced directly in controllers:

• PORT – used to call the Internal short-link service before handing back SHORT_DOMAIN URLs.
• SHORT_DOMAIN – base domain appended to shortened request links.

All outbound notifications run through centralized helpers, so SendGrid/Twilio credentials remain in shared utilities rather than this module.

🗄️ MongoDB collections touched

• forms – Canonical authored form document (formsModel).
• forms.sent.requests – One document per share/request link (formSentRequest).
• forms.userresponse – Stored submission payloads plus analytics metadata (UserResponseModel).
• forms-user-categories & forms-user-tags – Per-account tag/category catalogs.
• campaign.data – Campaign automation rules for inbound follow-up.
• crm.contacts – Recipient metadata for send/resend flows.
• _users – Used when pending requests were sent to internal users instead of CRM contacts.
• _store.orders – Updated to mark onboarding milestones when responses are received.

🔁 Request & automation flow

flowchart TD
    A[Agency user creates form] --> B[formsController.saveForm]
    B --> C{is clone?}
    C -->|yes| D[getFormCopyName resolves unique name]
    C -->|no| E[Persist draft form]
    E --> F[Return secret link + domain]

    F --> G[Agency configures + publishes]
    G --> H[formsController.updateForm]
    H --> I[Status: published]
    I --> J[formsController.sendRequest]
    J --> K[forms.sent.requests save]
    K --> L[sendMail/sendSMS]
    L --> M[Recipient clicks link]

    M --> N[userresponseController.saveResponse]
    N --> O[forms.userresponse insert]
    O --> P[inbound.inHouseForms]
    P --> Q[Create/update CRM contact + deals]
    Q --> R[Trigger Instasite/Instareport builds when configured]
    R --> S[Update Store Orders / Support Rooms / Notifications]

📚 Sub-module documentation

• Form authoring & request workflows – categories, tags, CRUD, sending requests, widget metrics.
• Template catalogue & onboarding defaults – list/save/update templates and restricted scopes.
• Response ingest & post-processing – submission handling, pending queues, analytics, onboarding linkage.

Each page enumerates controller methods, Joi schemas, related models, and edge cases.

🔗 Routing overview

File	Mount point	Highlights
Routes/forms.js	/forms	Secured by verifyAuthorization, scope enforcement (forms.*), and Joi validation. Supports user tag/category CRUD, bulk deletes, request sending, widget stats, and public response submission (unauthenticated POST).
Routes/templates.js	/forms/templates	Special scope dashclicks.forms.templates protects template authoring. Standard pagination via validateLimit.

When the Internal API is proxied through the router service, these endpoints surface under /v1/forms and /v1/forms/templates.

🧠 Validation

• validators/index.js guards almost every route (params/query/body). Notable rules:
  • Accept both Mongo ObjectId and UUID for :formid (public share URLs use form_secret_id).
  • Bulk delete requires total to match server-side count unless explicit IDs are provided.
  • POST /forms supports cloning (?isclone=true&form_id=<id>) and ensures incoming structures match Instasite/Instareport expectations when form_type is set.
• validators/templates.js requires special template scopes and validates template IDs.

🧭 Operational notes

• Notification prerequisites – Email delivery falls back to a generic sender if the recipient record is missing a name; SMS requires the account to own a Twilio number or the request fails with NO_NUMBER_PURCHASED.
• Campaign automation – Form deletion cascades to campaign.data documents by setting is_deleted=true, preventing zombie automations.
• Widget metrics – forms.getFormsWidgetData computes counts via aggregation, filtering only status: 'published' forms.
• Short URL service – shortenUrl makes an authenticated POST back into the Internal service (loopback) using the same JWT; ensure the short-link service is enabled in non-production environments before testing send/resend flows.

✅ Next steps

Head over to the sub-documents for deep dives on specific controllers:

• Forms controller
• Templates controller
• User response controller

They document request contracts, background side effects, and integration touch points in detail.