🎯 Hubspot - CRM Data Export

📖 Overview

The Hubspot CRM data export feature provides unified access to contacts, companies, deals, and notes (engagements) via a single controller with dynamic type routing. All exports support pagination and automatic token refresh.

Source Files:

Controller: external/Integrations/Hubspot/Controllers/contactController.js
Provider: external/Integrations/Hubspot/Providers/crmData.js, Providers/api.js
Routes: external/Integrations/Hubspot/Routes/exportRoutes.js

Hubspot API Endpoints:

Contacts: GET https://api.hubapi.com/contacts/v1/lists/all/contacts/all
Companies: GET https://api.hubapi.com/companies/v2/companies/paged
Deals: GET https://api.hubapi.com/deals/v1/deal/paged
Notes: GET https://api.hubapi.com/engagements/v1/engagements/paged

🗄️ Collections Used

`integrations.hubspot.key`

Operations: Read, Update
Model: shared/models/hubspot-key.js
Usage Context: Retrieve OAuth tokens and automatically refresh expired access tokens

🔄 Data Export Flow

sequenceDiagram
    participant Client as DashClicks Client
    participant Controller as Contact Controller
    participant APIProvider as API Provider
    participant CRMProvider as CRM Data Provider
    participant DB as MongoDB
    participant Hubspot as Hubspot API

    Client->>Controller: GET /export/{type}?limit=100&offset=0
    Controller->>DB: Find token (account_id, owner)
    DB-->>Controller: Return token with refresh_token
    Controller->>APIProvider: getRefreshAccessToken(tokenData)
    APIProvider->>Hubspot: POST /oauth/v1/token (refresh)
    Hubspot-->>APIProvider: New access_token
    APIProvider->>DB: Update token + generated_at
    APIProvider-->>Controller: Return fresh access_token
    Controller->>CRMProvider: retrieveDATA(access_token, type, query)
    CRMProvider->>CRMProvider: Build endpoint URL based on type
    CRMProvider->>Hubspot: GET /contacts|companies|deals|engagements
    Hubspot-->>CRMProvider: CRM data + pagination
    CRMProvider->>CRMProvider: Normalize pagination fields
    CRMProvider-->>Controller: data + pagination
    Controller-->>Client: JSON response with data

🔧 Controller Functions

`readContactHubspot(req, res, next)`

Purpose: Unified export function for all Hubspot CRM data types

Source: Controllers/contactController.js

Endpoint: GET /v1/integrations/hubspot/export/:type

Path Parameters:

type (String) - Data type to export: contacts, companies, deals, or notes

Query Parameters:

limit (Number, optional) - Number of records to return (default: 10)
offset (Number, optional) - Pagination offset for next page
Additional query parameters are passed through for deals endpoint

Authorization:

Requires valid JWT in req.auth
Requires contacts or contacts.external scope

Business Logic Flow:

Verify Authentication
- Check req.auth for valid JWT
- Extract account_id and uid
Retrieve Stored Token
- Query integrations.hubspot.key by account_id and owner
- Return 404 if no token found
Refresh Access Token
- Call providers.getRefreshAccessToken(query)
- Obtain fresh access token
- Token is automatically saved to database
Retrieve CRM Data
- Call exportData.retrieveDATA(accessToken, type, query)
- Pass access token, data type, and query parameters
Return Response
- Format response with data and pagination
- Handle errors with next() middleware

Success Response:

{
  "success": true,
  "message": "DATA Recieved",
  "data": {
    "has-more": true,
    "offset": 3412996225,
    "contacts": [...],  // or companies, deals, engagements
    "vid-offset": 3412996225  // for contacts only
  },
  "pagination": {
    "hasMore": true,
    "offset": 3412996225
  }
}

Error Responses:

// Token not found
{
  "success": false,
  "errno": 400,
  "message": "Api Token Not Found"
}

// Unauthorized
{
  "success": false,
  "errno": 400,
  "message": "Unauthorized User"
}

// Invalid type
{
  "success": false,
  "errno": 400,
  "message": "data not found on object of type"
}

Example Usage:

// Export contacts
GET /v1/integrations/hubspot/export/contacts?limit=100&offset=0

// Export companies
GET /v1/integrations/hubspot/export/companies?limit=50

// Export deals with associations
GET /v1/integrations/hubspot/export/deals?limit=200&offset=400

// Export notes (engagements)
GET /v1/integrations/hubspot/export/notes?limit=100

🔧 CRM Data Provider Functions

`retrieveDATA(accessToken, getType, reqQuery)`

Purpose: Fetch CRM data from Hubspot API with type-specific endpoint and pagination handling

Source: Providers/crmData.js

Parameters:

accessToken (String) - Fresh OAuth access token
getType (String) - Data type: contacts, companies, deals, or notes
reqQuery (Object) - Express query parameters (limit, offset, etc.)

Returns: Promise<Object> - Data with normalized pagination

Business Logic Flow:

Determine Endpoint and Parameters

Based on getType, configure URL and query parameters:

Contacts:

url = 'https://api.hubapi.com/contacts/v1/lists/all/contacts/all';
queryParam = {
  count: reqQuery.limit ? reqQuery.limit : 10,
  vidOffset: reqQuery.offset ? reqQuery.offset : null,
};

Companies:

url = 'https://api.hubapi.com/companies/v2/companies/paged?properties=name&properties=website';
queryParam = {
  limit: reqQuery.limit ? reqQuery.limit : 10,
  offset: reqQuery.offset ? reqQuery.offset : null,
};

Deals:

url = 'https://api.hubapi.com/deals/v1/deal/paged?properties=dealname&includeAssociations=true';
queryParam = {
  limit: reqQuery.limit ? reqQuery.limit : 10,
  offset: reqQuery.offset ? reqQuery.offset : null,
  ...reqQuery, // Pass through additional query parameters
};

Notes (Engagements):

url = 'https://api.hubapi.com/engagements/v1/engagements/paged';
queryParam = {
  limit: reqQuery.limit ? reqQuery.limit : 10,
  offset: reqQuery.offset ? reqQuery.offset : null,
};

Validate Type
- Reject promise if getType is not one of the supported types

Call Hubspot API

const response = await axios.get(url, {
  headers: {
    Authorization: `Bearer ${accessToken}`,
    'Content-Type': 'application/json',
  },
  params: queryParam,
});

Normalize Pagination

Hubspot uses different field names for different endpoints:

Contacts: Uses vid-offset (visitor ID offset)
Companies, Deals, Notes: Use offset
All: Use has-more for more data indicator

const hasMore = data['has-more'] ? data['has-more'] : null;
let offset;
if (data['vid-offset']) {
  offset = data['vid-offset'];
} else if (data['offset']) {
  offset = data['offset'];
} else {
  offset = null;
}

const pagination = {
  hasMore,
  offset,
};

Return Normalized Response
- Return raw data from Hubspot plus normalized pagination object

API Request Examples:

# Contacts
GET https://api.hubapi.com/contacts/v1/lists/all/contacts/all?count=100&vidOffset=0
Authorization: Bearer CJzg_asd9fa8s7df98a7sdf...
Content-Type: application/json

# Companies
GET https://api.hubapi.com/companies/v2/companies/paged?properties=name&properties=website&limit=50&offset=0
Authorization: Bearer CJzg_asd9fa8s7df98a7sdf...
Content-Type: application/json

# Deals
GET https://api.hubapi.com/deals/v1/deal/paged?properties=dealname&includeAssociations=true&limit=200&offset=400
Authorization: Bearer CJzg_asd9fa8s7df98a7sdf...
Content-Type: application/json

# Notes (Engagements)
GET https://api.hubapi.com/engagements/v1/engagements/paged?limit=100&offset=0
Authorization: Bearer CJzg_asd9fa8s7df98a7sdf...
Content-Type: application/json

Hubspot API Response (Contacts):

{
  "contacts": [
    {
      "vid": 123456,
      "canonical-vid": 123456,
      "portal-id": 62515,
      "is-contact": true,
      "properties": {
        "firstname": { "value": "John" },
        "lastname": { "value": "Doe" },
        "email": { "value": "john.doe@example.com" }
      }
    }
  ],
  "has-more": true,
  "vid-offset": 123456
}

Hubspot API Response (Companies):

{
  "companies": [
    {
      "portalId": 62515,
      "companyId": 123456789,
      "properties": {
        "name": { "value": "Example Corp" },
        "website": { "value": "https://example.com" }
      }
    }
  ],
  "has-more": true,
  "offset": 123456789
}

Hubspot API Response (Deals):

{
  "deals": [
    {
      "portalId": 62515,
      "dealId": 987654321,
      "properties": {
        "dealname": { "value": "Q1 2024 Contract" },
        "amount": { "value": "50000" },
        "dealstage": { "value": "closedwon" }
      },
      "associations": {
        "associatedVids": [123456],
        "associatedCompanyIds": [123456789]
      }
    }
  ],
  "has-more": true,
  "offset": 987654321
}

Hubspot API Response (Notes/Engagements):

{
  "results": [
    {
      "engagement": {
        "id": 555555555,
        "portalId": 62515,
        "type": "NOTE",
        "timestamp": 1704844800000
      },
      "metadata": {
        "body": "Follow-up call scheduled for next week"
      },
      "associations": {
        "contactIds": [123456],
        "companyIds": [123456789]
      }
    }
  ],
  "has-more": true,
  "offset": 555555555
}

Error Handling:

// Unsupported type
{
  message: 'Given type is not supported. supported types are companies, contacts, deals, notes'
}

// API errors passed through
{
  response: {
    data: {
      status: "error",
      message: "Invalid access token",
      correlationId: "abc-123"
    }
  }
}

🔀 Route Configuration

Source: Routes/exportRoutes.js

router.get(
  '/:type',
  verifyScope(['contacts', 'contacts.external']),
  verifyAccessAndStatus({ accountIDRequired: true }),
  contactController.readContactHubspot,
);

// 404 handler for invalid routes
router.all('/*', (req, res) => {
  res.status(404).json({
    success: false,
    errno: 404,
    message: 'RESOURCE_NOT_FOUND',
  });
});

Dynamic Type Parameter:

The :type parameter allows a single route to handle multiple CRM data types:

/export/contacts → Returns contacts
/export/companies → Returns companies
/export/deals → Returns deals
/export/notes → Returns engagements

📊 Data Type Details

Contacts Export

Endpoint: GET /v1/integrations/hubspot/export/contacts

Hubspot API: GET /contacts/v1/lists/all/contacts/all

Query Parameters:

count - Number of contacts (default: 10)
vidOffset - Visitor ID offset for pagination

Response Fields:

contacts[] - Array of contact objects
vid-offset - Next page offset (visitor ID)
has-more - Boolean indicating more data

Contact Object Structure:

{
  "vid": 123456,                    // Visitor ID (contact ID)
  "canonical-vid": 123456,          // Canonical visitor ID
  "portal-id": 62515,               // Hubspot portal ID
  "is-contact": true,
  "properties": {
    "firstname": { "value": "John" },
    "lastname": { "value": "Doe" },
    "email": { "value": "john.doe@example.com" },
    "phone": { "value": "+1234567890" },
    "company": { "value": "Example Corp" }
  },
  "form-submissions": [],
  "list-memberships": [],
  "identity-profiles": []
}

Companies Export

Endpoint: GET /v1/integrations/hubspot/export/companies

Hubspot API: GET /companies/v2/companies/paged?properties=name&properties=website

Query Parameters:

limit - Number of companies (default: 10)
offset - Company ID offset for pagination

Response Fields:

companies[] - Array of company objects
offset - Next page offset (company ID)
has-more - Boolean indicating more data

Company Object Structure:

{
  "portalId": 62515,
  "companyId": 123456789,
  "isDeleted": false,
  "properties": {
    "name": {
      "value": "Example Corp",
      "timestamp": 1704844800000
    },
    "website": {
      "value": "https://example.com",
      "timestamp": 1704844800000
    }
  },
  "additionalDomains": [],
  "stateChanges": [],
  "mergeAudits": []
}

Custom Properties

The endpoint requests name and website properties. Additional properties can be requested by modifying HUBSPOT_COMPANIES_ENDPOINT environment variable.

Deals Export

Endpoint: GET /v1/integrations/hubspot/export/deals

Hubspot API: GET /deals/v1/deal/paged?properties=dealname&includeAssociations=true

Query Parameters:

limit - Number of deals (default: 10)
offset - Deal ID offset for pagination
Additional query parameters are passed through (e.g., properties, includeAssociations)

Response Fields:

deals[] - Array of deal objects
offset - Next page offset (deal ID)
has-more - Boolean indicating more data

Deal Object Structure:

{
  "portalId": 62515,
  "dealId": 987654321,
  "isDeleted": false,
  "properties": {
    "dealname": {
      "value": "Q1 2024 Contract",
      "timestamp": 1704844800000
    },
    "amount": {
      "value": "50000",
      "timestamp": 1704844800000
    },
    "dealstage": {
      "value": "closedwon",
      "timestamp": 1704844800000
    },
    "pipeline": {
      "value": "default",
      "timestamp": 1704844800000
    }
  },
  "associations": {
    "associatedVids": [123456],          // Associated contact IDs
    "associatedCompanyIds": [123456789], // Associated company IDs
    "associatedDealIds": []
  },
  "imports": [],
  "stateChanges": []
}

Deal Associations

includeAssociations=true parameter ensures related contacts and companies are included in the response.

Notes (Engagements) Export

Endpoint: GET /v1/integrations/hubspot/export/notes

Hubspot API: GET /engagements/v1/engagements/paged

Query Parameters:

limit - Number of engagements (default: 10)
offset - Engagement ID offset for pagination

Response Fields:

results[] - Array of engagement objects
offset - Next page offset (engagement ID)
has-more - Boolean indicating more data

Engagement Object Structure:

{
  "engagement": {
    "id": 555555555,
    "portalId": 62515,
    "active": true,
    "createdAt": 1704844800000,
    "lastUpdated": 1704844800000,
    "createdBy": 123456,
    "modifiedBy": 123456,
    "ownerId": 123456,
    "type": "NOTE",
    "timestamp": 1704844800000
  },
  "associations": {
    "contactIds": [123456],
    "companyIds": [123456789],
    "dealIds": [987654321],
    "ownerIds": [123456],
    "ticketIds": []
  },
  "attachments": [],
  "metadata": {
    "body": "Follow-up call scheduled for next week"
  }
}

Engagement Types

While this endpoint is used for "notes", Hubspot engagements include other types: NOTE, EMAIL, CALL, MEETING, TASK. Filter by type if needed.

🧪 Edge Cases & Special Handling

Different Pagination Fields

Issue: Hubspot uses inconsistent pagination field names across endpoints

Handling: Provider normalizes to consistent pagination.offset field:

// Contacts use vid-offset
if (data['vid-offset']) {
  offset = data['vid-offset'];
}
// Others use offset
else if (data['offset']) {
  offset = data['offset'];
}

Default Limit Values

Issue: Users may not provide limit parameter

Handling: Default to 10 records for all endpoints:

count: reqQuery.limit ? reqQuery.limit : 10;

Token Expiration During Request

Issue: Access token may expire between storage check and API call

Handling: Token is refreshed automatically before every data request:

accessToken = await providers.getRefreshAccessToken(query);

Unsupported Type

Issue: User may request invalid data type

Handling: Promise rejection with clear error message:

Promise.reject({
  message: 'Given type is not supported. supported types are companies, contacts, deals, notes',
});

Additional Query Parameters for Deals

Issue: Deals may need custom properties or filters

Handling: Spread operator passes through all query parameters:

queryParam = {
  limit: reqQuery.limit ? reqQuery.limit : 10,
  offset: reqQuery.offset ? reqQuery.offset : null,
  ...reqQuery, // Include all additional parameters
};

⚠️ Important Notes

🔄 Automatic Token Refresh: Access tokens are refreshed on every request, ensuring fresh credentials
📊 Pagination Normalization: Different Hubspot endpoints use different pagination fields; provider normalizes to consistent format
🔢 Default Limits: All endpoints default to 10 records if limit not specified
🔗 Deal Associations: Deals include associated contacts and companies via includeAssociations=true
📝 Custom Properties: Companies and deals request specific properties in URL; modify environment variables to request additional fields
🚨 Error Propagation: Hubspot API errors are caught and passed to Express error middleware
🎯 Type Validation: Invalid types return helpful error message listing supported types
📦 Raw Data: Provider returns raw Hubspot API responses without transformation

Integration Overview: Hubspot Index
Authentication: OAuth 2.0 Flow
Hubspot Contacts API: Contacts API Reference
Hubspot Companies API: Companies API Reference
Hubspot Deals API: Deals API Reference
Hubspot Engagements API: Engagements API Reference

🚀 Quick Start Examples

Export First 100 Contacts

curl -X GET \
  'https://api.dashclicks.com/v1/integrations/hubspot/export/contacts?limit=100&offset=0' \
  -H 'Authorization: Bearer {jwt_token}'

Paginate Through All Companies

let offset = 0;
let hasMore = true;

while (hasMore) {
  const response = await fetch(
    `/v1/integrations/hubspot/export/companies?limit=250&offset=${offset}`,
    {
      headers: { Authorization: `Bearer ${jwt}` },
    },
  );

  const result = await response.json();
  processCompanies(result.data.companies);

  hasMore = result.pagination.hasMore;
  offset = result.pagination.offset;
}

Export Deals with Custom Filters

curl -X GET \
  'https://api.dashclicks.com/v1/integrations/hubspot/export/deals?limit=50&properties=dealname&properties=amount&properties=closedate&includeAssociations=true' \
  -H 'Authorization: Bearer {jwt_token}'

Fetch Recent Engagement Notes

curl -X GET \
  'https://api.dashclicks.com/v1/integrations/hubspot/export/notes?limit=200' \
  -H 'Authorization: Bearer {jwt_token}'

📖 Overview​

🗄️ Collections Used​

integrations.hubspot.key​

🔄 Data Export Flow​

🔧 Controller Functions​

readContactHubspot(req, res, next)​

🔧 CRM Data Provider Functions​

retrieveDATA(accessToken, getType, reqQuery)​

🔀 Route Configuration​

📊 Data Type Details​

Contacts Export​

Companies Export​

Deals Export​

Notes (Engagements) Export​

🧪 Edge Cases & Special Handling​

Different Pagination Fields​

Default Limit Values​

Token Expiration During Request​

Unsupported Type​

Additional Query Parameters for Deals​

⚠️ Important Notes​

🔗 Related Documentation​

🚀 Quick Start Examples​

Export First 100 Contacts​

Paginate Through All Companies​

Export Deals with Custom Filters​

Fetch Recent Engagement Notes​

Documentation Assistant

📖 Overview

🗄️ Collections Used

`integrations.hubspot.key`

🔄 Data Export Flow

🔧 Controller Functions

`readContactHubspot(req, res, next)`

🔧 CRM Data Provider Functions

`retrieveDATA(accessToken, getType, reqQuery)`

🔀 Route Configuration

📊 Data Type Details

Contacts Export

Companies Export

Deals Export

Notes (Engagements) Export

🧪 Edge Cases & Special Handling

Different Pagination Fields

Default Limit Values

Token Expiration During Request

Unsupported Type

Additional Query Parameters for Deals

⚠️ Important Notes

🔗 Related Documentation

🚀 Quick Start Examples

Export First 100 Contacts

Paginate Through All Companies

Export Deals with Custom Filters

Fetch Recent Engagement Notes