Zoho - CRM Data Export

📖 Overview

Zoho CRM data export enables retrieval of contacts, accounts, deals, and notes from Zoho CRM with automatic token refresh and pagination support. Each API call automatically obtains a fresh access token using the stored refresh token to ensure uninterrupted data access.

Source Files:

• Provider: external/Integrations/Zoho/Providers/crmData/index.js
• Controller: external/Integrations/Zoho/Controller/contactController/index.js
• Routes: external/Integrations/Zoho/Routes/contactRoutes/index.js

External API: Zoho CRM API v2 (https://www.zohoapis.{domain}/crm/v2/)

🗄️ Collections Used

integrations.zoho.key

• Operations: Read
• Model: shared/models/zoho-key.js
• Usage Context: Fetch stored OAuth tokens (refresh_token, api_domain) for API authentication

Token Document Structure:

{
  "_id": ObjectId,
  "token": {
    "refresh_token": "1000.50b9048...", // Used to get new access_token
    "api_domain": "https://www.zohoapis.in" // Data center specific domain
  },
  "account_id": "12345",
  "owner": "user_Lwh9EzeD8"
}

🔄 Data Flow

CRM Data Export Flow

sequenceDiagram
    participant Client as API Consumer
    participant Controller as Contact Controller
    participant AuthProvider as Auth Provider
    participant CRMProvider as CRM Data Provider
    participant DB as MongoDB (zoho-key)
    participant ZohoAuth as Zoho OAuth API
    participant ZohoCRM as Zoho CRM API

    Client->>Controller: GET /v1/e/zoho/export/:type?page=1&limit=10
    Controller->>Controller: Validate type parameter
    Controller->>DB: Find token by account_id & owner
    DB-->>Controller: Return token with refresh_token

    Controller->>AuthProvider: getAccessToken(refresh_token)
    AuthProvider->>ZohoAuth: POST /oauth/v2/token
    ZohoAuth-->>AuthProvider: New access_token
    AuthProvider-->>Controller: Return access_token

    Controller->>CRMProvider: crmDATA(access_token, api_domain, type, query)
    CRMProvider->>ZohoCRM: GET /crm/v2/{type}?page=1&per_page=10
    ZohoCRM-->>CRMProvider: CRM data + pagination info
    CRMProvider->>CRMProvider: Transform response
    CRMProvider-->>Controller: Formatted data

    Controller-->>Client: JSON response with data & pagination

🔧 Business Logic & Functions

---

CRM Provider Functions

crmDATA(access_token, api_domain, type, queryParams)

Purpose: Fetch CRM data from Zoho API with pagination

Source: Providers/crmData/index.js

External API Endpoint: GET https://www.zohoapis.{domain}/crm/v2/{type}

Parameters:

• access_token (String) - Fresh access token from token refresh
• api_domain (String) - Zoho API domain (e.g., https://www.zohoapis.in)
• type (String) - CRM module type: Contacts, Accounts, Deals, Notes
• queryParams (Object) - Pagination parameters
  • page (Number) - Page number (default: 1)
  • per_page (Number) - Records per page (default: 10)

Returns: Promise<Object> - Formatted CRM data with pagination

{
  "data": [...],        // Array of CRM records
  "pagination": {       // Pagination metadata
    "page": 1,
    "per_page": 10,
    "more_records": true,
    "sort_by": "id",
    "sort_order": "asc"
  }
}

Business Logic Flow:

1. Configure Request Headers

   • Set Content-Type: application/json
   • Set Authorization: Bearer {access_token}

2. Build API URL

   • Construct URL: {api_domain}/crm/v2/{type}
   • Example: https://www.zohoapis.in/crm/v2/Contacts

3. Add Query Parameters

   • Include pagination params (page, per_page)

4. Execute API Request

   • Send GET request to Zoho CRM API

5. Transform Response

   • Extract data array from response
   • Extract info object as pagination metadata
   • Rename info to pagination

6. Return Formatted Data

   • Return object with data and pagination properties

API Request Example:

GET https://www.zohoapis.in/crm/v2/Contacts?page=1&per_page=10
Headers:
  Authorization: Bearer 1000.2cf2aef25f4404a7...
  Content-Type: application/json

API Response Example (from Zoho):

{
  "data": [
    {
      "id": "3652397000000624013",
      "Owner": { "name": "John Doe", "id": "3652397000000186017" },
      "Email": "contact@example.com",
      "First_Name": "Jane",
      "Last_Name": "Smith",
      "Phone": "+1234567890",
      "Created_Time": "2023-10-01T12:00:00+00:00",
      "Modified_Time": "2023-10-05T14:30:00+00:00"
    }
  ],
  "info": {
    "page": 1,
    "per_page": 10,
    "count": 1,
    "more_records": true,
    "sort_by": "id",
    "sort_order": "asc"
  }
}

Transformed Response (returned by function):

{
  "data": [
    {
      "id": "3652397000000624013",
      "Owner": { "name": "John Doe", "id": "3652397000000186017" },
      "Email": "contact@example.com",
      "First_Name": "Jane",
      "Last_Name": "Smith",
      "Phone": "+1234567890",
      "Created_Time": "2023-10-01T12:00:00+00:00",
      "Modified_Time": "2023-10-05T14:30:00+00:00"
    }
  ],
  "pagination": {
    "page": 1,
    "per_page": 10,
    "count": 1,
    "more_records": true,
    "sort_by": "id",
    "sort_order": "asc"
  }
}

Error Handling:

• 401 Unauthorized: Invalid or expired access token
• 403 Forbidden: Insufficient permissions for CRM module
• 404 Not Found: Invalid module type
• 429 Rate Limit: Zoho rate limit exceeded
• Network Errors: Axios failures propagated to caller

Example Usage:

const data = await crmProvider.crmDATA('access_token_here', 'https://www.zohoapis.in', 'Contacts', {
  page: 1,
  per_page: 10,
});
// Returns: { data: [...], pagination: {...} }

Provider API Rate Limits:

• Zoho enforces ~100 API calls per minute per organization
• Rate limit headers included in response:
  • X-RateLimit-Limit: Maximum requests per minute
  • X-RateLimit-Remaining: Remaining requests
  • X-RateLimit-Reset: Time until rate limit reset

Side Effects:

• ⚠️ External API Call: Consumes Zoho CRM API quota
• ℹ️ No Database Writes: Read-only operation

---

Controller Functions

readContactZoho(req, res, next)

Purpose: Main controller for CRM data export with automatic token refresh

Source: Controller/contactController/index.js

External API Endpoint: Orchestrates calls to auth and CRM providers

Parameters:

• req.params.type (String) - Export type: contacts, accounts, deals, notes
• req.query.page (Number, optional) - Page number (default: 1)
• req.query.limit (Number, optional) - Records per page (default: 10)
• req.auth.account_id (String) - DashClicks account ID from JWT
• req.auth.uid (String) - DashClicks user ID from JWT

Returns: JSON response with CRM data and pagination

Business Logic Flow:

1. Map Type Parameter

   • Convert URL parameter to Zoho API format:
     • contacts → Contacts
     • accounts → Accounts
     • deals → Deals
     • notes → Notes

2. Fetch Stored Token

   • Query integrations.zoho.key by account_id and owner
   • Return 404 if no token found

3. Validate Type Parameter

   • Check if type is one of the supported values
   • Return 404 if invalid type

4. Refresh Access Token

   • Call authProvider.getAccessToken(refresh_token)
   • Obtain fresh access token for API call

5. Build Query Parameters

   • Set default pagination: page=1, per_page=10
   • Override with request query params if provided

6. Fetch CRM Data

   • Call crmProvider.crmDATA() with:
     • Fresh access token
     • API domain from stored token
     • Mapped type parameter
     • Query parameters

7. Format Response

   • Extract data and pagination from provider response
   • Return formatted JSON response

Type Parameter Mapping:

URL Parameter	Zoho API Module
contacts	Contacts
accounts	Accounts
deals	Deals
notes	Notes

Request Example:

GET /v1/e/zoho/export/contacts?page=1&limit=20
Authorization: Bearer {jwt_token}

Success Response:

{
  "success": true,
  "message": "Received Information",
  "data": [
    {
      "id": "3652397000000624013",
      "Email": "contact@example.com",
      "First_Name": "Jane",
      "Last_Name": "Smith",
      "Phone": "+1234567890",
      "Created_Time": "2023-10-01T12:00:00+00:00"
    }
  ],
  "pagination": {
    "page": 1,
    "per_page": 20,
    "count": 1,
    "more_records": true,
    "sort_by": "id",
    "sort_order": "asc"
  }
}

Error Response (No Token):

{
  "success": false,
  "errno": 400,
  "message": "Access Token is not Found"
}

Error Response (Invalid Type):

{
  "success": false,
  "errno": 400,
  "message": "Type is not Selected yet"
}

Error Handling:

• No Token Found: Return 404 with "Access Token is not Found"
• Invalid Type: Return 404 with "Type is not Selected yet"
• Zoho API Errors: Passed to error middleware via next(error)
• Token Refresh Errors: Propagated from auth provider
• Token Invalidation: Caught by main error handler, sets token_invalidated: true

Example Usage:

// Export contacts with pagination
const contacts = await fetch('/v1/e/zoho/export/contacts?page=1&limit=50', {
  headers: { Authorization: `Bearer ${token}` },
});

// Export deals
const deals = await fetch('/v1/e/zoho/export/deals?page=2&limit=25', {
  headers: { Authorization: `Bearer ${token}` },
});

Side Effects:

• ⚠️ Token Refresh: Generates new access token on every call
• ⚠️ External API Call: Fetches data from Zoho CRM
• ⚠️ Rate Limit Consumption: Counts toward Zoho API quota (2 calls: token + data)

---

🔀 Integration Points

Internal Services

Used By:

• DashClicks data import/sync workflows
• CRM integration dashboards
• Automated contact synchronization

Common Integration Pattern:

// Typical usage in internal services
async function syncZohoCRM(accountId, userId) {
  // 1. Fetch contacts
  const contacts = await fetch(`/v1/e/zoho/export/contacts?limit=100`, {
    headers: { Authorization: token },
  });

  // 2. Process and store in DashClicks
  for (const contact of contacts.data) {
    await saveContact(contact);
  }

  // 3. Handle pagination
  if (contacts.pagination.more_records) {
    // Fetch next page
  }
}

External API Dependencies

Provider: Zoho Corporation
Endpoints:

• OAuth Token Refresh: POST https://accounts.zoho.{domain}/oauth/v2/token
• CRM Data: GET https://www.zohoapis.{domain}/crm/v2/{module}

Authentication: Bearer token (auto-refreshed per request)
Rate Limits: ~100 API calls per minute per organization

Supported CRM Modules:

Module	Description
Contacts	Individual contact records
Accounts	Company/organization records
Deals	Sales opportunities and deals
Notes	Notes attached to CRM records

Data Structure Examples

Contacts Response

{
  "id": "3652397000000624013",
  "Owner": { "name": "John Doe", "id": "3652397000000186017" },
  "Email": "contact@example.com",
  "First_Name": "Jane",
  "Last_Name": "Smith",
  "Phone": "+1234567890",
  "Mobile": "+0987654321",
  "Account_Name": { "name": "Acme Corp", "id": "3652397000000624000" },
  "Lead_Source": "Web",
  "Created_Time": "2023-10-01T12:00:00+00:00",
  "Modified_Time": "2023-10-05T14:30:00+00:00"
}

Accounts Response

{
  "id": "3652397000000624000",
  "Owner": { "name": "John Doe", "id": "3652397000000186017" },
  "Account_Name": "Acme Corporation",
  "Account_Type": "Customer",
  "Phone": "+1234567890",
  "Website": "https://acme.com",
  "Industry": "Technology",
  "Annual_Revenue": 1000000,
  "Created_Time": "2023-09-15T10:00:00+00:00",
  "Modified_Time": "2023-10-01T11:00:00+00:00"
}

Deals Response

{
  "id": "3652397000000625000",
  "Owner": { "name": "Sales Rep", "id": "3652397000000186020" },
  "Deal_Name": "Q4 Enterprise Deal",
  "Account_Name": { "name": "Acme Corp", "id": "3652397000000624000" },
  "Amount": 50000,
  "Stage": "Negotiation/Review",
  "Closing_Date": "2023-12-31",
  "Probability": 75,
  "Created_Time": "2023-10-10T09:00:00+00:00",
  "Modified_Time": "2023-10-12T16:00:00+00:00"
}

Notes Response

{
  "id": "3652397000000626000",
  "Owner": { "name": "John Doe", "id": "3652397000000186017" },
  "Parent_Id": { "name": "Acme Corp", "id": "3652397000000624000" },
  "Note_Title": "Follow-up call",
  "Note_Content": "Discussed product requirements and timeline.",
  "Created_Time": "2023-10-05T14:00:00+00:00",
  "Modified_Time": "2023-10-05T14:00:00+00:00"
}

---

🧪 Edge Cases & Special Handling

Automatic Token Refresh

Issue: Access tokens expire after 1 hour

Handling:

• Every API call first refreshes access token
• Uses stored refresh_token to get new access_token
• No token expiration errors for end users

// Automatic refresh pattern
const storedToken = await findTokenQuery(account_id, owner);
const freshToken = await authProvider.getAccessToken(storedToken.token.refresh_token);
// Use freshToken.access_token for API call

Multi-Page Data Export

Issue: Large datasets require multiple API calls

Handling:

• Zoho returns more_records: true in pagination when more data available
• Client must make sequential requests with incremented page parameter
• Default per_page is 10, maximum is typically 200

Pagination Logic:

let allContacts = [];
let page = 1;
let hasMore = true;

while (hasMore) {
  const response = await fetch(`/v1/e/zoho/export/contacts?page=${page}&limit=100`);
  allContacts.push(...response.data);
  hasMore = response.pagination.more_records;
  page++;
}

Data Center Specific Domains

Issue: Zoho uses different API domains per data center

Handling:

• api_domain stored in token document during OAuth
• Examples:
  • https://www.zohoapis.com (US)
  • https://www.zohoapis.in (India)
  • https://www.zohoapis.eu (Europe)
  • https://www.zohoapis.com.au (Australia)
• CRM provider uses stored api_domain for API calls

Token Invalidation During Export

Issue: User might revoke access mid-export

Handling:

• Zoho returns error with code: 'INVALID_TOKEN'
• Error handler in index.js catches this error
• Updates all account tokens with token_invalidated: true
• Returns error message: TOKEN_INVALIDATED
• Client must restart OAuth flow

Empty Result Sets

Issue: CRM module might have no records

Handling:

• Zoho returns empty data array
• Pagination object still included
• Controller returns:

{
  "success": true,
  "message": "Received Information",
  "data": [],
  "pagination": { "page": 1, "per_page": 10, "count": 0, "more_records": false }
}

Invalid Module Types

Issue: User requests unsupported CRM module

Handling:

• Type mapping returns empty string for invalid types
• Controller returns 404: "Type is not Selected yet"
• Valid types: contacts, accounts, deals, notes

---

⚠️ Important Notes

• 🔄 Automatic Token Refresh: Every API call refreshes access token (2 API calls per request)
• 📊 Pagination Required: Large datasets require multiple requests with incremented page numbers
• 🌍 Data Center Aware: API domain varies per Zoho account's data center location
• 🚨 Rate Limit Impact: Token refresh + data fetch = 2 API calls toward rate limit
• 📝 Default Pagination: Default is 10 records per page, maximum typically 200
• 🔒 Token Validation: No pre-validation - token errors caught during API call
• ⏱️ Fresh Tokens: Access tokens generated on-demand, not cached
• 🎯 Type Case Sensitive: URL parameters lowercase (contacts), API requires PascalCase (Contacts)

---

🔗 Related Documentation

• Integration Overview: Zoho Integration
• Authentication: OAuth 2.0 Flow
• Zoho CRM API: Zoho CRM v2 Documentation
• Database Collections: MongoDB Collections
• Rate Limiting: Zoho API Limits