Pipedrive Integration

📋 Overview

Pipedrive integration providing OAuth 2.0 authentication and CRM data export capabilities. Supports exporting deals, contacts (persons), organizations, notes, pipelines, and stages with automatic token refresh and cursor-based pagination.

Provider: Pipedrive (https://www.pipedrive.com)
API Base URL: Dynamic per account (https://{company}-sandbox.pipedrive.com or https://{company}.pipedrive.com)
Integration Type: OAuth 2.0 with automatic token refresh
OAuth Scopes: base, deals:full, activities:full, contacts:full, products:full, users:read, recents:read, search:read

📚 Documentation Structure

This integration is organized into the following sections:

1. Authentication - OAuth 2.0 flow with automatic token refresh
2. CRM Data Export - Export deals, persons, organizations, notes, pipelines, stages

✨ Features

• ✅ OAuth 2.0: Secure authentication with Pipedrive accounts
• ✅ Automatic Token Refresh: Middleware refreshes tokens on every request (proactive approach)
• ✅ Dynamic API Domain: Each account has unique API domain (returned in OAuth response)
• ✅ CRM Data Export: Deals, persons, organizations, notes, pipelines, stages
• ✅ Cursor-based Pagination: Use start parameter with next_start value for pagination
• ✅ Pipeline Details: Get individual pipeline and stage information
• ✅ Token Invalidation Detection: Automatic detection of revoked tokens
• ✅ Scope Protection: Required scopes enforced on endpoints
• ✅ Sub-account Support: Works with DashClicks sub-accounts

🏗️ Architecture

Frontend Request
      ↓
OAuth Flow (Pipedrive consent)
      ↓
Token Storage (MongoDB)
      ↓
Middleware: Auto Token Refresh
      ↓
Pipedrive REST API (dynamic domain)
      ↓
CRM Data (deals, persons, etc.)

🗄️ MongoDB Collections

📖 Detailed Schema: See Database Collections Documentation

integrations.pipedrive.key

Purpose: Store OAuth 2.0 tokens for Pipedrive access

Schema: Flexible schema (strict: false) for storing token data

Key Fields:

• account_id (String, required) - DashClicks account reference
• owner (String, required) - DashClicks user ID (uid)
• token (Object, required) - Token information
  • access_token (String) - OAuth access token
  • refresh_token (String) - OAuth refresh token
  • expires_in (Number) - Token lifetime in seconds (3600 = 1 hour)
  • generated_at (Number) - Unix timestamp of token generation
  • token_type (String) - Token type ("Bearer")
  • scope (String) - Granted OAuth scopes
  • api_domain (String) - Account-specific API endpoint URL
• token_invalidated (Boolean) - Flag for invalidated tokens

Indexes:

• { account_id: 1, owner: 1 } (implied unique) - Primary lookup

Document Example:

{
  _id: ObjectId("507f1f77bcf86cd799439011"),
  account_id: "507f191e810c19729de860ea",
  owner: "user_Lwh9EzeD8",
  token: {
    access_token: "7507356:11465942:72cdfd552a1c4c2659fd8395aaf0da3e14934874",
    refresh_token: "7507356:11465942:cf3d769527455ee0beb3dd3fcf68276a45039570",
    expires_in: 3600,
    generated_at: 1728547200,
    token_type: "Bearer",
    scope: "base,deals:full,activities:full,contacts:full,products:full,users:read,recents:read,search:read",
    api_domain: "https://dashclicksllc.pipedrive.com"
  },
  token_invalidated: false
}

📁 Directory Structure

Source Code Location:

external/Integrations/Pipedrive/
├── Controller/
│   ├── authController/     # OAuth authentication handlers
│   │   └── index.js
│   └── contactController/  # CRM data export controllers
│       └── index.js
├── Middleware/
│   └── getToken.js         # Automatic token refresh middleware
├── Model/
│   └── PipedriveCollection.js # Database operations wrapper
├── Providers/
│   └── contactProvider/    # Pipedrive API wrapper functions
│       └── index.js
├── Routes/
│   ├── authRoutes/         # Auth endpoints
│   │   └── index.js
│   └── exportRoutes/       # CRM data endpoints
│       └── index.js
└── index.js                # Route registration and error handling

Shared Models Used:

• shared/models/pipedrive-key.js
• shared/models/account.js
• shared/models/user.js

🔌 API Endpoints Summary

Method	Endpoint	Description
GET	/auth/login	Initiate OAuth 2.0 flow
GET	/auth/callback	Handle OAuth callback
DELETE	/auth	Disconnect integration (delete tokens)
GET	/export/:type	Export CRM data (persons, organizations, deals, notes, pipelines, stages)
GET	/export/pipelines/:pipelineid	Get specific pipeline details
GET	/export/stages/:stageid	Get specific stage details

⚙️ Environment Variables

# OAuth credentials
PIPE_DRIVE_CLIENT_ID=your_pipedrive_client_id
PIPE_DRIVE_CLIENT_SECRET=your_pipedrive_client_secret
PIPE_DRIVE_REDIRECT_URL=https://api.dashclicks.com/v1/e/pipedrive/auth/callback

# JWT secret for state parameter
APP_SECRET=your_app_secret

🚀 Quick Start

1. Configure Environment Variables

Set up the required Pipedrive OAuth credentials in your .env file.

2. Initiate OAuth Flow

GET /v1/e/pipedrive/auth/login?forward_url=https://app.dashclicks.com/integrations
Authorization: Bearer {jwt_token}

Response: Redirects to Pipedrive OAuth consent screen

3. Handle Callback

After user grants permission, Pipedrive redirects to the callback URL, which stores the tokens and redirects to your forward_url with status.

# Success redirect
https://app.dashclicks.com/integrations?status=success&integration=pipedrive

# Error redirect
https://app.dashclicks.com/integrations?status=error&integration=pipedrive&reason={error_message}

4. Export CRM Data

GET /v1/e/pipedrive/export/deals?limit=100
Authorization: Bearer {jwt_token}

Response:

{
  "success": true,
  "message": "SUCCESS",
  "data": [
    {
      "id": 12345,
      "title": "Enterprise Deal",
      "value": 50000,
      "currency": "USD",
      "status": "open",
      "stage_id": 5,
      "person_id": 678,
      "org_id": 910
    }
  ],
  "pagination": {
    "start": 0,
    "limit": 100,
    "more_items_in_collection": true,
    "next_start": 100
  }
}

🔄 Token Refresh Mechanism

Pipedrive integration uses a proactive token refresh approach (similar to Keap):

Refresh on Every Request

The getToken middleware refreshes the access token on every API request:

const getToken = async (req, res, next) => {
  const account_id = req.auth.account_id;
  const uid = req.auth.uid;

  // 1. Retrieve stored token
  const snapshot = await PipeDriveCollection.getData(account_id, uid);

  if (!snapshot) {
    return res.status(401).json({
      message: 'User oauth token not found. Please redirect to login',
    });
  }

  const doc = snapshot._doc;

  // 2. Always refresh token (commented out expiration check)
  try {
    const url = 'https://oauth.pipedrive.com/oauth/token';

    const requestData = qs.stringify({
      grant_type: 'refresh_token',
      refresh_token: doc.token.refresh_token,
    });

    // 3. Make refresh request with Basic Auth
    const base64Credentials = Buffer.from(
      `${process.env.PIPE_DRIVE_CLIENT_ID}:${process.env.PIPE_DRIVE_CLIENT_SECRET}`,
    ).toString('base64');

    const response = await axios.post(url, requestData, {
      headers: {
        Authorization: `Basic ${base64Credentials}`,
        'Content-Type': 'application/x-www-form-urlencoded',
      },
    });

    // 4. Update stored token
    await PipeDriveCollection.updateData(doc._id, {
      token: {
        generated_at: moment().unix(),
        access_token: response.data.access_token,
        refresh_token: response.data.refresh_token,
        expires_in: response.data.expires_in,
        api_domain: response.data.api_domain,
      },
    });

    // 5. Attach fresh token and API domain to request
    req.access_token = response.data.access_token;
    req.api_domain = response.data.api_domain;
    next();
  } catch (error) {
    next(error);
  }
};

Why refresh on every request?

• Ensures tokens are always fresh
• Avoids expiration timing issues
• Guarantees correct api_domain for each request

🌐 Dynamic API Domain

Each Pipedrive account has a unique API domain:

• Sandbox: https://{company}-sandbox.pipedrive.com
• Production: https://{company}.pipedrive.com

The api_domain is returned in the OAuth token response and must be used for all API requests:

// Wrong: Hard-coded domain
const url = 'https://api.pipedrive.com/v1/deals';

// Correct: Dynamic domain from token
const url = `${req.api_domain}/v1/deals`;

📊 Export Types

The integration supports 6 export types via /export/:type:

Type	Description	Pipedrive API Endpoint
persons	Contact records	/v1/persons
organizations	Company records	/v1/organizations
deals	Deals/opportunities	/v1/deals
notes	Notes attached to records	/v1/notes
pipelines	Sales pipelines	/v1/pipelines
stages	Pipeline stages	/v1/stages

🔄 Pagination

Pipedrive uses cursor-based pagination with start and next_start:

// First request (no start parameter)
GET /v1/e/pipedrive/export/deals?limit=100

// Response includes next_start
{
  "data": [...],
  "pagination": {
    "start": 0,
    "limit": 100,
    "more_items_in_collection": true,
    "next_start": 100  // Use this for next request
  }
}

// Next request (use next_start value)
GET /v1/e/pipedrive/export/deals?limit=100&start=100

🎯 OAuth Scopes

The integration requests comprehensive scopes:

Scope	Permission
base	Basic account information
deals:full	Full access to deals
activities:full	Full access to activities
contacts:full	Full access to contacts (persons)
products:full	Full access to products
users:read	Read user information
recents:read	Read recent items
search:read	Search functionality

⚠️ Error Handling

Token Invalidation Detection

The integration detects invalidated tokens from Pipedrive API error responses:

// Error response structure
{
  "success": false,
  "error": "Unauthorized",
  "errorCode": "401"
}

// Error handler in index.js
if (error.response.data && error.response.data.errorCode === '401') {
  // Mark all tokens for this account as invalidated
  await pipedriveKeys.updateMany(
    { account_id: accountId.toString() },
    { $set: { token_invalidated: true } }
  );

  error.message = 'TOKEN_INVALIDATED';
}

When Tokens Become Invalid

1. User revokes access in Pipedrive account settings
2. User changes password (invalidates all tokens)
3. App permissions changed by admin
4. Account suspended or deactivated

💡 Key Differences from Other Integrations

1. Dynamic API Domain

Unlike most integrations with fixed API endpoints, Pipedrive provides a unique domain per account in the OAuth response.

2. Proactive Token Refresh

Always refreshes tokens on every request (same approach as Keap), not checking expiration first.

3. Cursor-based Pagination

Uses start and next_start parameters (not page numbers or offsets).

4. Basic Auth for Token Refresh

Token refresh requests use Basic Authentication with base64-encoded client credentials.

5. Comprehensive Scopes

Requests full access to multiple modules (deals, contacts, activities, products).

🔗 Related Documentation

• Pipedrive API Documentation
• Pipedrive OAuth Guide
• Authentication Documentation
• CRM Data Export Documentation

🛠️ Use Cases

1. Export All Deals

// Fetch all deals with pagination
async function getAllDeals(jwtToken) {
  let allDeals = [];
  let start = 0;
  let hasMoreItems = true;

  while (hasMoreItems) {
    const response = await axios.get(
      `/v1/e/pipedrive/export/deals?limit=100${start ? `&start=${start}` : ''}`,
      { headers: { Authorization: `Bearer ${jwtToken}` } },
    );

    allDeals = [...allDeals, ...response.data.data];

    if (response.data.pagination.more_items_in_collection) {
      start = response.data.pagination.next_start;
    } else {
      hasMoreItems = false;
    }
  }

  return allDeals;
}

2. Get Pipeline Structure

// Retrieve pipeline with all stages
async function getPipelineStructure(pipelineId, jwtToken) {
  const pipeline = await axios.get(`/v1/e/pipedrive/export/pipelines/${pipelineId}`, {
    headers: { Authorization: `Bearer ${jwtToken}` },
  });

  return pipeline.data;
}

3. Sync CRM Data to Database

// Sync persons, organizations, and deals
async function syncPipedriveData(accountId, jwtToken) {
  const types = ['persons', 'organizations', 'deals'];

  for (const type of types) {
    let start = 0;
    let hasMoreItems = true;

    while (hasMoreItems) {
      const response = await axios.get(
        `/v1/e/pipedrive/export/${type}?limit=200${start ? `&start=${start}` : ''}`,
        { headers: { Authorization: `Bearer ${jwtToken}` } },
      );

      // Store in database
      await CRMDataModel.insertMany(
        response.data.data.map(item => ({
          accountId,
          type,
          pipedriveId: item.id,
          data: item,
          lastSynced: new Date(),
        })),
      );

      if (response.data.pagination.more_items_in_collection) {
        start = response.data.pagination.next_start;
      } else {
        hasMoreItems = false;
      }
    }
  }
}

📝 Best Practices

1. Always Use Dynamic API Domain

// ✅ Good: Use api_domain from middleware
const url = `${req.api_domain}/v1/deals`;

// ❌ Bad: Hard-coded domain
const url = 'https://api.pipedrive.com/v1/deals';

2. Handle Pagination Correctly

// Check more_items_in_collection flag
if (pagination.more_items_in_collection) {
  nextStart = pagination.next_start;
  // Fetch next page
}

3. Implement Error Handling

try {
  const response = await axios.get(url);
} catch (error) {
  if (error.response?.data?.message === 'TOKEN_INVALIDATED') {
    // Redirect to re-authenticate
  }
}

4. Use Appropriate Limit Values

• Recommended: 100-200 records per page
• Maximum: Check Pipedrive API documentation

5. Cache API Domain

Store the api_domain from the token to avoid unnecessary lookups.