Keap Integration

📋 Overview

Keap (formerly Infusionsoft) integration providing OAuth 2.0 authentication and CRM data export capabilities. Supports exporting contacts, companies, opportunities (deals), and notes with automatic token refresh and pagination.

Provider: Keap (https://keap.com)
API Base URL: https://api.infusionsoft.com/crm/rest/v1
Integration Type: OAuth 2.0 with automatic token refresh
OAuth Scope: full (complete access to Keap account)

📚 Documentation Structure

This integration is organized into the following sections:

1. Authentication - OAuth 2.0 flow with automatic token refresh
2. CRM Data Export - Export contacts, companies, opportunities, and notes

✨ Features

• ✅ OAuth 2.0: Secure authentication with Keap accounts
• ✅ Automatic Token Refresh: Middleware refreshes tokens on every request
• ✅ CRM Data Export: Contacts, companies, opportunities, notes
• ✅ Pagination Support: Offset-based pagination for large datasets
• ✅ Token Invalidation Detection: Automatic detection of revoked tokens
• ✅ Scope Protection: Required scopes enforced on endpoints
• ✅ Sub-account Support: Works with DashClicks sub-accounts

🏗️ Architecture

Frontend Request
      ↓
OAuth Flow (Keap consent)
      ↓
Token Storage (MongoDB)
      ↓
Middleware: Auto Token Refresh
      ↓
Keap REST API
      ↓
CRM Data (contacts, companies, etc.)

🗄️ MongoDB Collections

📖 Detailed Schema: See Database Collections Documentation

integrations.keap.key

Purpose: Store OAuth 2.0 tokens for Keap access

Schema: Flexible schema (strict: false) for storing token data

Key Fields:

• account_id (String, required) - DashClicks account reference
• owner (String, required) - DashClicks user ID (uid)
• token (Object, required) - Token information
  • access_token (String) - OAuth access token
  • refresh_token (String) - OAuth refresh token
  • expires_in (Number) - Token lifetime in seconds (86400 = 24 hours)
  • generated_at (Number) - Unix timestamp of token generation
  • scope (String) - Granted OAuth scope
  • token_type (String) - Token type ("bearer")
• token_invalidated (Boolean) - Flag for invalidated tokens

Indexes:

• { account_id: 1, owner: 1 } (implied unique) - Primary lookup

Document Example:

{
  _id: ObjectId("507f1f77bcf86cd799439011"),
  account_id: "507f191e810c19729de860ea",
  owner: "user_Lwh9EzeD8",
  token: {
    access_token: "eHbSz8f2aNgBMmSMCKflyvCDFxpF",
    refresh_token: "inxRTuhyitSHFnrzx9yVKyBbIHIGdFdi",
    expires_in: 86400,
    generated_at: 1728547200,
    scope: "full|rm844.infusionsoft.com",
    token_type: "bearer"
  },
  token_invalidated: false
}

📁 Directory Structure

Source Code Location:

external/Integrations/Keap/
├── Controllers/
│   ├── auth.js          # OAuth authentication handlers
│   └── keap.js          # CRM data export controllers
├── Middleware/
│   └── getToken.js      # Automatic token refresh middleware
├── Models/
│   └── KeapCollection.js # Database operations wrapper
├── Providers/
│   └── keap.js          # Keap API wrapper functions
├── Routes/
│   ├── auth.js          # Auth endpoints
│   └── keap.js          # CRM data endpoints
└── index.js             # Route registration and error handling

Shared Models Used:

• shared/models/keap-key.js
• shared/models/account.js
• shared/models/user.js

🔌 API Endpoints Summary

Method	Endpoint	Description
GET	/auth/login	Initiate OAuth 2.0 flow
GET	/auth/callback	Handle OAuth callback
DELETE	/auth	Disconnect integration (delete tokens)
GET	/export/:type	Export CRM data (contacts, companies, opportunities, notes)

⚙️ Environment Variables

# OAuth credentials
KEAP_CLIENT_ID=your_keap_client_id
KEAP_CLIENT_SECRET=your_keap_client_secret
KEAP_REDIRECT_URL=https://api.dashclicks.com/v1/e/keap/auth/callback

# API configuration
KEAP_BASE_URL=https://api.infusionsoft.com/crm/rest/v1

# JWT secret for state parameter
APP_SECRET=your_app_secret

🚀 Quick Start

1. Configure Environment Variables

Set up the required Keap OAuth credentials in your .env file.

2. Initiate OAuth Flow

GET /v1/e/keap/auth/login?forward_url=https://app.dashclicks.com/integrations
Authorization: Bearer {jwt_token}

Response: Redirects to Keap OAuth consent screen

3. Handle Callback

After user grants permission, Keap redirects to the callback URL, which stores the tokens and redirects to your forward_url with status.

# Success redirect
https://app.dashclicks.com/integrations?status=success&integration=keap&token={token_id}

# Error redirect
https://app.dashclicks.com/integrations?status=error&integration=keap&reason={error_message}

4. Export CRM Data

GET /v1/e/keap/export/contacts?limit=100&page=1
Authorization: Bearer {jwt_token}

Response:

{
  "success": true,
  "message": "Data exported",
  "data": {
    "contacts": [
      {
        "id": 12345,
        "given_name": "John",
        "family_name": "Doe",
        "email_addresses": [
          {
            "email": "john.doe@example.com",
            "field": "EMAIL1"
          }
        ]
      }
    ]
  },
  "pagination": {
    "page": 1,
    "limit": 100,
    "total": 5,
    "next_page": null,
    "pre_page": null
  }
}

🔄 Token Refresh Mechanism

Keap integration uses a proactive token refresh approach via middleware:

Refresh on Every Request

The getToken middleware refreshes the access token on every API request, regardless of expiration:

// Middleware flow
const getToken = async (req, res, next) => {
  const account_id = req.auth.account_id;
  const owner = req.auth.uid;

  // 1. Retrieve stored token
  const doc = await KeapCollection.getData(account_id, owner);

  if (!doc) {
    return res.status(401).json({
      message: 'User oauth token not found. Please redirect to login',
    });
  }

  // 2. Always refresh token (commented out expiration check)
  const url = 'https://api.infusionsoft.com/token';
  const requestData = {
    grant_type: 'refresh_token',
    refresh_token: doc.token.refresh_token,
  };

  // 3. Request new token with Basic Auth
  const response = await axios.post(url, requestData, {
    headers: {
      Authorization: `Basic ${base64ClientCredentials}`,
      'Content-Type': 'application/x-www-form-urlencoded',
    },
  });

  // 4. Update stored token
  await KeapCollection.updateData(doc._id, {
    token: {
      generated_at: moment().unix(),
      access_token: response.data.access_token,
      refresh_token: response.data.refresh_token,
      expires_in: response.data.expires_in,
    },
  });

  // 5. Attach fresh token to request
  req.access_token = response.data.access_token;
  next();
};

Why refresh on every request?

• Ensures tokens are always fresh
• Avoids expiration timing issues
• Simplifies token management logic

🔐 OAuth Flow Details

Authorization URL

https://signin.infusionsoft.com/app/oauth/authorize?
  response_type=code&
  state={jwt_state_token}&
  client_id={KEAP_CLIENT_ID}&
  redirect_uri={KEAP_REDIRECT_URL}&
  scope=full

State Parameter

The state parameter contains encrypted account context:

const state = {
  account_id: req.auth.account_id.toString(),
  owner: req.auth.uid.toString(),
  forward_url: req.query.forward_url,
};

const stateToken = jwt.sign(state, process.env.APP_SECRET, {
  expiresIn: '1h',
});

Token Exchange

// POST https://api.infusionsoft.com/token
{
  client_id: KEAP_CLIENT_ID,
  client_secret: KEAP_CLIENT_SECRET,
  code: authorization_code,
  grant_type: 'authorization_code',
  redirect_uri: KEAP_REDIRECT_URL
}

Response:

{
  "access_token": "eHbSz8f2aNgBMmSMCKflyvCDFxpF",
  "refresh_token": "inxRTuhyitSHFnrzx9yVKyBbIHIGdFdi",
  "expires_in": 86400,
  "token_type": "bearer",
  "scope": "full|rm844.infusionsoft.com"
}

🎯 Export Types

The integration supports 4 export types via /export/:type:

Type	Description	Keap API Endpoint
contacts	Contact records	/contacts
companies	Company records	/companies
opportunities	Deals/opportunities	/opportunities
notes	Notes attached to records	/notes

📊 Pagination

The integration uses offset-based pagination:

// Request parameters
const limit = parseInt(req.query.limit) || 10;
const page = parseInt(req.query.page) || 1;
const offset = (page - 1) * limit;

// Query parameters sent to Keap API
{
  limit: limit,    // Number of records per page
  offset: offset   // Starting position (0-indexed)
}

Response format:

{
  "pagination": {
    "page": 2,
    "limit": 100,
    "total": 5, // Total pages
    "next_page": 3, // Next page number (null if last)
    "pre_page": 1 // Previous page number (null if first)
  }
}

⚠️ Error Handling

Token Invalidation Detection

The integration detects invalidated tokens from Keap API error responses:

// Error response from Keap
{
  "fault": {
    "faultstring": "Invalid Access Token",
    "detail": {
      "errorcode": "keymanagement.service.invalid_access_token"
    }
  }
}

// Handler marks token as invalidated
if (error?.response?.data?.fault?.detail?.errorcode ===
    'keymanagement.service.invalid_access_token') {
  await keapKeys.updateMany(
    { account_id: accountId },
    { $set: { token_invalidated: true } }
  );
  error.message = 'TOKEN_INVALIDATED';
}

Token Invalidation Flag

When a token is detected as invalid:

1. token_invalidated flag is set to true in database
2. Error message TOKEN_INVALIDATED is returned
3. Next login attempt deletes the invalidated token
4. User must re-authenticate

🔒 Scope Protection

The integration uses scope-based access control:

// Required scopes for CRM operations
verifyScope(['contacts', 'contacts.external']);

Scopes:

• contacts - Access to contacts module
• contacts.external - External contacts access

🛠️ Use Cases

1. Export All Contacts

// Fetch first page of contacts
const response = await axios.get('/v1/e/keap/export/contacts?limit=100&page=1', {
  headers: { Authorization: `Bearer ${jwt_token}` },
});

const contacts = response.data.data.contacts;
const totalPages = response.data.pagination.total;

console.log(`Found ${contacts.length} contacts on page 1 of ${totalPages}`);

2. Paginate Through All Records

// Fetch all contacts across multiple pages
async function getAllContacts(jwtToken) {
  let allContacts = [];
  let page = 1;
  let hasMorePages = true;

  while (hasMorePages) {
    const response = await axios.get(`/v1/e/keap/export/contacts?limit=100&page=${page}`, {
      headers: { Authorization: `Bearer ${jwtToken}` },
    });

    allContacts = [...allContacts, ...response.data.data.contacts];

    if (response.data.pagination.next_page) {
      page = response.data.pagination.next_page;
    } else {
      hasMorePages = false;
    }
  }

  return allContacts;
}

// Usage
const allContacts = await getAllContacts(jwt_token);
console.log(`Total contacts: ${allContacts.length}`);

3. Export Multiple Data Types

// Export contacts, companies, and opportunities
const types = ['contacts', 'companies', 'opportunities', 'notes'];

const exportData = async type => {
  const response = await axios.get(`/v1/e/keap/export/${type}?limit=100&page=1`, {
    headers: { Authorization: `Bearer ${jwt_token}` },
  });
  return response.data.data;
};

// Fetch all types in parallel
const results = await Promise.all(types.map(exportData));

const [contacts, companies, opportunities, notes] = results;
console.log(
  `Exported: ${contacts.contacts.length} contacts, ${companies.companies.length} companies`,
);

4. Handle Token Invalidation

// Gracefully handle token invalidation
async function exportWithRetry(type, jwtToken, forwardUrl) {
  try {
    const response = await axios.get(`/v1/e/keap/export/${type}?limit=100&page=1`, {
      headers: { Authorization: `Bearer ${jwtToken}` },
    });
    return response.data;
  } catch (error) {
    if (error.response?.data?.message === 'TOKEN_INVALIDATED') {
      // Redirect to re-authenticate
      window.location.href = `/v1/e/keap/auth/login?forward_url=${encodeURIComponent(forwardUrl)}`;
    }
    throw error;
  }
}

5. Check Connection Status

// Check if user has existing valid connection
async function checkKeapConnection(jwtToken) {
  try {
    const response = await axios.get('/v1/e/keap/export/contacts?limit=1&page=1', {
      headers: { Authorization: `Bearer ${jwtToken}` },
    });
    return { connected: true, tokenValid: true };
  } catch (error) {
    if (error.response?.status === 401) {
      return { connected: false, tokenValid: false };
    }
    if (error.response?.data?.message === 'TOKEN_INVALIDATED') {
      return { connected: true, tokenValid: false };
    }
    throw error;
  }
}

🔗 Related Documentation

• Keap REST API Documentation
• Keap OAuth 2.0 Guide
• Authentication Documentation
• CRM Data Export Documentation

💡 Best Practices

1. Token Management

• Always handle TOKEN_INVALIDATED errors by prompting re-authentication
• The middleware handles token refresh automatically - no manual refresh needed
• Store token IDs (not tokens themselves) in frontend state

2. Pagination

• Use appropriate limit values (10-200 recommended)
• Always check next_page to determine if more pages exist
• Handle pagination client-side for large datasets

3. Error Handling

• Check for 401 status (no token found)
• Check for TOKEN_INVALIDATED message
• Provide clear re-authentication flow for users

4. Scope Management

• Ensure user has required scopes (contacts, contacts.external)
• Check scope errors before attempting exports

5. Rate Limiting

• Keap has API rate limits (exact limits not documented in integration)
• Implement retry logic with exponential backoff
• Cache exported data when possible

6. Data Validation

• Verify export type is valid before making request
• Handle empty result sets gracefully
• Validate pagination parameters (limit > 0, page > 0)

🔍 Differences from Other Integrations

Proactive Token Refresh

Unlike other integrations that check expiration first, Keap always refreshes tokens on every request.

Basic Auth for Token Refresh

Token refresh requests use Basic Authentication with base64-encoded client credentials:

Authorization: Basic {base64(client_id:client_secret)}

Full Scope

Keap integration requests full scope, granting complete access to the account (unlike other integrations with granular scopes).

Offset-Based Pagination

Uses offset/limit pagination (not cursor-based like some modern APIs).