Skip to main content

๐Ÿ”ป Subscription Downgrade Processing

๐Ÿ“– Overviewโ€‹

The subscription downgrade module handles tier downgrades for DashClicks accounts. When a customer downgrades their subscription plan, this processor manages the deletion of resources that exceed the new tier's limits (users, pipelines, forms, campaigns, etc.), removes custom domains for free-tier downgrades, and ensures data integrity through MongoDB transactions with rollback support.

Source Files:

  • Cron: queue-manager/crons/store/subscriptions/downgrade.js
  • Service: queue-manager/services/store/subscriptions/downgrade.js
  • Queue Processor: queue-manager/queues/store/subscriptions/downgrade/software.js

๐ŸŽฏ Purposeโ€‹

  • Tier Enforcement: Delete resources exceeding new tier limits
  • Domain Management: Remove custom domains for free-tier downgrades
  • Data Cleanup: Clean up users, pipelines, forms, campaigns, templates, SEO keywords
  • Transaction Safety: Use MongoDB transactions for atomic operations with rollback
  • Audit Trail: Log all deletions for tracking and debugging
  • User Reassignment: Transfer ownership before user deletion

โš™๏ธ Configurationโ€‹

Environment Variablesโ€‹

# Proxy API for custom domain management
PROXY_API_URL=http://proxy-server:6003

# Execution control
QM_SUBSCRIPTION_DOWNGRADE=true # Enable this module

# Node environment (domain removal skipped in development)
NODE_ENV=production

Cron Scheduleโ€‹

'*/30 * * * * *'; // Every 30 seconds

Frequency: Every 30 seconds Concurrency: Single execution (in-progress locking) Retry Strategy: 3 attempts with 4-second backoff

๐Ÿ“‹ Processing Flowโ€‹

High-Level Architectureโ€‹

sequenceDiagram
participant CRON as Cron (30s)
participant SERVICE as Downgrade Service
participant QUEUE_MODEL as Queue Collection
participant ACCOUNT as Account Model
participant PROCESSOR as Software Processor
participant MONGO as MongoDB Transaction
participant PROXY as Proxy API
participant LOG as Downgrade Logs

CRON->>SERVICE: Trigger every 30s
SERVICE->>QUEUE_MODEL: Find pending downgrades
QUEUE_MODEL-->>SERVICE: subscription entries

loop For each subscription
SERVICE->>ACCOUNT: Verify downgrade.pending=true
alt Account needs downgrade
SERVICE->>QUEUE_MODEL: Set in_progress=true
SERVICE->>PROCESSOR: Start queue processor
PROCESSOR->>PROCESSOR: Add job to Bull queue

PROCESSOR->>MONGO: Start transaction

alt Free tier downgrade
PROCESSOR->>PROXY: DELETE custom domain
PROXY-->>PROCESSOR: Domain removed
PROCESSOR->>ACCOUNT: Set domain.custom=null
end

opt Users to delete
PROCESSOR->>PROCESSOR: Delete users & reassign
end

opt Pipelines to delete
PROCESSOR->>PROCESSOR: Delete pipelines, deals, stages, automations
end

opt Forms to delete
PROCESSOR->>PROCESSOR: Delete forms
end

opt Campaigns to delete
PROCESSOR->>PROCESSOR: Delete campaigns & leads
end

opt SEO keywords to delete
PROCESSOR->>PROCESSOR: Delete SEO configs & data
end

opt Templates to delete
PROCESSOR->>PROCESSOR: Delete templates
end

PROCESSOR->>PROCESSOR: Delete custom smart lists (contacts & deals)
PROCESSOR->>PROCESSOR: Deactivate deal automations

PROCESSOR->>MONGO: Commit transaction
PROCESSOR->>ACCOUNT: Set downgrade.pending=false
PROCESSOR->>QUEUE_MODEL: Update status=success
PROCESSOR->>LOG: Log deletion results

else Transaction fails
MONGO->>MONGO: Rollback transaction
PROCESSOR->>ACCOUNT: Set downgrade.prompt=true
PROCESSOR->>QUEUE_MODEL: Update status=failed
PROCESSOR->>LOG: Log error details
end
end

๐Ÿ”ง Component Detailsโ€‹

1. Cron Schedulerโ€‹

File: crons/store/subscriptions/downgrade.js

const subscriptionDowngrade = require('../../../services/store/subscriptions/downgrade');
const cron = require('node-cron');
const logger = require('../../../utilities/logger');

let inProgress = false;
exports.start = async () => {
try {
cron.schedule('*/30 * * * * *', async () => {
if (!inProgress) {
logger.log({ initiator: 'QM/store/subscriptions/downgrade', message: 'Execution Started' });
inProgress = true;
await subscriptionDowngrade();
inProgress = false;
logger.log({
initiator: 'QM/store/subscriptions/downgrade',
message: 'Execution Finished',
});
}
});
} catch (err) {
logger.error({ initiator: 'QM/store/subscriptions/downgrade', error: err });
}
};

In-Progress Locking:

  • inProgress flag prevents concurrent execution
  • Ensures only one downgrade process runs at a time
  • Releases lock after completion (success or error)

2. Service Layerโ€‹

File: services/store/subscriptions/downgrade.js

const Queue = require('../../../models/queues');
const Account = require('../../../models/account');
const subscriptionDowngradeQueue = require('../../../queues/store/subscriptions/downgrade/software');

module.exports = async () => {
try {
// Find pending downgrades not yet in progress
const subscriptions = await Queue.find({
source: 'subscription-downgrade',
status: 'pending',
in_progress: false,
});

if (subscriptions.length) {
await Promise.all(
subscriptions.map(async subscription => {
try {
const { _id, account_id, user_id, additional_data } = subscription;

// Verify account needs downgrade
const accountData = await Account.findOne({
_id: account_id,
$and: [
{ 'downgrade.pending': true },
{
$or: [{ 'downgrade.prompt': false }, { 'downgrade.prompt': { $exists: false } }],
},
],
});

if (!accountData) return; // Skip if not ready

// Mark as in progress
await Queue.updateOne({ _id }, { in_progress: true });

// Start Bull queue processor
let queue = await subscriptionDowngradeQueue.start(subscription._id.toString());

// Add job to queue
await addData({
id: _id,
accountId: account_id,
userId: user_id,
downgradeData: additional_data,
queue,
});

console.log('Subscription downgrade data added to queue for deleting.');
} catch (err) {
err.queue_id = subscription._id;
throw err;
}
}),
);
} else {
console.log('No subscription downgrade data to delete.');
}
} catch (err) {
if (err.queue_id) {
await Queue.updateMany({ _id: err.queue_id }, { in_progress: false });
}
console.log(`Error occurred while processing downgrade.`);
}
};

const addData = async ({ id, accountId, userId, downgradeData, queue }) => {
await queue.add(
{ accountId, userId, downgradeData },
{
jobId: id.toString(),
attempts: 3, // Retry 3 times
backoff: 4000, // 4 second delay between retries
},
);
};

Query Logic:

// Find downgrades ready to process
{
source: "subscription-downgrade", // Queue type
status: "pending", // Not processed yet
in_progress: false // Not currently being processed
}

// Verify account is ready
{
_id: account_id,
'downgrade.pending': true, // Downgrade confirmed
$or: [
{ 'downgrade.prompt': false }, // No user prompt needed
{ 'downgrade.prompt': { $exists: false } } // Prompt field not set
]
}

3. Queue Processor (Software Downgrade)โ€‹

File: queues/store/subscriptions/downgrade/software.js

This 400+ line processor handles the complex business logic of deleting resources.

Data Structuresโ€‹

Job Data:

{
accountId: ObjectId,
userId: ObjectId,
downgradeData: {
price: ObjectId, // New price/tier
delete: {
users: [ // Users to delete
{ id: ObjectId, assigned_to: ObjectId }
],
pipelines: [ObjectId], // Pipeline IDs
forms: [ObjectId], // Form IDs
campaigns: [ObjectId], // Campaign IDs
seo_keywords: [ // SEO keywords to remove
{ keyword: String, country: String, near: String }
],
templates: [ObjectId] // Template IDs
}
}
}

Records Deleted Tracking:

{
users: 0,
pipelines: 0,
forms: 0,
campaigns: 0,
templates: 0,
seo_keywords: 0
}

Processing Logicโ€‹

Step 1: Transaction Initializationโ€‹
const session = await mongoose.startSession();
session.startTransaction();

Transaction Benefits:

  • All-or-nothing execution
  • Automatic rollback on error
  • Data consistency guaranteed
Step 2: Free Tier Domain Removalโ€‹
const priceData = await Price.findOne({ _id: downgradeData.price }).session(session).lean().exec();

const { metadata, nickname } = priceData;

if (metadata?.software === 'true' && metadata?.tier === 'tier0' && nickname === 'Free') {
// Remove custom domain (production only)
if (process.env.NODE_ENV !== 'development' && accountInfo?.domain?.custom) {
// Call proxy API to detach custom domain
await axios.delete(`${process.env.PROXY_API_URL}/id/dashboard-custom-${account_id.toString()}`);

// Update account to remove custom domain
await Account.findByIdAndUpdate(account_id, {
'domain.custom': null,
}).session(session);
}
}

Why Remove Custom Domain?

  • Free tier doesn't include custom domain feature
  • Prevents domain misuse after downgrade
  • Reverts to default dashclicks.com subdomain
Step 3: User Deletionโ€‹
if (users?.length) {
const ownerUser = await User.findOne({
account: account_id,
is_owner: true,
})
.session(session)
.lean()
.exec();

for (let { id, assigned_to } of users) {
if (user && assigned_to) {
// Archive user and reassign their resources
let resp = await deleteUser(id, ownerUser._id.toString(), assigned_to);

if (!resp) {
throw new Error('REASSIGNED_ERROR: Error in reassigning user');
}
} else {
throw new Error('USER_OR_ASSIGNED_USER_NOT_FOUND');
}
}
}

User Deletion Process:

  1. Find account owner
  2. For each user to delete:
    • Call deleteUser utility (archives user)
    • Reassigns user's resources to assigned_to user
    • Throws error if reassignment fails
Step 4: Pipeline Deletionโ€‹
if (pipelines?.length) {
// Verify pipelines exist
const pipelineData = await Pipeline.find({
account_id,
_id: { $in: pipelines },
})
.session(session)
.lean()
.exec();

if (!pipelineData.length) {
throw new Error('PIPELINE_NOT_FOUND');
}

// Get deals to sync to contacts
const dealsData = await Deals.find({
pipeline_id: { $in: pipelines },
account_id,
created_by: user,
})
.session(session)
.exec();

// Delete pipeline and related data
const promises = [
dealsData.map(deal => deal.syncToContacts(null, true)), // Sync deals to contacts
Pipeline.deleteMany({ _id: { $in: pipelines }, account_id, created_by: user }).session(session),
Deals.deleteMany({ pipeline_id: { $in: pipelines }, account_id, created_by: user }).session(
session,
),
PipelineStage.deleteMany({
pipeline_id: { $in: pipelines },
account_id,
created_by: user,
}).session(session),
Automation.deleteMany({
pipeline_id: { $in: pipelines },
module: 'DEAL',
account_id,
created_by: user,
}).session(session),
];

await Promise.all(promises);
}

Pipeline Cleanup Includes:

  • Sync deals to contacts before deletion
  • Delete pipeline records
  • Delete all deals in pipeline
  • Delete all pipeline stages
  • Delete pipeline automations
Step 5: Form Deletionโ€‹
if (forms?.length) {
const formOptions = {
account_id,
_id: { $in: forms },
};

const formData = await Forms.find(formOptions).session(session).lean().exec();

if (!formData) {
throw new Error('FORM_NOT_FOUND');
}

await Forms.deleteMany(formOptions).session(session);
}
Step 6: Campaign Deletionโ€‹
if (campaigns?.length) {
const inboundOptions = {
account_id: account_id.toString(),
is_deleted: { $ne: true },
_id: { $in: campaigns },
};

const campaign_ids = campaigns.map(campaign => new mongoose.Types.ObjectId(campaign));

const leadOptions = {
campaign_id: { $in: campaign_ids },
user_id: new mongoose.Types.ObjectId(user),
account_id: account_id.toString(),
};

// Verify campaigns and leads exist
const { 0: inboundResult, 1: leadResult } = await Promise.all([
Inbound.find(inboundOptions).session(session).lean().exec(),
LeadsData.find(leadOptions).session(session).lean().exec(),
]);

if (!inboundResult.length) {
throw new Error('CAMPAIGN_NOT_FOUND');
}

if (!leadResult.length) {
throw new Error('LEADS_ENTRIES_NOT_FOUND');
}

// Delete campaigns and leads
const promises = [
Inbound.deleteMany(inboundOptions).session(session),
LeadsData.deleteMany(leadOptions).session(session),
];

await Promise.all(promises);
}

Campaign Cleanup:

  • Deletes campaign configuration
  • Deletes all lead entries for the campaign
  • Verifies both exist before deletion
Step 7: Round Robin User Cleanupโ€‹
// Remove deleted user from round robin and default to owner
const ownerUser = await User.findOne({
account: account_id,
is_owner: true,
})
.session(session)
.lean()
.exec();

const inboundOptions = {
$expr: { $gt: [{ $size: '$selected_user' }, 1] }, // Multiple users
$or: [{ owner: user }, { 'selected_user.id': new mongoose.Types.ObjectId(user) }],
};

const inboundData = await Inbound.find(inboundOptions).session(session).lean().exec();

if (inboundData.length) {
// Set to owner user only
await Inbound.updateMany(inboundOptions, [
{
$set: {
selected_user: [
{
_id: { $first: '$selected_user._id' },
id: ownerUser._id,
name: ownerUser.name,
custom_number: ownerUser.additional_info.phones[0].number,
},
],
},
},
])
.session(session)
.lean()
.exec();
}

Purpose: When a user is deleted, remove them from campaign round robin assignments.

Step 8: SEO Keyword Deletionโ€‹
if (seo_keywords?.length) {
const seoData = await AnalyticsSeoConfig.findOne({ account: account_id })
.session(session)
.lean()
.exec();

const existingKeywords = seoData?.keywords || [];
let matchedKeywords = [];

// Match keywords by keyword, country, and near fields
for (let keyword of seo_keywords) {
const resultKeywords = existingKeywords.filter(k => {
return (
keyword.keyword === k.keyword && keyword.country === k.country && keyword.near === k.near
);
});
matchedKeywords.push(resultKeywords);
}

if (!matchedKeywords.length) {
throw new Error('KEYWORDS_NOT_FOUND');
}

const promises = [
AnalyticsSeoConfig.deleteOne({
account: account_id,
keywords: { $in: matchedKeywords },
}).session(session),
AnalyticsSeo.deleteMany({ account: account_id }).session(session),
];

await Promise.all(promises);
}
Step 9: Template Deletionโ€‹
if (templates?.length) {
const options = {
_id: { $in: templates },
account: account_id,
user,
};

const templateData = await Templates.find(options).session(session).lean().exec();

if (!templateData.length) {
throw new Error('TEMPLATE_NOT_FOUND');
}

await Templates.deleteMany(options).session(session);
}
Step 10: Custom Smart List Cleanupโ€‹

Contact Smart Lists:

const smartOptions = {
account: account_id,
user: user,
$and: [
{
$or: [{ is_default: { $exists: false } }, { is_default: false }],
},
{
$or: [{ type: 'people' }, { type: 'businesses' }, { type: 'contacts' }],
},
],
};

const filter = await Filters.find(smartOptions).session(session);

if (filter.length) {
await Filters.deleteMany(smartOptions).session(session);
}

Deal Smart Lists:

const smartOptions = {
account: account_id,
user: user,
type: 'deal',
$or: [{ is_default: { $exists: false } }, { is_default: false }],
};

const filter = await Filters.find(smartOptions).session(session);

if (filter.length) {
await Filters.deleteMany(smartOptions).session(session);
}

Purpose: Remove custom filters created by deleted users.

Step 11: Deal Automation Deactivationโ€‹
const options = {
account_id,
created_by: user,
module: 'DEAL',
};

const automationData = await Automation.find(options).session(session).lean().exec();

if (automationData.length) {
const updatedAutomation = await Automation.updateMany(options, {
$set: { status: 'INACTIVE' },
}).session(session);

if (updatedAutomation.modifiedCount === 0) {
throw new Error('FAILED_TO_TURN_OFF: Automation failed to turn-off');
}
}

Note: Automations are deactivated, not deleted (preserves configuration).

Step 12: Transaction Commitโ€‹
await session.commitTransaction();

If successful:

  • All changes are permanently applied
  • Account downgrade is complete

If any error occurs:

await session.abortTransaction();
  • All changes are rolled back
  • Account state remains unchanged
Step 13: Post-Transaction Cleanupโ€‹
// Verify resource usage is within new tier limits
const req = {
auth: {
account_id: account_id,
account: accountInfo,
},
};

const promises = [];
for (let [resource, count] of Object.entries(req.resource_usage)) {
resource = RESOURCES[resource];
if (count.total >= count.used) {
promises.push(Promise.resolve(true));
recordsDeleted[resource] = downgradeData.delete[resource].length;
} else {
promises.push(Promise.reject(false));
}
}

await Promise.all(promises);

// Mark downgrade as complete
await Account.findOneAndUpdate(
{ _id: account_id },
{ $set: { 'downgrade.pending': false } },
{ new: true },
)
.lean()
.exec();

done(); // Job complete

If verification fails:

// Prompt user to delete more resources
await Account.findOneAndUpdate(
{ _id: account_id },
{ $set: { 'downgrade.prompt': true } },
{ new: true },
)
.lean()
.exec();

done(error);

4. Completion Callbacksโ€‹

Success Callback:

const completedCb = async job => {
await QueueModel.updateOne(
{ _id: job.id },
{
status: 'success',
in_progress: false,
},
);

await logging(job.id, recordsDeleted, 'success', '', account_id, user);
};

Failure Callback:

const failedCb = async (job, err) => {
try {
await QueueModel.updateOne(
{ _id: job.id },
{
status: 'failed',
in_progress: false,
},
);

await logging(job.id, recordsDeleted, 'failed', err.message, account_id, user);
} catch (err) {
console.error('Failed to update status on queue item', job.id, err.message);
}

console.error('Subscription Downgrade Queue failed', job.id, err.message, err.stack);
};

๐Ÿ“Š Data Modelsโ€‹

Queue Entryโ€‹

{
_id: ObjectId,
source: 'subscription-downgrade',
status: 'pending' | 'success' | 'failed',
in_progress: Boolean,
account_id: ObjectId,
user_id: ObjectId,
additional_data: {
price: ObjectId,
delete: {
users: [{ id: ObjectId, assigned_to: ObjectId }],
pipelines: [ObjectId],
forms: [ObjectId],
campaigns: [ObjectId],
seo_keywords: [{ keyword: String, country: String, near: String }],
templates: [ObjectId]
}
},
createdAt: Date,
updatedAt: Date
}

Account Downgrade Stateโ€‹

{
_id: ObjectId,
downgrade: {
pending: Boolean, // Downgrade in progress
prompt: Boolean // User needs to delete more resources
},
domain: {
custom: String | null // Custom domain (removed for free tier)
}
}

Price/Tier Metadataโ€‹

{
_id: ObjectId,
nickname: 'Free' | 'Starter' | 'Pro' | 'Enterprise',
metadata: {
software: 'true' | 'false',
tier: 'tier0' | 'tier1' | 'tier2' | 'tier3'
}
}

๐Ÿšจ Error Handlingโ€‹

Transaction Errorsโ€‹

All errors during transaction cause automatic rollback:

try {
await session.commitTransaction();
} catch (error) {
await session.abortTransaction();
done(error);
} finally {
session.endSession();
}

Specific Error Typesโ€‹

Error CodeMessageCauseResolution
FAILED TO REMOVE CUSTOM DOMAINProxy API call failedDomain DNS issuesManual domain cleanup required
REASSIGNED_ERRORUser reassignment failedInvalid assigned_to userVerify user exists and has permissions
PIPELINE_NOT_FOUNDPipeline doesn't existInvalid pipeline IDRemove from delete list
FORM_NOT_FOUNDForm doesn't existInvalid form IDRemove from delete list
CAMPAIGN_NOT_FOUNDCampaign doesn't existInvalid campaign IDRemove from delete list
LEADS_ENTRIES_NOT_FOUNDNo leads for campaignCampaign has no dataSkip campaign deletion
KEYWORDS_NOT_FOUNDSEO keywords don't matchKeyword already removedSkip SEO deletion
TEMPLATE_NOT_FOUNDTemplate doesn't existInvalid template IDRemove from delete list
FAILED_TO_TURN_OFFAutomation update failedDatabase constraintRetry or manual deactivation

๐Ÿ“ˆ Performance Considerationsโ€‹

Transaction Performanceโ€‹

  • Single Transaction: All deletions in one transaction
  • Rollback Cost: Complete rollback on any error
  • Lock Duration: Holds locks for entire transaction (30-60 seconds typical)

Optimization Strategiesโ€‹

  1. Parallel Deletions: Uses Promise.all for independent operations
  2. Lean Queries: Uses .lean() for read-only queries
  3. Session Scoping: All queries use same session for consistency
  4. Batch Operations: deleteMany instead of individual deletes

Scaling Considerationsโ€‹

Current Limitations:

  • Processes one downgrade at a time (in-progress lock)
  • Long transaction duration for accounts with many resources
  • Proxy API is a single point of failure for domain removal

Improvements:

// Process multiple downgrades concurrently (with care)
const CONCURRENCY = 3;
const chunks = _.chunk(subscriptions, CONCURRENCY);

for (let chunk of chunks) {
await Promise.all(chunk.map(processDowngrade));
}

๐Ÿงช Testing Considerationsโ€‹

Test Scenariosโ€‹

  1. Free Tier Downgrade:

    • Verify custom domain removal
    • Verify domain.custom set to null
  2. User Deletion:

    • Verify reassignment to assigned_to user
    • Verify user resources transferred
  3. Pipeline Deletion:

    • Verify deals synced to contacts
    • Verify cascading deletes (stages, automations)
  4. Transaction Rollback:

    • Simulate error mid-transaction
    • Verify no partial deletions
    • Verify account state unchanged
  5. Resource Verification:

    • Verify downgrade.prompt=true if limits still exceeded
    • Verify downgrade.pending=false on success

Mock Setupโ€‹

jest.mock('../../../models/queues');
jest.mock('../../../models/account');
jest.mock('../../../utilities/delete-user.new');
jest.mock('axios');

describe('Subscription Downgrade', () => {
test('Removes custom domain for free tier', async () => {
// Setup mock data
const mockAccount = {
_id: 'account123',
domain: { custom: 'custom.example.com' },
};

const mockPrice = {
metadata: { software: 'true', tier: 'tier0' },
nickname: 'Free',
};

// Test domain removal
// ...
});

test('Rolls back on error', async () => {
// Simulate error during processing
// Verify transaction rollback
// Verify no data deleted
});
});

๐Ÿ“ Notesโ€‹

Why MongoDB Transactions?โ€‹

Downgrade involves multiple collections and must be atomic:

  • If pipeline deletion fails, user deletion should rollback
  • Prevents partial downgrades (inconsistent state)
  • Ensures data integrity

Custom Domain Removal Strategyโ€‹

Free tier users cannot use custom domains:

  1. Call proxy API to remove DNS configuration
  2. Update account to remove custom domain reference
  3. User automatically redirected to dashclicks.com subdomain

Resource Verification Post-Transactionโ€‹

Even after deletions, account may still exceed limits if:

  • User didn't select enough resources to delete
  • Other resources were created during downgrade
  • Calculation error in frontend

Solution: Set downgrade.prompt=true to ask user to delete more.

Automation Deactivation vs Deletionโ€‹

Automations are deactivated, not deleted:

  • Preserves automation configuration
  • User can reactivate if they upgrade
  • Prevents loss of complex workflow logic

Round Robin Cleanup Importanceโ€‹

When a user is deleted from round robin lead assignment:

  • Leads would fail to assign properly
  • Could cause lead loss
  • Default to owner ensures no disruption

Complexity: Very High (400+ lines, MongoDB transactions, multi-collection deletions)
Business Impact: CRITICAL - Revenue operations, must not fail
Dependencies: 15+ models, proxy API, delete-user utility, downgrade-logs
Transaction Safe: Yes (full rollback on error)
Last Updated: 2025-10-10

๐Ÿ’ฌ

Documentation Assistant

Ask me anything about the docs

Hi! I'm your documentation assistant. Ask me anything about the docs!

I can help you with:
- Code examples
- Configuration details
- Troubleshooting
- Best practices

Try asking: How do I configure the API?
09:31 AM