๐ป Subscription Downgrade Processing
๐ Overviewโ
The subscription downgrade module handles tier downgrades for DashClicks accounts. When a customer downgrades their subscription plan, this processor manages the deletion of resources that exceed the new tier's limits (users, pipelines, forms, campaigns, etc.), removes custom domains for free-tier downgrades, and ensures data integrity through MongoDB transactions with rollback support.
Source Files:
- Cron:
queue-manager/crons/store/subscriptions/downgrade.js - Service:
queue-manager/services/store/subscriptions/downgrade.js - Queue Processor:
queue-manager/queues/store/subscriptions/downgrade/software.js
๐ฏ Purposeโ
- Tier Enforcement: Delete resources exceeding new tier limits
- Domain Management: Remove custom domains for free-tier downgrades
- Data Cleanup: Clean up users, pipelines, forms, campaigns, templates, SEO keywords
- Transaction Safety: Use MongoDB transactions for atomic operations with rollback
- Audit Trail: Log all deletions for tracking and debugging
- User Reassignment: Transfer ownership before user deletion
โ๏ธ Configurationโ
Environment Variablesโ
# Proxy API for custom domain management
PROXY_API_URL=http://proxy-server:6003
# Execution control
QM_SUBSCRIPTION_DOWNGRADE=true # Enable this module
# Node environment (domain removal skipped in development)
NODE_ENV=production
Cron Scheduleโ
'*/30 * * * * *'; // Every 30 seconds
Frequency: Every 30 seconds Concurrency: Single execution (in-progress locking) Retry Strategy: 3 attempts with 4-second backoff
๐ Processing Flowโ
High-Level Architectureโ
sequenceDiagram
participant CRON as Cron (30s)
participant SERVICE as Downgrade Service
participant QUEUE_MODEL as Queue Collection
participant ACCOUNT as Account Model
participant PROCESSOR as Software Processor
participant MONGO as MongoDB Transaction
participant PROXY as Proxy API
participant LOG as Downgrade Logs
CRON->>SERVICE: Trigger every 30s
SERVICE->>QUEUE_MODEL: Find pending downgrades
QUEUE_MODEL-->>SERVICE: subscription entries
loop For each subscription
SERVICE->>ACCOUNT: Verify downgrade.pending=true
alt Account needs downgrade
SERVICE->>QUEUE_MODEL: Set in_progress=true
SERVICE->>PROCESSOR: Start queue processor
PROCESSOR->>PROCESSOR: Add job to Bull queue
PROCESSOR->>MONGO: Start transaction
alt Free tier downgrade
PROCESSOR->>PROXY: DELETE custom domain
PROXY-->>PROCESSOR: Domain removed
PROCESSOR->>ACCOUNT: Set domain.custom=null
end
opt Users to delete
PROCESSOR->>PROCESSOR: Delete users & reassign
end
opt Pipelines to delete
PROCESSOR->>PROCESSOR: Delete pipelines, deals, stages, automations
end
opt Forms to delete
PROCESSOR->>PROCESSOR: Delete forms
end
opt Campaigns to delete
PROCESSOR->>PROCESSOR: Delete campaigns & leads
end
opt SEO keywords to delete
PROCESSOR->>PROCESSOR: Delete SEO configs & data
end
opt Templates to delete
PROCESSOR->>PROCESSOR: Delete templates
end
PROCESSOR->>PROCESSOR: Delete custom smart lists (contacts & deals)
PROCESSOR->>PROCESSOR: Deactivate deal automations
PROCESSOR->>MONGO: Commit transaction
PROCESSOR->>ACCOUNT: Set downgrade.pending=false
PROCESSOR->>QUEUE_MODEL: Update status=success
PROCESSOR->>LOG: Log deletion results
else Transaction fails
MONGO->>MONGO: Rollback transaction
PROCESSOR->>ACCOUNT: Set downgrade.prompt=true
PROCESSOR->>QUEUE_MODEL: Update status=failed
PROCESSOR->>LOG: Log error details
end
end
๐ง Component Detailsโ
1. Cron Schedulerโ
File: crons/store/subscriptions/downgrade.js
const subscriptionDowngrade = require('../../../services/store/subscriptions/downgrade');
const cron = require('node-cron');
const logger = require('../../../utilities/logger');
let inProgress = false;
exports.start = async () => {
try {
cron.schedule('*/30 * * * * *', async () => {
if (!inProgress) {
logger.log({ initiator: 'QM/store/subscriptions/downgrade', message: 'Execution Started' });
inProgress = true;
await subscriptionDowngrade();
inProgress = false;
logger.log({
initiator: 'QM/store/subscriptions/downgrade',
message: 'Execution Finished',
});
}
});
} catch (err) {
logger.error({ initiator: 'QM/store/subscriptions/downgrade', error: err });
}
};
In-Progress Locking:
inProgressflag prevents concurrent execution- Ensures only one downgrade process runs at a time
- Releases lock after completion (success or error)
2. Service Layerโ
File: services/store/subscriptions/downgrade.js
const Queue = require('../../../models/queues');
const Account = require('../../../models/account');
const subscriptionDowngradeQueue = require('../../../queues/store/subscriptions/downgrade/software');
module.exports = async () => {
try {
// Find pending downgrades not yet in progress
const subscriptions = await Queue.find({
source: 'subscription-downgrade',
status: 'pending',
in_progress: false,
});
if (subscriptions.length) {
await Promise.all(
subscriptions.map(async subscription => {
try {
const { _id, account_id, user_id, additional_data } = subscription;
// Verify account needs downgrade
const accountData = await Account.findOne({
_id: account_id,
$and: [
{ 'downgrade.pending': true },
{
$or: [{ 'downgrade.prompt': false }, { 'downgrade.prompt': { $exists: false } }],
},
],
});
if (!accountData) return; // Skip if not ready
// Mark as in progress
await Queue.updateOne({ _id }, { in_progress: true });
// Start Bull queue processor
let queue = await subscriptionDowngradeQueue.start(subscription._id.toString());
// Add job to queue
await addData({
id: _id,
accountId: account_id,
userId: user_id,
downgradeData: additional_data,
queue,
});
console.log('Subscription downgrade data added to queue for deleting.');
} catch (err) {
err.queue_id = subscription._id;
throw err;
}
}),
);
} else {
console.log('No subscription downgrade data to delete.');
}
} catch (err) {
if (err.queue_id) {
await Queue.updateMany({ _id: err.queue_id }, { in_progress: false });
}
console.log(`Error occurred while processing downgrade.`);
}
};
const addData = async ({ id, accountId, userId, downgradeData, queue }) => {
await queue.add(
{ accountId, userId, downgradeData },
{
jobId: id.toString(),
attempts: 3, // Retry 3 times
backoff: 4000, // 4 second delay between retries
},
);
};
Query Logic:
// Find downgrades ready to process
{
source: "subscription-downgrade", // Queue type
status: "pending", // Not processed yet
in_progress: false // Not currently being processed
}
// Verify account is ready
{
_id: account_id,
'downgrade.pending': true, // Downgrade confirmed
$or: [
{ 'downgrade.prompt': false }, // No user prompt needed
{ 'downgrade.prompt': { $exists: false } } // Prompt field not set
]
}
3. Queue Processor (Software Downgrade)โ
File: queues/store/subscriptions/downgrade/software.js
This 400+ line processor handles the complex business logic of deleting resources.
Data Structuresโ
Job Data:
{
accountId: ObjectId,
userId: ObjectId,
downgradeData: {
price: ObjectId, // New price/tier
delete: {
users: [ // Users to delete
{ id: ObjectId, assigned_to: ObjectId }
],
pipelines: [ObjectId], // Pipeline IDs
forms: [ObjectId], // Form IDs
campaigns: [ObjectId], // Campaign IDs
seo_keywords: [ // SEO keywords to remove
{ keyword: String, country: String, near: String }
],
templates: [ObjectId] // Template IDs
}
}
}
Records Deleted Tracking:
{
users: 0,
pipelines: 0,
forms: 0,
campaigns: 0,
templates: 0,
seo_keywords: 0
}
Processing Logicโ
Step 1: Transaction Initializationโ
const session = await mongoose.startSession();
session.startTransaction();
Transaction Benefits:
- All-or-nothing execution
- Automatic rollback on error
- Data consistency guaranteed
Step 2: Free Tier Domain Removalโ
const priceData = await Price.findOne({ _id: downgradeData.price }).session(session).lean().exec();
const { metadata, nickname } = priceData;
if (metadata?.software === 'true' && metadata?.tier === 'tier0' && nickname === 'Free') {
// Remove custom domain (production only)
if (process.env.NODE_ENV !== 'development' && accountInfo?.domain?.custom) {
// Call proxy API to detach custom domain
await axios.delete(`${process.env.PROXY_API_URL}/id/dashboard-custom-${account_id.toString()}`);
// Update account to remove custom domain
await Account.findByIdAndUpdate(account_id, {
'domain.custom': null,
}).session(session);
}
}
Why Remove Custom Domain?
- Free tier doesn't include custom domain feature
- Prevents domain misuse after downgrade
- Reverts to default dashclicks.com subdomain
Step 3: User Deletionโ
if (users?.length) {
const ownerUser = await User.findOne({
account: account_id,
is_owner: true,
})
.session(session)
.lean()
.exec();
for (let { id, assigned_to } of users) {
if (user && assigned_to) {
// Archive user and reassign their resources
let resp = await deleteUser(id, ownerUser._id.toString(), assigned_to);
if (!resp) {
throw new Error('REASSIGNED_ERROR: Error in reassigning user');
}
} else {
throw new Error('USER_OR_ASSIGNED_USER_NOT_FOUND');
}
}
}
User Deletion Process:
- Find account owner
- For each user to delete:
- Call
deleteUserutility (archives user) - Reassigns user's resources to
assigned_touser - Throws error if reassignment fails
- Call
Step 4: Pipeline Deletionโ
if (pipelines?.length) {
// Verify pipelines exist
const pipelineData = await Pipeline.find({
account_id,
_id: { $in: pipelines },
})
.session(session)
.lean()
.exec();
if (!pipelineData.length) {
throw new Error('PIPELINE_NOT_FOUND');
}
// Get deals to sync to contacts
const dealsData = await Deals.find({
pipeline_id: { $in: pipelines },
account_id,
created_by: user,
})
.session(session)
.exec();
// Delete pipeline and related data
const promises = [
dealsData.map(deal => deal.syncToContacts(null, true)), // Sync deals to contacts
Pipeline.deleteMany({ _id: { $in: pipelines }, account_id, created_by: user }).session(session),
Deals.deleteMany({ pipeline_id: { $in: pipelines }, account_id, created_by: user }).session(
session,
),
PipelineStage.deleteMany({
pipeline_id: { $in: pipelines },
account_id,
created_by: user,
}).session(session),
Automation.deleteMany({
pipeline_id: { $in: pipelines },
module: 'DEAL',
account_id,
created_by: user,
}).session(session),
];
await Promise.all(promises);
}
Pipeline Cleanup Includes:
- Sync deals to contacts before deletion
- Delete pipeline records
- Delete all deals in pipeline
- Delete all pipeline stages
- Delete pipeline automations
Step 5: Form Deletionโ
if (forms?.length) {
const formOptions = {
account_id,
_id: { $in: forms },
};
const formData = await Forms.find(formOptions).session(session).lean().exec();
if (!formData) {
throw new Error('FORM_NOT_FOUND');
}
await Forms.deleteMany(formOptions).session(session);
}
Step 6: Campaign Deletionโ
if (campaigns?.length) {
const inboundOptions = {
account_id: account_id.toString(),
is_deleted: { $ne: true },
_id: { $in: campaigns },
};
const campaign_ids = campaigns.map(campaign => new mongoose.Types.ObjectId(campaign));
const leadOptions = {
campaign_id: { $in: campaign_ids },
user_id: new mongoose.Types.ObjectId(user),
account_id: account_id.toString(),
};
// Verify campaigns and leads exist
const { 0: inboundResult, 1: leadResult } = await Promise.all([
Inbound.find(inboundOptions).session(session).lean().exec(),
LeadsData.find(leadOptions).session(session).lean().exec(),
]);
if (!inboundResult.length) {
throw new Error('CAMPAIGN_NOT_FOUND');
}
if (!leadResult.length) {
throw new Error('LEADS_ENTRIES_NOT_FOUND');
}
// Delete campaigns and leads
const promises = [
Inbound.deleteMany(inboundOptions).session(session),
LeadsData.deleteMany(leadOptions).session(session),
];
await Promise.all(promises);
}
Campaign Cleanup:
- Deletes campaign configuration
- Deletes all lead entries for the campaign
- Verifies both exist before deletion
Step 7: Round Robin User Cleanupโ
// Remove deleted user from round robin and default to owner
const ownerUser = await User.findOne({
account: account_id,
is_owner: true,
})
.session(session)
.lean()
.exec();
const inboundOptions = {
$expr: { $gt: [{ $size: '$selected_user' }, 1] }, // Multiple users
$or: [{ owner: user }, { 'selected_user.id': new mongoose.Types.ObjectId(user) }],
};
const inboundData = await Inbound.find(inboundOptions).session(session).lean().exec();
if (inboundData.length) {
// Set to owner user only
await Inbound.updateMany(inboundOptions, [
{
$set: {
selected_user: [
{
_id: { $first: '$selected_user._id' },
id: ownerUser._id,
name: ownerUser.name,
custom_number: ownerUser.additional_info.phones[0].number,
},
],
},
},
])
.session(session)
.lean()
.exec();
}
Purpose: When a user is deleted, remove them from campaign round robin assignments.
Step 8: SEO Keyword Deletionโ
if (seo_keywords?.length) {
const seoData = await AnalyticsSeoConfig.findOne({ account: account_id })
.session(session)
.lean()
.exec();
const existingKeywords = seoData?.keywords || [];
let matchedKeywords = [];
// Match keywords by keyword, country, and near fields
for (let keyword of seo_keywords) {
const resultKeywords = existingKeywords.filter(k => {
return (
keyword.keyword === k.keyword && keyword.country === k.country && keyword.near === k.near
);
});
matchedKeywords.push(resultKeywords);
}
if (!matchedKeywords.length) {
throw new Error('KEYWORDS_NOT_FOUND');
}
const promises = [
AnalyticsSeoConfig.deleteOne({
account: account_id,
keywords: { $in: matchedKeywords },
}).session(session),
AnalyticsSeo.deleteMany({ account: account_id }).session(session),
];
await Promise.all(promises);
}
Step 9: Template Deletionโ
if (templates?.length) {
const options = {
_id: { $in: templates },
account: account_id,
user,
};
const templateData = await Templates.find(options).session(session).lean().exec();
if (!templateData.length) {
throw new Error('TEMPLATE_NOT_FOUND');
}
await Templates.deleteMany(options).session(session);
}
Step 10: Custom Smart List Cleanupโ
Contact Smart Lists:
const smartOptions = {
account: account_id,
user: user,
$and: [
{
$or: [{ is_default: { $exists: false } }, { is_default: false }],
},
{
$or: [{ type: 'people' }, { type: 'businesses' }, { type: 'contacts' }],
},
],
};
const filter = await Filters.find(smartOptions).session(session);
if (filter.length) {
await Filters.deleteMany(smartOptions).session(session);
}
Deal Smart Lists:
const smartOptions = {
account: account_id,
user: user,
type: 'deal',
$or: [{ is_default: { $exists: false } }, { is_default: false }],
};
const filter = await Filters.find(smartOptions).session(session);
if (filter.length) {
await Filters.deleteMany(smartOptions).session(session);
}
Purpose: Remove custom filters created by deleted users.
Step 11: Deal Automation Deactivationโ
const options = {
account_id,
created_by: user,
module: 'DEAL',
};
const automationData = await Automation.find(options).session(session).lean().exec();
if (automationData.length) {
const updatedAutomation = await Automation.updateMany(options, {
$set: { status: 'INACTIVE' },
}).session(session);
if (updatedAutomation.modifiedCount === 0) {
throw new Error('FAILED_TO_TURN_OFF: Automation failed to turn-off');
}
}
Note: Automations are deactivated, not deleted (preserves configuration).
Step 12: Transaction Commitโ
await session.commitTransaction();
If successful:
- All changes are permanently applied
- Account downgrade is complete
If any error occurs:
await session.abortTransaction();
- All changes are rolled back
- Account state remains unchanged
Step 13: Post-Transaction Cleanupโ
// Verify resource usage is within new tier limits
const req = {
auth: {
account_id: account_id,
account: accountInfo,
},
};
const promises = [];
for (let [resource, count] of Object.entries(req.resource_usage)) {
resource = RESOURCES[resource];
if (count.total >= count.used) {
promises.push(Promise.resolve(true));
recordsDeleted[resource] = downgradeData.delete[resource].length;
} else {
promises.push(Promise.reject(false));
}
}
await Promise.all(promises);
// Mark downgrade as complete
await Account.findOneAndUpdate(
{ _id: account_id },
{ $set: { 'downgrade.pending': false } },
{ new: true },
)
.lean()
.exec();
done(); // Job complete
If verification fails:
// Prompt user to delete more resources
await Account.findOneAndUpdate(
{ _id: account_id },
{ $set: { 'downgrade.prompt': true } },
{ new: true },
)
.lean()
.exec();
done(error);
4. Completion Callbacksโ
Success Callback:
const completedCb = async job => {
await QueueModel.updateOne(
{ _id: job.id },
{
status: 'success',
in_progress: false,
},
);
await logging(job.id, recordsDeleted, 'success', '', account_id, user);
};
Failure Callback:
const failedCb = async (job, err) => {
try {
await QueueModel.updateOne(
{ _id: job.id },
{
status: 'failed',
in_progress: false,
},
);
await logging(job.id, recordsDeleted, 'failed', err.message, account_id, user);
} catch (err) {
console.error('Failed to update status on queue item', job.id, err.message);
}
console.error('Subscription Downgrade Queue failed', job.id, err.message, err.stack);
};
๐ Data Modelsโ
Queue Entryโ
{
_id: ObjectId,
source: 'subscription-downgrade',
status: 'pending' | 'success' | 'failed',
in_progress: Boolean,
account_id: ObjectId,
user_id: ObjectId,
additional_data: {
price: ObjectId,
delete: {
users: [{ id: ObjectId, assigned_to: ObjectId }],
pipelines: [ObjectId],
forms: [ObjectId],
campaigns: [ObjectId],
seo_keywords: [{ keyword: String, country: String, near: String }],
templates: [ObjectId]
}
},
createdAt: Date,
updatedAt: Date
}
Account Downgrade Stateโ
{
_id: ObjectId,
downgrade: {
pending: Boolean, // Downgrade in progress
prompt: Boolean // User needs to delete more resources
},
domain: {
custom: String | null // Custom domain (removed for free tier)
}
}
Price/Tier Metadataโ
{
_id: ObjectId,
nickname: 'Free' | 'Starter' | 'Pro' | 'Enterprise',
metadata: {
software: 'true' | 'false',
tier: 'tier0' | 'tier1' | 'tier2' | 'tier3'
}
}
๐จ Error Handlingโ
Transaction Errorsโ
All errors during transaction cause automatic rollback:
try {
await session.commitTransaction();
} catch (error) {
await session.abortTransaction();
done(error);
} finally {
session.endSession();
}
Specific Error Typesโ
| Error Code | Message | Cause | Resolution |
|---|---|---|---|
FAILED TO REMOVE CUSTOM DOMAIN | Proxy API call failed | Domain DNS issues | Manual domain cleanup required |
REASSIGNED_ERROR | User reassignment failed | Invalid assigned_to user | Verify user exists and has permissions |
PIPELINE_NOT_FOUND | Pipeline doesn't exist | Invalid pipeline ID | Remove from delete list |
FORM_NOT_FOUND | Form doesn't exist | Invalid form ID | Remove from delete list |
CAMPAIGN_NOT_FOUND | Campaign doesn't exist | Invalid campaign ID | Remove from delete list |
LEADS_ENTRIES_NOT_FOUND | No leads for campaign | Campaign has no data | Skip campaign deletion |
KEYWORDS_NOT_FOUND | SEO keywords don't match | Keyword already removed | Skip SEO deletion |
TEMPLATE_NOT_FOUND | Template doesn't exist | Invalid template ID | Remove from delete list |
FAILED_TO_TURN_OFF | Automation update failed | Database constraint | Retry or manual deactivation |
๐ Performance Considerationsโ
Transaction Performanceโ
- Single Transaction: All deletions in one transaction
- Rollback Cost: Complete rollback on any error
- Lock Duration: Holds locks for entire transaction (30-60 seconds typical)
Optimization Strategiesโ
- Parallel Deletions: Uses
Promise.allfor independent operations - Lean Queries: Uses
.lean()for read-only queries - Session Scoping: All queries use same session for consistency
- Batch Operations:
deleteManyinstead of individual deletes
Scaling Considerationsโ
Current Limitations:
- Processes one downgrade at a time (in-progress lock)
- Long transaction duration for accounts with many resources
- Proxy API is a single point of failure for domain removal
Improvements:
// Process multiple downgrades concurrently (with care)
const CONCURRENCY = 3;
const chunks = _.chunk(subscriptions, CONCURRENCY);
for (let chunk of chunks) {
await Promise.all(chunk.map(processDowngrade));
}
๐งช Testing Considerationsโ
Test Scenariosโ
-
Free Tier Downgrade:
- Verify custom domain removal
- Verify domain.custom set to null
-
User Deletion:
- Verify reassignment to assigned_to user
- Verify user resources transferred
-
Pipeline Deletion:
- Verify deals synced to contacts
- Verify cascading deletes (stages, automations)
-
Transaction Rollback:
- Simulate error mid-transaction
- Verify no partial deletions
- Verify account state unchanged
-
Resource Verification:
- Verify downgrade.prompt=true if limits still exceeded
- Verify downgrade.pending=false on success
Mock Setupโ
jest.mock('../../../models/queues');
jest.mock('../../../models/account');
jest.mock('../../../utilities/delete-user.new');
jest.mock('axios');
describe('Subscription Downgrade', () => {
test('Removes custom domain for free tier', async () => {
// Setup mock data
const mockAccount = {
_id: 'account123',
domain: { custom: 'custom.example.com' },
};
const mockPrice = {
metadata: { software: 'true', tier: 'tier0' },
nickname: 'Free',
};
// Test domain removal
// ...
});
test('Rolls back on error', async () => {
// Simulate error during processing
// Verify transaction rollback
// Verify no data deleted
});
});
๐ Related Documentationโ
- Store Module Overview
- Subscription Cancel
- Subscription Activate
- Downgrade Logs Utility
- Queue Wrapper
๐ Notesโ
Why MongoDB Transactions?โ
Downgrade involves multiple collections and must be atomic:
- If pipeline deletion fails, user deletion should rollback
- Prevents partial downgrades (inconsistent state)
- Ensures data integrity
Custom Domain Removal Strategyโ
Free tier users cannot use custom domains:
- Call proxy API to remove DNS configuration
- Update account to remove custom domain reference
- User automatically redirected to dashclicks.com subdomain
Resource Verification Post-Transactionโ
Even after deletions, account may still exceed limits if:
- User didn't select enough resources to delete
- Other resources were created during downgrade
- Calculation error in frontend
Solution: Set downgrade.prompt=true to ask user to delete more.
Automation Deactivation vs Deletionโ
Automations are deactivated, not deleted:
- Preserves automation configuration
- User can reactivate if they upgrade
- Prevents loss of complex workflow logic
Round Robin Cleanup Importanceโ
When a user is deleted from round robin lead assignment:
- Leads would fail to assign properly
- Could cause lead loss
- Default to owner ensures no disruption
Complexity: Very High (400+ lines, MongoDB transactions, multi-collection deletions)
Business Impact: CRITICAL - Revenue operations, must not fail
Dependencies: 15+ models, proxy API, delete-user utility, downgrade-logs
Transaction Safe: Yes (full rollback on error)
Last Updated: 2025-10-10