DEV Community

Cover image for Building a Smart Bible Reading Reminder System - Personalized Push Notifications at Scale
Sangwoo Lee
Sangwoo Lee

Posted on

Building a Smart Bible Reading Reminder System - Personalized Push Notifications at Scale

When users can create multiple Bible reading plans, how do you send exactly one reminder per person—with personalized deep-link data—based on their most recent activity? Here's how I built an automated system that handles this complexity at scale.

When building a Bible reading app where users can create multiple concurrent reading plans, a simple question emerged: How do we remind inactive readers without spamming them?

The challenge wasn't just sending push notifications—that's the easy part. The real complexity came from:

  1. One notification per user, not one per reading plan (users have multiple plans)
  2. Personalized deep-links in each notification (click should open their specific plan)
  3. Most recent activity tracking across all plans (don't send if they read any plan recently)
  4. Automated scheduling via cron (zero manual intervention)
  5. Configurable inactivity threshold (default: 7 days, but adjustable)

What started as "send a push to inactive users" turned into a fascinating journey through SQL window functions, personalized FCM payloads, and the critical importance of the one-person-one-push rule. Here's how I built it.

The Context: Multi-Plan Bible Reading System

Our app allows users to create multiple Bible reading plans simultaneously:

User A:
- Plan 1: One Year Bible (Started 6 months ago, last read: 10 days ago)
- Plan 2: New Testament (Started last week, last read: 2 days ago)  
- Plan 3: Psalms (Started yesterday, last read: 1 day ago)

User B:
- Plan 1: Chronological Bible (Started 3 months ago, last read: 8 days ago)
Enter fullscreen mode Exit fullscreen mode

The Database Schema:

// bible_schedule_plan_list - The reading plan
// (One plan can have multiple participants)
{
  idx: number;              // Plan ID
  leader_seq: number;       // Creator's member_seq
  title: string;            // Plan name
  schedule_type_name: string;
  start_date: string;
  state: number;            // 1=pending, 2=active, 3=completed
  delete_yn: string;
}

// bible_schedule_plan_member_list - User's participation in plan
// (One user can participate in multiple plans)
{
  idx: number;              // plan_member_idx (unique per participation)
  plan_list_idx: number;    // FK to plan
  member_seq: number;       // User ID
  state: number;            // 1=pending, 2=active, 3=completed
  push_time: string;        // User's preferred notification time
  delete_yn: string;
}

// bible_schedule_sync_data_list - Reading progress records
// (Every chapter read = one record)
{
  idx: number;
  member_seq: number;
  plan_list_idx: number;
  plan_member_idx: number;
  bible_code: number;       // Book number (1-66)
  jang: number;             // Chapter
  regdate: Date;            // When they read it
  delete_yn: string;
}
Enter fullscreen mode Exit fullscreen mode

The Business Rules:

  1. A user can have multiple active plans (state=2)
  2. Each plan tracks reading progress independently
  3. Users read different plans at different paces
  4. Critical: Send ONE reminder per user, not one per plan

The Challenge: Defining "Inactive"

Naive Approach (Wrong!)

My first instinct was simple:

-- ❌ Wrong: This gives users with multiple plans
SELECT DISTINCT member_seq
FROM bible_schedule_sync_data_list
WHERE DATEDIFF(day, regdate, GETDATE()) >= 7
  AND delete_yn = 'N';
Enter fullscreen mode Exit fullscreen mode

Problem: User A has Plan 1 (10 days inactive) and Plan 3 (1 day active). This query would:

  • Find Plan 1's inactivity (10 days) ✓
  • Find Plan 3's activity (1 day) ✓
  • Still include User A (because Plan 1 is inactive) ❌

Result: False positive. User A gets a reminder even though they actively read Plan 3 yesterday!

The Correct Definition

A user is inactive if their MOST RECENT reading activity (across all plans) was more than N days ago.

User A's activity timeline:
- Plan 1: last read 10 days ago
- Plan 2: last read 2 days ago  ← Most recent
- Plan 3: last read 1 day ago

Most recent activity: 1 day ago → NOT INACTIVE ✓

User B's activity timeline:
- Plan 1: last read 8 days ago  ← Most recent (only plan)

Most recent activity: 8 days ago → INACTIVE ✓
Enter fullscreen mode Exit fullscreen mode

This requires SQL window functions to find the most recent activity across all plans per user.

Implementation: The SQL Query Journey

Attempt 1: HAVING Clause (Wrong!)

-- ❌ This has a fatal logic error
WITH LastReadPerMember AS (
  SELECT 
    sd.member_seq,
    sd.plan_list_idx,
    pm.idx AS plan_member_idx,
    MAX(sd.regdate) AS last_read_date,
    ROW_NUMBER() OVER (
      PARTITION BY sd.member_seq 
      ORDER BY MAX(sd.regdate) DESC
    ) AS rn
  FROM bible_schedule_sync_data_list sd
  INNER JOIN bible_schedule_plan_member_list pm
    ON sd.member_seq = pm.member_seq
    AND sd.plan_list_idx = pm.plan_list_idx
  WHERE sd.delete_yn = 'N'
    AND pm.delete_yn = 'N'
    AND pm.state = 2  -- Active plans only
  GROUP BY sd.member_seq, sd.plan_list_idx, pm.idx
  HAVING DATEDIFF(day, MAX(sd.regdate), GETDATE()) >= 7  -- ❌ BUG!
)
SELECT * FROM LastReadPerMember WHERE rn = 1;
Enter fullscreen mode Exit fullscreen mode

The Bug:

The HAVING clause filters OUT plans with recent activity BEFORE applying ROW_NUMBER().

Example:

User A's plans:
- Plan 1: 10 days inactive → HAVING passes → included
- Plan 2: 5 days inactive  → HAVING passes → included
- Plan 3: 1 day active     → HAVING fails  → EXCLUDED

ROW_NUMBER() now sees only Plan 1 and Plan 2:
- Plan 2 gets rn=1 (more recent)

Result: User A is marked inactive with Plan 2 (5 days)
Reality: User A is ACTIVE (read Plan 3 yesterday)
Enter fullscreen mode Exit fullscreen mode

Actual production data:

  • Wrong query: 71,682 users flagged as inactive
  • Correct query: 56,671 users actually inactive
  • Difference: 15,011 false positives!

Attempt 2: Correct Logic (✅)

-- ✅ Correct: Check inactivity AFTER finding most recent activity
WITH LastReadPerMember AS (
  SELECT 
    sd.member_seq,
    sd.plan_list_idx,
    pm.idx AS plan_member_idx,
    MAX(sd.regdate) AS last_read_date,
    ROW_NUMBER() OVER (
      PARTITION BY sd.member_seq 
      ORDER BY MAX(sd.regdate) DESC  -- Most recent first
    ) AS rn
  FROM bible_schedule_sync_data_list sd
  INNER JOIN bible_schedule_plan_member_list pm
    ON sd.member_seq = pm.member_seq
    AND sd.plan_list_idx = pm.plan_list_idx
  WHERE sd.delete_yn = 'N'
    AND pm.delete_yn = 'N'
    AND pm.state = 2
  GROUP BY sd.member_seq, sd.plan_list_idx, pm.idx
  -- No HAVING clause here!
)
SELECT 
  member_seq,
  plan_list_idx,
  plan_member_idx,
  last_read_date
FROM LastReadPerMember
WHERE rn = 1  -- Get most recent plan per user
  AND DATEDIFF(day, last_read_date, GETDATE()) >= @0  -- Check inactivity here!
ORDER BY member_seq;
Enter fullscreen mode Exit fullscreen mode

How It Works:

  1. Find all active plans for all users (no date filtering yet)
  2. Get last read date for each plan
  3. Rank by recency per user (PARTITION BY member_seq ORDER BY MAX(regdate) DESC)
  4. Select most recent plan per user (WHERE rn = 1)
  5. Check inactivity on that most recent plan only

Result:

User A:
- Plan 1: 10 days, rn=3
- Plan 2: 5 days, rn=2  
- Plan 3: 1 day, rn=1   ← Selected
- Check: 1 day < 7 days → NOT included ✓

User B:
- Plan 1: 8 days, rn=1  ← Selected  
- Check: 8 days >= 7 days → INCLUDED ✓
Enter fullscreen mode Exit fullscreen mode

Verified: Matches the independent query that counts DISTINCT member_seq (56,671 users).

The Critical Piece: Personalized Deep-Links

The Problem: Generic vs Personalized

Generic approach (wrong for multi-plan users):

// ❌ All users get the same FCM data
const message = {
  token: user.push_token,
  notification: {
    title: "Come back to Bible reading!",
    body: "It's been a while since your last reading."
  },
  data: {
    division: "screen",
    screen_name: "ReadPlanHome"  // Opens generic screen
  }
};
Enter fullscreen mode Exit fullscreen mode

Problem: User clicks notification → Opens generic plan list → User confused ("Which plan was I reading?")

The Solution: Individual Plan Data

Each user should get their specific plan's deep-link:

// ✅ Each user gets personalized FCM data
const messages = chunk.map((token, idx) => {
  const memberSeqIndex = chunkIndex * params.chunkSize + idx;
  const memberSeq = params.memberSeqs[memberSeqIndex];
  const planListIdx = params.planListIdxs[memberSeqIndex];      // ← User's plan
  const planMemberIdx = params.planMemberIdxs[memberSeqIndex];  // ← User's participation

  return {
    token,
    notification: {
      title: params.title,
      body: params.content,
    },
    data: {
      division: "screen",
      screen_name: "ReadPlanHome",
      plan_idx: String(planListIdx),        // ✅ Opens THIS user's plan
      plan_member_idx: String(planMemberIdx),
      member_seq: String(memberSeq),
    },
  };
});
Enter fullscreen mode Exit fullscreen mode

Result:

  • User A clicks notification → Opens Plan 3 directly (their most recent)
  • User B clicks notification → Opens Plan 1 directly (their only plan)

This is crucial for user experience in a multi-plan system!

NestJS Implementation: The Complete Service

Step 1: Module Setup

// bible-read.module.ts
@Module({
  imports: [
    ScheduleModule.forRoot(),  // Enable cron

    TypeOrmModule.forFeature([
      Member, 
      MemberNewmediaDigital,
      PushNotificationLog,
    ], 'mssqlConnection'),

    TypeOrmModule.forFeature([
      BibleScheduleSyncDataList, 
      BibleSchedulePlanMemberList
    ], 'mobileConnection'),

    BullModule.registerQueue({ name: 'push-message-queue' }),
  ],
  controllers: [BibleReadController],
  providers: [BibleReadService],
})
export class BibleReadModule {}
Enter fullscreen mode Exit fullscreen mode

Step 2: Configurable Constants

// bible-read.service.ts
@Injectable()
export class BibleReadService {
  private readonly logger = new Logger(BibleReadService.name);

  // ===== Configuration =====
  private readonly CRON_ENABLED = true;
  private readonly DEFAULT_INACTIVE_DAYS = 7;  // ← Adjustable!
  private readonly DEFAULT_TITLE = 'Come back to Bible reading!';
  private readonly DEFAULT_CONTENT = 'Continue your spiritual journey today.';

  constructor(
    @InjectRepository(Member, 'mssqlConnection')
    private readonly memberRepository: Repository,

    @InjectRepository(PushNotificationLog, 'mssqlConnection')
    private readonly pushLogRepository: Repository,

    @InjectDataSource('mobileConnection')
    private readonly mobileDataSource: DataSource,

    @InjectQueue('push-message-queue')
    private readonly pushQueue: Queue,

    @Inject('FIREBASE_ADMIN')
    private readonly firebaseApp: admin.app.App,
  ) {}
}
Enter fullscreen mode Exit fullscreen mode

Step 3: Cron Job Scheduler

/**
 * Cron Job: Daily Inactive User Reminder
 * Schedule: Every day at 9:00 AM KST
 * 
 * To change schedule, modify the cron expression:
 * '0 9 * * *'  = 9:00 AM daily
 * '30 14 * * *' = 2:30 PM daily
 * '0 6 * * 1'   = 6:00 AM every Monday
 */
@Cron('0 9 * * *', {
  name: 'daily-bible-read-reminder',
  timeZone: 'Asia/Seoul',
})
async handleDailyInactiveReminder() {
  if (!this.CRON_ENABLED) {
    this.logger.log('[Cron] Reminder disabled');
    return;
  }

  try {
    this.logger.log('[Cron] ===== Daily Reminder Started =====');
    this.logger.log(`[Cron] Execution time: ${new Date().toLocaleString('ko-KR')}`);

    const dto: SendInactiveReminderDto = {
      title: this.DEFAULT_TITLE,
      content: this.DEFAULT_CONTENT,
      inactive_days: this.DEFAULT_INACTIVE_DAYS,
      chunkSize: 500,
      chunkDelay: 2000,
      data: {
        division: 'screen',
        screen_name: 'ReadPlanHome',
      },
    };

    await this.sendInactiveReminder(dto, 'CRON_JOB');

    this.logger.log('[Cron] ===== Daily Reminder Completed =====');
  } catch (error) {
    this.logger.error('[Cron] Execution failed:', error);
  }
}
Enter fullscreen mode Exit fullscreen mode

Step 4: Database Query (The Core Logic)

private async getInactiveUsersWithTokens(
  inactiveDays: number,
): Promise<Array> {
  this.logger.log(`[getInactiveUsers] Querying ${inactiveDays}-day inactive users`);

  // Step 1: Find inactive users with their most recent plan
  const query = `
    WITH LastReadPerMember AS (
      SELECT 
        sd.member_seq,
        sd.plan_list_idx,
        pm.idx AS plan_member_idx,
        MAX(sd.regdate) AS last_read_date,
        ROW_NUMBER() OVER (
          PARTITION BY sd.member_seq 
          ORDER BY MAX(sd.regdate) DESC
        ) AS rn
      FROM bible_schedule_sync_data_list sd
      INNER JOIN bible_schedule_plan_member_list pm
        ON sd.member_seq = pm.member_seq
        AND sd.plan_list_idx = pm.plan_list_idx
      WHERE sd.delete_yn = 'N'
        AND pm.delete_yn = 'N'
        AND pm.state = 2
      GROUP BY sd.member_seq, sd.plan_list_idx, pm.idx
    )
    SELECT 
      member_seq,
      plan_list_idx,
      plan_member_idx,
      last_read_date
    FROM LastReadPerMember
    WHERE rn = 1
      AND DATEDIFF(day, last_read_date, GETDATE()) >= @0
    ORDER BY member_seq
  `;

  const inactiveMembers = await this.mobileDataSource.query(query, [inactiveDays]);

  this.logger.log(`[getInactiveUsers] Found ${inactiveMembers.length} inactive users`);

  if (inactiveMembers.length === 0) return [];

  // Step 2: Get push tokens (paginated for large datasets)
  const memberSeqs = inactiveMembers.map(m => m.member_seq);
  const tokenToDataMap = new Map();

  let page = 0;
  const PAGE_SIZE = 10000;

  while (page * PAGE_SIZE < memberSeqs.length) {
    const seqChunk = memberSeqs.slice(page * PAGE_SIZE, (page + 1) * PAGE_SIZE);

    const members = await this.memberRepository
      .createQueryBuilder('member')
      .select(['member.seq', 'member.push_token'])
      .where('member.seq IN (:...seqs)', { seqs: seqChunk })
      .andWhere('member.push_onoff = :onoff', { onoff: 'Y' })
      .andWhere('member.push_token IS NOT NULL')
      .andWhere("member.push_token != ''")
      .getMany();

    for (const member of members) {
      const inactiveMember = inactiveMembers.find(m => m.member_seq === member.seq);
      if (inactiveMember && member.push_token) {
        tokenToDataMap.set(member.push_token, {
          member_seq: member.seq,
          last_read_date: inactiveMember.last_read_date,
          plan_list_idx: inactiveMember.plan_list_idx,
          plan_member_idx: inactiveMember.plan_member_idx,
        });
      }
    }

    page++;
    await delay(100);  // Prevent DB overload
  }

  const result = Array.from(tokenToDataMap.entries()).map(([token, data]) => ({
    push_token: token,
    ...data,
  }));

  this.logger.log(`[getInactiveUsers] Final: ${result.length} users with valid tokens`);

  return result;
}
Enter fullscreen mode Exit fullscreen mode

Step 5: Personalized FCM Payload

async sendInactiveReminderNotifications(params: {
  jobId: string;
  title: string;
  content: string;
  chunkSize: number;
  chunkDelay: number;
  pushTokens: string[];
  memberSeqs: number[];
  planListIdxs: number[];      // ← Individual plan IDs
  planMemberIdxs: number[];    // ← Individual participation IDs
  inactiveDays: number;
}): Promise {
  const uniqueTokens = Array.from(new Set(params.pushTokens));
  const chunks = chunkArray(uniqueTokens, params.chunkSize);

  this.logger.log(`[FCM] Sending to ${uniqueTokens.length} users in ${chunks.length} chunks`);

  let totalSent = 0;
  let totalFailed = 0;

  for (let chunkIndex = 0; chunkIndex < chunks.length; chunkIndex++) {
    const chunk = chunks[chunkIndex];

    if (chunkIndex > 0) {
      await delay(params.chunkDelay);
    }

    const messaging = this.firebaseApp.messaging();

    // ✅ Build personalized messages
    const messages = chunk.map((token, idx) => {
      const globalIdx = chunkIndex * params.chunkSize + idx;
      const memberSeq = params.memberSeqs[globalIdx];
      const planIdx = params.planListIdxs[globalIdx];        // ← This user's plan
      const planMemberIdx = params.planMemberIdxs[globalIdx]; // ← This user's participation

      return {
        token,
        notification: {
          title: params.title,
          body: params.content,
        },
        data: {
          division: 'screen',
          screen_name: 'ReadPlanHome',
          member_seq: String(memberSeq),
          plan_idx: String(planIdx),              // ✅ Personalized!
          plan_member_idx: String(planMemberIdx), // ✅ Personalized!
        },
      };
    });

    // Send via FCM with retry logic
    const response = await sendEachWithRetry(messaging, messages, false, {
      maxRetries: 3,
      initialDelayMs: 1000,
      maxDelayMs: 5000,
    });

    totalSent += response.successCount;
    totalFailed += response.failureCount;

    // ✅ Save logs asynchronously
    await saveBibleReadInactiveReminderLogs(
      this.pushLogRepository,
      params.jobId,
      params.inactiveDays,
      params.title,
      params.content,
      messages,
      response,
      chunkIndex,
      params.chunkSize,
      params.chunkDelay,
      false  // isDryRun
    );

    this.logger.log(
      `[FCM] Chunk ${chunkIndex + 1}/${chunks.length}: ${response.successCount}/${chunk.length} sent`
    );
  }

  this.logger.log(`[FCM] Total: ${totalSent} sent, ${totalFailed} failed`);

  return totalFailed === 0;
}
Enter fullscreen mode Exit fullscreen mode

Testing and Verification

Manual Trigger API (For Testing)

// bible-read.controller.ts
@Post('trigger-daily-reminder')
@Public()
async triggerDailyReminder() {
  this.logger.log('[Manual] Triggering daily reminder');

  await this.bibleReadService.handleDailyInactiveReminder();

  return {
    statusCode: 200,
    message: 'Daily reminder executed manually',
  };
}
Enter fullscreen mode Exit fullscreen mode

Test in development:

# Don't wait for cron - trigger immediately
curl -X POST http://localhost:3002/bible-read/trigger-daily-reminder
Enter fullscreen mode Exit fullscreen mode

Count API (Preview Recipients)

@Get('inactive-users/count')
async getInactiveUsersCount(
  @Query('inactive_days') inactiveDays: number = 7,
) {
  const result = await this.bibleReadService.getInactiveUsersCount(inactiveDays);

  return {
    statusCode: 200,
    data: {
      inactive_users_count: result.inactive_users_count,
      inactive_days: inactiveDays,
    },
  };
}
Enter fullscreen mode Exit fullscreen mode

Test before production:

# Check how many users would receive notifications
curl "http://localhost:3002/bible-read/inactive-users/count?inactive_days=7"

# Response:
{
  "statusCode": 200,
  "data": {
    "inactive_users_count": 56671,
    "inactive_days": 7
  }
}
Enter fullscreen mode Exit fullscreen mode

Production Results

After deploying to production with 900,000+ registered users:

Metric Result Notes
Query Performance < 3 seconds Cursor pagination + indexed queries
Daily Reminders Sent ~56,000 7-day inactive users
One-User-One-Push Rate 100% Verified via logs
Personalized Deep-Link Rate 100% All users get their plan_idx
False Positive Rate 0% Corrected SQL logic
Notification Delivery Rate 98.5% FCM standard delivery rate
User Engagement Lift +23% Compared to generic reminders

Database Query Verification:

-- Verification query: Should match service count
WITH LastReadPerMember AS (
  SELECT 
    sd.member_seq,
    MAX(sd.regdate) AS last_read_date,
    ROW_NUMBER() OVER (
      PARTITION BY sd.member_seq 
      ORDER BY MAX(sd.regdate) DESC
    ) AS rn
  FROM bible_schedule_sync_data_list sd
  INNER JOIN bible_schedule_plan_member_list pm
    ON sd.member_seq = pm.member_seq
    AND sd.plan_list_idx = pm.plan_list_idx
  WHERE sd.delete_yn = 'N'
    AND pm.delete_yn = 'N'
    AND pm.state = 2
  GROUP BY sd.member_seq, sd.plan_list_idx
)
SELECT COUNT(*) AS inactive_count
FROM LastReadPerMember
WHERE rn = 1
  AND DATEDIFF(day, last_read_date, GETDATE()) >= 7;

-- Result: 56,671 ✓ (matches service count)
Enter fullscreen mode Exit fullscreen mode

Key Implementation Patterns

Pattern 1: SQL Window Functions for Multi-Plan Users

-- Use ROW_NUMBER() to rank activities per user
ROW_NUMBER() OVER (
  PARTITION BY sd.member_seq      -- Group by user
  ORDER BY MAX(sd.regdate) DESC   -- Most recent first
) AS rn

-- Then filter: WHERE rn = 1 (most recent only)
Enter fullscreen mode Exit fullscreen mode

Why This Matters:

  • Handles users with 1 plan (simple case)
  • Handles users with 10+ plans (complex case)
  • Single query for all scenarios

Pattern 2: Parameterized Inactivity Threshold

// Configurable constant
private readonly DEFAULT_INACTIVE_DAYS = 7;

// Parameterized SQL
const query = `... WHERE DATEDIFF(day, last_read_date, GETDATE()) >= @0`;

// Pass value at runtime
await this.mobileDataSource.query(query, [inactiveDays]);
Enter fullscreen mode Exit fullscreen mode

Benefits:

  • Easy A/B testing (5 days vs 7 days vs 10 days)
  • No code changes needed
  • SQL injection safe

Pattern 3: Personalized FCM Data Arrays

// Parallel arrays for individual data
{
  pushTokens: ['token1', 'token2', 'token3'],
  memberSeqs: [123, 456, 789],
  planListIdxs: [10, 20, 30],        // ← User 123's plan is 10
  planMemberIdxs: [100, 200, 300],   // ← User 123's participation is 100
}

// Map by index
const messages = chunk.map((token, idx) => {
  const globalIdx = chunkIndex * chunkSize + idx;
  return {
    token,
    data: {
      plan_idx: String(planListIdxs[globalIdx]),  // ← Individual!
    },
  };
});
Enter fullscreen mode Exit fullscreen mode

Alternative (rejected): Map of maps

// ❌ More memory, harder to chunk
{
  'token1': { member_seq: 123, plan_idx: 10, ... },
  'token2': { member_seq: 456, plan_idx: 20, ... },
}
Enter fullscreen mode Exit fullscreen mode

Pattern 4: Dry-Run Mode for Testing

const isDryRun = true;  // ← Toggle for testing

const response = await sendEachWithRetry(
  messaging,
  messages,
  isDryRun,  // ← FCM validates but doesn't send
);

// Test the full pipeline without spamming users
Enter fullscreen mode Exit fullscreen mode

Lessons Learned

1. SQL Window Functions Are Your Friend

When you need "per-group rankings" (e.g., most recent activity per user), ROW_NUMBER() OVER (PARTITION BY ...) is the elegant solution. Alternative approaches (subqueries, multiple joins) are harder to read and slower.

2. HAVING vs WHERE Placement Matters

-- ❌ Filters BEFORE ranking (wrong)
GROUP BY user
HAVING last_date >= 7
THEN rank by date

-- ✅ Ranks FIRST, filters AFTER (correct)
GROUP BY user
THEN rank by date
WHERE rn = 1 AND last_date >= 7
Enter fullscreen mode Exit fullscreen mode

Rule of thumb: Apply filters on derived columns (like rn) in the outer query, not in HAVING.

3. Personalization At Scale Requires Planning

Sending 50,000 personalized notifications means:

  • Storing 50,000 plan_idx values in memory
  • Mapping them correctly during chunking
  • Verifying index alignment (easy to get off-by-one errors)

Test with small datasets first (10-100 users) before scaling.

4. Cron Expressions Are Powerful But Subtle

'0 9 * * *'   // 9:00 AM daily
'30 14 * * *' // 2:30 PM daily  
'0 6 * * 1'   // 6:00 AM every Monday
'0 */2 * * *' // Every 2 hours
Enter fullscreen mode Exit fullscreen mode

Pro tip: Use crontab.guru to validate expressions.

5. Database Logging Is Non-Negotiable

Every sent notification should be logged to the database with:

  • member_seq (who received it)
  • job_id (which batch)
  • is_success (delivered or failed)
  • sent_at (when)
  • error_message (if failed)

This enables:

  • Analytics (engagement rates)
  • Debugging (why did User X not receive it?)
  • Compliance (proof of delivery)

6. Dry-Run Mode Saves Production Incidents

Before enabling cron in production:

  1. Set isDryRun = true
  2. Trigger manually via API
  3. Verify logs show correct count
  4. Verify FCM returns dryRun: true in response
  5. Only then enable isDryRun = false

Real story: My dry-run test showed 71,682 users instead of expected 56,671. Caught the SQL bug before spamming 15,000 users!

Scalability Considerations

Current Architecture (Synchronous Cron)

Cron (9 AM) → Query DB → Send via BullMQ → FCM
              ↓ 3s      ↓ Queue        ↓ 2-3 min
Enter fullscreen mode Exit fullscreen mode

Pros:

  • Simple architecture
  • Easy to debug
  • Sufficient for 50K-100K users

Cons:

  • Cron blocks until query completes
  • Single point of failure

Future Architecture (Async Queue)

For 500K+ users, consider decoupling query from cron:

Cron (9 AM) → Add "query job" to queue
               ↓
            Worker queries DB → Adds "send jobs" to queue
                                 ↓
                              Workers send via FCM
Enter fullscreen mode Exit fullscreen mode

Benefits:

  • Cron completes in <1 second
  • Query can take minutes without blocking
  • Horizontal scaling (more workers)

Implementation sketch:

@Cron('0 9 * * *')
async handleDailyReminder() {
  // Just schedule the query job
  await this.pushQueue.add('query-inactive-users', {
    inactiveDays: 7,
  });

  this.logger.log('[Cron] Query job scheduled');
}

// Separate worker processes the query
case 'query-inactive-users': {
  const users = await this.bibleReadService.getInactiveUsers(data.inactiveDays);

  // Add individual send jobs
  for (const userChunk of chunkArray(users, 500)) {
    await this.pushQueue.add('send-bible-reminder', {
      users: userChunk,
    });
  }
  break;
}
Enter fullscreen mode Exit fullscreen mode

Conclusion

Building an automated Bible reading reminder system taught me that the devil is in the business logic. What seemed like a straightforward "send push to inactive users" requirement turned into a fascinating journey through:

  • SQL window functions for complex multi-plan logic
  • Personalized FCM payloads for better user experience
  • The critical importance of "one person, one push"
  • Parameterized queries for flexibility
  • Dry-run testing for confidence

The final system now runs daily at 9 AM KST, automatically identifying ~56,000 inactive users, sending each exactly one notification with their personalized deep-link, and logging every delivery—all with zero manual intervention.

The key insight: In a multi-plan system, "inactive" doesn't mean "hasn't used Plan A"—it means "hasn't used ANY plan." SQL window functions (ROW_NUMBER() OVER (PARTITION BY ...)) elegantly solve this by finding the most recent activity across all plans per user, then checking inactivity on that single timestamp.

By carefully structuring the SQL logic (rank first, filter second), leveraging TypeScript's type safety, and building in observability (logs, dry-run mode), I built a reminder system that's both robust and maintainable.

In the next part of this series, I'll explore how I optimized FCM delivery for 900,000+ tokens using cursor-based pagination, deduplicated tokens with Redis, and handled rate limits gracefully.

Key Takeaways

  • Use SQL window functions (ROW_NUMBER() OVER (PARTITION BY ...)) for per-group rankings
  • In multi-entity systems, define "inactive" carefully (most recent across ALL entities)
  • Always filter on derived columns (like rn) in the outer query, not in HAVING
  • Personalized data (plan_idx) significantly improves user engagement (+23% in our case)
  • Parameterize everything (inactive days, cron schedule, chunk size)
  • Implement dry-run mode and test thoroughly before production
  • Log every notification for debugging and analytics
  • Database query optimization matters at scale (cursor pagination, indexes)

Top comments (0)