Where do prices come from?
Bad: Crawl every page
Their server load, broken parsers, forbidding terms, a bot arms race you lose.
Good: Cheapest allowed channel
Delta feeds, extension reports, the affiliate API, polite page fetches last.
28 h
between checks if every product gets equal turns
Assume 5,000 refreshes a second, 432M a day, for 500M products. A midnight drop is noticed the next night.
A TV checked every 15 minutes drops its price. How long until phones buzz?
Where time goes
About six minutes waiting for the next check, two seconds for everything else. Freshness is a scheduling problem.
Same 5,000 a second, spent where alerts might fire
Score each product by watchers, the chance its price moved, a target close by and the sale calendar. Hot products get checked every 15 minutes.
100 million alerts. Which ones does this drop fire?
Keep each product’s armed alerts sorted by target. A drop from ₹54,999 to ₹47,999 fires exactly the targets in between: one range scan.
6M
alerts fired in a sale’s first five minutes
At 00:00 on Big Billion Days, 40,000 a second in the first minute. Pre-scale senders by the calendar, give live alerts a lane, re-check stale prices.
Never alert 200,000 people about a ₹499 TV
- Check units, stock and 90-day history
- Confirm suspicious drops through a second channel
- Huge fan-out plus deep drop? Confirm anyway
- Pause adapters whose parse failures jump
Remember
Spend a fixed fetch budget where alerts might fire. Let each price change find its alerts in one range scan. Confirm before you cry wolf.