Transcript
You unlock your phone. Two hundred candidates are already waiting. The retrieval stage did its job. Now comes the harder question: in what exact order do those candidates appear? That decision happens in milliseconds. It shapes the top results the user sees. The ranking stage assigns a numeric relevance score to each candidate item, then sorts them in descending order to select the top results shown to the user. Get the order wrong, and even a perfect shortlist feels broken. The retrieval stage has completed, providing a diverse shortlist of a few hundred items to the ranker within the latency budget. The ranker's job is precision. It uses supervised learning to predict a utility signal, something like click-through rate or watch time, from user features, item features, and contextual signals. That means the model is trained on what people actually did, not what they said they wanted. Consider a user who interacts with various content types. A simple logistic regression views these interactions as flat features, while a deep ranking model interprets them as a sequence. It learns low-dimensional embeddings for users and items, then computes neural network-based scores that correlate with future interactions. The mechanism that makes this powerful is attention. A Deep Interest Network, for example, weights a user's historical interactions differently depending on the candidate item being scored. Recent behavior gets amplified. Older, less relevant history fades. Static preferences are only half the picture. Real-time adaptation means the ranker incorporates fresh contextual information: the latest user interactions, current session context, time of day, device state, and even recent negative feedback like skips. Session-based ranking models treat the short sequence of actions within a single session as the primary input, predicting the next best item in real time. Online systems must update user features on very short timescales, sometimes within seconds, so rankings reflect the latest behavior rather than yesterday's model. After the deep ranker scores items, a re-ranking layer takes over. This is where business logic enters. Re-ranking enforces diversity, content freshness, safety filters, and avoidance of near-duplicate items. [short pause] Listwise diversification algorithms intentionally lower the rank of highly similar items to avoid monotonous feeds, even at a small cost in predicted click probability. That tradeoff is deliberate. A feed of five consecutive clips from the same creator is technically high-scoring but experientially exhausting. Re-ranking corrects for that. Ranking models focused only on click-through can unintentionally amplify popularity bias, leading to under-exposure of niche content. Worse, maximizing immediate clicks can reduce long-term retention. A user who gets only sensational content eventually churns. That means production systems must balance multiple objectives simultaneously: engagement metrics like viewing duration and session length, but also return visits, freshness rate, and complaint rate. In video and media recommendation, ranking that explicitly optimizes for session length and return visits leads to improved user retention. The takeaway that ties everything together is this. The full mobile recommendation loop, from retrieval through ranking through re-ranking, must complete within a tight latency budget. Ranking models are more complex than retrieval models but still optimized for millisecond-level inference. To prevent the feed from narrowing over time, systems inject exploration, deliberately surfacing less-certain candidates so the model keeps learning. User interactions with ranked items are logged, aggregated, and fed back to update future ranking decisions. That closed feedback loop is what keeps the system alive. This concludes the Sun Team's technical briefing on mobile recommendation systems. Over to you, Greg.