People of ACM - Amey Dharwadker

March 4, 2025

How did you initially become interested in recommender systems for videos?

My journey into video recommender systems began during my early days at Meta more than a decade ago. I became fascinated by the unique challenges of video recommendations when I joined as one of the first machine learning engineers on Facebook's video ranking initiative. What captivated me was the multifaceted nature of video ranking—it wasn't just about analyzing user preferences, but about comprehending the intricate interplay between visual content, video engagement patterns, and user interests at an unprecedented scale.

The technical complexity of serving personalized video recommendations to billions of users, combined with the immediate impact on how people discover and consume content was intellectually stimulating. I was particularly drawn to the challenge of building systems that could tackle fascinating problems including multi-task learning, neural network architecture design, and bias mitigation effectively while directly influencing how billions of people globally experience digital content.

From a bird's eye perspective, what is the most important way in which machine learning techniques have improved recommendation systems?

I believe the most transformative improvement in large-scale recommendation systems has been our ability to model complex user-content interactions through advanced neural network architectures. The shift from traditional collaborative filtering to sophisticated multi-task neural networks has fundamentally changed how we capture latent user preferences and content relationships in high-dimensional spaces, delivering highly personalized and contextually relevant recommendations.

Equally impactful is the shift toward leveraging sequential engagement data. Modern ranking models incorporate the temporal patterns of user behavior, learning from both the order and timing of interactions to predict future preferences. Analyzing how a user transitions between different types of content, allows systems to uncover evolving interests and adapt in real time, delivering recommendations that feel intuitive and timely.

Your paper "CAM2: Conformity-Aware Multi-Task Ranking Model for Large-Scale Recommender Systems" notes that not all popularity biases are necessarily bad. For example, viral content could be genuinely relevant. How did you approach this nuanced challenge when designing CAM2's architecture?

The challenge of distinguishing helpful from unhelpful popularity signals was central to our approach in developing CAM2. Instead of eliminating popularity bias entirely, we focused on separating users' conformity—driven by trends or engagement patterns—from their genuine interests through key architectural innovations.

The model was designed to process statistical interaction features like view counts and engagement rates separately from content-based features such as video attributes and user preferences. This was achieved by creating two specialized modules: one for conformity and another for relevance. The conformity module analyzed engagement-driven patterns, while the relevance module focused on aligning content with user preferences.

To ensure these modules learned independently, we introduced a novel loss formulation. This allowed the model to capture distinct conformity and relevance signals without interference. For example, when a viral video genuinely matched a user’s interests, CAM2 could recognize both its popularity and relevance.

This approach enabled CAM2 to recommend relevant content while preserving the visibility of viral content that authentically resonated with users, balancing personalization and popularity dynamics effectively.

What is an ongoing challenge in your field of billion-user scale recommender systems, and how might this be surmounted in the near future?

An ongoing challenge in billion-user scale recommender systems is balancing model sophistication with computational efficiency. Advanced architectures like transformers offer exceptional personalization but demand significant computational resources, especially at scale, where latency and cost are critical.

Techniques like knowledge distillation, model quantization and hardware acceleration—areas where my team has made notable progress—are essential. Distillation simplifies complex models into lightweight versions without sacrificing performance, while quantization reduces costs using low-precision data types for efficient inference. Specialized hardware such as GPUs and TPUs combined with distributed training frameworks further enable efficient scaling. Continued innovation in these areas is crucial to ensure efficient, scalable and accessible systems.

As co-organizer of the  VideoRecsys workshop at the ACM RecSys conference, what motivated you to create this forum focusing on video recommendation systems and how has the workshop evolved since its inception?

The motivation for creating the VideoRecSys workshop came from recognizing the unique challenges in large-scale video recommendation systems, often overlooked in broader recommender systems discussions. Having worked extensively on video recommendations, I saw the need for a dedicated forum to explore issues like processing libraries containing billions of videos, understanding complex user-content interactions and balancing diverse multi-stakeholder needs amid the rapid growth of video platforms.

Since its inception in 2023, the workshop has grown significantly. The first edition drew over 200 attendees and validated the strong demand for such a platform. In 2024, we expanded by combining with LargeRecSys, broadening the scope to include large-scale recommender systems while retaining a focus on video-specific challenges. We also introduced paper and poster sessions to encourage contributions from the broader community.

Today, our VideoRecSys workshop has evolved into a vibrant platform for researchers, practitioners, and industry leaders to collaboratively tackle the complexities of building responsible, scalable, and engaging video recommendation systems.