From Hacks to Harmony: Structuring Product Rules in Recommendations
In today's data-driven landscape, recommendation systems power everything from social media feeds to e-commerce. While it's tempting to think that machine learning algorithms do all the heavy lifting, that's only half the story. Real-world systems often rely on a mix of machine learning and heuristic rules – commonly referred to as product rules, business rules, or simply hacks – to generate the most relevant recommendations.
For example:
- You can't recommend tracks from the same artist too often;
- You should include content from subscriptions in the feed, but not overwhelm it;
- If a user has already disliked a certain category or author, the related content should be penalized or even filtered out;
- Explicit content can't be recommended – except when appropriate.
Rules come in two types: hard and soft. Hard rules act as filters, prohibiting certain documents from being recommended in specific contexts; failing to comply is considered a product bug. There's nothing inherently wrong with these rules, but their number should be limited. Moreover, they should be applied as early as possible in the ranking process, either at the candidate generation stage or even during index construction. Soft rules, on the other hand, are more like guidelines: you can recommend such items, but preferably not too much (or the opposite, more is better). Having too many of these soft rules can make system debugging and development highly challenging.
Rules are technical debt.
I find that the quantity of such rules in a system often depends on the internal power dynamics within the team. Product managers usually find it convenient to express constraints through rules, while engineers typically dislike these hacks. In my previous team, I took pride in our ability to keep the number of such rules to a minimum.
Throughout my career, I've frequently encountered a recurring pattern. The engineering team struggles to train the system in a way that produces good recommendations (either overall or in specific aspects). The product team then resorts to what they know best – adding new rules. Such patches are justified when quick fixes are needed, but they're hard to remove later. The system often remains in this patched state until a major refactoring takes place, much like regular technical debt.
The moral of the story – don't skimp on hiring strong engineers
