Feedback loops and exploration

I’ve touted the virtues of feedback loops on this blog before. So I found it surprising to read a blog post on “Technical Debt in Machine Learning” and find feedback loops listed as an anti-pattern (post based on NIPS papers from 2014 and 2015).

Thinking about this assertion more, I’d rephrase the concern to be around the most straightforward (exploit-only) feedback loop’s tendency to quickly “collapse” the space of possible predictions for a user.

This is actually something I was thinking about during the task extraction prototype… for example, if we eventually got this great feedback loop going where users are telling us whether they agreed or disagreed with a suggested task, we could eventually end up with too narrow a definition of a task, and the system could stop learning anything useful (hence the importance of exploring while you exploit 😄).