Today I received an email containing a survey request from NPD on behalf of Rhapsody. I have been a Rhapsody customer for several years, and gift two subscriptions to others. This survey was annoying for two reasons.1. The first reason is a KBS consideration. The KBS rationale is a reminder of how poorly the recommender engines perform. Whether this is because engines are forced to act upon minimal data, or because the engines themselves are deeply flawed, or because tastes even within a single individual can be complex and variable -- is unclear. But the recommender engines at Amazon and Rhapsody never fail to suggest artists or works that I find especially objectionable. Also at work is the problem of the glaring outlier: a single terrible recommendation tends to overshadow the acceptable ones. Peer-produced playlists tend to have similar flaws: low fidelity, vaguely connected lists that contain so many over-recommended artists that the energy required to skim offsets one's motivation to discover the jewels.
2. The other reason has to do with the survey. Up one side and down the next, this poorly constructed survey by NPD tried to assess my interest in the behavior of music recommender engines: for playlists (automatic, peer-produced, professionally produced, celebrity-produced), streamed music stations, and so on. Unfortunately, since the underlying recommender engines are so poor, it's not possible to respond to the survey properly. If might have been different if they'd worded the question differently. "If the recommender engine worked to recommend tunes that you were interested in . . ." Lastly, like many surveys, there was no free form text box to feed back to the survey designer a list of perceived survey flaws and why one felt one wasn't surveyed properly. I am not unaware that such surveys are sometimes constructed with forced choice, and that interpreting free text answers is not always cost-effective, but if a natural language scoring system can be used for the SAT, one could be developed that works for surveys, especially considering the narrow domain of most surveys.
◦
0 comments:
Post a Comment