APIE Project: SWIMM swimming in my music

As access to digital music catalogs become widespread and personal music collections grow the possibilities for personalized playlists and even personal radio stations also grows. But as is the case with everything else in the information age, the growing amount of data comes with a correspondingly growing burden of organizing and dealing with all that data.

In our initial work and user studies, we have realized two things. First, users aren't very good at describing the sort of music they want to listen to. Second, users are at least as interested (if not moreso) in having radio stations as they are in having short playlists. From a modeling point of view this raises a number of interesting mathematical questions as well as user interface questions.

SWIMM (SWimming In My Music) is a system that attempts to address these issues. Among other things, it provides adaptive music streams. A stream, while analogous to a radio station, draws from a user's digital music library, preferences expressed by the user, and the user's empirical listening habits to provide an unbounded stream of music customized to the userÕs tastes, even as those tastes evolve over time.

SWIMM draws on additional results from our initial study suggesting features that affect users' listening preferences. In particular, it leverages ID3 tags, listening patterns, acquisition dates, and a wide array of signal processing features extracted from individual songs. Those signal-processing features are well-understood by music theorists and provide an underlying representation of the variety of music available to a user beyond the relatively unsophisticated and error-prone human-generated tags such as genre and artist. SWIMM then uses an information theoretic modeling technique (based on maximum entropy) and MCMC sampling to deal with the otherwise intractable problem of representing a user's preferences and generating (possibly unbounded) sequences consistent with those preferences.

The user's interaction is simple. Users may create as many streams as they wish. In creating a new music stream, the user simply selects a few sample songs. This creates an initial probability distribution over songs. Using Metropolis-Hastings, SWIMM continually updates the ordered sequence of songs to be played on the stream. While users are able to explicitly accept, reject, or rate upcoming songs, SWIMM also extracts implicit feedback as the user listens to, removes, or adds songs; reorders the list; adjusts the volume; etc. SWIMM uses that feedback to adjust its model of the appropriate songs for that stream. Furthermore, SWIMM's model captures not just the kinds of songs that are appropriate, but also the relationships that describe their relative ordering. For example, SWIMM develops a notion of how smoothly the tempo of songs should change as the sequence continues.

In addition to the software itself, we are interested in evaluating both the quality of the user experience and the efficacy of the learned model. Critical questions will be what rate of convergence to a sufficiently "good" result is necessary, what amount of feedback is required to achieve it, and how much of that feedback must be explicit. Other questions include how to structure the interface to maximize the amount of implicit feedback, what explicit and implicit feedback users provide in practice, how the balance between the two changes over time, how to effectively model distributions over infinitely many unbounded streams, and how to model distributions over sequences of objects when some of those objects do not as yet exist.