Helpfulness Voting and the Search for Excellence

Matt Leo

On Friday, November 4, Assistant Professor of Information Science at Cornell University David Mimno visited Colgate to present a lecture titled “Trendiness, Conformity and Stackoverflow: Searching for Excellence When Everything is Connected.” During the discussion, Mimno argued that Internet consumers are strongly influenced by the various ways in which online content is presented. 

To illustrate this point, Mimno discussed how content is displayed on websites such as programming website StackOverflow.com, Amazon.com and even Ask.com. These websites use helpfulness voting, where users are able to rate comments on forums based on the comment’s overall ability to answer various questions posed. During the lecture, Mimno described a model that seeks to illuminate how helpfulness voting works and to characterize behaviors of different communities based on these votes. Ultimately, Mimno sought to find which answers and online material are truly excellent and have ratings that are not skewed by technicalities of online voting.

To develop a frame for his discussion, Mimno presented a few basic definitions of terms he used. He defined “item” as a topic of discussion, “response” as a user-submitted reaction to an item and “vote” as a user-submitted reaction to the response. With these defined terms in mind, he discussed a few examples to help contextualize what his research does.

Mimno first discussed helpfulness voting on Amazon.com, manifested as customer reviews. He posed a question about why certain reviews pop up initially, and others take a lot of effort to find. For an example, he shared online reviews of the book “A Million Random Digits”. Using this example, Mimno pointed out the 655 reviews and the four-star average of reviews. However, he discussed the quality of these reviews, and he pointed out that the top-rated reviews, the first to show up, tended to lack seriousness. 

“While the printed version is good, I would have expected the publisher to have an audiobook version as well. A perfect companion for one’s iPod,” wrote one reviewer.

“Such a terrific reference work! But with so many terrific random digits, it’s a shame they didn’t sort them to make it easier to find the one you’re looking for,” wrote another sarcastic reviewer.

This example demonstrated how insincere reviews could still be highly ranked by consumers. They may not necessarily be considered “excellent” or even “helpful” from an objective standing.

Mimno presented a couple of other examples of this and he largely focused on Stack Overflow, a website that is a very important resource for the programming community. He discussed the voting process that determines how different answers and comments are displayed, and it is a simple upvote/downvote system. He pointed out how this system is efficient in presenting the correct answers first, yet there are many ways for the information to be deviated and that the correct answer is not always the one that is accepted as such and displayed at the top.

The discussion shifted to a broader viewpoint regarding the bridge between psychology and modeling as the twenty-first century is the era of big data. Mimno discussed the different biases that arise from the way that information is presented, such as the Music Lab Study by Salganic, Dodds and Watts. This study demonstrated that people’s individual beliefs about a song are largely affected by the rankings other people have assigned to those songs. The study demonstrates Mimno’s argument that people’s opinions are hugely influenced by those of their peers.