There's a body of research on this, and it suggests that ratings are more meaningful if you add options, up to about 5 or 6 ratings.
That is, if you asked people to do the ratings once, and then asked them 1 hour later, there would be more consistency across time as you add options from 2 to 3 to 4, up to about 5 or 6.
The problem with binary ratings is that, as much as you might think otherwise, you're forcing a kind of hazy, grey experiential assessment into 0 or 1. And in doing so, people near the boundary (whatever that might be) will vacillate between them. E.g., people who feel "meh" about something are forced to choose something else, and sometimes they'll say 0 and sometimes 1. The more options you give, the more reliable / meaningful the ratings will be.
This example is interesting to me because it's something most people can relate to and illustrates the complications of utility-based and Bayesian formulations of the problem. You end up having to decide on utilities and/or priors.
To me the answer is to weight the data maximally in forming a posterior, in which case you end up using a reference prior. Similar kinds of arguments about utilities lead to reference priors. Reference priors can be complicated to compute, but for things like multinomials over ordinal ratings, reference priors have been worked out fairly well.
To me it always made sense to allow people to sort by the center of the estimate, or the lower bound (maybe using different language).
I think 1-4 stars is the ideal rating style. I wish that were used more often.
A choice of 1-4 stars gives you enough freedom to express your opinion, without being overwhelming. It's a small enough range to be reasonably objective (almost everybody will interpret it as 1 star = bad, 2 = passable, 3 = good, 4 = great). And with an even number of choices there's no middle "meh" option -- you're forced to make a choice between 2 and 3.
Of course it's important not to ruin it by adding extra options, like 0 stars or half-stars. (That was Ebert's big mistake!)
Edit to add: to relate this to the parent post, I'm thinking that maybe ranking things as 1-4 stars in several categories could be the best if both worlds.
The rationale that seems to be put forth now is that there's so much competition between mediums that it's ok to allow monopolies of each one of them.
I think this is absurd of course, but that's the argument that's always made implicitly or explicitly.
It's also scary as hell because ISPs have disproportionate control.
And once they're granted all sorts of freedoms to screw over consumers, it's going to be a lot more difficult to take it away from them.
All Pai has to do is set the ball rolling and it won't stop.
I don't really see a political landscape in the near future where you have anti-monopoly laws being enforced. All the current corporate monopoly whores will have to die of old age first.
That is, if you asked people to do the ratings once, and then asked them 1 hour later, there would be more consistency across time as you add options from 2 to 3 to 4, up to about 5 or 6.
The problem with binary ratings is that, as much as you might think otherwise, you're forcing a kind of hazy, grey experiential assessment into 0 or 1. And in doing so, people near the boundary (whatever that might be) will vacillate between them. E.g., people who feel "meh" about something are forced to choose something else, and sometimes they'll say 0 and sometimes 1. The more options you give, the more reliable / meaningful the ratings will be.
This example is interesting to me because it's something most people can relate to and illustrates the complications of utility-based and Bayesian formulations of the problem. You end up having to decide on utilities and/or priors.
To me the answer is to weight the data maximally in forming a posterior, in which case you end up using a reference prior. Similar kinds of arguments about utilities lead to reference priors. Reference priors can be complicated to compute, but for things like multinomials over ordinal ratings, reference priors have been worked out fairly well.
To me it always made sense to allow people to sort by the center of the estimate, or the lower bound (maybe using different language).