Welcome, Guest! Why not create a free forum account today and join in with the world's friendliest bunch of Kindle enthusiasts

Amazon's change to 'highly rated' book lists. Writers and readers beware.

Thread Rating:
  • 0 Votes - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
 
Offline TresaC Reading Submissions! A N Other
13 Jun 2012, 04:15 PM | Post: #11

Accidental Publisher
******

Posts: 754
Joined: Sep 2011

Thanked 5 times

What I Read

RE: Amazon's change to 'highly rated' book lists. Writers and readers beware.

(13 Jun 2012 03:57 PM)Rosen Trevithick Wrote:  I've got into a debate with mathematicians about the statistics behind this. Uh oh...

But we are all very impressed, Rosen! Smile

Do you think it is a case of the new guy on the team has just finished a degree in statistics and is trying to say 'look at what I can do.'
Offline Rosen Trevithick Reading
13 Jun 2012, 04:49 PM | Post: #12

Totally Splendid Hotshot Author
********

Posts: 5,201
Joined: Oct 2011

Thanked 278 times

What I Read

RE: Amazon's change to 'highly rated' book lists. Writers and readers beware.

(13 Jun 2012 04:15 PM)TresaC Wrote:  
(13 Jun 2012 03:57 PM)Rosen Trevithick Wrote:  I've got into a debate with mathematicians about the statistics behind this. Uh oh...
Do you think it is a case of the new guy on the team has just finished a degree in statistics and is trying to say 'look at what I can do.'

No, it's entirely my fault. I thought 'there's a maths problem I wish I knew the answer to' and it escalated.
[Image: 51we%2BNvX2aL._SL95.jpg]  [Image: 51dlN%2BeBifL._SL95.jpg]  [Image: 51Y3bTpE8eL._SL95.jpg]  [Image: 51gmgMPfnBL._SL95.jpg]  [Image: 51qdNKyGE9L._SL95.jpg]  [Image: 61TE0dQyxjL._SL95.jpg]   [Image: 51Fbvd7KyWL._SL95.jpg]  [Image: 51YepJnT-dL._SL95.jpg]
Offline TresaC Reading Submissions! A N Other
13 Jun 2012, 05:17 PM | Post: #13

Accidental Publisher
******

Posts: 754
Joined: Sep 2011

Thanked 5 times

What I Read

RE: Amazon's change to 'highly rated' book lists. Writers and readers beware.

(13 Jun 2012 04:49 PM)Rosen Trevithick Wrote:  
(13 Jun 2012 04:15 PM)TresaC Wrote:  
(13 Jun 2012 03:57 PM)Rosen Trevithick Wrote:  I've got into a debate with mathematicians about the statistics behind this. Uh oh...
Do you think it is a case of the new guy on the team has just finished a degree in statistics and is trying to say 'look at what I can do.'

No, it's entirely my fault. I thought 'there's a maths problem I wish I knew the answer to' and it escalated.

Your homework for tonight is.... hee hee Smile
Offline Notoriety Reading Five Days in May by Andrew Adonis
13 Jun 2012, 09:34 PM | Post: #14

Posting Freak
******

Posts: 643
Joined: May 2011

Thanked 2 times

What I Read

RE: Amazon's change to 'highly rated' book lists. Writers and readers beware.

This post was last modified: 13 Jun 2012 09:47 PM by Notoriety.
(13 Jun 2012 03:00 PM)Rosen Trevithick Wrote:  That's statistically invalid.

They should apply mathematical tests to check for confidence and rank books by the probability that a book's quality matches 5 stars. This would take the number of reviews into consideration.

Getting a bit nerdy and as your maths mates may have said - the first bit is not about probability it's about test "validity". In this case does the Amazon star rating system measure "quality" - whatever that is? Answer: very much no. It's a measure of customer satisfaction which is not at all the same thing. The second question is the one of non-random sampling and response rate and is faced by various sites, famously TripAdvisor, but most notably Ebay where customer confidence in satisfaction measures is the most crucial aspect of the site. The problem is that on Amazon the responders are self-selected and can be any old people with the vast majority of book readers never bothering to post a review. You can post a review without even having read the book, shock, horror. Ebay on the other hand works hard to get high rates of response by asking both sides of the transaction to rate each other, including whether they have given a rating, which makes their USP so strong. If memory serves it even gives the rating as percent of total purchasers, not just responders. So authors would not want "over all the sales of this book the average rating is 0.0001 star" and neither on the other hand can they reciprocate by rating readers - " One star, cr*p reader, no taste at all"- however much they may want to Wink.

So whatever algorithm you create it will always be limited by the responders and at best might just even things up a tiny bit. In any statistical discussion Amazon's system would not be worth the paper it isn't written on and talk of "probability/confidence intervals" in the calculations is a complete waste of time as the survey methodology is so rubbish. No more use than the current TV ad that says that 96 out of 114 women asked preferred Baldo shampoo. (That shouldn't stop Lou or others writing algorithms - as long as you get well paid for your troubles!)

The other difficulty which has so upset Steve and others is Amazon overstating the significance of the "results" and thereby affecting authors' sales. Amazon couldn't care less about your sales - it's their sales that matter to them.

Readers should see the star system as utterly worthless and potentially misleading. The most we can do is read the book description and the actual reviewers' comments, apply a large quantity of salt and hope for the best. Sales figures may be a slightly better guide but not much. Authors should take 4 and 5 star ratings as gospel/scientific truth, depending on preference, and totally ignore the others as biased and unreliable.

Tony
Arab proverb: Only a fool lends his books and only a fool returns them.
Offline Rosen Trevithick Reading
13 Jun 2012, 09:57 PM | Post: #15

Totally Splendid Hotshot Author
********

Posts: 5,201
Joined: Oct 2011

Thanked 278 times

What I Read

RE: Amazon's change to 'highly rated' book lists. Writers and readers beware.

Thanks Tony. I am sure I will find your response fascinating when I re-read it tomorrow morning. At 11pm, it's a bit daunting.

I see what you're saying about the star rating system being misleading in general, but some ways of presenting the 'data' are more misleading than others.
[Image: 51we%2BNvX2aL._SL95.jpg]  [Image: 51dlN%2BeBifL._SL95.jpg]  [Image: 51Y3bTpE8eL._SL95.jpg]  [Image: 51gmgMPfnBL._SL95.jpg]  [Image: 51qdNKyGE9L._SL95.jpg]  [Image: 61TE0dQyxjL._SL95.jpg]   [Image: 51Fbvd7KyWL._SL95.jpg]  [Image: 51YepJnT-dL._SL95.jpg]
Offline Notoriety Reading Five Days in May by Andrew Adonis
13 Jun 2012, 10:19 PM | Post: #16

Posting Freak
******

Posts: 643
Joined: May 2011

Thanked 2 times

What I Read

RE: Amazon's change to 'highly rated' book lists. Writers and readers beware.

Oh dear, Rosen,

I didn't want you to worry about my comment tomorrow in that other thread. I was making some other dubious point about my highly tuned and unbiased approach to books (LOL). I will probably be a bit late getting to the discussion tomorrow - but so you sleep easy it will all be pretty positive!

All data is almost always presented in misleading ways, unfortunately. If you read read the most respected medical journals' reports of major drug trials difficult aspects of the study are played down and bits of the findings that show that new drug b is better than old drug a are talked up. You can even see where sentences are constructed to respond to previous published trials. Peer review is supposed to reduce this and does to some extent but professional consumers of research are trained to see the loopholes. It's about big money and big reputations.

Tony
Arab proverb: Only a fool lends his books and only a fool returns them.
Offline Rosen Trevithick Reading
13 Jun 2012, 10:33 PM | Post: #17

Totally Splendid Hotshot Author
********

Posts: 5,201
Joined: Oct 2011

Thanked 278 times

What I Read

RE: Amazon's change to 'highly rated' book lists. Writers and readers beware.

I guess I think that the limitations of peer reviews are largely irrelevant in a discussion about whether the books with one review should or should not be listed ahead of Steve's, because whatever algorithm they use in a table sorted by average rating, it will be subject to the same limitations.

And I'd forgotten about your comment in the other thread. I was going to sleep well but
anxiety caused by anticipating your comments of 'Seesaw' + preoccupation with the maths behind Steve's predicament
might just meet the threshold for mild insomnia. ;-)
[Image: 51we%2BNvX2aL._SL95.jpg]  [Image: 51dlN%2BeBifL._SL95.jpg]  [Image: 51Y3bTpE8eL._SL95.jpg]  [Image: 51gmgMPfnBL._SL95.jpg]  [Image: 51qdNKyGE9L._SL95.jpg]  [Image: 61TE0dQyxjL._SL95.jpg]   [Image: 51Fbvd7KyWL._SL95.jpg]  [Image: 51YepJnT-dL._SL95.jpg]
Offline Jennie Lee Reading Course notes for Pilates Teacher Training
14 Jun 2012, 04:12 AM | Post: #18

Aussie Kiwi
********

Posts: 2,034
Joined: Mar 2012

Thanked 156 times

What I Read

RE: Amazon's change to 'highly rated' book lists. Writers and readers beware.

In simple terms, Amazon needs to add a sub-level of criteria to their reporting query.

It seems to me they are getting the end result by querying their database for 'overall' star rating in descending order, whereas they should be factoring in the number of reviews as well.
Jennie's Website


[Image: 51XG2UNKzFL._SL95.jpg][Image: 51odD7YudkL._SL95.jpg][Image: 61wVrKkiTaL._SL95.jpg][Image: 61071hCtWXL._SL95.jpg][Image: 51IlO6c3bpL._SL95.jpg][Image: 61KsWLjj9rL._SL95.jpg][Image: 51hyuXTcVUL._SL95.jpg][Image: 41J-kkf3cQL._SL95.jpg]
Breathe, Relax, Read.
Offline LindaGruchy Reading The Chandelier Ballroom by Elizabeth Lord
14 Jun 2012, 06:13 AM | Post: #19

Queen of the Typos
********

Posts: 2,943
Joined: Jan 2012

Thanked 41 times

What I Read

RE: Amazon's change to 'highly rated' book lists. Writers and readers beware.

Quite. All they need to do is add in a weighting for each review based on the overall number of reviews. up to an upper limit of say, 100 reviews, then fine tune each of those based on the total number of reviews.

Ten 4 star reviews ought to have more significance than one 5 star review.

Do people actually look for books based on their 5 star reviews? Are we worrying unnecessarily? Surely any means of ranking has its problems?

What is important to me is that people find my books when browsing, and that's where Amazon are failing with their also-boughts. When I (rather obsessively) check my rankings my also-boughts never seem to change. When you consider that 10,000 have been downloaded, you would expect there to be lots of different permutations of those particular pages.
Offline Steve Robinson Reading
14 Jun 2012, 07:15 AM | Post: #20

Not Jefferson Tayte
*******

Posts: 1,711
Joined: Sep 2011

Thanked 85 times

What I Read

RE: Amazon's change to 'highly rated' book lists. Writers and readers beware.

I'm sure people do search for books based on how highly rated they are. Amazon provide the option so they should make those lists a true reflection of the overall opinion of its customers. I got a reply back this morning to my question where I cited the Crime & Punishment example and they've somehow misunderstood it, answering something I didn't even ask.
Steve

 [Image: 51-1lqA8V3L._SL95.jpg]    [Image: 61%2BKa%2BDUqsL._SL95.jpg]    [Image: 51QybUoiHjL._SL95.jpg]    [Image: 51OzJND69QL._SL95.jpg]    [Image: 51YouKuxEJL._SL95.jpg] 
Website: http://www.steve-robinson.me
Blog: http://www.ancestryauthor.blogspot.com
Twitter: https://twitter.com/SteveRobinson01
Facebook: https://www.facebook.com/SteveRobinsonAuthor
Amazon KDP film - 'Behind the Cover': http://youtu.be...

Currently Reading:Submissions! A N Other Last Book I Read:Conversations in the Abyss by Michael Brookes Favourite Genres:Thought provoking fiction/biographies of adventurous people
Currently Reading: Last Book I Read: Favourite Genres:Comedy; Crime, thrillers and mysteryFavourite eBooks:
See my recommendations
Currently Reading:Five Days in May by Andrew Adonis Last Book I Read:The Sense of an Ending by Julian Barnes Favourite Genres:Classics, politics, foreign language, "lit fic", funFavourite eBooks:
See my recommendations
Currently Reading:Course notes for Pilates Teacher Training Last Book I Read:Tell Me I'm Here - Anne Deveson Favourite Genres:Historical, mystery, suspense, drama, romance
Currently Reading:The Chandelier Ballroom by Elizabeth Lord Last Book I Read:Fake Kate Favourite Genres:crime. women's fiction
Currently Reading: Last Book I Read: Favourite Genres:Crime, mystery & thriller