stevenpiziks (![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png)
stevenpiziks) wrote2026-04-23 11:35 am
stevenpiziks) wrote2026-04-23 11:35 am
Rating Systems: Perfection or Garbage?
Can we talk rating systems?
This came up elsewhere. A short-term rental host was fishing for higher ratings by putting up a door sign that "translated" what the ratings meant. 5/5 meant nothing was wrong. A 4/5 meant there were several things wrong with the place but it was still okay. A 3/5 meant it was only barely tolerable. A 2/5 meant the place should never have been listed in the first place. A 1/5 meant you refused to cross the threshold because it was so awful.
Various people commented on what a rotten person the landlord was for trying to gimmick the system and make his place seem better than it was.
But ... not really.
At AirBnB, for example, a single 4/5 rating can "demote" you from Superhost status and cost you customers. A 3/5 means plenty of people will skip over your rental altogether.
Online rating demands perfection. If it isn't 5/5, readers tell themselves, "They did something wrong. I don't want to stay there/buy that/hire them. I'll find a 5/5." And it really hurts the business.
It doesn't help that some online reviewers take glee in dumping on short-term landlords or small businesses. "I could hear street noise. 3/5." "The garbage truck came through at 8 AM and woke me up. 2/5." "I found a human hair on the floor behind the toilet. 1/5." They see themselves as Anthony Bourdain reviewing a restaurant and need to get over themselves.
This is also true when a company asks you to rate employees, and the rating is used in evaluations. If you give anything less than 5/5 or 10/10, the employee is questioned. "What could you have done better? How can you improve?" Enough less-than-perfect reviews, and the employee can be fired. Never mind that some people will give the cable company 1/5 on service because they're mad the bill went up, something the employee has no control over.
When I gave my students instructions for a project or essay, I always pointed out that if they followed the directions to the letter, they were looking at a C. "What??? But if we did everything, why isn't it an A?" "A C means you meet the basic requirements. To get a B, you need to do better than that, with an interesting writing style or extra sources, for example. To get an A, you need to go way above and beyond and do something pretty awesome. Another way to look at it: if you follow the basic recipe on a package of chocolate chips, you'll get some average cookies. A C. If you refrigerate the dough before baking and sprinkle some kosher salt on them, you've got some GOOD cookies. A B. If you frost each cookie with home-made frosting and pipe on an individualized design based on the eater's favorite color, you have some EXCELLENT cookies. An A. Doing the minimum is never the road to excellence."
Rating systems for short-term rentals (and other companies) should be the same. A 3/5 or 7/10 should mean "It was good enough. I would stay there / hire them again." 4/5 or 8-9/10 should mean "Very nice place / service. Various little touches were helpful. I hope I can stay there /hire them again next time." 5/5 or 10/10 should mean "Fantastic! Above and beyond! I felt a king waited on by servants. I would climb over my own grandmother to stay there /hire them again."
But internet culture is already well-entrenched, and I can't see this happening. If it's not perfection, it's garbage!
So when you do rate someone or something, keep this in mind. If you would stay there or hire them again, give them a 5/5, and if there are things they could improve on, talk about it in the commentary part of your review. And if you would stay there or hire them again, say so at the end of your review.
This came up elsewhere. A short-term rental host was fishing for higher ratings by putting up a door sign that "translated" what the ratings meant. 5/5 meant nothing was wrong. A 4/5 meant there were several things wrong with the place but it was still okay. A 3/5 meant it was only barely tolerable. A 2/5 meant the place should never have been listed in the first place. A 1/5 meant you refused to cross the threshold because it was so awful.
Various people commented on what a rotten person the landlord was for trying to gimmick the system and make his place seem better than it was.
But ... not really.
At AirBnB, for example, a single 4/5 rating can "demote" you from Superhost status and cost you customers. A 3/5 means plenty of people will skip over your rental altogether.
Online rating demands perfection. If it isn't 5/5, readers tell themselves, "They did something wrong. I don't want to stay there/buy that/hire them. I'll find a 5/5." And it really hurts the business.
It doesn't help that some online reviewers take glee in dumping on short-term landlords or small businesses. "I could hear street noise. 3/5." "The garbage truck came through at 8 AM and woke me up. 2/5." "I found a human hair on the floor behind the toilet. 1/5." They see themselves as Anthony Bourdain reviewing a restaurant and need to get over themselves.
This is also true when a company asks you to rate employees, and the rating is used in evaluations. If you give anything less than 5/5 or 10/10, the employee is questioned. "What could you have done better? How can you improve?" Enough less-than-perfect reviews, and the employee can be fired. Never mind that some people will give the cable company 1/5 on service because they're mad the bill went up, something the employee has no control over.
When I gave my students instructions for a project or essay, I always pointed out that if they followed the directions to the letter, they were looking at a C. "What??? But if we did everything, why isn't it an A?" "A C means you meet the basic requirements. To get a B, you need to do better than that, with an interesting writing style or extra sources, for example. To get an A, you need to go way above and beyond and do something pretty awesome. Another way to look at it: if you follow the basic recipe on a package of chocolate chips, you'll get some average cookies. A C. If you refrigerate the dough before baking and sprinkle some kosher salt on them, you've got some GOOD cookies. A B. If you frost each cookie with home-made frosting and pipe on an individualized design based on the eater's favorite color, you have some EXCELLENT cookies. An A. Doing the minimum is never the road to excellence."
Rating systems for short-term rentals (and other companies) should be the same. A 3/5 or 7/10 should mean "It was good enough. I would stay there / hire them again." 4/5 or 8-9/10 should mean "Very nice place / service. Various little touches were helpful. I hope I can stay there /hire them again next time." 5/5 or 10/10 should mean "Fantastic! Above and beyond! I felt a king waited on by servants. I would climb over my own grandmother to stay there /hire them again."
But internet culture is already well-entrenched, and I can't see this happening. If it's not perfection, it's garbage!
So when you do rate someone or something, keep this in mind. If you would stay there or hire them again, give them a 5/5, and if there are things they could improve on, talk about it in the commentary part of your review. And if you would stay there or hire them again, say so at the end of your review.
Thoughts
A good assignment should tell students what the different grades look like. I've seen them, and I've written them that way. If they follow all the instructions, of course they'll be upset at only getting a C. They wanted an A but the instructions didn't tell them how to do that. It doesn't matter what the goals are; it matters that students know what all of the goals are.
I agree that current online rating systems will probably not change. But anyone who wishes to create a website with a rating system could choose to include a rubric to improve their accuracy. So can anyone scoring something in meatspace. One that I created was for model rockets, because some of the judges were not rocket hobbyists, so the rubric explained what to look for regarding fins, paint job, etc. and then other things like stability and flight time/altitude were factored into other sections.
Re: Thoughts