Portkey ratings system discussion

[sarea]

For those of you who care about such things -- Portkey is reviewing its ratings system. I've put in my $0.02 on a replacement idea, but I'm not sure one is necessary. Bottom line is that I don't find the ratings helpful in the least (not as a writer nor as a reader), and even if it's not replaced with something else, I'd rather see it gone entirely than exist as-is.

You can vote and log your own opinion here: http://talk.portkey.org/index.php?showtopic=6392

I'm really glad they're considering things, at least. All the complaints they've gotten over the years has finally worn them down. <g> One of the first things I did when I got an LJ was complain about Portkey ratings, and even then they'd said they wanted to revisit it. Yay for progress!

Comments

[kstanley.livejournal.com]

I registered for that board and logged in and it would not let me reply over there. Apparently I don't have permission.

The ratings over there are bad, but it's a not problem that is exclusive to Portkey. I see ratings problems with most fanfic sites and even places like Amazon.

The problem is, people do not understand what ratings represent or how to give them. Reading a piece critically is not something that people intuitively know how to do. It requires training. Fortunately, I think that a mini-primer on reviewing someone's work is all that will be required for Portkey.

In such a document, I would stress the fact that an honest review is helpful to the writer, even when you are pointing out weaknesses in the story. I would also stress how unethical it is to give a false review (positive or negative) to someone you especially like or dislike. One of the things I like about Amazon that that little "Was this review helpful" feature. It's sort of a review of the reviewers and I think it keeps people more honest.

I think ratings can be very useful. What I would do over at Portkey is create a more sophisticated, yet straightfoward rating system. For example, stories could be rated in different categories (characterization, plot, dialogue, narrative, or whatever), but the ratings for each of these categories should be very simple (needs work, adequate, quite good, very good, or some variation).

Now obviously these ratings would have to be explained to people. This could be done easily by making the categories and ratings hotlinked to a pop-up that describes what each of them means.

There may be some people who find the system too complicated to use, but frankly do you really want those people reviewing your work? It seems to me that a few high quality reviews are better than dozens of oversimplified useless ones.

[sarea-okelani.livejournal.com]

Theoretically, I agree w/ everything you said. A few high-quality reviews does mean more than many that don't necessarily get into critique. As a writer, however, I have to say that just knowing people read my stories goes a long way in making me feel that it's worth spending long hours at my keyboard to create a story. So if people don't feel comfortable leaving long reviews, or don't have time, or whatever -- even a small gesture of appreciation, or letting me know they're reading, goes a long way. And a system like the one you're describing -- while awesome -- would probably discourage people from reviewing at all. And I can safely say that that would not be a good motivational tool if I'm not feeling the impetus to write. Because frankly, the high-quality reviews are few and far between in any case; making it even more complicated might even taper those off.

As a writer, there are only so many ways that you get visibility on whether people are even reading your work, and sometimes it's frankly disheartening when no one says anything. Then you wonder if you should bother (angiej went through this, and with good reason), because you aren't sure whether anyone cares, or wants to read your writing, or enjoys it. That's where "even the smallest bit of feedback" comes in, to help buoy the writer, to help her know that there are people reading her work, and that she should keep posting. I write for myself, but there's no reason I should post anything I write if no one's going to read it.

... and I can feel myself starting to go into another tangent about how people ought to therefore be better about feedbacking stories, period, but I won't there. That's a whole other discussion. : )

[kstanley.livejournal.com]

And a system like the one you're describing -- while awesome -- would probably discourage people from reviewing at all.

I don't see why. It's still radio buttons, you are simply making four choices instead of one.

And I'll tell you something else, I rarely bother with the simple 1-10 system, because I know it is meaningless. I am far more likely review with this 4-choice system (which is the one they use on eopinions.com), than the single choice.

The system could be set up so that folks are not required to answer all the ratings. So you still have people with their very simple feedback, and then you have others with slightly less, but still very simple, and yet more useful feedback.

This system that I am referring to is in use, (or at least a variation of it is), on a site I regularly visit to get product reviews: eopinions.com.

I think it's a pretty effective model that could be adapted for reviewing fanfic.

Although in the interest of full disclosure, your Draco/Ginny fanfiction is just about the only fanfiction I read and I always review that.

I always focus on the positive aspects of your writing, because that seems appropriate. But I've taken writing classes every semester (I'm getting a literature degree) and so I frequently work in peer and mentoring groups where we review each others' work. Any time you want a more robust review of something, just let me know.

[sarea-okelani.livejournal.com]

I don't see why.

I honestly can't answer that one. <g> I only know what review/feedback trends are, and I can tell you that getting people to respond these days is a bit like pulling teeth. It's rather depressing. At the same time, I also sympathize; leaving thoughtful, constructive reviews isn't an easy task. Also, there are times when you read a story that you don't want to leave a review, because you wouldn't have anything helpful to say, or maybe you didn't enjoy it, but don't want to make the writer feel bad for no reason, and what might be based on personal preference. However, that means that writers will interpret silence from their readers as being the latter -- because if they did enjoy it, wouldn't they say so?

And that may not necessarily be true, but human nature being what it is...

It'd be fantastic if, as a whole, fandom used a system like that on eopinions (there are many reasons why the 1-10 ratings system doesn't work, but the biggest thing is that the numbers have no true meaning), but I suppose since I don't have any say in how Portkey is run, it'll be hard to institute/test.

I am extremely flattered that you're reading my fiction, and yes, the positives are always appreciated. However, constructive criticism is also very much desired. You're really generous to offer more robust reviews, and I can say that any time you feel like doing one, I'd be more than happy to get your feedback. However, I don't think I could ask you to do it specifically, because it's fanfic and my expectations/approach is different from how it would be for, say, original fiction. If it were original fiction, that's the only kind of feedback I'd want; fanfic being mostly about escape, I don't expect people to read/respond to it the same way they would an original work.

Anyway, I've always really appreciated your insightful and well-spoken comments. Thanks so much.