Announcing our new scoring system

Broken link? Feature request? Anything related to the Spectrum Computing website here.
catmeows
Manic Miner
Posts: 718
Joined: Tue May 28, 2019 12:02 pm
Location: Prague

Re: Announcing our new scoring system

Post by catmeows »

Juan F. Ramirez wrote: Sat Oct 24, 2020 10:53 am
Vampyre wrote: Sat Oct 24, 2020 10:21 am At least it's registered to users so they can be named and shamed :-)
And insulted!

It's a question of time , when more people vote, most games will have sensible scores.
Exactly!
Proud owner of Didaktik M
User avatar
Morkin
Bugaboo
Posts: 3278
Joined: Mon Nov 13, 2017 8:50 am
Location: Bristol, UK

Re: Announcing our new scoring system

Post by Morkin »

Vampyre wrote: Sat Oct 24, 2020 12:02 pm
PeterJ wrote: Sat Oct 24, 2020 11:03 am Indeed @Vampyre. I will be starting a consultation shortly on the publishing of each users scores publicly. I've just looked who gave Cassette 50 the 10.
Is the system able to deny users from voting? I think a yellow/red card system might be the way to stop things like this. As mentioned earlier I'd hate a good idea like this to become the joke that WoS's became.
...I personally think that the WoS top 100, despite that argument about "Abadia" that happened years ago, looks like the most 'authentic' 100 list I've seen anywhere.

Anonymous voting has been criticised but its strength is the number of votes it can acquire - if you look at the list, the games have hundreds of votes.

I think the issue you can't really avoid with both average magazine ratings and member-only ratings is that the number of votes is small, which makes it easily to skew scores with one or two votes. And I'm not sure exactly what you can do about it - the whole idea of voting/rating is subjective, so if someone wants to give a game you like 2 out of 10, it's a valid vote.
My Speccy site: thirdharmoniser.com
User avatar
RWAC
Manic Miner
Posts: 702
Joined: Sun Aug 18, 2019 9:59 pm

Re: Announcing our new scoring system

Post by RWAC »

Vampyre wrote: Sat Oct 24, 2020 12:11 pm
RWAC wrote: Sat Oct 24, 2020 12:08 pm Most people think Knight Lore is one of the best games ever? I think it's little more than a tech demo with boring gameplay. Should I be penalised for not voting with the masses?
There's a huge difference between that and what you did, so don't come that rubbish. You were taking the mick and admitted it.
It was a joke and I have since amended my score. I just wanted to see if anyone noticed!
It also helps to hilight any issues with the system.
I don't want it to be a joke either but I certainly don't want to go down the route of policing what people vote for.
User avatar
PeterJ
Site Admin
Posts: 6888
Joined: Thu Nov 09, 2017 7:19 pm
Location: Surrey, UK

Re: Announcing our new scoring system

Post by PeterJ »

I'm out and about today, so quick reply. It is technically possible [mention]Vampyre[/mention], but I agree with [mention]RWAC[/mention] that we shouldn't police people's votes. I obviously if someone was being malicious we could deal with that, but it's all a bit of fun and shouldn't be taken to seriously.

Again, sorry for the short reply.
hikoki
Manic Miner
Posts: 576
Joined: Thu Nov 16, 2017 10:54 am

Re: Announcing our new scoring system

Post by hikoki »

anonymous 1 to 10 doesnt mean anything to me. it's useful to have informed opinions from sensible voters who you resonate with. When reading imdb users opinions I can guess the reasoning of juddgements and keep reading opinions from a voter in particular and rule jokers out of my sight
User avatar
RWAC
Manic Miner
Posts: 702
Joined: Sun Aug 18, 2019 9:59 pm

Re: Announcing our new scoring system

Post by RWAC »

Morkin wrote: Sat Oct 24, 2020 11:59 am
Well that's no good, we insult @RWAC regularly so no change there..! ;)

Oops - I think I may have just voted the Saga Emperor Keyboard top of the list...! :lol: Is there any way of removing a vote..?

[Edit: Genuine mistake, I thought the top 100 would have filtered 'all software']
Maximum players 0
Sounds interesting!
User avatar
Morkin
Bugaboo
Posts: 3278
Joined: Mon Nov 13, 2017 8:50 am
Location: Bristol, UK

Re: Announcing our new scoring system

Post by Morkin »

:lol:

..Although "Rating your hardware" sounds a bit like a smutty euphemism, I guess it might still be nice to do, just perhaps filtered out from the ratings table if possible...?
My Speccy site: thirdharmoniser.com
User avatar
PeterJ
Site Admin
Posts: 6888
Joined: Thu Nov 09, 2017 7:19 pm
Location: Surrey, UK

Re: Announcing our new scoring system

Post by PeterJ »

Yes [mention]Morkin[/mention],

I can delete that vote tonight.
User avatar
PeterJ
Site Admin
Posts: 6888
Joined: Thu Nov 09, 2017 7:19 pm
Location: Surrey, UK

Re: Announcing our new scoring system

Post by PeterJ »

hikoki wrote: Sat Oct 24, 2020 2:22 pm anonymous 1 to 10 doesnt mean anything to me. it's useful to have informed opinions from sensible voters who you resonate with. When reading imdb users opinions I can guess the reasoning of juddgements and keep reading opinions from a voter in particular and rule jokers out of my sight
It's just meant as a bit of interactive fun. People should use Spectrum2.0 to leave serious reviews.
User avatar
RWAC
Manic Miner
Posts: 702
Joined: Sun Aug 18, 2019 9:59 pm

Re: Announcing our new scoring system

Post by RWAC »

Morkin wrote: Sat Oct 24, 2020 2:45 pm :lol:

..Although "Rating your hardware" sounds a bit like a smutty euphemism, I guess it might still be nice to do, just perhaps filtered out from the ratings table if possible...?
I can upload a picture of my dongle if you like?
akeley
Dynamite Dan
Posts: 1049
Joined: Sat Feb 01, 2020 5:47 pm

Re: Announcing our new scoring system

Post by akeley »

You can try and police some really malicious attempts at skewing the system, but it will always be tricky, and perhaps only applicable in some obvious cases, like when somebody posts 50 1/10 votes. But even then, you won't know that maybe that person simply felt like voting for their most hated choices.

Basically, you will never be able to make an ideal voting system, so it's best just to leave it as it is, and like I said earlier not to take it too seriously.

My only appeal is for folks to think it through before slamming something with below-4/10 vote. For me to award such a low score a game would have to be inherently broken or the authors' taking a mick in some way. So please try to remember we all have varied tastes and preferred genres, and different games have different value factors . And so, Kosmic Kanga really is not a 1/10 game, even if you dearly hate it. Knight Lore is not just a tech demo and Movie is totally playable, despite the fact control scheme might not be to everyone's liking.

Few examples:
-I really hate joystick wagglers like Hypersports, Daley Thompson's etc. But I can also recognise that these games are well made and that this style has its fans. 5 or 6/10
-Manic Miner and the like - I hate unforgiving precision platforming too. Still, they were hugely influential and others love them and excel in this style - 8/10
-Hunchback - arrrgh, 30 tries and I can't make it past second screen. I think jumping and collision detection is a joke - 1/10. But...other people seem to have no problem racking big scores in this game. Ok, 4/10 then.
User avatar
RWAC
Manic Miner
Posts: 702
Joined: Sun Aug 18, 2019 9:59 pm

Re: Announcing our new scoring system

Post by RWAC »

I rate games on how enjoyable they are to play.
Sqij can't even be played without breaking into the program and turning caps on (or off - I can't remember). So I scored it a 0.

EastEnders and Underwurlde are excruciating to play so scored a 1.

Deliverance is marginally less excruciating and so scores a 2.

Knight Lore is better still but not by much and scores a 3.

Shadow of the Beast is below average and scores a 4.

Sabre Wulf is the very definition of bang average so it's a 5.

Bounder is above average and gets a 6.

Curse of Sherwood is good but flawed so scores 7.

Moontorc is a very good game and gets an 8.

Chaos is excellent and scores 9.

Rebelstar 2 is the best game on the system and scores 10.

These are just my opinions of course and no more valid than anyone elses. But that's how I've done it.
User avatar
Juan F. Ramirez
Bugaboo
Posts: 5139
Joined: Tue Nov 14, 2017 6:55 am
Location: Málaga, Spain

Re: Announcing our new scoring system

Post by Juan F. Ramirez »

Things are going very well up to now as to scoring system, as La Abadia del Crimen currently have an average score of 6... :mrgreen:




(I voted 7)
User avatar
Vampyre
Manic Miner
Posts: 839
Joined: Wed Nov 15, 2017 2:51 pm
Contact:

Re: Announcing our new scoring system

Post by Vampyre »

Apologies all round to everyone for earlier on, particularly RWAC. My inner Victor Meldrew keeps coming out, as the missus keeps reminding me, so please accept my apologies for being an uptight, grumpy git earlier.
ZX Spectrum Reviews REST API: http://zxspectrumreviews.co.uk/
akeley
Dynamite Dan
Posts: 1049
Joined: Sat Feb 01, 2020 5:47 pm

Re: Announcing our new scoring system

Post by akeley »

Chaos - crap graphics, no clue what's going on, too many stats, what a joke. 1/10.
Rebelstar 2 - ditto, only with even more stats, 1/10.
Curse of Sherwood - just run and run in some corridors, shoot the same enemies over and over, what a snoozefest. 2/10.

These aren't my real scores of course, just a sample of this logic working both ways.

All I'm saying is we could try and be a bit more objective towards games we don't like.
voxeltower
Drutt
Posts: 24
Joined: Wed Oct 21, 2020 12:00 pm

Re: Announcing our new scoring system

Post by voxeltower »

it is very difficult for everyone to agree to a voting system. I believe that the more votes it has, the fairer the system is.
In any case, the game that has less than 5 votes is not decisive.
User avatar
RWAC
Manic Miner
Posts: 702
Joined: Sun Aug 18, 2019 9:59 pm

Re: Announcing our new scoring system

Post by RWAC »

akeley wrote: Sat Oct 24, 2020 3:56 pm Chaos - crap graphics, no clue what's going on, too many stats, what a joke. 1/10.
Rebelstar 2 - ditto, only with even more stats, 1/10.
Curse of Sherwood - just run and run in some corridors, shoot the same enemies over and over, what a snoozefest. 2/10.

These aren't my real scores of course, just a sample of this logic working both ways.

All I'm saying is we could try and be a bit more objective towards games we don't like.
I fully support anyone who awards Rebelstar 1 out of 10. I fully accept some people will think it's rubbish.

I have tried to be as objective as possible but some games are just rubbish (in my opinion) and do not deserve anymore than 3 or 4.
I'm all for an open and transparent vote. I don't like games like Knight Lore so people knowing that can then disregard my ranking.
User avatar
Morkin
Bugaboo
Posts: 3278
Joined: Mon Nov 13, 2017 8:50 am
Location: Bristol, UK

Re: Announcing our new scoring system

Post by Morkin »

voxeltower wrote: Sat Oct 24, 2020 4:12 pm it is very difficult for everyone to agree to a voting system.
I agree.. :lol:
I believe that the more votes it has, the fairer the system is.
In any case, the game that has less than 5 votes is not decisive.
I think having a minimum vote number before a game is ranked was mentioned for consideration? Perhaps it may discourage people from downvoting suddenly high ranking games with 1 vote.
My Speccy site: thirdharmoniser.com
hikoki
Manic Miner
Posts: 576
Joined: Thu Nov 16, 2017 10:54 am

Re: Announcing our new scoring system

Post by hikoki »

It's just meant as a bit of interactive fun. People should use Spectrum2.0 to leave serious reviews.
I meant that it'd be useful to see the votes from any forum member as everyone votes with different likings, logic, scale rating, etc. Some users rank games higher for nostalgia, replayability, genre, graphics, gameplay, cheesiness, humour, difficulty..
akeley
Dynamite Dan
Posts: 1049
Joined: Sat Feb 01, 2020 5:47 pm

Re: Announcing our new scoring system

Post by akeley »

RWAC wrote: Sat Oct 24, 2020 4:18 pm I fully support anyone who awards Rebelstar 1 out of 10. I fully accept some people will think it's rubbish.

I have tried to be as objective as possible but some games are just rubbish (in my opinion) and do not deserve anymore than 3 or 4.
I'm all for an open and transparent vote. I don't like games like Knight Lore so people knowing that can then disregard my ranking.
My point was that an objectively thinking person wouldn't give Rebelstar 1/10. Even somebody who doesn't like the genre should consider how lauded it was and stop at 5-6/10 or thereabouts, because this is "not for me territory". 1/10 suggests it's a terrible, broken strategy game, which should be avoided at all costs. The same goes for Knight Lore. If we all executed some restraint in these negative votes, then it would perhaps help to build a better top 100 or whatever list.

Anyway, 'twas just my 2 cents. It's also the reason why I never take these "people's rankings" seriously. I mean, the best ZX game according to Spectrum 2.0 is Hypersports ;) On WoS 1st ranked Gollop's game is Rebelstar @ #33 (WTF). These lists work best when you remove the ranks and just take them as "bunch of games probably worth checking out".
User avatar
PeterJ
Site Admin
Posts: 6888
Joined: Thu Nov 09, 2017 7:19 pm
Location: Surrey, UK

Re: Announcing our new scoring system

Post by PeterJ »

Thanks [mention]hikoki[/mention],

We are looking at the options for displaying users votes.
User avatar
Juan F. Ramirez
Bugaboo
Posts: 5139
Joined: Tue Nov 14, 2017 6:55 am
Location: Málaga, Spain

Re: Announcing our new scoring system

Post by Juan F. Ramirez »

akeley wrote: Sat Oct 24, 2020 4:52 pm [...] It's also the reason why I never take these "people's rankings" seriously
That's (or should be) the key of this topic. I think this rank is made, as someone said, just for fun. And I like it.

I have my choices on films, and whenever I see Imdb's rank, and see The Godfather in the first positions, I understand the value of it for people but I DON'T like it (yes, sorry). And I won't question the rank. There are thousands of films and millions of people voting for them. I'm sure many voted 1 or 2 for it (my no. 1 film is Young Frankenstein, by the way).

If someone gives a quality game 1 or 2 points, there's always a reason, even a question of hate (how many times have we read here 'I hate this game'?) because our opinions are always a subjetive thing. On the other hand, there people who give 9 or 10 points to a (for many) mediocre game.

It's a question of statistics: many, many people never fail, and seeing the current SC rank gives us the tendency of most people here.
catmeows
Manic Miner
Posts: 718
Joined: Tue May 28, 2019 12:02 pm
Location: Prague

Re: Announcing our new scoring system

Post by catmeows »

RWAC wrote: Sat Oct 24, 2020 3:23 pm I rate games on how enjoyable they are to play.
I do same thing.
EastEnders and Underwurlde are excruciating to play so scored a 1
And I consider Underwurlde solid 7 game. I played it, I enjoyed it and I finished it.
Curse of Sherwood is good but flawed so scores 7.
I found it too frustrating, 4 at most.

My point is that with enough votes we can hope for reasonable scores And it Is probably best thing one could get.
Proud owner of Didaktik M
User avatar
p13z
Manic Miner
Posts: 611
Joined: Sun Feb 17, 2019 10:41 pm
Location: UK
Contact:

Re: Announcing our new scoring system

Post by p13z »

Seems a bit mad to give a game with a single 10 vote a higher ranking than one with hundreds of 10 votes and one vote of 9.
Shouldn't a scoring system like this assume an initial vote of 5 for everything when working out a score by averages?

IE. Assume a game is average until proven otherwise. Rather than assume it is the best game ever because it's first rating is a 10.
User avatar
Lethargeek
Manic Miner
Posts: 746
Joined: Wed Dec 11, 2019 6:47 am

Re: Announcing our new scoring system

Post by Lethargeek »

or + one 5 for the 1st vote, then one 5 for the next 2 votes, then for the next 4 votes etc
Post Reply