New WoS and ZXDB

This is the place for general discussion and updates about the ZXDB Database. This forum is not specific to Spectrum Computing.

Moderator: druellan

Post Reply
User avatar
Einar Saukas
Dynamite Dan
Posts: 1303
Joined: Wed Nov 15, 2017 2:48 pm

New WoS and ZXDB

Post by Einar Saukas »

It turns out new WoS is using ZXDB! More precisely ZXDB version 1.08 released on September 2, 2018.

Don't you believe me? Then visit new WoS software page and click on "EXPORT CSV (ALL)", to download some of the data from all titles stored in new WoS. Afterwards see if you can find any differences between new WoS content and the ZXDB version I mentioned... Or visit a few game pages at random to compare old WoS (search for a game in Google then click "cached"), new WoS, and any other site running ZXDB like SpectrumComputing or ZXInfo.

Don't take the wrong, I'm glad that new WoS is using ZXDB. Rebuilding WoS was exactly the reason ZXDB was created. I have been trying for years to convince Lee Fogarty that it would be a terrible idea otherwise (as recently as last week). The community doesn't deserve to have everyone's contributions thrown away, restarting everything from scratch for absolutely no reason. Besides, ZXDB is an open database, everyone is welcome to use it, there's no need to even ask my permission.

So what's the problem?

The problem is using ZXDB while pretending otherwise. Last month (May 2020), Lee Fogarty explicitly claimed new WoS would initially use "old WoS data" (although WoS hasn't been updated since 2013) and they were "going through the data applying fixes", fixing "thousands and thousands of errors", etc. However this claim is false: new Wos is not using either. It's using ZXDB from 2018. So now you know where thousands and thousands of fixes came from.

Even worse, there was clearly an attempt to disguise using ZXDB. For instance, ZXDB version 1.08 contained 24369 non-Timex programs from old WoS (from ID=1 to ID=28187), 6011 programs added in ZXDB (from ID=28201 to ID=34414), and 137 Timex programs from old WoS (from ID=4000001 to ID=4000143). New WoS is using all data from ZXDB 2018, but all IDs above 28187 were removed, so it only shows the same 24369 titles that already existed in old WoS. Well, actually new WoS indicates 24368 titles because somehow they lost Trans-Disc Express along the way. Therefore you currently won't find any titles in new WoS that were added after 2013... although it still contain several references to them! A few examples:

* Comercio Cosmico was released in 1987. Later I re-released it myself in English as Cosmic Commerce in 2015. Old WoS from 2013 didn't know it, but this re-release appears at new WoS (click on "RELEASES" to see it).

* En Busca de Mortadelo was re-released by Team Siglo XXI in 2017. Again old WoS didn't know it, but new WoS does.

* Catacombs of Balachor was a single game in old WoS. After a sequel was released in 2014, this game was added to series "Balachor" in ZXDB. You won't find the sequel at new WoS, but Catacombs of Balachor at new WoS lists series "Balachor", that only contains "Catacombs of Balachor". Isn't it weird to see a series of only one game?

* Alter Ego was a single game in old WoS. After a sequel was released in 2014, this game was added to series "Alter Ego" in ZXDB. You won't find the sequel at new WoS, but Alter Ego at new WoS lists a series called "Alter Ego", that only contains game "Alter Ego".

There are other ways to identify this data came from ZXDB. For instance, old WoS didn't distinguish text adventures, therefore Colossal Adventure and The Hobbit were both classified as "Adventure: Text". In ZXDB 2018, we added the distinction between "Adventure Game: Text-Only" (like Colossal Adventure) and "Adventure Game: Text/Illustrated" (like The Hobbit). Coincidentally that's what you see at new WoS pages for Colossal Adventure and The Hobbit.

But wait, last week I insisted once again that new WoS should use ZXDB. So perhaps my argument finally convinced Lee Fogarty to change his mind, but then he "forgot" to mention it? Not really. I wrote it last week (June 2020), when I was accused of "stealing" WoS data (proved wrong), copying another database and not crediting other people properly for their participation on creating it (also proved wrong). But if you visit any game page at new WoS at random, you will notice almost all of them are dated May 2020. Which means they falsely accused me of doing it, while they were already doing themselves exactly this! Incredible.

But wait, perhaps ZXDB 2018 was not copied directly, these are just coincidences? I mean, it's theoretically possible that every single change made in ZXDB for the first 2 years was added independently to the old WoS database. After all, most of these changes came from posts publicly available in discussion forums. Or perhaps someone simply saw information in other sites using ZXDB, then typed them again into old WoS... Right?

Wrong. New WOS is a straight copy directly from ZXDB 2018, period. How do we know? Well, let's dive a little deeper.

Martijn's WoS internal file maindb.dat contained the following comment for BIFROST* ENGINE:

Code: Select all

Inspired by {ZXodus Engine|Andrew Owen [2]}.
Based on it, old WoS produced the following HTML:

Code: Select all

Inspired by <a href="http://www.worldofspectrum.org/infoseek.cgi?regexp=^ZXodus+Engine$&pub=^Andrew+Owen+[2]$&loadpics=1">ZXodus Engine</a>.
By comparison, ZXDB database file contains the following comment for the same title:

Code: Select all

Multicolor 8x1 Graphics Engine (18 char columns x 18 char rows). Inspired by {ZXodus Engine|Andrew Owen [2]|0026639}.
Although this is not what you see at websites, because each site uses this information from ZXDB to generate their own HTML. This is actually what you see at SpectrumComputing:

Code: Select all

Multicolor 8x1 Graphics Engine (18 char columns x 18 char rows). Inspired by  <a href="index.php?cat=96&id=0026639">ZXodus Engine</a>.
This is what you see at ZXInfo:

Code: Select all

Multicolor 8x1 Graphics Engine (18 char columns x 18 char rows). Inspired by <a href="https://zxinfo.dk/details/0026639">ZXodus Engine</a> (Andrew Owen [2]).
And this is what you see at new WoS site:

Code: Select all

Multicolor 8x1 Graphics Engine (18 char columns x 18 char rows). Inspired by {ZXodus Engine|Andrew Owen [2]|0026639}.
It means this line wasn't even copied from another site. It was imported straight from ZXDB source file. Just like everything else.

Also according to new WoS, BIFROST* belongs to a group called "BIFROST/NIRVANA" (exactly like in ZXDB), although NIRVANA itself was removed from new WoS after importing ZXDB.

Another interesting point is, there are 2 Spectrum authors called "Andrew Owen", so this Andrew was originally added to old WoS as "Andrew Owen [2]". Years ago Andrew told me by email that his middle initial was "S", so I changed ZXDB from "Andrew Owen [2]" to "Andrew S. Owen". However I forgot to update this comment... Coincidentally new WoS made the same mistake! Thus leading to our next example:

I co-authored "FZX" with Andrew in 2013. He was originally credited as "Andrew Owen [2]" in old WoS. He's now credited as "Andrew S. Owen" in ZXDB. But if you look at new WoS, you will notice he's credited both ways: author "Andrew S. Owen" and publisher "Andrew Owen [2]" (open "RELEASES" below to see it). So what's the reason for this inconsistency? Because the list of publishers in new WoS came mostly from Martijn's WoS internal file publishers.dat, that Lee Fogarty imported back in May 2017 and never fixed, but all actual game data came from ZXDB. How do we know? Because game data at new WoS contains all information that was updated in ZXDB 2018, but the list of publishers at new WoS still contains all the bugs I reported when I started ZXDB in 2016. For instance Waner Brothers was reported in July 2016.

OK, so I made it clear that new WoS is using an old version of ZXDB. But wait, these people also helped me to import old WoS content into ZXDB at the beginning. Therefore we should excuse them for using the result of their own work without crediting ZXDB, right? The problem is, I imported old WoS content with their help from July 2016 to August 2016. If they had used one of the early versions of ZXDB that they participated, without crediting ZXDB, I would leave it alone. However they chose to use a version of ZXDB from September 2018, in order to take advantage of over 2 years of other people's work, without crediting anybody. That's a ZXDB version released 2 years after WoS stopped supporting ZXDB and started attacking my work. About 1 year after ZXDB and SpectrumComputing were censored at WoS thus forcing ZXDB to move to another forum. Months after I was personally censored at WoS without ever receiving any explanation.

This leads to our next question. Here's a list of ZXDB releases since we started numbering them in early 2018. Why did they chose to use ZXDB version 1.08 from September 2018? It cannot be that they got the most recent version available at the time, because if they took it before ZXDB 1.09 (October 2018), it would mean they have been lying for the last 2 years about importing data from old WoS and working on thousands of fixes themselves. It cannot be that they only started using ZXDB recently but deliberately downloaded an older version of ZXDB to make it harder for people to notice, because that would be really dishonest...

The bottom line is, it's now clear WoS made little progress for years, despite claiming they were busy working on a much better database containing thousands and thousands of fixes. In the meantime, they initially made excuses for not using ZXDB, then attacked our work with false accusations. Half a decade later, when a new WoS is finally launched, it's simply using ZXDB in disguise. And it happened only 4 days after they falsely accused ZXDB of copying another database without crediting people's work! How ironic.

What now? Should new WoS stop using ZXDB? Of course not! To stop doing it now, after getting caught, would only make it look even worse. A new site using even older data, full of errors already fixed for the last few years, won't help anybody in the community. Instead, I would simply expect an apology for this stunt, and to properly credit somewhere in new WoS that it's using content originally from ZXDB (according to the ZXDB license). Easy, right?

Also for heaven's sake, use an updated version of ZXDB! Why take 2 years of other people's work while trying to avoid crediting them, when you could have taken all 4 years!!!


NOTE: I have just posted this text above at WoS forum, then I thought this subject would be of interest here too...

EDIT: Just fixed a missing link.
28 x

User avatar
Juan F. Ramirez
Rick Dangerous
Posts: 2789
Joined: Tue Nov 14, 2017 6:55 am
Location: Málaga, Spain

Re: New WoS and ZXDB

Post by Juan F. Ramirez »

Einar Saukas wrote:
Thu Jun 18, 2020 12:03 pm
It turns out new WoS is using ZXDB!
I can't believe it! :D
0 x

User avatar
Pegaz
Manic Miner
Posts: 493
Joined: Mon Nov 13, 2017 1:44 pm

Re: New WoS and ZXDB

Post by Pegaz »

Einar Saukas wrote:
Thu Jun 18, 2020 12:03 pm

Also for heaven's sake, use an updated version of ZXDB! Why take 2 years of other people's work while trying to avoid crediting them, when you could have taken all 4 years!!!
Why?
We all know why, Einar and I will not repeat again what I have written many times on similar occasions.
Maybe this way, this topic will avoid the fate of being locked.
If you still have some illusions or you haven't understood who you are dealing with, then it is pointless to say anything else.
It was enough to look at the reaction to your recent messages there - just total ignorance.
Real life is not a fairy tale and good does not always defeat evil, it's usually the opposite.
I really appreciate your work and effort and I would not like to give advices.
Still, maybe it's time to remember the good old Bible saying - Don't throw your pearls before the pigs...
2 x

User avatar
StooB
Manic Miner
Posts: 372
Joined: Mon Oct 22, 2018 11:03 am

Re: New WoS and ZXDB

Post by StooB »

I think it's more likely to be using data taken from ZXDB at that point of time, rather than actually running from an instance of ZXDB itself.
1 x

User avatar
bob_fossil
Manic Miner
Posts: 231
Joined: Mon Nov 13, 2017 6:09 pm

Re: New WoS and ZXDB

Post by bob_fossil »

Einar, any chance you can share the POKES you're using to give yourself infinite patience and infinite reasonability? Thanks. :)
10 x

User avatar
PeterJ
Site Admin
Posts: 2466
Joined: Thu Nov 09, 2017 7:19 pm
Location: Surrey, UK

Re: New WoS and ZXDB

Post by PeterJ »

Maybe @StooB. Regardless, there is still a requirement in the open source license to mention ZXDB, and a request to make the created database (of one was created) open too.

It's obviously good though that it's being used. Although to have rubbished all the work of Einar and others previously (even up to last week) and make numerous false claims, then not mention it turns out ZXDB has been used is disgraceful.
5 x

User avatar
Pegaz
Manic Miner
Posts: 493
Joined: Mon Nov 13, 2017 1:44 pm

Re: New WoS and ZXDB

Post by Pegaz »

StooB wrote:
Thu Jun 18, 2020 12:43 pm
I think it's more likely to be using data taken from ZXDB at that point of time, rather than actually running from an instance of ZXDB itself.
Well, Einar gave quite detailed arguments for his claims.
We will see soon what the response will be and who was right...
2 x

User avatar
Rorthron
Dynamite Dan
Posts: 1159
Joined: Sun Nov 12, 2017 10:35 pm

Re: New WoS and ZXDB

Post by Rorthron »

Juan F. Ramirez wrote:
Thu Jun 18, 2020 12:35 pm
Einar Saukas wrote:
Thu Jun 18, 2020 12:03 pm
It turns out new WoS is using ZXDB!
I can't believe it! :D
I can, and am not in the least surprised. I have long suspected Lee would pull such a stunt. There seemed to be no other way to catch up with the massive backlog of changes the WoS archive needed.
0 x

User avatar
Rorthron
Dynamite Dan
Posts: 1159
Joined: Sun Nov 12, 2017 10:35 pm

Re: New WoS and ZXDB

Post by Rorthron »

Pegaz wrote:
Thu Jun 18, 2020 1:01 pm
Well, Einar gave quite detailed arguments for his claims.
We will see soon what the response will be and who was right...
What are the odds that Lee will just ignore Einar's arguments and respond with some feeble and false whataboutery?
2 x

User avatar
StooB
Manic Miner
Posts: 372
Joined: Mon Oct 22, 2018 11:03 am

Re: New WoS and ZXDB

Post by StooB »

Pegaz wrote:
Thu Jun 18, 2020 1:01 pm
StooB wrote:
Thu Jun 18, 2020 12:43 pm
I think it's more likely to be using data taken from ZXDB at that point of time, rather than actually running from an instance of ZXDB itself.
Well, Einar gave quite detailed arguments for his claims.
We will see soon what the response will be and who was right...
I'm not saying Einar's wrong! I'm saying that Lee Fogarty would rather take the data and put it in his own database because he doesn't like the fact he didn't design ZXDB, despite it making any updates into a huge data conversion mess. Which means WoS is now broken and frozen in time in 2018, rather than broken and frozen in time in 2014.
1 x

Post Reply