New Database Model ZXDB

This is the place for general discussion and updates about the ZXDB Database. This forum is not specific to Spectrum Computing.

Moderator: druellan

Post Reply
User avatar
PeterJ
Site Admin
Posts: 6854
Joined: Thu Nov 09, 2017 7:19 pm
Location: Surrey, UK

Re: New Database Model ZXDB

Post by PeterJ »

Thank you again to [mention]druellan[/mention], [mention]Einar Saukas[/mention], [mention]R-Tape[/mention], [mention]pavero[/mention] and all the others who spend their free time on ZXDB. Also thanks to all of you submitting little bugs

I'm not sure what others think, but I always imagined the WoS database was pretty accurate, and I never expected the number of issues and corrections to surface.

Please be patient, and rest assured the team are doing their very best.

Regarding the marking of changes as 'done' let me speak to the others, and I'm sure we can find a solution.
User avatar
Stefan
Manic Miner
Posts: 793
Joined: Mon Nov 13, 2017 9:51 pm
Location: Belgium
Contact:

Re: New Database Model ZXDB

Post by Stefan »

Just thinking about the (lack of) relationship between issues found and fixes.

No point reinventing the wheel of an issue tracker.

Could zxdb be "hosted" on github with each record being a json file or so?
User avatar
Einar Saukas
Bugaboo
Posts: 3070
Joined: Wed Nov 15, 2017 2:48 pm

Re: New Database Model ZXDB

Post by Einar Saukas »

Stefan wrote: Thu May 23, 2019 9:09 pm Just thinking about the (lack of) relationship between issues found and fixes.

No point reinventing the wheel of an issue tracker.

Could zxdb be "hosted" on github with each record being a json file or so?
ZXDB is already hosted on Github:

https://github.com/zxdb/ZXDB

I mentioned it at the beginning of this thread :)

I agree it would be more appropriate to manage reported bugs using an issue tracker. However I very much doubt users here would bother to visit it regularly. The advantage of posting bug reports directly in this forum is that more people can see and help validate (or dispute) each report.

The main reason we have so many bugs still pending is that, for a long time, I was the only person working on reported issues. For instance, I have added most missing cover artist credits reported in this thread until recently. However I had to split my free time with other activities (implementing ZX-Ed, improving ZXDB structure, organizing releases, etc) so I was lagging behind. Fortunately [mention]druellan[/mention] recently joined the ZXDB team specifically to work on these issues. His main priority is processing reported bugs and he's fixing them faster than people are reporting new ones. There's a huge backlog to process so it's going to take a while, but at current rate he's going to catch up eventually. His contributions are not as visible as new titles added by [mention]R-Tape[/mention] but he's doing a great job, we just need to give him more time!
User avatar
Einar Saukas
Bugaboo
Posts: 3070
Joined: Wed Nov 15, 2017 2:48 pm

Re: New Database Model ZXDB

Post by Einar Saukas »

djnzx48 wrote: Thu May 23, 2019 10:01 amI was sort of wondering the same thing about these bugfixes in general. I know ZXDB doesn't deal with loading screens (or does it?), but there was a report made about a wrong loading screen over a year ago and nothing happened since then ;)
This particular case is tricky, because the loading screen is wrong but fixing it would make it look worse... Frankly we have been prioritizing other "easier" fixes for now while we give this case more thought.

djnzx48 wrote: Thu May 23, 2019 10:01 amI guess the problem is related to what was brought up in this thread about marking ZXDB posts, in that you don't actually know what happens with these fixes, whether they get noticed or just left to be forgotten in the depths of some old thread.
Yes, that's part of the problem. [mention]druellan[/mention] is currently marking processed fixes with "likes" but they are hardly visible. Ideally we should have a more visible method to identify fixed issues, so everyone can notice our progress and validate if the fixes were applied correctly. Also it will make our work more productive. However I don't really like my own proposal to edit other people's posts... So I'm not sure what to do about that.
User avatar
Stefan
Manic Miner
Posts: 793
Joined: Mon Nov 13, 2017 9:51 pm
Location: Belgium
Contact:

Re: New Database Model ZXDB

Post by Stefan »

Einar Saukas wrote: Thu May 23, 2019 10:18 pm ZXDB is already hosted on Github:

https://github.com/zxdb/ZXDB

I mentioned it at the beginning of this thread :)
One SQL file containing all definitions and data is indeed "hosted", but also unmanageable.

I was thinking more something along the lines of a json / sql file per entry allowing direct linking between issues and "sources". On the other hand abusing github as a database and putting 100,000 (?) files into a repository may be a bit over the top (and may violate some github fair use policy)

But, whatever is easiest for the contributors is best.
User avatar
StooB
Dynamite Dan
Posts: 1073
Joined: Mon Oct 22, 2018 11:03 am
Contact:

Re: New Database Model ZXDB

Post by StooB »

Einar Saukas wrote: Thu May 23, 2019 11:12 pm Yes, that's part of the problem. @druellan is currently marking processed fixes with "likes" but they are hardly visible. Ideally we should have a more visible method to identify fixed issues, so everyone can notice our progress and validate if the fixes were applied correctly. Also it will make our work more productive. However I don't really like my own proposal to edit other people's posts... So I'm not sure what to do about that.
Isn't the easiest solution to have a Little Bugs sub-forum with a separate post for each bug or group of related bugs? You could track fixes by either adding prefixes like "FIXED" to post titles, or alternatively have further sub-forums for open/fixed/rejected issues and move posts between them.
User avatar
R-Tape
Site Admin
Posts: 6353
Joined: Thu Nov 09, 2017 11:46 am

Re: New Database Model ZXDB

Post by R-Tape »

StooB wrote: Fri May 24, 2019 7:21 am Isn't the easiest solution to have a Little Bugs sub-forum with a separate post for each bug or group of related bugs? You could track fixes by either adding prefixes like "FIXED" to post titles, or alternatively have further sub-forums for open/fixed/rejected issues and move posts between them.
That's going to be a lot of threads bumping up and down. Someone would still have to try and keep on top of things—it's a lot of work.

Personally I think bugs1, bugs2 etc can work well, as things are put into manageable chunks that can be blitzed when someone has time. The whole thread can be locked when it gets big enough.

As the main person currently doing corrections at the moment, what do you prefer [mention]druellan[/mention] ? Would it be useful to have a volunteer (perhaps [mention]StooB[/mention] ?) to keep on top of a finalised corrections spreadsheet, that is basically a list of ZXDB ID and a concise explanation of the correction needed? (Or would you rather be left alone to get on with it?!)
hikoki
Manic Miner
Posts: 576
Joined: Thu Nov 16, 2017 10:54 am

Re: New Database Model ZXDB

Post by hikoki »

Did you know google spreadsheets are nicely displayed in this forum? try to make one and post the shared link in here
User avatar
StooB
Dynamite Dan
Posts: 1073
Joined: Mon Oct 22, 2018 11:03 am
Contact:

Re: New Database Model ZXDB

Post by StooB »

R-Tape wrote: Fri May 24, 2019 9:57 am
That's going to be a lot of threads bumping up and down. Someone would still have to try and keep on top of things—it's a lot of work.
It's the same number of bugs whether they're in separate threads or in a couple of big threads. So it's a lot of work either way!

R-Tape wrote: Fri May 24, 2019 9:57 am Personally I think bugs1, bugs2 etc can work well, as things are put into manageable chunks that can be blitzed when someone has time. The whole thread can be locked when it gets big enough.
If you lock the threads then how does [mention]durellan[/mention], or anyone else for that matter, respond to any of the bugs? You'd end up with a situation where you could have a reply in the 5th bugs thread which is about a bug posted in the 1st bug thread. There's still the problem of marking bugs as fixed more visibly, without liking or editing.
User avatar
StooB
Dynamite Dan
Posts: 1073
Joined: Mon Oct 22, 2018 11:03 am
Contact:

Re: New Database Model ZXDB

Post by StooB »

hikoki wrote: Fri May 24, 2019 11:13 am Did you know google spreadsheets are nicely displayed in this forum? try to make one and post the shared link in here
Here's one. It doesn't seem to embed - perhaps there's a setting needs to be changed?

https://docs.google.com/spreadsheets/d/ ... ED/pubhtml
User avatar
djnzx48
Manic Miner
Posts: 729
Joined: Wed Dec 06, 2017 2:13 am
Location: New Zealand

Re: New Database Model ZXDB

Post by djnzx48 »

I think the one that got a built-in preview was an Excel spreadsheet stored in Google Drive. Maybe it doesn't work for Google Docs spreadsheets.
User avatar
R-Tape
Site Admin
Posts: 6353
Joined: Thu Nov 09, 2017 11:46 am

Re: New Database Model ZXDB

Post by R-Tape »

StooB wrote: Fri May 24, 2019 11:33 am If you lock the threads then how does @durellan, or anyone else for that matter, respond to any of the bugs? You'd end up with a situation where you could have a reply in the 5th bugs thread which is about a bug posted in the 1st bug thread.
The thread could be locked by consensus after the discussion has pretty much died down, or it doesn't have to be locked, as long as new bugs get put in the new thread. It's just an idea.
User avatar
Einar Saukas
Bugaboo
Posts: 3070
Joined: Wed Nov 15, 2017 2:48 pm

Re: New Database Model ZXDB

Post by Einar Saukas »

djnzx48 wrote: Thu May 23, 2019 10:12 am How about just changing the subject line of the post? It might be safer than modifying the post's contents.
This is better than modifying posts! Although this is not a perfect solution, because I assume that editing the subject of the first post in a thread to add "[FIXED]" would change the title of the entire thread...

There's also the problem that someone would still need to edit the post in order to change its subject. What if this person deletes part of the post content by mistake without noticing? Not too unlikely when editing it from mobile, for instance.

IMHO the ideal solution would be a clickable marking for "fixed" on each post, similar to "like" but only available for users working on fixes.
hikoki
Manic Miner
Posts: 576
Joined: Thu Nov 16, 2017 10:54 am

Re: New Database Model ZXDB

Post by hikoki »

djnzx48 wrote: Fri May 24, 2019 11:55 am I think the one that got a built-in preview was an Excel spreadsheet stored in Google Drive. Maybe it doesn't work for Google Docs spreadsheets.
indeed, xls excel files are embedded in and can be edited by any collaborator from within google drive.

https://drive.google.com/open?id=1poQ4p ... V-cTAryEk7
User avatar
StooB
Dynamite Dan
Posts: 1073
Joined: Mon Oct 22, 2018 11:03 am
Contact:

Re: New Database Model ZXDB

Post by StooB »

Einar Saukas wrote: Fri May 24, 2019 12:40 pm IMHO the ideal solution would be a clickable marking for "fixed" on each post, similar to "like" but only available for users working on fixes.
This extension almost does that, but it only works on topics rather than individual posts.

Image
User avatar
druellan
Dynamite Dan
Posts: 1466
Joined: Tue Apr 03, 2018 7:19 pm

Re: New Database Model ZXDB

Post by druellan »

R-Tape wrote: Fri May 24, 2019 9:57 am As the main person currently doing corrections at the moment, what do you prefer @druellan ? Would it be useful to have a volunteer (perhaps @StooB ?) to keep on top of a finalised corrections spreadsheet, that is basically a list of ZXDB ID and a concise explanation of the correction needed? (Or would you rather be left alone to get on with it?!)
I really don't like the idea to have the information spread on different mediums, that usually end up with discrepancies and an overhead of work. I like the flexibility of the public spreadsheet, but:

A spreadsheet is really not a good bug reporting platform at all, it makes the information difficult to post, edit and read. Not everyone has a Google account, that means to leave everything open and editable, and that's kind of concerning, not because someone is going to sabotage or post nonsense (can happen), but it is easy delete or alter the information by mistake if you are just looking.

The sheet works best as an index for all the bugs reported and solved (or not) on the forum. I'm already doing that myself, just to keep track on everything I'm looking at, perhaps is not a big deal to open a public spreadsheet and add everything there.

Could be nice to use such a sheet as a triage process for the reports, but makes no sense of doing that if I'm the only one focusing on this area.

Besides that, I think we still need to inform something to the person that posted the report, without the hassle of looking on a spreadsheet, this also helps me to quickly catch posts I missed, something that can happen when a discussion fires up. If we can't find a plugin or a hack to "mark" a post, the best way to do it is just edit the post and add some kind of tag, like [IN PROGRESS] or [DONE]. Those tags can also be color PNGs and added to the forum as Smilies.

But for my own requirements, the "like" approach works, so we can start with a public sheet of reports solved or in progress if you like, and go on from there.
User avatar
druellan
Dynamite Dan
Posts: 1466
Joined: Tue Apr 03, 2018 7:19 pm

Re: New Database Model ZXDB

Post by druellan »

druellan wrote: Fri May 24, 2019 2:16 pm The sheet works best as an index for all the bugs reported and solved (or not) on the forum. I'm already doing that myself, just to keep track on everything I'm looking at, perhaps is not a big deal to open a public spreadsheet and add everything there.
Just to be clear, I'm doing this by REPORT, and not by ENTRY, using the post ID and author as reference. By entry can be problematic, since a report can target a company or a publisher, not only a title.
User avatar
Einar Saukas
Bugaboo
Posts: 3070
Joined: Wed Nov 15, 2017 2:48 pm

Re: New Database Model ZXDB

Post by Einar Saukas »

druellan wrote: Fri May 24, 2019 2:16 pmThe sheet works best as an index for all the bugs reported and solved (or not) on the forum. I'm already doing that myself, just to keep track on everything I'm looking at, perhaps is not a big deal to open a public spreadsheet and add everything there.
Instead of opening a new public spreadsheet, you could simply share your already existing spreadsheet, as read-only. Thus you wouldn't have any extra work besides what you are already doing.

Although I'm still convinced the ideal solution would be some kind of "fixed" tag in each post.
User avatar
R-Tape
Site Admin
Posts: 6353
Joined: Thu Nov 09, 2017 11:46 am

Re: New Database Model ZXDB

Post by R-Tape »

Sorry [mention]druellan[/mention] [mention]Einar Saukas[/mention] , I actually meant someone else (not me, you, or Dru) does the work of distilling correction discussions into a concise list (spreadsheet or whatever), so that it's easier for Dru (or myself when I find time) to work through them.

My thinking was that it is already enough work for someone to create the SQL for the corrections without having to constantly keep on top of the forum discussions.

But maybe Dru is already happy with the current situation. Sorry! I don't know enough about what's happening there, corrections are not my priority, and I shouldn't have poked my nose in :D
hikoki
Manic Miner
Posts: 576
Joined: Thu Nov 16, 2017 10:54 am

Re: New Database Model ZXDB

Post by hikoki »

[mention]druellan[/mention] you may want to give bookmarks.google.com a try as it allows to add tags and comments to every post url that you bookmark. From time to time you could export to html and publish that page. An example: speccy21.updog.co/googlebookmarks.html
User avatar
PeterJ
Site Admin
Posts: 6854
Joined: Thu Nov 09, 2017 7:19 pm
Location: Surrey, UK

Re: New Database Model ZXDB

Post by PeterJ »

Lets wait for [mention]druellan[/mention] to reply until we go into detailed options. I'm a fan of keep it simple.
User avatar
druellan
Dynamite Dan
Posts: 1466
Joined: Tue Apr 03, 2018 7:19 pm

Re: New Database Model ZXDB

Post by druellan »

PeterJ wrote: Fri May 24, 2019 4:02 pm Lets wait for @druellan to reply until we go into detaied options. I'm a fan of keep it simple.
I'm OK on keeping a spreadsheet about the reports I'm working on. Just for the record, I'm currently not using a sheet but just a plain simple TXT file, but I have no problem on using a sheet instead.

About a way to proper "mark" the posts I'm working on, the "like" approach is not ideal but works for me, but for other users is kind of confusing, since all "likes" look identical, so, a better way to mark those reports is welcome.

If you don't want to mess with the forum and install a MOD, there are a couple of solutions involving Javascript that might work (just detect a post was liked by me an trigger some actions), but the simplest way is to edit the original POST (I'm going to need permissions to do that on the topic) and add a label like [DONE], [WON'T FIX], etc. We can use the smilies to add a nice graphical one.
User avatar
PeterJ
Site Admin
Posts: 6854
Joined: Thu Nov 09, 2017 7:19 pm
Location: Surrey, UK

Re: New Database Model ZXDB

Post by PeterJ »

druellan wrote: Sun May 26, 2019 12:07 pm
PeterJ wrote: Fri May 24, 2019 4:02 pm Lets wait for @druellan to reply until we go into detailed options. I'm a fan of keep it simple.
About a way to proper "mark" the posts I'm working on, the "like" approach is not ideal but works for me, but for other users is kind of confusing, since all "likes" look identical, so, a better way to mark those reports is welcome.

If you don't want to mess with the forum and install a MOD, there are a couple of solutions involving Javascript that might work (just detect a post was liked by me an trigger some actions), but the simplest way is to edit the original POST (I'm going to need permissions to do that on the topic) and add a label like [DONE], [WON'T FIX], etc. We can use the smilies to add a nice graphical one.
OK, [mention]druellan[/mention] now has appropriate rights to this sub-forum so will be able to edit posts to mark as appropriate.
User avatar
Einar Saukas
Bugaboo
Posts: 3070
Joined: Wed Nov 15, 2017 2:48 pm

Re: New Database Model ZXDB

Post by Einar Saukas »

A new ZXDB update is available!

This new version contains game review scores from all major magazines (many thanks to Chris from ZXSR for this information!), an updated list of all available RZX recordings (many thanks to Daren from RZX Archive for this information!), and additional hires inlays (many thanks to Pavel as usual!)
User avatar
PeterJ
Site Admin
Posts: 6854
Joined: Thu Nov 09, 2017 7:19 pm
Location: Surrey, UK

Re: New Database Model ZXDB

Post by PeterJ »

This latest update has been applied. Thanks again to Chris AKA [mention]Vampyre[/mention] for giving us access to the ZXSR database, [mention]Daren[/mention] for all his videos, and [mention]pavero[/mention] for the continuous flow of updated images. We will be working on adding the additional fields from ZXSR to the game pages.
Post Reply