New Database Model ZXDB

This is the place for general discussion and updates about the ZXDB Database. This forum is not specific to Spectrum Computing.
User avatar
Einar Saukas
Manic Miner
Posts: 692
Joined: Wed Nov 15, 2017 2:48 pm

Re: New Database Model ZXDB

Post by Einar Saukas » Thu Apr 18, 2019 8:26 pm

A new ZXDB update is available!

It contains lots of new titles (thanks @R-Tape!), lots of corrections (thanks @druellan!), lots of improved inlays and instructions (thanks @pavero!), and probably lots of other stuff I can't remember right now :)
3 x

User avatar
PeterJ
Site Admin
Posts: 1133
Joined: Thu Nov 09, 2017 7:19 pm
Location: Surrey, UK

Re: New Database Model ZXDB

Post by PeterJ » Thu Apr 18, 2019 9:06 pm

Einar Saukas wrote:
Thu Apr 18, 2019 8:26 pm
A new ZXDB update is available!

It contains lots of new titles (thanks @R-Tape!), lots of corrections (thanks @druellan!), lots of improved inlays and instructions (thanks @pavero!), and probably lots of other stuff I can't remember right now :)
Thanks everyone. I will update SC over the weekend.
1 x

User avatar
Einar Saukas
Manic Miner
Posts: 692
Joined: Wed Nov 15, 2017 2:48 pm

Re: New Database Model ZXDB

Post by Einar Saukas » Thu Apr 25, 2019 4:05 pm

Here's some technical information that's only really relevant to websites based on ZXDB:

As part of our ongoing effort to improve ZXDB structure, we will soon remove table "formattypes", and the corresponding column "formattype_id" from tables "downloads", "extras", "labelfiles", and "magfiles".

This column currently stores the data format of each file. For instance, if it's a +3 disk image, then it contains the ID for "+3 disk (DSK)". If it's a PDF, it contains the ID for "Document (PDF)". And so on...

The problem is, this information is completely redundant. Just looking at the filename extension will give you exactly the same information. All +3 disk files have extension ".dsk.zip", all PDF documents have extension ".pdf", all TXT documents have extension ".txt", etc. Sometimes the people contributing with ZXDB updates have made mistakes filling this information, so it doesn't match the actual file type. Therefore this is a field that's unnecessary at best, plain wrong at worst.

There's now a new table called "extensions", that store an explanation for each file extension supported by ZXDB. Websites based on ZXDB can simply check the data formats themselves. If anyone needs help with SQL changes here (or anything else really), please let me know!

The main file extensions supported by ZXDB (as detailed in table "extensions") are listed below:

Code: Select all

.avi.zip            Video (AVI)
.ay.zip             Music (AY)
.d80.zip            Didaktik disk (D80)
.dck.zip            Timex expansion (DCK)
.dsk.zip            +3 disk (DSK)
.fdi.zip            TR-DOS disk (FDI)
.gif                Picture (GIF)
.html               Webpage (HTML)
.ifl                Screen dump 8x2 (IFL)
.img.zip            Disciple/+D disk (IMG)
.jpg                Picture (JPG)
.mdr.zip            Microdrive cartridge (MDR)
.mgt.zip            Disciple/+D disk (MGT)
.mid.zip            Music (MIDI)
.mlt                Screen dump 8x1 (MLT)
.mp3.zip            Music (MP3)
.mpg.zip            Video (MPG)
.opd.zip            Opus Discovery disk (OPD)
.p.zip              ZX81 tape (P)
.pdf                Document (PDF)
.png                Picture (PNG)
.pok                Pokes (POK)
.rom.zip            ROM image dump (ROM)
.rzx.zip            Game recording (RZX)
.scl.zip            TR-DOS disk (SCL)
.scr                Screen dump (SCR)
.slt.zip            Snapshot (SLT)
.sna.zip            Snapshot (SNA)
.szx.zip            Snapshot (SZX)
.tap.zip            Tape (TAP)
.trd.zip            TR-DOS disk (TRD)
.txt                Document (TXT)
.tzx.zip            Perfect tape (TZX)
.wdr.zip            Wafadrive wafer disk (WDR)
.wmv.zip            Video (WMV)
.z80.zip            Snapshot (Z80)
.z81.zip            ZX81 tape (81)
.zxk                ZX Vega keymap (ZXK)
_Fonts.zip          Fonts package
_SourceCode.zip     Source code package
This modification will happen gradually. For now, both formattypes 54 (document ".pdf") and 55 (document ".txt") will be removed in the next ZXDB update. OK?
4 x

User avatar
PeterJ
Site Admin
Posts: 1133
Joined: Thu Nov 09, 2017 7:19 pm
Location: Surrey, UK

Re: New Database Model ZXDB

Post by PeterJ » Thu Apr 25, 2019 6:50 pm

Thanks @Einar Saukas

Sounds good to me.
0 x

pavero
Microbot
Posts: 156
Joined: Sat Dec 09, 2017 11:49 pm
Location: Czech Republic
Contact:

Re: New Database Model ZXDB

Post by pavero » Fri May 17, 2019 6:13 pm

I agree, this information was really redundant.

But big number of PHP scripts as SC site will be probably complete nonfunctional after removing formattype_id attribute from ZXDB.
0 x

User avatar
Einar Saukas
Manic Miner
Posts: 692
Joined: Wed Nov 15, 2017 2:48 pm

Re: New Database Model ZXDB

Post by Einar Saukas » Fri May 17, 2019 6:51 pm

pavero wrote:
Fri May 17, 2019 6:13 pm
I agree, this information was really redundant.

But big number of PHP scripts as SC site will be probably complete nonfunctional after removing formattype_id attribute from ZXDB.
That's the reason I always announce these changes plenty of time in advance, implement changes as gradually as possible, and wait for feedback so site owners can let me know if there are any other issues I need to consider.

This way, even if sites like SC forget to make the required changes, they will probably just stop showing a text file icon or something similar. I doubt this would break anything important.
0 x

User avatar
PeterJ
Site Admin
Posts: 1133
Joined: Thu Nov 09, 2017 7:19 pm
Location: Surrey, UK

Re: New Database Model ZXDB

Post by PeterJ » Fri May 17, 2019 8:36 pm

This is especially important as ZXDB gets used in more and more projects.
0 x

pavero
Microbot
Posts: 156
Joined: Sat Dec 09, 2017 11:49 pm
Location: Czech Republic
Contact:

Re: New Database Model ZXDB

Post by pavero » Fri May 17, 2019 9:13 pm

Einar Saukas wrote:
Fri May 17, 2019 6:51 pm
This way, even if sites like SC forget to make the required changes, they will probably just stop showing a text file icon or something similar. I doubt this would break anything important.
I can't agree. Many SQL queries will be invalid and the page won't be loaded at all. Especially formattype_id in downloads table is critical.

Maybe some backward compatibility should be considered for the future ...
0 x

User avatar
Einar Saukas
Manic Miner
Posts: 692
Joined: Wed Nov 15, 2017 2:48 pm

Re: New Database Model ZXDB

Post by Einar Saukas » Fri May 17, 2019 10:05 pm

pavero wrote:
Fri May 17, 2019 9:13 pm
I can't agree. Many SQL queries will be invalid and the page won't be loaded at all. Especially formattype_id in downloads table is critical.
That could happen if I was simply dropping column "formattype_id" right away. But that's not the plan that I presented. Instead, this change will happen gradually.

In the next ZXDB update, this column will still exist, but it won't be populated anymore for formattypes 54 (document ".pdf") and 55 (document ".txt"). That's all. This change won't invalidate current SQL queries or break any pages.

Afterwards, I will let everybody know in advance about the next couple formattype values that won't be populated in the following ZXDB release. And so on.

Each website can choose to remove all references to formattypes at once. Or make the changes gradually as I announce them. Or simply leave it for now and deal with everything later when they have more free time, since the impact will be minimal for now.

pavero wrote:
Fri May 17, 2019 9:13 pm
Maybe some backward compatibility should be considered for the future ...
That's exactly what's being already considered!

We cannot stop improving ZXDB, just because many projects are using it. However every change we make is always planned carefully, announced in advance, explained in detail, and implemented gradually, in order to make it as easy as possible for others to adapt accordingly. That's the best trade-off possible between evolution and backward compatibility IMHO.

In this particular case, simply removing a redundant column is going to take several ZXDB releases, in order to make it easier for everybody else. I'm not in a hurry :)
2 x

User avatar
Einar Saukas
Manic Miner
Posts: 692
Joined: Wed Nov 15, 2017 2:48 pm

Re: New Database Model ZXDB

Post by Einar Saukas » Thu May 23, 2019 4:01 am

A new ZXDB update is available!

As usual, it contains several additional titles and hi-res inlays. Special thanks to @R-Tape, @druellan, and @pavero for their contributions :)
5 x

Post Reply