ZXDB based Open API

This is the place for general discussion and updates about the ZXDB Database. This forum is not specific to Spectrum Computing.

Moderator: druellan

User avatar
kolbeck
Manic Miner
Posts: 309
Joined: Mon Nov 13, 2017 9:04 pm

ZXDB based Open API

Post by kolbeck »

The other ZXDB based website http//zxinfo.dk - is driven by an API based backend, which makes it possible to build other types of apps without the need to understand the SQL structure of ZXDB, installing required software such as SQL database and having to maintain your own running instance of ZXDB.

The idea behind the API from the beginning was to make a common foundation for accessing ZXDB information (it actually started when only W*S was available), that could be used for mutiple purposes - from simple lookup to full featured websites such as ZXInfo. Openess has been a key factor from the beginning, so the code is available on Github and it is implemented on modern open source cross platform technologies such as NodeJS, Express-JS, REST, Swagger.

The current API behind ZXInfo is a swagger documented REST API, which you can access there:

http://api.zxinfo.dk/doc/

And the acutal code can be found on Github https://github.com/thomasheckmann/zxinfo-services, as with the code behind the ZXInfo website.

With this post I would like to open up for discussion of future requirements of such API, if it should be used by more than just my own website. What does it take to bring the API to a more community driven API for accessing ZXDB data?
https://api.zxinfo.dk/v3/ - ZXDB API for developers
zxinfo-file-browser - Cross platform app to manage your files
https://zxinfo.dk - another ZXDB frontend
hikoki
Manic Miner
Posts: 576
Joined: Thu Nov 16, 2017 10:54 am

Re: ZXDB based Open API

Post by hikoki »

Hi Thomas. Is there a way to sort new releases by date? year-month-day
Nomad
Manic Miner
Posts: 600
Joined: Thu Dec 28, 2017 12:38 pm

Re: ZXDB based Open API

Post by Nomad »

Just throwing it out there but being able to restrict searches to one language only would be fantastic.
User avatar
PeterJ
Site Admin
Posts: 6854
Joined: Thu Nov 09, 2017 7:19 pm
Location: Surrey, UK

Re: ZXDB based Open API

Post by PeterJ »

This looks great [mention]kolbeck[/mention] . Bringing ZXDB to a wider audience is fantastic.
hikoki
Manic Miner
Posts: 576
Joined: Thu Nov 16, 2017 10:54 am

Re: ZXDB based Open API

Post by hikoki »

hikoki wrote: Sun Mar 11, 2018 3:47 am Hi Thomas. Is there a way to sort new releases by date? year-month-day
Apart from Updates sorted by date, it would be nice getting a number of random games from any result list. Something like display only 5 random games from any narrowed query.
User avatar
kolbeck
Manic Miner
Posts: 309
Joined: Mon Nov 13, 2017 9:04 pm

Re: ZXDB based Open API

Post by kolbeck »

hikoki wrote: Sun Mar 11, 2018 1:29 pm
hikoki wrote: Sun Mar 11, 2018 3:47 am Hi Thomas. Is there a way to sort new releases by date? year-month-day
Apart from Updates sorted by date, it would be nice getting a number of random games from any result list. Something like display only 5 random games from any narrowed query.
The frontpage of ZXInfo already does that but it's not part of the API. Putting it on the TODO list

https://github.com/thomasheckmann/zxinf ... s/issues/3

/Thomas
https://api.zxinfo.dk/v3/ - ZXDB API for developers
zxinfo-file-browser - Cross platform app to manage your files
https://zxinfo.dk - another ZXDB frontend
User avatar
kolbeck
Manic Miner
Posts: 309
Joined: Mon Nov 13, 2017 9:04 pm

Re: ZXDB based Open API

Post by kolbeck »

hikoki wrote: Sun Mar 11, 2018 3:47 am Hi Thomas. Is there a way to sort new releases by date? year-month-day
Not yet - added to TODO list :-)

https://github.com/thomasheckmann/zxinf ... s/issues/4

/Thomas
https://api.zxinfo.dk/v3/ - ZXDB API for developers
zxinfo-file-browser - Cross platform app to manage your files
https://zxinfo.dk - another ZXDB frontend
User avatar
kolbeck
Manic Miner
Posts: 309
Joined: Mon Nov 13, 2017 9:04 pm

Re: ZXDB based Open API

Post by kolbeck »

Nomad wrote: Sun Mar 11, 2018 4:41 am Just throwing it out there but being able to restrict searches to one language only would be fantastic.
Great idea! Adding to TODO list

https://github.com/thomasheckmann/zxinf ... s/issues/5

/Thomas
https://api.zxinfo.dk/v3/ - ZXDB API for developers
zxinfo-file-browser - Cross platform app to manage your files
https://zxinfo.dk - another ZXDB frontend
User avatar
kolbeck
Manic Miner
Posts: 309
Joined: Mon Nov 13, 2017 9:04 pm

Re: ZXDB based Open API

Post by kolbeck »

So.. after some input - the ZXInfo API has been updated.

* Added a compact view mode option for getGameById and Search making it more friendly on low-end devices
* General data cleanup, removing duplicate data etc.
* getGameId now accepts ID with less than 7 digits with improved error handling
* Fixed some endpoints preventing timeout if parameters offset or size not supplied
* Updated API documentation

More info including test page here: http://api.zxinfo.dk/doc/
https://api.zxinfo.dk/v3/ - ZXDB API for developers
zxinfo-file-browser - Cross platform app to manage your files
https://zxinfo.dk - another ZXDB frontend
User avatar
kolbeck
Manic Miner
Posts: 309
Joined: Mon Nov 13, 2017 9:04 pm

Re: ZXDB based Open API

Post by kolbeck »

kolbeck wrote: Mon Mar 12, 2018 7:55 am
Nomad wrote: Sun Mar 11, 2018 4:41 am Just throwing it out there but being able to restrict searches to one language only would be fantastic.
Great idea! Adding to TODO list

https://github.com/thomasheckmann/zxinf ... s/issues/5

/Thomas
[mention]Nomad[/mention] Language filter has been implemented in API - http://api.zxinfo.dk/doc/#!/zxinfo/searchAndFilter
You can try it out on http://zxinfo.dk as well

/Thomas
https://api.zxinfo.dk/v3/ - ZXDB API for developers
zxinfo-file-browser - Cross platform app to manage your files
https://zxinfo.dk - another ZXDB frontend
Nomad
Manic Miner
Posts: 600
Joined: Thu Dec 28, 2017 12:38 pm

Re: ZXDB based Open API

Post by Nomad »

Image

This is good news! Thanks Thomas.
Ralf
Rick Dangerous
Posts: 2279
Joined: Mon Nov 13, 2017 11:59 am
Location: Poland

Re: ZXDB based Open API

Post by Ralf »

Great to see this project advancing!
User avatar
kolbeck
Manic Miner
Posts: 309
Joined: Mon Nov 13, 2017 9:04 pm

Re: ZXDB based Open API

Post by kolbeck »

hikoki wrote: Sun Mar 11, 2018 3:47 am Hi Thomas. Is there a way to sort new releases by date? year-month-day
[mention]hikoki[/mention]
Yes, just added 'sort' parameter to search (defaults to newest first) - see more at http://api.zxinfo.dk/doc/#!/zxinfo/searchAndFilter

To get newest 10 releases (SOFTWARE) just do this call: (in fact mode & sort can be left out, as default values are compact & date_desc)

http://api.zxinfo.dk/api/zxinfo/v2/sear ... 0&offset=0

BTW: Also updated to latest ZXDB just released.
https://api.zxinfo.dk/v3/ - ZXDB API for developers
zxinfo-file-browser - Cross platform app to manage your files
https://zxinfo.dk - another ZXDB frontend
hikoki
Manic Miner
Posts: 576
Joined: Thu Nov 16, 2017 10:54 am

Re: ZXDB based Open API

Post by hikoki »

^^^
Thanks!!
Is it possible to get a list from a certain month and year?
User avatar
kolbeck
Manic Miner
Posts: 309
Joined: Mon Nov 13, 2017 9:04 pm

Re: ZXDB based Open API

Post by kolbeck »

kolbeck wrote: Mon Mar 12, 2018 7:52 am
hikoki wrote: Sun Mar 11, 2018 1:29 pm Apart from Updates sorted by date, it would be nice getting a number of random games from any result list. Something like display only 5 random games from any narrowed query.
The frontpage of ZXInfo already does that but it's not part of the API. Putting it on the TODO list

https://github.com/thomasheckmann/zxinf ... s/issues/3

/Thomas
[mention]hikoki[/mention]

To get random results, use offset=random (size specifies number of documents, max 10 at the moment)
For example to fetch 8 random entries matching 'miner' use this call - try refresh browser :-)

http://api.zxinfo.dk/api/zxinfo/v2/sear ... set=random

/Thomas
https://api.zxinfo.dk/v3/ - ZXDB API for developers
zxinfo-file-browser - Cross platform app to manage your files
https://zxinfo.dk - another ZXDB frontend
User avatar
kolbeck
Manic Miner
Posts: 309
Joined: Mon Nov 13, 2017 9:04 pm

Re: ZXDB based Open API

Post by kolbeck »

hikoki wrote: Wed Mar 21, 2018 4:25 pm ^^^
Thanks!!
Is it possible to get a list from a certain month and year?
Not official, but I'll look into it :-)

/Thomas
https://api.zxinfo.dk/v3/ - ZXDB API for developers
zxinfo-file-browser - Cross platform app to manage your files
https://zxinfo.dk - another ZXDB frontend
Ralf
Rick Dangerous
Posts: 2279
Joined: Mon Nov 13, 2017 11:59 am
Location: Poland

Re: ZXDB based Open API

Post by Ralf »

Is it possible to get a list from a certain month and year?
Remember that 99% of titles won't have any data about month of release.
hikoki
Manic Miner
Posts: 576
Joined: Thu Nov 16, 2017 10:54 am

Re: ZXDB based Open API

Post by hikoki »

Thanks Thomas.
Many thanks again!!
To be frank, I don't know how to get a list of a certain year. Looking into months is just an interesting curiosity.

Yes, Ralf. I'd say that even dates have not been taken into account in some games from last years.
User avatar
kolbeck
Manic Miner
Posts: 309
Joined: Mon Nov 13, 2017 9:04 pm

Re: ZXDB based Open API

Post by kolbeck »

hikoki wrote: Wed Mar 21, 2018 4:55 pm Thanks Thomas.
Many thanks again!!
To be frank, I don't know how to get a list of a certain year. Looking into months is just an interesting curiosity.

Yes, Ralf. I'd say that even dates have not been taken into account in some games from last years.
[mention]hikoki[/mention] I've added 'year' as an option, for example to search for 'Andrew' in 2017 releases use:

http://api.zxinfo.dk/api/zxinfo/v2/sear ... 0&offset=0

As Ralf mentioned only a very tiny part of entries have month & day info, so I decided it was not worth adding these as filters - however the sort options will take them into consideration, if they exist.

/Thomas
https://api.zxinfo.dk/v3/ - ZXDB API for developers
zxinfo-file-browser - Cross platform app to manage your files
https://zxinfo.dk - another ZXDB frontend
User avatar
kolbeck
Manic Miner
Posts: 309
Joined: Mon Nov 13, 2017 9:04 pm

Re: ZXDB based Open API

Post by kolbeck »

So sum up ZXInfo API v1.3.0 has been released at http://api.zxinfo.dk/doc/

Different level of output details
All methods support a mode flag for output. Mode can be full or compact, and default is compact
For methods returning a single entry, compact mode returns the following fields:
fulltitle, yearofrelease, monthofrelease, dayofrelease, machinetype, numberofplayers, multiplayermode, multiplayermode, type, subtype, isbn, messagelanguage, originalprice, availability, knownerrors, remarks, spotcomments, score, publisher, releases, authors, roles, authored, authoring, controls, series, othersystems, contents, incompilations, booktypeins, additionals, mod_of, modified_by
For methods returning multiple entries, compact mode returns the following fields:
fulltitle, yearofrelease, monthofrelease, dayofrelease, type, subtype, authors, publishers, machinetype
Sorting
For methods returning multiple entries a sort flag is supported to specify sort order. Sort order can be:
  • title_asc - sort by title ascending
  • title_desc - sort by title descending
  • date_asc - sort by date (year, month, day) ascending (oldest first)
  • date_desc - sort by date (year, month, day) descending (newest first)
By default date_desc is used.

Display X random entries
The search method now supports returning X random entries from the search. Use size and offset to control result:
  • offset=random - return random documents
  • size=n - number of documents to return, max value for n is 10
New search filters
For the search method the following filters are added:
  • year - filter by release year
  • language - filter by messagelanguage
More flexible ID parsing
The method getGameById now supports id with less than 7 digits. It also handles invalid id and nothing found by returning http status 400 & 404

No more timeout if size or offset is missing
Parameters are now required for all methods returning multiple entries, and can be used for pagination.

/Thomas
https://api.zxinfo.dk/v3/ - ZXDB API for developers
zxinfo-file-browser - Cross platform app to manage your files
https://zxinfo.dk - another ZXDB frontend
User avatar
kolbeck
Manic Miner
Posts: 309
Joined: Mon Nov 13, 2017 9:04 pm

Re: ZXDB based Open API

Post by kolbeck »

IMPORTANT:

For now the API is https only, let me know if that's a problem.

/Thomas
https://api.zxinfo.dk/v3/ - ZXDB API for developers
zxinfo-file-browser - Cross platform app to manage your files
https://zxinfo.dk - another ZXDB frontend
User avatar
Einar Saukas
Bugaboo
Posts: 3070
Joined: Wed Nov 15, 2017 2:48 pm

Re: ZXDB based Open API

Post by Einar Saukas »

This API is awesome!!!

Also it's nice to see how much attention Thomas gives to people's suggestions and how fast he implements them. :)

This API is already being used by Dario's SpeccyBot, an interactive chat bot for Telegram that allows searching online for Spectrum related information. You can see it in action here:

Image

Unfortunately it's currently available in Spanish only. Hopefully it won't take too long to get an English version... For further information:

https://github.com/druellan/speccybot
User avatar
kolbeck
Manic Miner
Posts: 309
Joined: Mon Nov 13, 2017 9:04 pm

Re: ZXDB based Open API

Post by kolbeck »

Thank you [mention]Einar Saukas[/mention] - part of openness is to pay attention to the users, this case (potentially) all of you.

Let me also inform, that from now on the ZXDB API is hosted on a professional web hosting / datacenter - to ensure top quality performance, stability & availability.

/Thomas
https://api.zxinfo.dk/v3/ - ZXDB API for developers
zxinfo-file-browser - Cross platform app to manage your files
https://zxinfo.dk - another ZXDB frontend
User avatar
kolbeck
Manic Miner
Posts: 309
Joined: Mon Nov 13, 2017 9:04 pm

Re: ZXDB based Open API

Post by kolbeck »

Hello

The ZXInfo API has been updated to latest ZXDB as of March 28th.
https://api.zxinfo.dk/v3/ - ZXDB API for developers
zxinfo-file-browser - Cross platform app to manage your files
https://zxinfo.dk - another ZXDB frontend
User avatar
Einar Saukas
Bugaboo
Posts: 3070
Joined: Wed Nov 15, 2017 2:48 pm

Re: ZXDB based Open API

Post by Einar Saukas »

That's fast! Thank you!
Locked