Testing emulators using z80test-1.0. Quick results

Struggling with Fuse or trying to find an emulator with a specific feature. Ask your questions here.
User avatar
druellan
Dynamite Dan
Posts: 1466
Joined: Tue Apr 03, 2018 7:19 pm

Testing emulators using z80test-1.0. Quick results

Post by druellan »

A friend shared with me this little tool designed by Patrik Rak for testing Z80 instructions on emulated environments.
Download: https://www.dropbox.com/s/33he0chzu6v7x ... 0.zip?dl=0

The full set consists of 152 individual tests, that I decided to run on several of the emulators I have at hand. I captured only the end result, but the tool provides a concise list of both passed and failed tests.

SpecEmu v3.1 build 16/03/18
Image
Result: all tests passed (!)

Retro Virtual Machine 2.00 Alpha
Image
Result: 001 of 152 tests failed.

FUSE 1.5.3
Image
Result: 002 of 152 tests failed.

ZXSpin 0.7
Image
Result: 004 of 152 tests failed.

Spud 0.250
Image
Result: 005 of 152 tests failed.

ZERO 0.7.1
Image
Result: 003 of 152 tests failed.

ZEsarUX 6.1-RC
Image
Result: 008 of 152 tests failed.

UnrealSpeccy 0.38.2
Image
Result: 010 of 152 tests failed.

EightyOne 1.9
Image
Result: 014 of 152 tests failed.

EmuZWin 2.7
Image
Result: 015 of 152 tests failed.

Speccy 4.6
Image
Result: 088 of 152 tests failed.

And some DOS ones, thanks @Equinoxe for the testing!

X128
Image
Result: 086 of 152 tests failed.

R80 0.30
Image
Result: 013 of 152 tests failed.

RealSpec
Image
Result: 019 of 152 tests failed.
zx81
Microbot
Posts: 138
Joined: Sat Feb 17, 2018 9:33 pm

Re: Testing emulators using z80test-1.0. Quck results

Post by zx81 »

You has forget to test JSpeccy... ;)
User avatar
djnzx48
Manic Miner
Posts: 729
Joined: Wed Dec 06, 2017 2:13 am
Location: New Zealand

Re: Testing emulators using z80test-1.0. Quck results

Post by djnzx48 »

Interesting results!
I tried this out with Spectaculator, and it gave exactly the same results as Fuse: only the SCF and CCF tests failed, and with identical CRC values too.
Image
User avatar
Ast A. Moore
Rick Dangerous
Posts: 2640
Joined: Mon Nov 13, 2017 3:16 pm

Re: Testing emulators using z80test-1.0. Quck results

Post by Ast A. Moore »

Well, this one is not exactly a be-all and end-all test—and it only tests the CPU, not Spectrum emulation itself—but, yes, SpecEmu is a very accurate emulator. I’m curious, though, how’d you come across the alpha version of Retro Virtual Machine? Could you share a link, perhaps?
Every man should plant a tree, build a house, and write a ZX Spectrum game.

Author of A Yankee in Iraq, a 50 fps shoot-’em-up—the first game to utilize the floating bus on the +2A/+3,
and zasm Z80 Assembler syntax highlighter.
User avatar
druellan
Dynamite Dan
Posts: 1466
Joined: Tue Apr 03, 2018 7:19 pm

Re: Testing emulators using z80test-1.0. Quck results

Post by druellan »

zx81 wrote: Tue May 22, 2018 9:16 am You has forget to test JSpeccy... ;)
JSpeccy 0.93.1
Image
Result: all tests passed.

My respects!
User avatar
druellan
Dynamite Dan
Posts: 1466
Joined: Tue Apr 03, 2018 7:19 pm

Re: Testing emulators using z80test-1.0. Quck results

Post by druellan »

Ast A. Moore wrote: Tue May 22, 2018 10:56 am Well, this one is not exactly a be-all and end-all test—and it only tests the CPU, not Spectrum emulation itself—but, yes, SpecEmu is a very accurate emulator.
I agree. The CPU is a very important part of the package, but the emulator still needs to translate that to the screen, provide tools and interpreters for the different snaps and file formats, manage expansions and interfaces and other enhancements. I rather use a no-so-perfect CPU emulation with extra stuff that a perfect one that is barebone.

So, it is interesting to know how the different emulators compare using the same test tool, but if one emulator is better than another, that is really a personal opinion based on many factors.
Ast A. Moore wrote: Tue May 22, 2018 10:56 amI’m curious, though, how’d you come across the alpha version of Retro Virtual Machine? Could you share a link, perhaps?
Ah! That was a test version I have, and was provided by the author and is not for distribution (sorry). To be fair I was not supposed to post that screenshot here, but the results are so good that I thought it only shows how good that emulator is going to be.
Last edited by druellan on Tue May 22, 2018 2:55 pm, edited 1 time in total.
User avatar
Ast A. Moore
Rick Dangerous
Posts: 2640
Joined: Mon Nov 13, 2017 3:16 pm

Re: Testing emulators using z80test-1.0. Quck results

Post by Ast A. Moore »

druellan wrote: Tue May 22, 2018 2:41 pm That was a test version I have, and was provided by the author and not for distribution (sorry).
No biggie, man. I understand.
Every man should plant a tree, build a house, and write a ZX Spectrum game.

Author of A Yankee in Iraq, a 50 fps shoot-’em-up—the first game to utilize the floating bus on the +2A/+3,
and zasm Z80 Assembler syntax highlighter.
User avatar
druellan
Dynamite Dan
Posts: 1466
Joined: Tue Apr 03, 2018 7:19 pm

Re: Testing emulators using z80test-1.0. Quck results

Post by druellan »

+F Spectrum Emulator
Image
Q parameter error, 20

Qaop
Image
Result: 002 of 152 tests failed.
Impressive!
User avatar
RMartins
Manic Miner
Posts: 776
Joined: Thu Nov 16, 2017 3:26 pm

Re: Testing emulators using z80test-1.0. Quck results

Post by RMartins »

The description of Zero 0.7.1 says 005 but the image states only 3 errors.
Last edited by RMartins on Tue May 22, 2018 3:33 pm, edited 3 times in total.
User avatar
druellan
Dynamite Dan
Posts: 1466
Joined: Tue Apr 03, 2018 7:19 pm

Re: Testing emulators using z80test-1.0. Quck results

Post by druellan »

RMartins wrote: Tue May 22, 2018 3:07 pm The description of Zero 0.7.1 says 005 but the image states only 3 errors.
Oh! My bad! Sadly, I can't change it anymore. I also misspelled "quick" on the title :roll:
User avatar
RMartins
Manic Miner
Posts: 776
Joined: Thu Nov 16, 2017 3:26 pm

Re: Testing emulators using z80test-1.0. Quck results

Post by RMartins »

Which instruction are these ?

ALO A, A
ALO something

SLIA A

SRO (xy)
Last edited by RMartins on Tue May 22, 2018 4:04 pm, edited 1 time in total.
User avatar
RMartins
Manic Miner
Posts: 776
Joined: Thu Nov 16, 2017 3:26 pm

Re: Testing emulators using z80test-1.0. Quck results

Post by RMartins »

There are some inconsistent results too

Running `Z80Full.tap`
Gives 002 errors, for example.

and I see that instruction INC RR an INC XY (apparently they are different things (could it be second is for iX and iY ?) are returned as OK.
But these return as errors, when running `Z80ccf.tap`

It's not running a subset of the full test, or something fishy is going on.

NOTE: I was using FUSE, for test.
zx81
Microbot
Posts: 138
Joined: Sat Feb 17, 2018 9:33 pm

Re: Testing emulators using z80test-1.0. Quck results

Post by zx81 »

RMartins wrote: Tue May 22, 2018 3:43 pm There are some inconsistent results too

Running `Z80Full.tap`
Gives 002 errors, for example.

and I see that instruction INC RR an INC XY (apparently they are different things (could it be second is for iX and iY ?) are returned as OK.
But these return as errors, when running `Z80ccf.tap`

It's not running a subset of the full test, or something fishy is going on.

NOTE: I was using FUSE, for test.
That's true. I have Fuse 1.5.3 compiled from sources by myself running Fedora 28 (GCC 8.1.1). z80full fails two tests. But z80ccf fails 65 tests, first the 000 SELF TEST and then from INC RR fails many tests, JMP instructions included.

JSpeccy pass all tests OK from z80ccf. I'm very confident with Patrik's tests. I suspect from some problem with Fuse.
User avatar
RMartins
Manic Miner
Posts: 776
Joined: Thu Nov 16, 2017 3:26 pm

Re: Testing emulators using z80test-1.0. Quck results

Post by RMartins »

Independently of the quality of the test tool, it might have some specific dependency on a specific instruction, that if it fails, it might trigger a lot of other fails.

What I mean, is a simple fail, might trigger a lot more fails, because the test tool relies on that specific bad instruction, to make other tests.
User avatar
druellan
Dynamite Dan
Posts: 1466
Joined: Tue Apr 03, 2018 7:19 pm

Re: Testing emulators using z80test-1.0. Quck results

Post by druellan »

RMartins wrote: Tue May 22, 2018 3:43 pm It's not running a subset of the full test, or something fishy is going on.
The readme says: "- z80ccf - tests all flags after executing CCF after each instruction tested.", so I understand is not a subset of the z80full but a variation of the test?
User avatar
druellan
Dynamite Dan
Posts: 1466
Joined: Tue Apr 03, 2018 7:19 pm

Re: Testing emulators using z80test-1.0. Quck results

Post by druellan »

Z80Stealth 1.23
Image
Result: 025 of 152 tests failed.
User avatar
druellan
Dynamite Dan
Posts: 1466
Joined: Tue Apr 03, 2018 7:19 pm

Re: Testing emulators using z80test-1.0. Quck results

Post by druellan »

Patrik Rak however later discovered that the way how the flags 5 and 3 are affected after SCF/CCF actually depends on the previous instruction completed. In case of genuine Zilog CPU, if an instruction modifies the flags, the immediately following SCF/CCF does move of bits 5 and 3 from A to F, whereas if an instruction doesn't modify the flags (and after interrupt), the SCF/CCF does OR of bits 5 and 3 from A to F. In case of NEC and other clones, it is similar, except that instead of OR it does AND with some unknown value, making the result unreliable.
Source: https://faqwiki.zxnet.co.uk/wiki/Z80

The z80ccf.tap is perhaps related to that?
zx81
Microbot
Posts: 138
Joined: Sat Feb 17, 2018 9:33 pm

Re: Testing emulators using z80test-1.0. Quck results

Post by zx81 »

druellan wrote: Tue May 22, 2018 5:57 pm
Patrik Rak however later discovered that the way how the flags 5 and 3 are affected after SCF/CCF actually depends on the previous instruction completed. In case of genuine Zilog CPU, if an instruction modifies the flags, the immediately following SCF/CCF does move of bits 5 and 3 from A to F, whereas if an instruction doesn't modify the flags (and after interrupt), the SCF/CCF does OR of bits 5 and 3 from A to F. In case of NEC and other clones, it is similar, except that instead of OR it does AND with some unknown value, making the result unreliable.
Source: https://faqwiki.zxnet.co.uk/wiki/Z80

The z80ccf.tap is perhaps related to that?
Yes, it's related.
User avatar
RMartins
Manic Miner
Posts: 776
Joined: Thu Nov 16, 2017 3:26 pm

Re: Testing emulators using z80test-1.0. Quck results

Post by RMartins »

OK, then a great test to do, is save the app into a snapshot after break, and then run it on a real Spectrum, using DIVIde, DivMMC or similar.

I'm curious to run this on the ZX Next too :)
User avatar
MonkZy
Manic Miner
Posts: 278
Joined: Thu Feb 08, 2018 1:01 pm

Re: Testing emulators using z80test-1.0. Quck results

Post by MonkZy »

ZXBaremulator (version 3.H.5)

Image

All tests pass!!
Last edited by MonkZy on Tue May 22, 2018 6:51 pm, edited 1 time in total.
zx81
Microbot
Posts: 138
Joined: Sat Feb 17, 2018 9:33 pm

Re: Testing emulators using z80test-1.0. Quck results

Post by zx81 »

[mention]MonkZy[/mention]

Thanks by the test. JSpeccy shares the Z80 core with ZX Baremulator, simply translated from Java to C++.

Are you using a mini-LCD monitor or a Raspberry's LCD touch screen?. Your system is really cool... :D
User avatar
MonkZy
Manic Miner
Posts: 278
Joined: Thu Feb 08, 2018 1:01 pm

Re: Testing emulators using z80test-1.0. Quck results

Post by MonkZy »

The screen is a cheap Chinese clone from eBay, it is a copy of the Waveshare 5" display. It does have a touch screen but I don't use it as the driver support is not good. The case is designed and printed by me 8-) I will open a thread about it, one day. I am a huge ZXBaremulator fan.
Magnus
Dizzy
Posts: 61
Joined: Sat Jan 06, 2018 6:47 am
Location: Sweden

Re: Testing emulators using z80test-1.0. Quck results

Post by Magnus »

May I add the results for SoftSpectrum48 (it was a pain though :geek: ):
Image
My ZX Spectrum emulator project: https://softspectrum48.weebly.com.
User avatar
druellan
Dynamite Dan
Posts: 1466
Joined: Tue Apr 03, 2018 7:19 pm

Re: Testing emulators using z80test-1.0. Quck results

Post by druellan »

Magnus wrote: Sat May 26, 2018 5:58 pm May I add the results for SoftSpectrum48 (it was a pain though :geek: ):
Image
Nice work. I'm testing your emulator and I can say it looks promising. It plays the Gyroscope music quite well, something not all emulators are able to do.
User avatar
Pegaz
Dynamite Dan
Posts: 1209
Joined: Mon Nov 13, 2017 1:44 pm

Re: Testing emulators using z80test-1.0. Quck results

Post by Pegaz »

Spectramine v1.0.1b, passed all tests.

Image

I'm not surprised, Woody has already praised this emulator, as very precise...
Post Reply