Testing emulators using z80test-1.0. Quick results

[Lethargeek]

afaik memptr test is very reliable for all the range and might fail some subtests on eastern Z80 clones only; scf/ccf effects is another story

[PeterJ]

ES.pectrum

Not an emulator I had used before but seems to have a support for a large range of machines:

Failed 9 of 152

http://www.habisoft.com/espectrum/EN.htm#Descargas

http://www.habisoft.com/espectrum/Manual/EN.htm



Emulation of official models: 16, 48, 48+ (Spanish, Swedish, Arab), 48 NTSC, 128, 128+ (Spanish), grey +2 (Spanish, English, French, Arab), black +2 (Spanish, English, Arab) v4.0 and v4.1, +3 (Spanish, English) v4.0 and v4.1, Timex TC2048, Timex TC2068, Timex TS2068, Unipolbrit Komputer 2086.
Emulation of clones: Inves+, Tk90x (Portuguese, Argentinian), Tk95 (Portuguese, Argentinian), Orel BK-08, Pentagon 128, Pentagon 512, Pentagon 1024 SL v2.x, Scorpion ZS-256, Scorpion ZS-256 Turbo+, Leningrad, Dubna 48K, ATM-Turbo, ATM-Turbo 2+, Ice-Felix HC-91, BK-0001, Foton-IK03.
Emulation of mods: +2e (Spanish, English), +3e (Spanish, English), Spanish Pentagon.
Emulation of virtual ones: Spec256-48, Spec256-128.
Recognized formats:
Snapshot: Z80, SNA, SP, FSU, SZX, EZX, ESP (native).
Tape: TAP, TZX, PZX, CSW, WAV, VOC.
Disc: DSK, FDI, TRD, SCL, Hobbeta, MDR (if it can be considered as one).
Actions: RZX.
Screen: PNG, GIF (animated), SCR.
Cartridge: DCK, ROM (IF2, Dandanator), BIN.
Video: WMV.
Storage: HDF, RAW / BIN, IMG.
Emulation of hardware:
DivMMC.
Multiface 1, 128, +3, SpecMate.
Dandanator, including old versions.
Data-Gear (DMA).
PDS (development system). Requires a special DosBox version for linking.
Turbo Sound.
Covox, Specdrum.
16c modes in Pentagon.
64 / 4096 colour palette in Pentagon.
ULA+.
ZX1 extras (raster interruption, turbo, 128x96 mode).
Timex video modes.
Z80_GFX and 256 colour modes (Spec256, EmuzWin).
Betadisk: Beta, Beta+, Beta 128, and clones: Apina, CAS, Arcade, IDS 91, CBI 95.
Joysticks: Cursor / Protek / AGF, Sinclair 1 y 2, Fuller, Kempston.
Mouse: Kempston.
Guns: Gunstick, Lightgun.
Turbo modes.

[Patrik Rak]

They're a useful tool for emulator authors, but a less useful tool for judging the authenticity of designs. The most compatible Z80 code is that which uses only officially documented instructions. Software that seeks to trip up emulators by making use of private implementation details only succeeds in making itself incompatible with genuine Z80s.

Well, I would say that you can still use it to judge the quality of the Z80 emulation to some extent regardless.

If the "full" test fails after any of SCF/CCF/SCF+CCF/CCF+SCF, it means it implements other SCF/CCF behaviour than authentic NMOS Zilog Z80, but it's rather esoteric indeed. In such case, the "postccf" test will inevitably fail half of the tests, that's expected. However, until 2012, none of the emulators got this right, and some real CPUs differ as well, so I guess it's something we can live with.

If the "full" test fails after BIT N,(HL/IX/IY) instructions, it means it doesn't implement MEMPTR properly. In such case, the "memptr" test will very likely fail many tests, that's expected too. Of course, MEMPTR was nailed down much earlier, so even if it is obscure, authors should consider emulating it. Unlike the SCF/CCF, I don't think there is any real Z80 CPU which differs regarding MEMPTR.

If the "full" test fails after all IN instructions, the emulator quite likely incorrectly emulates bit 6 of port #FE. While it's not Z80 feature per-se, this should be obviously fixed, as it can impact real games.

Anything else means the z80 emulation is far from perfect and it should be fixed if possible.

So, while you can argue the test is synthetic, seeing numbers greater than 4 means real incompatibilities... The more of those, the worse Z80 emulation, at least that's how I see it (despite the fact that Z80 emulation alone doesn't make a great emulator, of course).

[Pegaz]

I just tested the ZX ULAX emulator and I'm not surprised that it passed all the tests.
Great job Lethargeek, I hope you'll add emulation of classic Sinclair 48k/128k models in the future.

[Lethargeek]

Thanks, [mention]Pegaz[/mention]! Ihope i will add original models too. I chose Pentagon to have faster and simpler start without the contention troubles and other things non-essential for the main goal. As for these tests, i mostly followed the FUSE source code for flag effects.

[azesmbog]

ES.pectrum

Failed 9 of 152

In defense of a good emulator
This test has not determined which machine to run it on.
On zx48 - 9 errors, but if you run on zx128 - only 2 errors, insignificant.

And this is one of the few emulators that can run (and pass!) Another interesting test.
Timing_Tests-48k_v1.0
And which works great with z80DMA, it can generally be units

[PeterJ]

Agreed [mention]azesmbog[/mention]. It's an excellent emulator with lots of nice features.

[Lethargeek]

This test has not determined which machine to run it on.
On zx48 - 9 errors, but if you run on zx128 - only 2 errors, insignificant.

ummm... this test is pure cpu thing afaiu, so different results with different models is another bug in itself

[azesmbog]

I partially agree, but in the test there is no mention anywhere that it is only zx48, and that different results depending on the configuration are probably the problems of the test, and not the emulator being tested. is not it ?

[Lethargeek]

I partially agree, but in the test there is no mention anywhere that it is only zx48,

because it is not "only zx48", it's "only z80" - doesn't depend on the 48 rom contents (like another cpu test) and doesn't use any 128+ ports

and that different results depending on the configuration are probably the problems of the test, and not the emulator being tested. is not it ?

definitely must be an emulator problem (and a very strange one)

[PeterJ]

In defense of a good emulator

Just noticed that among it's many great features, this emulator supports the Russian ATM2 machines. This will allow me to start looking at the titles that [mention]Alone Coder[/mention] mentioned in another thread.

The only issue I have found is the Microdrive support is broken, which I have been told will be fixed in a future release. It's a very good all round emulator

[ketmar]

sorry for shameless self-promotion (but nobody knows about my Z80 emulation core anyway, so i had to do it myself ;-): Zymosis (and ZXEmuT emulator based on it) only failing SCF/CCF tests. one test failed in "full", no failed tests in "memptr".

[ketmar]

upd: the bug with SCF/CCF was actually in interrupt acknowledgement code, so Zymosis passes all tests now.

[azesmbog]

sorry for shameless self-promotion (but nobody knows about my Z80 emulation core anyway, so i had to do it myself

Well, not really to anyone )
I know and remember. And I remember how you set up Linux for me 5-10 years ago by remote and the emulator started Since then, ownership of Linux has not gotten better for me.
Can you check a couple of my tests in 48k mode?)

[ketmar]

sorry for shameless self-promotion (but nobody knows about my Z80 emulation core anyway, so i had to do it myself ;-)

Well, not really to anyone :))
I know and remember. And I remember how you set up Linux for me 5-10 years ago by remote and the emulator started :) Since then, ownership of Linux has not gotten better for me.
Can you check a couple of my tests in 48k mode?)

yeah, i remember you too! you helped me alot back then, and your name is in THANKS file for Zymosis. ;-)

but i think we'd better move to Zymosis topic, to avoid derailing.

[arjun]

[mention]druellan[/mention] Wow. Thanks for the comprehensive tests. I am aware of the SCF tests failing on Zero, of course. I had consciously decided not to implement the "fixes" because at that point I was burnt out with the emulator and there seemed to be little interest in it (the emulator) anyway. I didn't see the point in trying to implement yet another esoteric feature that no one else would care about apart from me. And like I said, I was just too lazy at that point.

Over the past couple of years I have been tinkering with some parts of Zero (for eg, I rewrote the RZX emulation so that it's a bit more accurate and the RZX file output is smaller in size). But it's been slow going and v0.7.2 has been stuck in development hell. I've also been re-tooling the core emulation and that's broken a few things in the process (refactoring goal achieved!). For eg, now the dev build (unreleased) of Zero I have, fails 9 of the z80tests. Progress!

Anyway, I'll probably work on the emulator so that it at least passes the Z80 tests because I have a bit of OCD that way. Must. Pass. All. Tests. Although, I wouldn't hold my breath or anything. And in the end I may actually not bother with passing all the tests, but I'll give it a shot nevertheless.

[druellan]

Oh! Welcome to the forums [mention]arjun[/mention] So glad you are still working on the emulator, is one of my favorites for quick testing things.

About the test, I started the thread as a curiosity, but evolved into something pretty interesting, and yup!, Some authors felt the necessity to go back to their code (sorry/no-sorry about that, hehe) but ultimately and looking the improvements over the old projects, I think nobody is going to say the emulators tested here are lacking, and to the contrary, the level of accuracy is quite impressive considering even original hardware can fail the tests.

Hope to see you around!

[akeley]

I ran the "full" test on my MiSTer's ZX Spectrum core (FPGA). It failed 9 / 152 tests.

The failures were:
001 scf
002 scf
089 to 095 (IN something)

Can anybody comment on what these things mean? Do these "failures" have effect on running games etc?

There are also some settings in the core, like
port#FF: Timex / SAA1099
ULA+: on / off
Snow bug: on / off

I ran the test twice: one was with Timex & ULA+ on, the other wirh SAA1099 and ULA+ off. Do you think these settings can mess with the tests?
Did not change the Snow bug, I think it was "off" for both tests.

[ketmar]

I ran the "full" test on my MiSTer's ZX Spectrum core (FPGA). It failed 9 / 152 tests.

The failures were:
001 scf
002 scf
089 to 095 (IN something)

Can anybody comment on what these things mean? Do these "failures" have effect on running games etc?

nope. scf/ccf tests are testing the very obscure internal Z80 implementation detail. some genuine hardware Z80 fails those tests too (it depends mostly of cmos/nmos tech), and there are no programs exploiting those features, i believe (except maybe some code written to specifically defy emulators). it's nice to have that thing for a feature list, but it hardly has any visible effects.

[Lethargeek]

I ran the "full" test on my MiSTer's ZX Spectrum core (FPGA). It failed 9 / 152 tests.

The failures were:
001 scf
002 scf
089 to 095 (IN something)

Can anybody comment on what these things mean? Do these "failures" have effect on running games etc?

IN failures might affect even some games
it depends on what's exactly wrong with the flags
do these fail the z80doc test as well?

[azesmbog]

I ran the "full" test on my MiSTer's ZX Spectrum core (FPGA). It failed 9 / 152 tests.

MiSTer usually did not pass the first two tests, but Sorgelig fixed everything in the T80 long ago and should pass all tests. In ZX UNo, the T80 core is the same, and as far as I remember it passes all the tests.

[akeley]

do these fail the z80doc test as well?

The first two (scf) pass in z80doc, the INs still fail.

Here are 2 scr$, first is from full, second from doc.

MiSTer usually did not pass the first two tests, but Sorgelig fixed everything in the T80 long ago and should pass all tests. In ZX UNo, the T80 core is the same, and as far as I remember it passes all the tests.

Can't be that long ago, because some big fixes came only recently (cycle accuracy/conention) from the Mist core. In any case, I've just updated everything and as you can see it still happens.

Like I said in my previous post there are some settings in the core, can they affect the results?

[Lethargeek]

The first two (scf) pass in z80doc, the INs still fail.

Bad news. Especially bad for "IN A, (N)" as it shouldn't affect any flags at all. Or it's even worse if it's not about the flags.

[azesmbog]

ZXSEC
http://cngsoft.no-ip.org/cpcec.htm

Interesting emulator, 2-in-1. I didn’t come across it before, and I didn’t find it in the search on the forum.
Passes all tests except my border tests. Timings are not very accurate.

[PeterJ]

ZXSEC
I didn’t come across it before, and I didn’t find it in the search on the forum.

The included CPC emulator is good. I like the idea of having two emulators in one package!