Testing emulators using z80test-1.0. Quick results

Struggling with Fuse or trying to find an emulator with a specific feature. Ask your questions here.
polomint
Drutt
Posts: 29
Joined: Mon Nov 13, 2017 1:50 pm

Re: Testing emulators using z80test-1.0. Quick results

Post by polomint »

DarkTrancer wrote: Fri May 01, 2020 5:27 am There a link to the SpudEmu V0.001?
From the developer:
Not yet, still under development, executable files issued to select few beta testers only, links should be available on World Of Spectrum within the next week or so
So far, so meh :)
User avatar
Pegaz
Dynamite Dan
Posts: 1209
Joined: Mon Nov 13, 2017 1:44 pm

Re: Testing emulators using z80test-1.0. Quick results

Post by Pegaz »

djnzx48 wrote: Thu Apr 30, 2020 11:52 pm
Pegaz wrote: Thu Apr 30, 2020 12:49 pm Last few versions I tried, work just fine on my old dual core laptop in full screen mode.
If you've only tested in windowed mode, it's still crawling.
Fortunately, full screen mode is the only thing that matters to me, and it works really well there.
I also tested 50hz sync via the hdmi port on the TV and it is great, I would say perfect.
Also, the emulator is rock solid stable, but until it has some sort of GUI, I doubt it will reach a larger user base.
btw, it's really refreshing that after many years, we've got a female author, an extremely talented programer, like we may not have had in the Spectrum scene, since Veronica Megler.
I tried the latest version in fullscreen mode to see if I would get better performance, but unfortunately it didn't really help. All four builds ran slowly and with stuttery sound. I will just have to accept that my computer can't handle the awesome power of this emulator.
Oddly enough, I tested it on Win 7 64 bit, with my 9 year old laptop, dual core Intel 6200, 2 Ghz and there are no issues in full screen mode.
If you are using older windows (XP/Vista/7), you should try out the MinGW version, because the standard 32 and 64 bit versions are customized for windows 10.
It may also matter what the cpu is about, maybe the emulator is not well optimized for amd, if that is the case.
You can't go wrong with the setup because there is virtually nothing to adjust.
 
User avatar
djnzx48
Manic Miner
Posts: 729
Joined: Wed Dec 06, 2017 2:13 am
Location: New Zealand

Re: Testing emulators using z80test-1.0. Quick results

Post by djnzx48 »

In my case, it's almost surely the CPU. I'm currently using a laptop that is about 8 years old, and due to severe thermal problems I have to clock the CPU down below 1.5GHz if I don't want the fans spinning constantly. Disassembling it in an attempt to remove dust from the case didn't fix the problem, so it struggles running some programs. (I did try the other builds just in case.)
User avatar
PeterJ
Site Admin
Posts: 6856
Joined: Thu Nov 09, 2017 7:19 pm
Location: Surrey, UK

Re: Testing emulators using z80test-1.0. Quick results

Post by PeterJ »

polomint wrote: Fri May 01, 2020 7:57 am From the developer:
Not yet, still under development, executable files issued to select few beta testers only, links should be available on World Of Spectrum within the next week or so
Thank you for the explanation [mention]polomint[/mention].
User avatar
Pegaz
Dynamite Dan
Posts: 1209
Joined: Mon Nov 13, 2017 1:44 pm

Re: Testing emulators using z80test-1.0. Quick results

Post by Pegaz »

djnzx48 wrote: Fri May 01, 2020 8:41 am In my case, it's almost surely the CPU. I'm currently using a laptop that is about 8 years old, and due to severe thermal problems I have to clock the CPU down below 1.5GHz if I don't want the fans spinning constantly. Disassembling it in an attempt to remove dust from the case didn't fix the problem, so it struggles running some programs. (I did try the other builds just in case.)
I had the exactly the same problem last year and cleaning the dust was unfortunately not enough.
In my case it was necessary to replace the cooling heatsink and after that CPU went back to normal operating temperature.
I don't know exactly hardware requirements for this emulator, but I think those 2Ghz are probably some lower limit.
edjones
Drutt
Posts: 33
Joined: Fri Feb 28, 2020 1:42 pm

Re: Testing emulators using z80test-1.0. Quick results

Post by edjones »

Sorry about no links yesterday. I hadn't been given any links, only pics to post. Reckon they were after starting the hype train for new releases!
User avatar
druellan
Dynamite Dan
Posts: 1466
Joined: Tue Apr 03, 2018 7:19 pm

Re: Testing emulators using z80test-1.0. Quick results

Post by druellan »

PeterJ wrote: Wed Apr 29, 2020 9:33 pm I missed this first time round druellan. Excellent work!
In the end is little more than a tech curiosity really, since an emulator is much more than a Z80 recreation, but we discovered interesting things, and some authors got inspired to improve on their projects, which is nice.

Another similar and very interesting thread is the AY Emulation Testing viewtopic.php?f=23&t=1045
User avatar
PeterJ
Site Admin
Posts: 6856
Joined: Thu Nov 09, 2017 7:19 pm
Location: Surrey, UK

Re: Testing emulators using z80test-1.0. Quick results

Post by PeterJ »

Looking at the results [mention]druellan[/mention] (with many getting very similar scores) do you think that many are sharing the same Z80 emulation code? No issue with that. I would not know where to start writing an emulator!
User avatar
druellan
Dynamite Dan
Posts: 1466
Joined: Tue Apr 03, 2018 7:19 pm

Re: Testing emulators using z80test-1.0. Quick results

Post by druellan »

Well, that is something an emulator author might answer. There are many emulators that are built around a Z80 core, like this java one by [mention]zx81[/mention] https://github.com/jsanchezv/Z80Core, or they used the same literature to create their own core.

But I think that the results you see here are similar because this test is designed to catch undocumented flags, specially the behavior discovered by Patrik Rak that is specific to only genuine Zilog Z80 CPUs, and fails on even real Spectrums (viewtopic.php?p=11038#p11038). Some developers decided not to emulate this because it has no clear benefit for the emulation, so, I think it is natural we see very similar results, because most of them fail on those particular tests.
User avatar
Seven.FFF
Manic Miner
Posts: 735
Joined: Sat Nov 25, 2017 10:50 pm
Location: USA

Re: Testing emulators using z80test-1.0. Quick results

Post by Seven.FFF »

Yes. The tests will pass on NMOS Z80s that are genuine Zilog, They may pass on some second-sourced from designs officially provided by Zilog (Mostek). They may fail on others second-sourced from designs officially provided by Zilog (SGS), reverse-engineered Z80s (some E.European/Russian), or official second-sources based on reverse engineering rather than designs (NEC). They may also fail on CMOS Z80s of any origin.

They're a useful tool for emulator authors, but a less useful tool for judging the authenticity of designs. The most compatible Z80 code is that which uses only officially documented instructions. Software that seeks to trip up emulators by making use of private implementation details only succeeds in making itself incompatible with genuine Z80s.
Robin Verhagen-Guest
SevenFFF / Threetwosevensixseven / colonel32
NXtel NXTP ESP Update ESP Reset CSpect Plugins
User avatar
Lethargeek
Manic Miner
Posts: 734
Joined: Wed Dec 11, 2019 6:47 am

Re: Testing emulators using z80test-1.0. Quick results

Post by Lethargeek »

afaik memptr test is very reliable for all the range and might fail some subtests on eastern Z80 clones only; scf/ccf effects is another story
User avatar
PeterJ
Site Admin
Posts: 6856
Joined: Thu Nov 09, 2017 7:19 pm
Location: Surrey, UK

Re: Testing emulators using z80test-1.0. Quick results

Post by PeterJ »

ES.pectrum

Not an emulator I had used before but seems to have a support for a large range of machines:

Failed 9 of 152

http://www.habisoft.com/espectrum/EN.htm#Descargas

http://www.habisoft.com/espectrum/Manual/EN.htm

Image

Emulation of official models: 16, 48, 48+ (Spanish, Swedish, Arab), 48 NTSC, 128, 128+ (Spanish), grey +2 (Spanish, English, French, Arab), black +2 (Spanish, English, Arab) v4.0 and v4.1, +3 (Spanish, English) v4.0 and v4.1, Timex TC2048, Timex TC2068, Timex TS2068, Unipolbrit Komputer 2086.
Emulation of clones: Inves+, Tk90x (Portuguese, Argentinian), Tk95 (Portuguese, Argentinian), Orel BK-08, Pentagon 128, Pentagon 512, Pentagon 1024 SL v2.x, Scorpion ZS-256, Scorpion ZS-256 Turbo+, Leningrad, Dubna 48K, ATM-Turbo, ATM-Turbo 2+, Ice-Felix HC-91, BK-0001, Foton-IK03.
Emulation of mods: +2e (Spanish, English), +3e (Spanish, English), Spanish Pentagon.
Emulation of virtual ones: Spec256-48, Spec256-128.
Recognized formats:
Snapshot: Z80, SNA, SP, FSU, SZX, EZX, ESP (native).
Tape: TAP, TZX, PZX, CSW, WAV, VOC.
Disc: DSK, FDI, TRD, SCL, Hobbeta, MDR (if it can be considered as one).
Actions: RZX.
Screen: PNG, GIF (animated), SCR.
Cartridge: DCK, ROM (IF2, Dandanator), BIN.
Video: WMV.
Storage: HDF, RAW / BIN, IMG.
Emulation of hardware:
DivMMC.
Multiface 1, 128, +3, SpecMate.
Dandanator, including old versions.
Data-Gear (DMA).
PDS (development system). Requires a special DosBox version for linking.
Turbo Sound.
Covox, Specdrum.
16c modes in Pentagon.
64 / 4096 colour palette in Pentagon.
ULA+.
ZX1 extras (raster interruption, turbo, 128x96 mode).
Timex video modes.
Z80_GFX and 256 colour modes (Spec256, EmuzWin).
Betadisk: Beta, Beta+, Beta 128, and clones: Apina, CAS, Arcade, IDS 91, CBI 95.
Joysticks: Cursor / Protek / AGF, Sinclair 1 y 2, Fuller, Kempston.
Mouse: Kempston.
Guns: Gunstick, Lightgun.
Turbo modes.
Patrik Rak
Microbot
Posts: 112
Joined: Mon Apr 13, 2020 3:07 pm

Re: Testing emulators using z80test-1.0. Quick results

Post by Patrik Rak »

Seven.FFF wrote: Fri May 01, 2020 4:32 pmThey're a useful tool for emulator authors, but a less useful tool for judging the authenticity of designs. The most compatible Z80 code is that which uses only officially documented instructions. Software that seeks to trip up emulators by making use of private implementation details only succeeds in making itself incompatible with genuine Z80s.
Well, I would say that you can still use it to judge the quality of the Z80 emulation to some extent regardless.

If the "full" test fails after any of SCF/CCF/SCF+CCF/CCF+SCF, it means it implements other SCF/CCF behaviour than authentic NMOS Zilog Z80, but it's rather esoteric indeed. In such case, the "postccf" test will inevitably fail half of the tests, that's expected. However, until 2012, none of the emulators got this right, and some real CPUs differ as well, so I guess it's something we can live with.

If the "full" test fails after BIT N,(HL/IX/IY) instructions, it means it doesn't implement MEMPTR properly. In such case, the "memptr" test will very likely fail many tests, that's expected too. Of course, MEMPTR was nailed down much earlier, so even if it is obscure, authors should consider emulating it. Unlike the SCF/CCF, I don't think there is any real Z80 CPU which differs regarding MEMPTR.

If the "full" test fails after all IN instructions, the emulator quite likely incorrectly emulates bit 6 of port #FE. While it's not Z80 feature per-se, this should be obviously fixed, as it can impact real games.

Anything else means the z80 emulation is far from perfect and it should be fixed if possible.

So, while you can argue the test is synthetic, seeing numbers greater than 4 means real incompatibilities... The more of those, the worse Z80 emulation, at least that's how I see it (despite the fact that Z80 emulation alone doesn't make a great emulator, of course).
User avatar
Pegaz
Dynamite Dan
Posts: 1209
Joined: Mon Nov 13, 2017 1:44 pm

Re: Testing emulators using z80test-1.0. Quick results

Post by Pegaz »

I just tested the ZX ULAX emulator and I'm not surprised that it passed all the tests. :)
Great job Lethargeek, I hope you'll add emulation of classic Sinclair 48k/128k models in the future.

Image
User avatar
Lethargeek
Manic Miner
Posts: 734
Joined: Wed Dec 11, 2019 6:47 am

Re: Testing emulators using z80test-1.0. Quick results

Post by Lethargeek »

Thanks, [mention]Pegaz[/mention]! Ihope i will add original models too. I chose Pentagon to have faster and simpler start without the contention troubles and other things non-essential for the main goal. As for these tests, i mostly followed the FUSE source code for flag effects.
azesmbog
Manic Miner
Posts: 306
Joined: Sat May 16, 2020 8:43 am

Re: Testing emulators using z80test-1.0. Quick results

Post by azesmbog »

PeterJ wrote: Sat May 02, 2020 8:24 pm ES.pectrum

Failed 9 of 152
In defense of a good emulator :)
This test has not determined which machine to run it on.
On zx48 - 9 errors, but if you run on zx128 - only 2 errors, insignificant.
Image
And this is one of the few emulators that can run (and pass!) Another interesting test.
Timing_Tests-48k_v1.0
And which works great with z80DMA, it can generally be units :)
User avatar
PeterJ
Site Admin
Posts: 6856
Joined: Thu Nov 09, 2017 7:19 pm
Location: Surrey, UK

Re: Testing emulators using z80test-1.0. Quick results

Post by PeterJ »

Agreed [mention]azesmbog[/mention]. It's an excellent emulator with lots of nice features.
User avatar
Lethargeek
Manic Miner
Posts: 734
Joined: Wed Dec 11, 2019 6:47 am

Re: Testing emulators using z80test-1.0. Quick results

Post by Lethargeek »

azesmbog wrote: Sat May 16, 2020 9:30 am This test has not determined which machine to run it on.
On zx48 - 9 errors, but if you run on zx128 - only 2 errors, insignificant.
ummm... this test is pure cpu thing afaiu, so different results with different models is another bug in itself :mrgreen:
azesmbog
Manic Miner
Posts: 306
Joined: Sat May 16, 2020 8:43 am

Re: Testing emulators using z80test-1.0. Quick results

Post by azesmbog »

I partially agree, but in the test there is no mention anywhere that it is only zx48, and that different results depending on the configuration are probably the problems of the test, and not the emulator being tested. is not it ?
User avatar
Lethargeek
Manic Miner
Posts: 734
Joined: Wed Dec 11, 2019 6:47 am

Re: Testing emulators using z80test-1.0. Quick results

Post by Lethargeek »

azesmbog wrote: Sat May 16, 2020 12:11 pm I partially agree, but in the test there is no mention anywhere that it is only zx48,
because it is not "only zx48", it's "only z80" - doesn't depend on the 48 rom contents (like another cpu test) and doesn't use any 128+ ports
azesmbog wrote: Sat May 16, 2020 12:11 pm and that different results depending on the configuration are probably the problems of the test, and not the emulator being tested. is not it ?
definitely must be an emulator problem (and a very strange one)
User avatar
PeterJ
Site Admin
Posts: 6856
Joined: Thu Nov 09, 2017 7:19 pm
Location: Surrey, UK

Re: Testing emulators using z80test-1.0. Quick results

Post by PeterJ »

azesmbog wrote: Sat May 16, 2020 9:30 am
In defense of a good emulator :)
Just noticed that among it's many great features, this emulator supports the Russian ATM2 machines. This will allow me to start looking at the titles that [mention]Alone Coder[/mention] mentioned in another thread.

The only issue I have found is the Microdrive support is broken, which I have been told will be fixed in a future release. It's a very good all round emulator
User avatar
ketmar
Manic Miner
Posts: 612
Joined: Tue Jun 16, 2020 5:25 pm
Location: Ukraine

Re: Testing emulators using z80test-1.0. Quick results

Post by ketmar »

sorry for shameless self-promotion (but nobody knows about my Z80 emulation core anyway, so i had to do it myself ;-): Zymosis (and ZXEmuT emulator based on it) only failing SCF/CCF tests. one test failed in "full", no failed tests in "memptr".
User avatar
ketmar
Manic Miner
Posts: 612
Joined: Tue Jun 16, 2020 5:25 pm
Location: Ukraine

Re: Testing emulators using z80test-1.0. Quick results

Post by ketmar »

upd: the bug with SCF/CCF was actually in interrupt acknowledgement code, so Zymosis passes all tests now.
azesmbog
Manic Miner
Posts: 306
Joined: Sat May 16, 2020 8:43 am

Re: Testing emulators using z80test-1.0. Quick results

Post by azesmbog »

ketmar wrote: Tue Jun 16, 2020 7:23 pm sorry for shameless self-promotion (but nobody knows about my Z80 emulation core anyway, so i had to do it myself ;-)
Well, not really to anyone :))
I know and remember. And I remember how you set up Linux for me 5-10 years ago by remote and the emulator started :) Since then, ownership of Linux has not gotten better for me.
Can you check a couple of my tests in 48k mode?)
User avatar
ketmar
Manic Miner
Posts: 612
Joined: Tue Jun 16, 2020 5:25 pm
Location: Ukraine

Re: Testing emulators using z80test-1.0. Quick results

Post by ketmar »

azesmbog wrote: Wed Jun 17, 2020 5:39 pm
ketmar wrote: Tue Jun 16, 2020 7:23 pm sorry for shameless self-promotion (but nobody knows about my Z80 emulation core anyway, so i had to do it myself ;-)
Well, not really to anyone :))
I know and remember. And I remember how you set up Linux for me 5-10 years ago by remote and the emulator started :) Since then, ownership of Linux has not gotten better for me.
Can you check a couple of my tests in 48k mode?)
yeah, i remember you too! you helped me alot back then, and your name is in THANKS file for Zymosis. ;-)

but i think we'd better move to Zymosis topic, to avoid derailing.
Post Reply