Proposal for RZX format update

Struggling with Fuse or trying to find an emulator with a specific feature. Ask your questions here.
Patrik Rak
Microbot
Posts: 117
Joined: Mon Apr 13, 2020 3:07 pm

Proposal for RZX format update

Post by Patrik Rak »

Hello,

here is my proposal for update of RZX format from version 0.13 to version 0.14.

TL;DR:

Allow proper RZX recording regardless of blocked interrupts.

The change itself:
Change the meaning of the 16 bit fetch counter of the IO recording frame in the following way:
bits 0-14 - the fetch counter itself
bit 15 - when set, the frame ends but the interrupt is not actually generated
Rationale:

The RZX format has served us for many years, but it has one annoying flaw, as we have already discussed some 13 years ago in this thread. As it is now, when the frame ends and the interrupt should be generated, the emulator can record the frame end only under the following conditions:

1) The interrupts are disabled.
2) The interrupts are enabled and the interrupt happens.

This seems to cover all the needed cases needed, but it actually does not. There is one more case which is missing:

3) The interrupts are enabled but blocked and the interrupt doesn't happen.

Under normal circumstances, the interrupts are not blocked for the entire duration of the interrupt pending period, so the emulator will be able to end the frame a bit later using the cases 1) or 2). But in case the interrupts were blocked for the entire duration, the recording emulator has to deal with that somehow. The choices are:

1) Abort the RZX recording. Not ideal for sure.
2) Emit the frame regardless. This leads to broken RZX which can't be played back reliably. As discussed before, when playing back the RZX, the emulator must disregard its own idea of blocking the interrupts. It can only use its own idea of disabled/enabled interrupts, but must generate the interrupt whenever it is enabled, regardless of if it is blocked or not.
3) Not finish the frame and combine it with the following one, creating a jumbo frame which actually spans several frames. This is perhaps the currently used solution in emulators, but even that has limited uses - this can be used for at most couple of frames before the 16 bit counters overflow, and then the recording emulator is back to square one - it will perhaps have to abort the RZX recording. Not to mention that such jumbo frames are pain for playback in the emulators with high possibility of not being handled properly.
4) Technically, it can start creating snapshots at each such frame. While this "works" in a way, it's hardly a desirable solution.

This proposal adds the preferable solution for this problem:

5) Simply emit a frame which relays the information that the interrupt didn't happen.

What now:

OK, that's the proposal, but what should you do now? There are several options:

1) Do nothing. I can understand that not everybody is willing to jump aboard. Fair enough. The chance of encountering a publicly distributed RZX file which would actually need this is pretty low, so feel free not to do anything.

2) To support the change and claim compatibility, the good news is that there is no need to modify the existing recording code unless you want to. All that is needed is to add support for playback of these frames. And that is actually extremely trivial. For example, these are essentially all the changes which were needed to do this in ZXDS:

Code: Select all

diff --git a/source/main.cpp b/source/main.cpp
index 91335e4..9cb0429 100644
--- a/source/main.cpp
+++ b/source/main.cpp
@@ -139,7 +139,9 @@ bool emulate_rzx_frame( void )
 
     if ( rzx_fetch_count <= RZX_SHORT_FRAME_LIMIT ) {
         z80_emulate_fetches( &z80, rzx_fetch_count ) ;
-        z80_force_irq( &z80 ) ;
+        if ( rzx_force_irq ) {
+            z80_force_irq( &z80 ) ;
+        }
         return false ;
     }
 
@@ -203,7 +205,9 @@ bool emulate_rzx_frame( void )
     // Now trigger an interrupt, even at places we would not normally
     // allow it due blocked instructions like delayed EI.
 
-    z80_force_irq( &z80 ) ;
+    if ( rzx_force_irq ) {
+        z80_force_irq( &z80 ) ;
+    }
     return true ;
 }
 
diff --git a/source/rzx.cpp b/source/rzx.cpp
index 577ba05..cb7e8bb 100644
--- a/source/rzx.cpp
+++ b/source/rzx.cpp
@@ -16,6 +16,10 @@ DTCM_BSS static uint start_count ;
 
 DTCM_BSS bool rzx_playing ;
 
+// Flag set when RZX should force IRQ at end of frame.
+
+DTCM_BSS bool rzx_force_irq ;
+
 // How many instructions (or more exactly, M1 fetch cycles) to execute in the current frame.
 //
 // Each instruction takes at least 4 T states, but may take as much as 23 (EX (SP),IX),
@@ -468,7 +473,9 @@ bool prepare_rzx_frame( Z80 * z )
         return false ;
     }
 
-    rzx_fetch_count = le( frame.fetch_count ) ;
+    const uint fetch_count = le( frame.fetch_count ) ;
+    rzx_fetch_count = ( fetch_count & 0x7FFF ) ;
+    rzx_force_irq = ( ( fetch_count & 0x8000 ) == 0 ) ;
     in_count = le( frame.in_count ) ;
 
 #ifdef DEBUG_RZX
That is, one change where the fetch count is decoded and then suppressing of the IRQ itself, once for short retriggered frames and once for the normal frames.

3) If you want to change your RZX recording code, or you are only starting to write your own for the first time, you may think about emitting these frames any time the frame ends and the interrupt is not taken. However, while the created RZX would work in any emulator which supports this, any older emulator would not be able to play it back correctly. So I suggest not to do this and look at the next option instead. Still, the ability of being able to play such frames back in compatible emulator can be great help while you are developing and testing your RZX recording code.

4) Use these frames internally any way you wish, but do not let them appear in the final RZX stream unless necessary. This is what ZXDS does. It uses these frames a lot (the ability to force the frame flush any time I need is extremely helpful, especially for the implementation of the rollback) but then it coalesces them with the following short frames whenever possibile, so they never appear in the resulting stream. Still, in case the finalization sweep pass would not work out for some reason (for example, the SD card becomes full while recording), it's nice that these frames can be played back as they are and the RZX can be independently finalized later.

So, these are the options - which one you choose is entirely up to you, but it would be appreciated if you consider at least the option 2).

For reference, here is the code which handles the RZX frame end and which is called after each instruction whenever the interrupt is pending:

Code: Select all

// Handle end-of-frame during RZX recording.

void finish_rzx_frame( Z80 * z )
{
    hope( z ) ;
    hope( rzx_recording ) ;

    // Do nothing if we are not recording frames.

    if ( input_block_offset == 0 ) {
        return ;
    }

    // Make sure this is called only when the interrupt is really pending.

    hope( ( z->regs.iff & F_IRQ ) != 0 ) ;

    // What we do depends on if the interrupts are enabled or not and blocked or not.

    const bool enabled = ( z->regs.iff & F_IFF1 ) != 0 ;
    const bool blocked = ( z->regs.iff & F_BLOCK ) != 0 ;

    // In case the interrupts are enabled and not blocked, interrupt will happen, so we have to finish
    // the frame no matter what so the RZX replay knows that it must generate the interrupt at this point.
    // This is where the short RZX frames may come from, for example because of IRQ retrigger.

    if ( enabled && ! blocked ) {
        flush_rzx_frame( z ) ;
        return ;
    }

    // In case the interrupts are disabled, interrupt will not happen, regardless of if it is blocked or not.
    // In this case we finish the frame so the RZX replay knows that the frame has ended at this point.
    // But we do it only if we didn't finish it already shortly before now, or the previous frame was forced.
    
    if ( ! enabled ) {
        if ( frame_fetches( z ) > RZX_SHORT_FRAME_LIMIT || last_frame_was_forced ) {
            flush_rzx_frame( z ) ;
        }
        return ;
    }

    // Otherwise force flush of the frame, unless it was already finished shortly before.
    //
    // If we didn't, we could miss the frame end entirely in case the interrupt is blocked by chain of EI/FD/DD,
    // creating a jumbo frame. Seems like the RZX authors simply forgot to deal with this case.
    // Technically, we could start storing the snapshots at each such point, but that's hardly desirable.
    // Therefore this support for forced frames was added to deal with it effectively.

    if ( frame_fetches( z ) > RZX_SHORT_FRAME_LIMIT ) {
        flush_rzx_frame( z, true ) ;
    }
}
BTW, as you can see, for the lack of better name, I internally call these frames force-flushed or simply forced, because I have forced the flush of the frame at points where it would not happen otherwise. I just wanted to mention this, as forced in this case is from the point of view of the recording emulator, not from the point of view of the playback.

Well, that's all, I guess. Comments?

Patrik
User avatar
ZjoyKiLer
Dizzy
Posts: 69
Joined: Thu Sep 09, 2021 3:20 pm

Re: Proposal for RZX format update

Post by ZjoyKiLer »

We would like to discuss with you your proposal and, in addition, share with you new info recently discovered about the Z80 that you may want to consider for this (several instruction flags, Q and HALT). We have new tests I think you will be happy to try.

Please, join the ZX Spectrum discord server, where you will be warmly welcomed: https://discord.gg/p8nDBxR2FM
User avatar
Weiv
Microbot
Posts: 178
Joined: Fri Apr 06, 2018 5:28 pm
Location: Ukraine

Re: Proposal for RZX format update

Post by Weiv »

Patrik Rak wrote: Mon Dec 27, 2021 9:55 am
3) The interrupts are enabled but blocked and the interrupt doesn't happen.

Under normal circumstances, the interrupts are not blocked for the entire duration of the interrupt pending period, so the emulator will be able to end the frame a bit later using the cases 1) or 2). But in case the interrupts were blocked for the entire duration, the recording emulator has to deal with that somehow.
As far as I understand, this is only possible by executing the chain of EI commands and/or #DD/#FD prefixes. In this case, the problem is solved simply - after performing a specified number of command fetches of the RZX frame, we check the interrupt blocking flag, and if interrupts are blocked, the interrupt will not perform. So no need to modify RZX format.
Patrik Rak
Microbot
Posts: 117
Joined: Mon Apr 13, 2020 3:07 pm

Re: Proposal for RZX format update

Post by Patrik Rak »

Weiv wrote: Sun Jan 02, 2022 4:10 am As far as I understand, this is only possible by executing the chain of EI commands and/or #DD/#FD prefixes. In this case, the problem is solved simply - after performing a specified number of command fetches of the RZX frame, we check the interrupt blocking flag, and if interrupts are blocked, the interrupt will not perform. So no need to modify RZX format.
With no offence, you were not paying attention:
As discussed before, when playing back the RZX, the emulator must disregard its own idea of blocking the interrupts. It can only use its own idea of disabled/enabled interrupts, but must generate the interrupt whenever it is enabled, regardless of if it is blocked or not.
In ideal world, what you propose would indeed be a solution, but unfortunately there are many emulators which don't/didn't implement interrupt blocking properly and were used to record many of the RZXs which are already out there. To playback these RZXs properly, the interrupt MUST be taken, so your solution can't be used.

Patrik
Ralf
Rick Dangerous
Posts: 2330
Joined: Mon Nov 13, 2017 11:59 am
Location: Poland

Re: Proposal for RZX format update

Post by Ralf »

I'll make a little bit offtopic, sorry for that.

What you discuss seems a very rare case for me. Files at Rzx Archive just work, you don't have to fix them.

I made many rzx recordings in the past and if you were to change rzx format I would like to see some another change,
which I believe would be more useful.

I'd like a possibility to insert a full memory snapshot somewhere into rzx file.

Why? Because you could immediately fast forward to that moment of gameplay. Imagine watching rzx which is 3
hours long. You cannot skip anything like you do when watching video. You have to watch it all and yes, you can speed up
emulator but it's still a pain.

And another thing, with such snapshots you could actually make a recording made of totally independent parts,
like levels loaded from tape. Today with have .zip files with each level recorded separately. With starting snapshots you could
probably glue them together into single files.

Such snapshots could have names and descriptions so you would know what you are jumping to.
The files would be a bit bigger but hey, it's year 2022 now.

I won't do it myself but feel inspired if you ever wanted to touch rzx format ;)
End of offtop.
User avatar
Guesser
Manic Miner
Posts: 649
Joined: Wed Nov 15, 2017 2:35 pm
Contact:

Re: Proposal for RZX format update

Post by Guesser »

Ralf wrote: Sun Jan 02, 2022 11:21 am What you discuss seems a very rare case for me. Files at Rzx Archive just work, you don't have to fix them.
Old RZX files work because emulators are inaccurate though, and stop working properly when they're fixed. That is (apparently) a big problem that people want to solve ;)
Patrik Rak
Microbot
Posts: 117
Joined: Mon Apr 13, 2020 3:07 pm

Re: Proposal for RZX format update

Post by Patrik Rak »

Ralf wrote: Sun Jan 02, 2022 11:21 am What you discuss seems a very rare case for me. Files at Rzx Archive just work, you don't have to fix them.
Well, nobody is talking about touching the RZX files in the archive at all. As you say, they don't need fixing.

This proposal is aimed entirely towards emulators authors so they can deal with this annoying aspect of the RZX format during recording. So I was hoping to hear back from people like Philip Kendall, Jon Needle, Woody, Richard Chandler, Arjun Nair, et al., rather than looking for public discussion.

(Because if there is one thing I have learned about formats, it's that you can discuss them as much as you want all day long, but in the end it's only these people who decide to implement something or not who matter.)
I'd like a possibility to insert a full memory snapshot somewhere into rzx file.
That's actually already possible. For example, SpecEmu does this, and I think that Spin allows jumping to snapshots as well.

FWIW, even ZXDS inserts snapshots whenever necessary, so you could do that multilevel example as well, simply by loading the snapshot of next level right after finishing the previous one. Or simply record playback of any sequence of RZX files, gluing them together.

OTOH, I think that having a zip file with multiple levels works perfectly fine, because anybody can play any level they want, and nobody has to program anything extra to enable that. So I prefer that over having one huge RZX.

Patrik
User avatar
Weiv
Microbot
Posts: 178
Joined: Fri Apr 06, 2018 5:28 pm
Location: Ukraine

Re: Proposal for RZX format update

Post by Weiv »

Patrik Rak wrote: Sun Jan 02, 2022 10:12 am
Weiv wrote: Sun Jan 02, 2022 4:10 am As far as I understand, this is only possible by executing the chain of EI commands and/or #DD/#FD prefixes. In this case, the problem is solved simply - after performing a specified number of command fetches of the RZX frame, we check the interrupt blocking flag, and if interrupts are blocked, the interrupt will not perform. So no need to modify RZX format.
With no offence, you were not paying attention:
As discussed before, when playing back the RZX, the emulator must disregard its own idea of blocking the interrupts. It can only use its own idea of disabled/enabled interrupts, but must generate the interrupt whenever it is enabled, regardless of if it is blocked or not.
In ideal world, what you propose would indeed be a solution, but unfortunately there are many emulators which don't/didn't implement interrupt blocking properly and were used to record many of the RZXs which are already out there. To playback these RZXs properly, the interrupt MUST be taken, so your solution can't be used.

Patrik
RZX format suggests that CPU emulation is perfect, so, if an emulator has imperfect CPU emulation, it's RZX can be incorrect for future emulators after the commands which its emulates imperfectly. It concerns not only interrupt blocking, but also undocumented flags behavior and other undocumented features. So your proposal can't solve this problem in general.

Created imperfect RZXs are created already, and your emulator must to know how to playback them. Also your emulator must to create new RZX-s with its own Z80 emulation representation. Your solution can be useful for future RZXs and future emulators - but future emulators will know about interrupt blocking and they will deal with it correctly without your solution. (And old emulators will not know about your solution and will not deal with it correctly of course.)
User avatar
ZjoyKiLer
Dizzy
Posts: 69
Joined: Thu Sep 09, 2021 3:20 pm

Re: Proposal for RZX format update

Post by ZjoyKiLer »

Patrik Rak wrote: Sun Jan 02, 2022 12:17 pm
Ralf wrote: Sun Jan 02, 2022 11:21 am What you discuss seems a very rare case for me. Files at Rzx Archive just work, you don't have to fix them.
This proposal is aimed entirely towards emulators authors so they can deal with this annoying aspect of the RZX format during recording. So I was hoping to hear back from people like Philip Kendall, Jon Needle, Woody, Richard Chandler, Arjun Nair, et al., rather than looking for public discussion.
Join our discord server, Woody, Richard Chandler and Arjun are always there.

And, as I said in my previous post, there are newly discovered things that have not yet been implemented in your emulator that you will be interested in, and those things may have some impact on RZX compatibility.
User avatar
Weiv
Microbot
Posts: 178
Joined: Fri Apr 06, 2018 5:28 pm
Location: Ukraine

Re: Proposal for RZX format update

Post by Weiv »

Ralf wrote: Sun Jan 02, 2022 11:21 am I'd like a possibility to insert a full memory snapshot somewhere into rzx file.
But you can insert as much file snapshots as you need in your RZX already. For example, emulators insert snapshots in its RZXs for each rollback point.
Ralf
Rick Dangerous
Posts: 2330
Joined: Mon Nov 13, 2017 11:59 am
Location: Poland

Re: Proposal for RZX format update

Post by Ralf »

For example, emulators insert snapshots in its RZXs for each rollback point.
Yes, but all these points are removed when you finish and close your recording.
You may say - so don't close it.

But that's not pretty ;)
And I'd like to have let's say 5 snapshots at chosen moments. When recording with rollback I often make
much, much more rollback points. When the game is hard you have to make rollbacks (which effectively
work as saves) every 20 seconds or so.

Anyway, I'm recoding mostly with Spectaculator so I don't hope much for any changes made to it ever.
User avatar
Weiv
Microbot
Posts: 178
Joined: Fri Apr 06, 2018 5:28 pm
Location: Ukraine

Re: Proposal for RZX format update

Post by Weiv »

Ralf wrote: Sun Jan 02, 2022 3:06 pm Anyway, I'm recoding mostly with Spectaculator so I don't hope much for any changes made to it ever.
So you need advanced emulator, not advanced RZX format)
Patrik Rak
Microbot
Posts: 117
Joined: Mon Apr 13, 2020 3:07 pm

Re: Proposal for RZX format update

Post by Patrik Rak »

Weiv wrote: Sun Jan 02, 2022 12:37 pm RZX format suggests that CPU emulation is perfect, so, if an emulator has imperfect CPU emulation, it's RZX can be incorrect for future emulators after the commands which its emulates imperfectly.
That is quite misguided opinion. Of course the emulation of the particular code has to be identical if you are to get the same outcome. But it doesn't mean it has to be perfect. One of the aims of RZX design was that neither the recording nor the replaying emulator has to have perfect timing. That's why it stores the number of instructions emulated in the first place. If all emulators had perfect emulation, we wouldn't need the fetch count at all.

Fortunately sufficient level of reasonable emulation was achieved long time ago. That's why we can enjoy all the RZXs out there, created with emulators most of which haven't had perfect emulation by any means when they were used to create those RZXs.
Created imperfect RZXs are created already, and your emulator must to know how to playback them. Also your emulator must to create new RZX-s with its own Z80 emulation representation. Your solution can be useful for future RZXs and future emulators - but future emulators will know about interrupt blocking and they will deal with it correctly without your solution.
No, they won't, because they can't tell the two cases apart.

Anyway, I am missing what are you trying to prove. I am an emulator author and there is one problem with RZX recording implementation which the RZX format 0.13 doesn't address. I propose reasonably simple way how to solve it. You telling me it doesn't need to be solved doesn't help anything, so you may just as well stop trying. Thanks.

Patrik
Patrik Rak
Microbot
Posts: 117
Joined: Mon Apr 13, 2020 3:07 pm

Re: Proposal for RZX format update

Post by Patrik Rak »

ZjoyKiLer wrote: Sun Jan 02, 2022 12:49 pm Join our discord server, Woody, Richard Chandler and Arjun are always there.

And, as I said in my previous post, there are newly discovered things that have not yet been implemented in your emulator that you will be interested in, and those things may have some impact on RZX compatibility.
Looking forward to, it's just that I am too busy with other IRL things these days. Will join as soon as things calm down a bit. Thanks.

Patrik
User avatar
Weiv
Microbot
Posts: 178
Joined: Fri Apr 06, 2018 5:28 pm
Location: Ukraine

Re: Proposal for RZX format update

Post by Weiv »

Patrik Rak wrote: Sun Jan 02, 2022 4:24 pm That is quite misguided opinion. Of course the emulation of the particular code has to be identical if you are to get the same outcome. But it doesn't mean it has to be perfect. One of the aims of RZX design was that neither the recording nor the replaying emulator has to have perfect timing. That's why it stores the number of instructions emulated in the first place. If all emulators had perfect emulation, we wouldn't need the fetch count at all.
I wrote "perfect Z80 emulation", not "perfect timings". Timings can be imperfect for RZX, of course. Timings are part of Z80 emulation, of course, but they are more like a part of whole Spectrum emulation in our case.

No, they won't, because they can't tell the two cases apart.
They can actually. The simplest way is to check the emulator version that saved in RZX file.

Anyway, I am missing what are you trying to prove. I am an emulator author and there is one problem with RZX recording implementation which the RZX format 0.13 doesn't address. I propose reasonably simple way how to solve it. You telling me it doesn't need to be solved doesn't help anything, so you may just as well stop trying. Thanks.
I tried to say that:
1) problem with interrupt blocking is a part of bigger problem with different Z80 emulation in various emulators, and your proposal can't solve this bigger problem. I know at least yet another one problem with RZX compatibility based on imperfect Z80 emulation.
2) emulator should create its RZXs with hope that a) its Z80 emulation is perfect, b) future emulators will have perfect Z80 emulation too; so you don't need to make RZX extension for each case of imperfect Z80 emulation. If Z80 emulation of your emulator is imperfect, but you created many RZXs, it can be solved in future without RZX format extension.
User avatar
ZjoyKiLer
Dizzy
Posts: 69
Joined: Thu Sep 09, 2021 3:20 pm

Re: Proposal for RZX format update

Post by ZjoyKiLer »

Weiv wrote: Sun Jan 02, 2022 5:50 pm
Patrik Rak wrote: Sun Jan 02, 2022 4:24 pm That is quite misguided opinion. Of course the emulation of the particular code has to be identical if you are to get the same outcome. But it doesn't mean it has to be perfect. One of the aims of RZX design was that neither the recording nor the replaying emulator has to have perfect timing. That's why it stores the number of instructions emulated in the first place. If all emulators had perfect emulation, we wouldn't need the fetch count at all.
I wrote "perfect Z80 emulation", not "perfect timings". Timings can be imperfect for RZX, of course. Timings are part of Z80 emulation, of course, but they are more like a part of whole Spectrum emulation in our case.

No, they won't, because they can't tell the two cases apart.
They can actually. The simplest way is to check the emulator version that saved in RZX file.

Anyway, I am missing what are you trying to prove. I am an emulator author and there is one problem with RZX recording implementation which the RZX format 0.13 doesn't address. I propose reasonably simple way how to solve it. You telling me it doesn't need to be solved doesn't help anything, so you may just as well stop trying. Thanks.
I tried to say that:
1) problem with interrupt blocking is a part of bigger problem with different Z80 emulation in various emulators, and your proposal can't solve this bigger problem. I know at least yet another one problem with RZX compatibility based on imperfect Z80 emulation.
2) emulator should create its RZXs with hope that a) its Z80 emulation is perfect, b) future emulators will have perfect Z80 emulation too; so you don't need to make RZX extension for each case of imperfect Z80 emulation. If Z80 emulation of your emulator is imperfect, but you created many RZXs, it can be solved in future without RZX format extension.
I understand your reasoning and, by the way, I congratulate you for the very accurate implementation of the snow effect of Spectramine, but I have to inform you that currently your Z80 implementation is not perfect. There are other emulators that are more accurate.
User avatar
Weiv
Microbot
Posts: 178
Joined: Fri Apr 06, 2018 5:28 pm
Location: Ukraine

Re: Proposal for RZX format update

Post by Weiv »

ZjoyKiLer wrote: Sun Jan 02, 2022 6:08 pm I understand your reasoning and, by the way, I congratulate you for the very accurate implementation of the snow effect of Spectramine, but I have to inform you that currently your Z80 implementation is not accurate enough. There are other emulators that are more accurate.
Thanks) I was informed about it already by [mention]azesmbog[/mention]. New emulation things are opening, and you should update your emulator accordingly. I will certainly share the info about correct snow effect emulation but only after (and if) the new version of my emulator will be released. Until that it's my consolation prize)
User avatar
ZjoyKiLer
Dizzy
Posts: 69
Joined: Thu Sep 09, 2021 3:20 pm

Re: Proposal for RZX format update

Post by ZjoyKiLer »

Weiv wrote: Sun Jan 02, 2022 6:28 pm
ZjoyKiLer wrote: Sun Jan 02, 2022 6:08 pm I understand your reasoning and, by the way, I congratulate you for the very accurate implementation of the snow effect of Spectramine, but I have to inform you that currently your Z80 implementation is not accurate enough. There are other emulators that are more accurate.
Thanks) I was informed about it already by @azesmbog. New emulation things are opening, and you should update your emulator accordingly. I will certainly share the info about correct snow effect emulation but only after (and if) the new version of my emulator will be released. Until that it's my consolation prize)
Thank you very much for sharing the technical info. It will be much appreciated.

Although I think azembsbog have informed you already, I was about to post the new test of Peter Helcmanovsky. It's here:
viewtopic.php?f=23&t=6102
User avatar
Weiv
Microbot
Posts: 178
Joined: Fri Apr 06, 2018 5:28 pm
Location: Ukraine

Re: Proposal for RZX format update

Post by Weiv »

ZjoyKiLer wrote: Sun Jan 02, 2022 7:02 pm Thank you very much for sharing the technical info. It's much appreciated.
I don't deserve it yet.
Although I think azesmbog have informed you already, I was about to post the new test of Peter Helcmanovsky. It's here:
viewtopic.php?f=23&t=6102
Thank you. Maybe I missed something else?
Patrik Rak
Microbot
Posts: 117
Joined: Mon Apr 13, 2020 3:07 pm

Re: Proposal for RZX format update

Post by Patrik Rak »

Weiv wrote: Sun Jan 02, 2022 5:50 pm
No, they won't, because they can't tell the two cases apart.
They can actually. The simplest way is to check the emulator version that saved in RZX file.
Well, good luck with maintaining the list of how each particular version of each emulator handled blocked interrupts after each instruction. In my opinion, this is complete nonsense. Feels like you are making an argument just for the sake of it, no matter how impractical it is.
I tried to say that:
1) problem with interrupt blocking is a part of bigger problem with different Z80 emulation in various emulators, and your proposal can't solve this bigger problem. I know at least yet another one problem with RZX compatibility based on imperfect Z80 emulation.
I don't try to solve any bigger problem. My problem was that my own emulator was not able to record and play back everything properly, regardless of any other emulator out there. With what I propose, it can. And so can others if they choose so. That's good enough to me. IMO better than scratching my head because of bigger problems and in the end doing nothing for more than 13 years.

(EDIT: it sounds more harsh than I intend to. Please try not to project any negative feelings into this. It's very easy to get the wrong impression based on written text. Thanks.)

Patrik
User avatar
ZjoyKiLer
Dizzy
Posts: 69
Joined: Thu Sep 09, 2021 3:20 pm

Re: Proposal for RZX format update

Post by ZjoyKiLer »

Weiv wrote: Sun Jan 02, 2022 7:20 pm
ZjoyKiLer wrote: Sun Jan 02, 2022 7:02 pm Thank you very much for sharing the technical info. It's much appreciated.
I don't deserve it yet.
Although I think azesmbog have informed you already, I was about to post the new test of Peter Helcmanovsky. It's here:
viewtopic.php?f=23&t=6102
Thank you. Maybe I missed something else?
I think azemsbog also shared with you the HALT stuff, as I remember he showed us screenshots of Spectramine passing the latest version of Woody's HALT2INT test.

But, try this to be sure, and this.

Also, we are thinking about testing the ccf and scf instructions again, since it seems that the behavior on NEC NMOS CPUs is different, and there are contradictions between Patrik Rak's findings regarding the behavior on Zilog CMOS CPUs and what other sources say.
User avatar
Weiv
Microbot
Posts: 178
Joined: Fri Apr 06, 2018 5:28 pm
Location: Ukraine

Re: Proposal for RZX format update

Post by Weiv »

ZjoyKiLer wrote: Sun Jan 02, 2022 7:29 pm I think azemsbog also shared with you the HALT stuff, as I remember he showed us screenshots of Spectramine passing the latest version of Woody's HALT2INT test.

But, try this to be sure, and this.

Also, we are thinking about testing the ccf and scf instructions again, since it seems that the behavior on NEC NMOS CPUs is different, and there are contradictions between Patrik Rak's findings regarding the behavior on Zilog CMOS CPUs and what other sources say.
Thank you very much. I don't fully understand what the tests test but I have the same results in my emulator as in last version of SpecEmu and on the photos from real machines.
User avatar
ZjoyKiLer
Dizzy
Posts: 69
Joined: Thu Sep 09, 2021 3:20 pm

Re: Proposal for RZX format update

Post by ZjoyKiLer »

Weiv wrote: Sun Jan 02, 2022 8:02 pm
ZjoyKiLer wrote: Sun Jan 02, 2022 7:29 pm I think azemsbog also shared with you the HALT stuff, as I remember he showed us screenshots of Spectramine passing the latest version of Woody's HALT2INT test.

But, try this to be sure, and this.

Also, we are thinking about testing the ccf and scf instructions again, since it seems that the behavior on NEC NMOS CPUs is different, and there are contradictions between Patrik Rak's findings regarding the behavior on Zilog CMOS CPUs and what other sources say.
Thank you very much. I don't fully understand what the tests test but I have the same results in my emulator as in last version of SpecEmu and on the photos from real machines.
You're welcome.

The first one tests the HALT state.

From the comments of the source code of my Z80 core:

The "halt" instruction enables the HALT state after PC is incremented during the opcode fetch. The CPU neither decrements nor avoids incrementing PC "so that the instruction is re-executed" as Sean Young writes in section 5.4 of "The Undocumented Z80 Documented". During the HALT state, the CPU repeatedly executes an internal NOP operation. Each NOP consists of 1 M1 cycle with 4 T-states that fetches and disregards the next opcode after "halt" without incrementing PC. This opcode is read again and again until an exit condition occurs (i.e., INT, NMI or RESET). This was first documented by Tony Brewer and later re-confirmed by the HALT2INT test written by Mark Woodmass.

Reference: http://primrosebank.net/computers/z80/z80_special_reset.htm


If the opcode following halt is read from an address that is not contended, but the halt opcode is, or viceversa, then the contention emulation will not be accurate if the emulator doesn't implement this correctly. The test places a "halt" opcode in the boundaries of the VRAM so that this can be tested.

Example:

Code: Select all

----------------------------------
Contended RAM | Uncontended RAM
--------------+-------------------
   ... | halt | nop  | nop  | ...
--------------+-------------------
  7FFE | 7FFF | 8000 | 8001 | ...


Correct implementation:

fetch(7FFFh)
-> CPU increments PC
-> CPU enables the HALT state
(void)fetch(8000h)
(void)fetch(8000h)
(void)fetch(8000h)
(void)fetch(8000h)
(void)fetch(8000h)
...
INT
-> CPU disables the HALT state
-> CPU responds to the interrupt
-> CPU returns from the ISR
fetch(8000h)
fetch(8001h)


Incorrect implementation:

fetch(7FFFh)
-> CPU enables the HALT state
fetch(7FFFh)
fetch(7FFFh)
fetch(7FFFh)
fetch(7FFFh)
fetch(7FFFh)
...
INT
-> CPU disables the HALT state
-> CPU responds to the interrupt
-> CPU returns from the ISR
fetch(8000h)
fetch(8001h)
Woody wrote a game that only works OK if this thing is correctly emulated, it's called Super HALT Invaders. The score counter only works if HALT is OK.

I don't remember what the second one tests, I think Woody can provide a detailed explanation if you ask him.
User avatar
Weiv
Microbot
Posts: 178
Joined: Fri Apr 06, 2018 5:28 pm
Location: Ukraine

Re: Proposal for RZX format update

Post by Weiv »

Patrik Rak wrote: Sun Jan 02, 2022 7:27 pm Well, good luck with maintaining the list of how each particular version of each emulator handled blocked interrupts after each instruction. In my opinion, this is complete nonsense. Feels like you are making an argument just for the sake of it, no matter how impractical it is.
Well, there are not much of problematic RZXs created in not much emulators, and there is the only one problematic instruction for the interrupt blocking case - EI, and, maybe, the index prefixes.
I don't try to solve any bigger problem. My problem was that my own emulator was not able to record and play back everything properly, regardless of any other emulator out there. With what I propose, it can. And so can others if they choose so. That's good enough to me. IMO better than scratching my head because of bigger problems and in the end doing nothing for more than 13 years.

(EDIT: it sounds more harsh than I intend to. Please try not to project any negative feelings into this. It's very easy to get the wrong impression based on written text. Thanks.)

Patrik
Your emulator is able to record RZXs as properly as it can with its representation of Z80 emulation. Good emulators know about interrupt blocking and will play back your RZXs as properly as they can. Also they know that there are some problematic RZXs and they solve this problem as properly as they can.

At least I decided to deal with the problem in this way. But if people will decide to solve the problem in your way (just for compatibility with some bad RZXs) - well, it's not a big deal to change the code for your (strange, as for me) extension of RZX format.
User avatar
Weiv
Microbot
Posts: 178
Joined: Fri Apr 06, 2018 5:28 pm
Location: Ukraine

Re: Proposal for RZX format update

Post by Weiv »

ZjoyKiLer wrote: Sun Jan 02, 2022 8:06 pm
Woody wrote a game that only works OK if this thing is correctly emulated, it's called Super HALT Invaders. The score counter only works if HALT is OK.
Yes, [mention]azesmbog[/mention] shared this game some time ago on zx-prk.ru and I solved its puzzle on my own) Thank you again, anyway.
Post Reply