Games behaving differently on the Next

[R-Tape]

Strange one this. Dingo slows down a lot whenever you (or the dingos) fire. On a real Speccy, or emulated, I have seen the game slow down occasionally, but on the Next it's every time, and treacle slow.

I wonder why?

[Lethargeek]

because the emulation accuracy is still worse than even most software emulators i guess

viewtopic.php?f=23&t=752&start=50#p32111

and who knows what's going on under the lid with timings and the rest of the system emulation

[akeley]

I thought Next has a well-developed (as in, ~99% accurate) FPGA core for the legacy versions of Spectrum?

[Lethargeek]

I thought Next has a well-developed (as in, ~99% accurate) FPGA core for the legacy versions of Spectrum?

most of the effort went into the case probably

ok, maybe that was too harsh and they will hopefully fix the firmware soon... ish...

[R-Tape]

because the emulation accuracy is still worse than even most software emulators i guess

viewtopic.php?f=23&t=752&start=50#p32111

and who knows what's going on under the lid with timings and the rest of the system emulation

Hmm. Dingo does use LD A,R a lot in its sound engine. I tried changing the code, but it looks like the timing issue is something else/more general.

A couple of other things I just noticed:

Shuttlebug is not drawing the sprites quickly enough on the Next, so they are often not seen at the top of the screen.

The AY music on Octukitty is playing fast (I think).

[PROSM]

How are you displaying your Next's image, [mention]R-Tape[/mention]? If I remember correctly, the Next has to fudge the display timings in HDMI mode so that the signal conforms to the standards, which results in a loss of emulation accuracy.

[R-Tape]

How are you displaying your Next's image, @R-Tape? If I remember correctly, the Next has to fudge the display timings in HDMI mode so that the signal conforms to the standards, which results in a loss of emulation accuracy.

Yes HDMI. Ah so it's the same reason the multicolour is out. I'll have to buy a VGA cable (please tell me that works properly!)

[Lethargeek]

i don't get why they fudge the display timings for hdmi instead of tweaking the system timings, esp when there is no real z80 anyway

[akeley]

""The Spectrum Next - an updated and enhanced version of the ZX Spectrum totally compatible with the original..."

[toot_toot]

The Nirvana+ Engine games mostly don't work on the Next over HDMI, which is down to reduced timing compatibility using the HDMI interface.

I've also seen random slowdown in some games, I've had it happen in Zynaps and Exolon. But reset the Next and the games work fine, so I'm not sure if it's down to something like a memory leak.

[Seven.FFF]

Memory leak? That doesn't even make sense unless you think the Next is running a software emulator on a modern CPU. The Next SRAM is accessed directly through 21 bit addresses by the core, and with usual 128K, +3, Timex or Pentagon banking methods by games. It's never allocated through a heap, so the concept of memory leaks doesn't apply any more than it does to 80s Spectrums.

HDMI 50Hz has a slightly different number of Ts per line and lines per frame, as bending the machine timings was the only way to output an image close enough to adhere to the HDMI spec to make most TVs happy. The only possible alternative to this is a hold-and-resample scanbuffer, but the Next was never designed with one of these on a separate chip, and there isn't (and never was) enough room in the FPGA to implement a virtual one at the same time as other promised features.

All 60Hz outputs have a significantly lower number of video lines per frame, as an unavoidable consequence of the historical number of lines in the NTSC standard which is carried forward even into modern HDMI 60Hz modes. Again, without a scanbuffer the Next has to bend to the TV standard, not the other way round. 60Hz modes are exactly as compatible as an original NTSC Spectrum was - i.e. not very if the game assumes there at least 70k Tstates in the frame to make use of.

Games running at half speed is caused by users running in 60Hz mode when the game requires 50Hz, and frames overrunning and missing every other interrupt, but still trying to sync with a halt. The overrun unexpectedly consumes one frame, and the halt consumes the second frame. Switch to 50Hz VGA or RGB, and you'll be configured as a Spectrum model that's compatible with what the game programmer expected.

Nirvana games work great in 50Hz or Pentagon modes too. HDMI completely breaks the timing due to the necessity of having slightly fewer Ts per line as explained above. 60Hz doesn't inherently break multicolour timings, but a lot of the games use almost all of the 70k Ts available in a 50Hz frame due to how expensive the raster chasing is, so these games will overrun the frame at 60Hz and crash or run half-speed, which then destroys the timings for a differerent reason. It would be quite possible to write a new version of the multicolour engines that adapts to the HDMI timings like Denis did for the Pentagon, but running them in 60Hz isn't really fixable.

If you really must use HDMI or 60Hz because you have a display that only supports those modes (which is a legimate thing - TVs only have to adhere to a minimum part of the HDMI spec, not the entirely of it), you can use an external scanbuffer device. I have a cheap RGB-SCART to HDMI converter which accepts input from the Next in one of its accurate timing modes, and outputs at various HDMI resolutions at both 50 and 60Hz. Other more elaborate buffers like the OSSC do a fine job too.

in future you'll also be able to run legacy games in an alternate core which does have room for a scanbuffer in the FPGA, because all the next-gen features are dropped. This already exists as a proof of concept and works great, but the multicore management system needs a little more work to make it ready for the mainstream. Once in this core, timings will be accurate even in HDMI and 60Hz modes, and you'll get the best of both worlds.

in short, seeing the Next as partly incompatible is mostly missing the point. It's compatible with existing 50Hz and 60Hz Spectrum models, plus introduces some new pseudo-models which have variant timings that match displays which didn't exist in the 80s. It's just a problem than not all games were designed to work on all models or with unknown-at-the time timing variations.

[Lethargeek]

HDMI 50Hz has a slightly different number of Ts per line and lines per frame, as bending the machine timings was the only way to output an image close enough to adhere to the HDMI spec to make most TVs happy. The only possible alternative to this is a hold-and-resample scanbuffer, but the Next was never designed with one of these on a separate chip, and there isn't (and never was) enough room in the FPGA to implement a virtual one.

Not the only one: the other way was to change the duration of these Ts (and/or even stopping the emulated CPU at certain times) making it completely transparent for spectrum software and for the user. Maybe it does complicate other things but surely the hardware is powerful enough to solve it.

[Seven.FFF]

Not the only one: the other way was to change the duration of these Ts (and/or even stopping the emulated CPU at certain times) making it completely transparent for spectrum software and for the user. Maybe it does complicate other things but surely the hardware is powerful enough to solve it.

That is being looked at too, as a separate exercise. There's some work being done in the MISTer project we're following interestedly. It does complicate other things for sure, especially as the DMA and copper are tied to the pixel clock, and are already used to output timed audio DAC samples. Perhaps this hypothetical mode might be more suitable as a 128K-only alternate core rather than incorporated into the main next-gen core. Manipulating the duration of the visible Ts in the pixel and border area to be TV friendly, and then compensating in the non-output flyback areas is certainly an interesting idea.

Unfortunately the Next is also a multicore machine, and most alternate machine cores by other authors are not even 60Hz or HDMI capable, let alone doing T-stetching. So it's likely that VGA0@50 and RGB0@50 will be the only core modes fully supported by the Next for the foreseeable future. Anything else, as I tried to allude to above, is going to be a nice but partial bonus.

The duration of the Ts is also being changed currently, but by changing the master clock speed. This is why VGA and RGB modes can run six other intermediate speeds between 50 and 60. The entire machine timing runs slightly fast, so music is pitched faster and the DMA audio mode and UART baud rates have to be rescaled, but the relative T-state timings remain the same and multicolour games also work in these configurations. Often VGA1 will work with a display that refuses to work with VGA0, and the speedup will be barely noticeable. Just occasionally a user has a screen that only works in VGA6, which is close to 60Hz, and then the speedup is almost 20%. Personally I don't think this is worth the candle, but people get fiercely attached to their displays and will put up with a lot other stuff to keep them.

[akeley]

in short, seeing the Next as partly incompatible is mostly missing the point.

Myself, I'm not actually seeing anything yet - just trying to understand what's up, because for me accuracy in such an advanced project is absoluteIy crucial.

I apppreciate your detailed explanation (even though it's way too technical for me to understand fully). If this is down to incompatibilities with the modern side (HDMI, displays, etc) and solvable by the way of various configurations then it's fine by me. Of course, if it can be improved by other means, as Lethargeek mentions, then it's rather interesting too.

I just think this is something that should be talked about more, so it's a good thing that R-Tape brought it up (pardon me if it is a known issue, it's just something I see mentioned for the first time anywhere). Because many users won't have a clue that things like that are actually happening in games and would think that it's normal behaviour.

How about tests from this thread? ZX Uno core sems to pass them all.

Also, I'm a CRT user. Would that simplify things or should I expect problems as well?

[Seven.FFF]

How about tests from this thread? ZX Uno core sems to pass them all.

This is a little bit of a straw man. Patrik Rak has done some great work modeling the internal undocumented intricacies of one particular Z80 model, and the failures you're talking about here are only related the interaction of two undocumented and unused flags on instructions that follow them. Plenty of real Z80s would fail these tests too, and no commercial games would ever make use of thus behaviour in such a way that failing the tests would stop the game working properly. If they did, it would be to deliberately trip up emulator writers at the expense of making their software incompatible with some actual Spectrums.

Mostly the tests are used by emulator authors who realise their emulator failed, then proudly make the few tweaks so they can go round saying their emulator works but somebody else's emulator doesn't. It doesn't actually have much to do with the real world at this point.

The Z80 used by the Next is an open source implementation which has quite a few other bugs in its published form, and the Next team has fixed all the ones which were stopping the Next working properly. For example NMIs were quite broken, and wait states used by the +3 timings also were. If you look at other retro projects, they're mostly using this T80 core without fixes, or have cobbled together their own partial fixes as things were noticed. Miguel has tweaked the Uno a little to pass the Rak tests, but the standard T80 fails them too.

Making the Next pass those particular remaining tests is definitely on the list, but not very high priority. There's some VHDL refactoring needed before it will be done. It's not stopping anything working right now, apart from stuff which has been artificially written to do so.

Also, I'm a CRT user. Would that simplify things or should I expect problems as well?

Simplifies things. Provided your CRT runs in normal 50Hz mode then you can buy a Next RGB-SCART cable from Retro Computer Shack or Cool Novelties, plug it in (or use BNC adaptors if it's a PVM or BVM, etc) and it should just work, with accurate machine timings. When running the initial testcard routine to set up your machine, you can skip all the VGA and HDMI modes by holding R down at power up, which will just give you a choice between 50Hz and 60Hz RGB. You should choose 50Hz if you can see the card.

A few people have said that their image is not quite centered on CRT by default, and have needed to use the OSD menu (or PVM knobs?) to manually get it is as close to the center as they can.

[Pegaz]

This is really disappointing.
If the ZX Uno can pass all the tests, why Next has 22 errors?
On the other hand, if the Next like this reaches $1,000 in auctions, who cares about timings...

[Seven.FFF]

I wrote a four paragraph explanation for that above, [mention]Pegaz[/mention]. You've literally just reiterated what I pointed out in the second paragraph, as if it made your point instead of mine. The timings have nothing to do with the test failures.

This is a little bit of a straw man. Patrik Rak has done some great work modeling the internal undocumented intricacies of one particular Z80 model, and the failures you're talking about here are only related the interaction of two undocumented and unused flags on instructions that follow them. Plenty of real Z80s would fail these tests too, and no commercial games would ever make use of thus behaviour in such a way that failing the tests would stop the game working properly. If they did, it would be to deliberately trip up emulator writers at the expense of making their software incompatible with some actual Spectrums.

Mostly the tests are used by emulator authors who realise their emulator failed, then proudly make the few tweaks so they can go round saying their emulator works but somebody else's emulator doesn't. It doesn't actually have much to do with the real world at this point.

The Z80 used by the Next is an open source implementation which has quite a few other bugs in its published form, and the Next team has fixed all the ones which were stopping the Next working properly. For example NMIs were quite broken, and wait states used by the +3 timings also were. If you look at other retro projects, they're mostly using this T80 core without fixes, or have cobbled together their own partial fixes as things were noticed. Miguel has tweaked the Uno a little to pass the Rak tests, but the standard T80 fails them too.

Making the Next pass those particular remaining tests is definitely on the list, but not very high priority. There's some VHDL refactoring needed before it will be done. It's not stopping anything working right now, apart from stuff which has been artificially written to do so.

It is the case and keyboard of the Next which is fetching the premiums in the current climate where demand outstretches supply. Nobody should be paying inflated prices for Nexts, as there will be a second kickstarter soon that will increase supply and bring second-hand prices down.

Some people are willing to purchase expensive Nexts because they want to develop now, before ks2 delivers. In those scenarios the case and keyboard is not important, and devs should purchase a 1MB ZX-DOS right now from Antonio Villena, and run Antonio Silva's Next core on the DOS hardware. I have one of these as well as my Next dev board, and it is fantastic!

A few people are just mega-rich and make impulsive purchases engaging in fierce bidding wars, but you can't really do much about that. You shouldn't gauge the true worth of the Next by those few auctions though.

The Next core is GPL3 open source, so you or Miguel or anybody could contribute a patch or pull request today for the Rak tests if you felt that passing the tests was causing you material problems. I'd genuinely like to see an explanation of what those problems are. In the absence of that, it's just a way to artifically contend the Uno is "better" than the Next, when they're not directly comparable. Or an excuse for people like you to say "It's really disappointing", as if your disappointment was a metric that measures a real-world deficiency But it's ok, you can be freely disappointed if you like.

[Alcoholics Anonymous]

The Next has exact timing for 48k, 128k and pentagon in 50Hz VGA/RGB. It is close with +3 and that will come when some time is spent on it. This is why you can run all 48k, all 128k, and all pentagon sw properly.

It does not have correct timing on HDMI and this has been repeated many times for the past year. "If you want to have exact video timing, use VGA or RGB." This is a normal state in the fpga community where most cores are VGA only. The Mister project is an exception and that's because it has a giant fpga (they have an article describing the situation https://github.com/MiSTer-devel/Main_Mi ... oth-output ). In the spectrum community this is normal too -- the uno is VGA only, the evo is VGA only. For the evo, it's only a subset of VGA displays that will work. The Next is the first spectrum fpga system to attempt HDMI.

The correct timing is a relationship between the z80 and the ula, which is in turn connected to PAL video timing. This is fine for analog systems like VGA and RGB because their vertical and horizontal scan rates are flexible. Even though the Spectrum's video signal was not correct PAL, it still worked on TVs back in the day.

HDMI is a different story. It is rigid and its timing is incompatible with PAL. The HDMI standard does not mean all TVs are the same -- it only guarantees some modes are supported by some TVs and the level of support varies widely among TVs. You have no choice but to support several documented modes or stick to two where at least one is supposed to be supported by all TVs. If you are out by even a cycle, some TVs will not display a picture or may flicker periodically.

You can do a few things to adapt the spectrum video timing to hdmi:

* You can do frame conversion which involves storing up to one frame of the next's video picture. The next would generate its video into a buffer at its rate and the hdmi source would read it out and deliver at a different rate. The next does not have enough memory resources in its fpga to do this. The pi could do this, however that would severely reduce the pi's usefulness. The zxhd is an example of a device that uses a pi to do frame conversion.

* You can alter the spectrum video timing to match hdmi by speeding up / slowing down the machine and altering the number of T states in a line and/or frame. The speed up/ slow down helps with altering the T states as little as possible. No buffering is needed as the source video is timed to match the hdmi. However the spectrum video timing is altered and that breaks the timing relationship expected by some programs. This is the approach currently taken by the Next.

* You can try to speed up the spectrum to fit the right number of Ts in one vertical frame and halt the clock during portions of the horizontal scan to match the hdmi horizontal rate. This doesn't work either as it breaks audio. Beeper sound is sh*t and ay doesn't work for sampled music or stuff that does "sid sound".

* The last approach, which I will be trying, is to match the vertical rate of the hdmi and the vertical rate of the spectrum exactly to try to minimize the amount of buffering needed in each frame. The buffering needed will expand to some maximum during the frame and reduce to zero at the end of each frame. Because the hdmi and spectrum resolutions will not match, some 2D filtering will be required to avoid periodic doubled up pixels. This has to be accomplished in limited available space.

There is another option too and that is to abandon proper video timing on the Next but to supply plain 48k, 128k, pentagon, etc cores that have space for hdmi buffering. These cores could be launched by nextzxos whenever running legacy programs. But this would not solve a larger issue, that of other cores. There are now arcade cores, bbc, amstrad, colecovision, 2600, intellivision, etc that can also be loaded up on the next. Almost all of these are VGA only exactly for the reasons listed above. A general solution for hdmi with the constraints in place would mean these too can run on hdmi.

This is really disappointing.
If the ZX Uno can pass all the tests, why Next has 22 errors?

Because the failing tests have little importance so it doesn't have high priority. These tests are testing obscure and undocumented behaviours and are comparing against the results from a few specific nmos z80 models. Some models of real Z80s can't pass these tests, including cmos z80s which have incompatibilities that affect some demos. What is important is undocumented behaviour that is actually useful and has been used by programmers. These are also part of these tests so I'm not saying they aren't useful -- it's just that you have to be careful when judging the importance of what is failing.

We have gone in and fixed several things, including some major errors in the T80 implementation which can only be seen if you try to attach real hardware to an fpga system, if you try to implement +3 contention or if you execute a specific class of instructions at high speed. These things have gone largely unnoticed because it is not common to attach peripherals to fpga projects and it's not common to implement the +3.

If you can find any examples of any 48k, 128k, pentagon programs that do not work you should let us know. The list is very short at this time and the compatibility of the Next is much higher than any original models.

[toot_toot]

It’s been mentioned before, but many general users won’t be aware of the technical nuances that are now being discussed, they will just see it as a game not working.

I have to say the whole HDMI issue has been the most disappointing for me. When I backed the Next, at no point was it stated that HDMI was going to have issues, meaning some games simply don’t work. I backed a Next that was promised to be “future proof” to work with modern TVs. It feels like a bit of a backtrack now where it’s being said that HDMI was never going to be the main output. My TV doesn’t even have a vga port, it only has HDMI, which I would imagine most general users also have. I also didn’t expect to have to buy an additional unit to connect VGA to then HDMI, which is very messy seeing as the Next has an HDMI output already.

What it means is that a lot of modern games don’t work as intended on the Next when using HDMI. I was looking forward to going through the “best modern games starting with” threads and trying out the many modern games that Ive not experienced, but I gave up because it was so hit and miss on the Next. Seeing as we’re in lockdown, I can go through those games and compile a list of those that aren’t working and post here, if it helps with fixing any issues.

Regarding the “memory leak” issue, I don’t know what the cause is, but some games seem to suffer from eventual random slowdown. If you soft reset the Next and load the exact same file in the Next, the game works as normal. While it might not be a memory leak, it feels like one of those sort of issues. Again, not great if you’re in the middle of a game and have to reset but thankfully you’ve got a save state option! I’ll try and replicate it and post the results on here if I can.

Also Football Manager Revisited doesn’t work as it gives a “nonsense in basic” error. Maybe the author will make a Next version as it would be great playing the updated version of the classic game.

[bob_fossil]

Full disclosure: I'm not a backer / owner of the ZX Next (I got a ZX Uno a couple of years ago) but like many in the community have been following the development of it.

As an outside observer, it's interesting, given the forensic level of effort (and time) that went into getting the keyboard 'just right', that issues with the HDMI video output and Z80 emulation were left as they were on release.

The main boards have been out in the wild for a while (end of 2017) . Didn't anyone test / try any of the recent multicolour or more demanding Spectrum games or demos on it and spot the problem? Given that there is now a software solution in the works a couple of months after release, this suggests it could have been sorted out in the period between the boards being issued and the final boxes being shipped this year.

[Pegaz]

@bob_fossil

I think the same way, but instead of getting an answer to that question, we get tons of technical essays on why hdmi is a bad boy.
Someone had to do a reality check first and not promise the HDMI feature at all, if it didn't work properly.
Or maybe we just expect too much from people, who haven't been able to fix the Next keyboard for three years.
Fortunately, we still have the right to be disappointed, until further notice.

[Ast A. Moore]

So far I’ve been abstaining from commenting on the Next, because it’s never been something I’ve been interested in. Nevertheless, I find most criticism aimed at it unjustified.

Let’s set aside the whys and wherefores of the very concept of such a machine and focus solely on what it is and what it is not. It is, essentially, an homage to the idea of a home computer that could have been produced sometime in the 80s or early 90s. There were plenty of both more, and less, powerful machines actually produced back in the day. But the Next itself is not a Spectrum. It’s a concept of an abstract gaming machine with a few nods to the Spectrum (mostly in terms of its industrial design and menu UI). The Spectrum compatibility layer is just that—a compatibility layer. I don’t believe the designers had the goal of making it a 100-percent compatible Spectrum clone as a priority. On the contrary, the key goals were to add higher resolution graphics, a richer color pallet, “hardware” sprites, a fast and easy to use storage system, and so on. That’s what the Next is. And from what I’ve seen and heard so far, games written specifically to take advantage of the Next’s unique features run just fine over HDMI.

So, I don’t really see the problem here. A non-Spectrum machine has a Spectrum compatibility option. That option has limitations. So? It’s not a Spectrum. I repeat—it’s not a Spectrum. Why is it getting so much flak for not being a Spectrum then?

[Swainy]

It feels very much like a Spectrum in hand. I love my Next but then again mine is hooked up to nice little 14” Sony Trinitron CRT.

[Lethargeek]

* You can try to speed up the spectrum to fit the right number of Ts in one vertical frame and halt the clock during portions of the horizontal scan to match the hdmi horizontal rate. This doesn't work either as it breaks audio. Beeper sound is sh*t and ay doesn't work for sampled music or stuff that does "sid sound".

Doubt there will be much audible difference, as the beeper usually needs to be toggled once in a few scanlines (more likely several scanlines), esp if these halts weren't lumped together. And even if it won't be satisfactory, buffering the sound is a lot easier than buffering the video frames.

[Lethargeek]

But the Next itself is not a Spectrum. It’s a concept of an abstract gaming machine with a few nods to the Spectrum (mostly in terms of its industrial design and menu UI). The Spectrum compatibility layer is just that—a compatibility layer. I don’t believe the designers had the goal of making it a 100-percent compatible Spectrum clone as a priority. On the contrary, the key goals were to add higher resolution graphics, a richer color pallet, “hardware” sprites, a fast and easy to use storage system, and so on. That’s what the Next is. And from what I’ve seen and heard so far, games written specifically to take advantage of the Next’s unique features run just fine over HDMI.

So, I don’t really see the problem here. A non-Spectrum machine has a Spectrum compatibility option. That option has limitations. So? It’s not a Spectrum. I repeat—it’s not a Spectrum. Why is it getting so much flak for not being a Spectrum then?

then why it is called "ZX Spectrum Next", when the key goals were to make something closer to MSX (and done by the MSX guy)

why not "Kinda Spectrum Too" or something