ZX Recoloring project

[geecab]

Yeah, that's why I'm amazed. He's hit his limit at 20 sprites - or his PC grinds to a crawl presumably. Unless this is being run on a 486, then a much better strategy is needed. And oh my, there are so many.

Remember that this is an Alpha build of software, still very much a work in progress / proof of concept. The recognition code is not optimized in any way, it searches for everything, everywhere, for every frame, so its expected to be processor hungry. A few posts ago I mentioned several obvious ways I could think of to boost performance, no doubt the developer has his own ideas too so I am not too concerned about high CPU usage at this stage

[ZXDunny]

As has been pointed out, the problem is that nobody uses the tool, so... nobody is inclined to use the tool.

Graphics detection is pretty easy to do, especially as you have (in the worst case) only 6kb of image data to search. And if it's written well, you only have to search it once per frame. As the emulator builds the frame byte by byte, that portion (the screen rendering) is the perfect place to either directly replace the graphics or to buffer a list of changes that need to be made once frame generation is complete. Honestly, if I can detect 50 sprites per frame (of 16x16 pixels) in interpreted PC BASIC then a compiled solution should be able to detect thousands per frame.

As you're testing screen bytes for bit patterns (which is blisteringly fast on any PC hardware) you can then check the attribute at the end of the frame to catch those "same sprite, different colour" issues that apparently plagued Spec256 (only they didn't, really).

To be perfectly honest, I think that the best method would be a blend of Spec256's implementation alongside this sprite-detection - would solve all the issues, both the shortcomings of this and that of Spec256. Generic sprites and background items could be immediately drawn as 256 colour items, and anything procedurally generated or colour-changing could be flagged as such as part of the detection algorithm.

One thing though - don't increase resolution - Spectrum games very rarely look better in higher res, as the i-ball example plainly shows, and with a lot of games using pixel-perfect collision detection you're going to muck that up as smoothed curves generally use more pixels.

[4thRock]

The problem with recognition on a running game screen is sprite overlap and false positives.

I think a better solution would be to use pattern recognition trick to find/replace the sprites in memory or even better, on a tape file.
That would give you two options:
» Change the original pixel data. Only original resolution and no color. No overlap issues and total compatibility. You'd get a new TAP/SNA.
» Watching the original sprite memory positions and feeding extra graphic data when accessed (like Spec256 I think). Needs a special emulator but has no limitations.

[ZXDunny]

The problem with recognition on a running game screen is sprite overlap and false positives.

I think a better solution would be to use pattern recognition trick to find/replace the sprites in memory or even better, on a tape file.
That would give you two options:
» Change the original pixel data. Only original resolution and no color. No overlap issues and total compatibility. You'd get a new TAP/SNA.
» Watching the original sprite memory positions and feeding extra graphic data when accessed (like Spec256 I think). Needs a special emulator but has no limitations.

The second one has legs, IMO. Sprite overlap won't be a problem for a Spec256 implementation, as you're reading/writing 256 colour byte strips directly to the 256 colour screen. As for detection instead, you could take it one step further - hook the z80 code that draws the sprite. It's that simple - you just hijack it and draw your own sprite to the display instead.

"But that would mean we need to be able to detect the game!" You all cry. Well yes, but your current implementation of auto-identifying screen graphics already does that - your replacement graphics won't work for any other game, will they?

Just insert an $ED00 instruction at the sprite draw routine and have your emulator pause emulation while you inspect what's being drawn, render your own sprite and then issue a RET to go back to the emulation.

DId I mention that there were many solutions to this that are superior to image recognition?

[4thRock]

Image recognition solves the problem of matching sprites with memory content.
That makes it a "universal" solution. In theory all we need is a sprite sheet, and then the software would map the original memory positions for all the graphics.
This would allow for the Spec256 approach on a user friendly way
User friendly is important (that's why previous efforts have failed, too hard to work with).

[Joefish]

You need an emulator that reads and writes memory using shadow RAM images.

When a memory read is performed, you read from three extra copies of the 64k that's paged in - and store them in shadow registers. When writing, you either write just the 8 bits you read from main memory (in the case of a general memory write), unless you're writing to the screen in which case you extract 8 bits for PAPER and 8 bits for INK and combine those to a screen that can handle 256 colours. If you have 7 extra banks of RAM, you can easily make graphics that are 256 colours per pixel.

Of course, you can't load games from tape this way, but snapshots are really useful. All you need then is an editor that allows you to identify graphical objects in memory and pre-load those regions with 256 colour representations of their 1s and 0s.

This has already been done, btw, many years ago. But it never gained any traction at all, despite being very simple for the average user to mess around with. You don't need to do image recognition, no need to trap screen display routines.

I quite liked the 256 colour project, but I think there were a few flaws that needed to be overcome. It didn't get updated much. And I've probably forgotten or got most of this wrong...

The added-in backgrounds just looked awful and should have been dropped.

The SNA editor wasn't very sophisticated, and I can't exactly remember but most 256 colour palettes I see are a mess. It may have been better to pick a smaller pallette, or do something like 16 shades of 16 common colours, rather than try and use some 'web-safe' jumble where it's impossible to find the colour you want.

It would also be much more interesting if part of the palette was set aside for e.g. 8 shades of the PAPER colour of a character cell and 8 shades of the INK colour, maybe hue-shifted or tinted by the BRIGHT bit. Then when you used the INK-related colours in a sprite, they would change depending on the colour that sprite was drawn in. So in Manic Miner you could draw a penquin in cyan with an orange beak and a white belly, but the main sprite colour would swap from cyan to yellow automatically for the other penguin.

The sprite editor would also need some sort of intermediate file format where you can designate the sprites and how they're stored in memory. Once that was defined for a game, more people could then get into editing its graphics.

[geecab]

I have used Emuzwin in spec256 mode a lot quite recently. It is very good, though it doesn't work for every game you throw at it, there are also some general bugs/usability/stability issues with it too (Which I mention in this post). I really like the sound of ZXDummy's solution, and the tool that 4thRock mentions (Software that would map the original memory positions for all the graphics from a tape image). Unfortunately, these things don't exist yet, would be nice though as I'd happily use them

Its been interesting listening to opinions regarding the best way to add color to Spectrum games and what does and doesn't look correct (Amount of colors and resolution). It sounds like that would make a good debate thread on its own (I.e. "What colors and resolution should spectrum remakes have?" thread). They way I see it though, there is no right or wrong, Sinclair never released a Spectrum 48K/128K with improved graphics, so with everyone 'imagining what might have been' you are likely to get a difference of opinion. Purists can make a remake that might have been achievable in the 80ties given the hardware available if that's what they desire, but that doesn't mean everyone else must be constrained to do the same. Some people might want to make a completely modern looking remake. I was thinking that I might digitize a few of my friends, and use the tool to replace the graphics on WayOfTheExplodingFist. I thought that'll be quite a fun project, I don't need perfect results, and would be very quick & easy to achieve with the sprite recognition tool.


With all the talk of the correct/theoretical ways of doing things, its easy to loose sight of the benefits this tool (a pure sprite recognition solution) has:-

- Easy to understand and you can get something working quickly

- No Z80 machine code literacy required

- Use any paint editor you like

- Could be developed into a general solution for all platforms (Not just the Speccy)


Sure, it has its limitations, but lots can be over come and we haven't scratched the surface:-

- Sprites hiding behind sprites issues
There is always the danger of so little of one sprite showing, that recognition algorithm won't find focus. However, who is to say the recognition algorithm can't be modified one day to make intelligent guesses as to what a partially hidden sprite is based on its historical position in previous frames. Your eye does this, then recognition algorithm could be trained to, maybe .

- Performance issues
Already mentioned a few ways the tool could be optimized, and with PCs getting faster and faster I don't see this as a long term issue.

[ZXDunny]

II really like the sound of ZXDummy's solution

<splutter>

- Sprites hiding behind sprites issues
There is always the danger of so little of one sprite showing, that recognition algorithm won't find focus. However, who is to say the recognition algorithm can't be modified one day to make intelligent guesses as to what a partially hidden sprite is based on its historical position in previous frames. Your eye does this, then recognition algorithm could be trained to, maybe .

Yeah, we could have an online connection to Google's deepmind.

[Ralf]

I believe a sprite recognition is generally a wrong approach. In Spectrum games, graphics often overlap and when they overlap often things like XOR are used which alter pixels even more. And even without any XOR - when a player stands in front of a tree then only a part of a tree is visible so how it's going to get recognized? It will mostly fail and not work instead of a few very special cases where nothing touches nothing.

I would say the proper approach was used in EmuZWin - paint directly over bytes in memory. Yes, it often failed too and would need quite a lot of work to make it work better and become more user friendly but that's the way it should be.

[geecab]

<splutter>

Sorry, honest mistake

[rg_software]

Well, I see the discussion goes into the direction of "whether it is right approach or not"

I think the problem boils down to the fact that Spectrum doesn't have any "hardware-supported sprites" or anything like that. While for each particular game you can perhaps point out a better solution (let's say that for games like Elite you don't need sprites at all -- we can scale vector graphics wth perfection), in general case I see no real choice here.

First, on performance. Given that Spectrum screen is just 6912 bytes, I think sprite search should not be an issue. Currently I use a naive approach to searching, but I can implement something like Aho-Corasick algorithm, which should make searching of multiple sprites really fast. I can again complain about lack of time to do it, but I see interest now, so I promise to reconsider my schedule

Next, on false positives, etc, etc. Yes, it's all tough. Also, think about cases when sprites overlap and you need to decide which one goes above the other (say, in games like Fairlight). I am aware that we can find a lot of potential issues. But I really see no other "magic" approach that would solve these issues. Every game is different, sprite handling is different and so on. If I could only be sure that somewhere in the memory there is a map of sprites that can be substituted... Alas, I don't think that's the case.

But, anyway, instead of expounding my theories I'd perhaps better focus on the code... Maybe within the next 2 weeks or so I'll try to show something new. Thanks for your interest, really, I am happy to be here.

[AndyC]

In terms of "what resolution should games use" I think it's important to distinguish between real remakes and just coloured in spectrum titles. A genuine remake of the game has a low of flexibility to change how things look and adjust collision detection, movement etc to compensate. Recolouring a spectrum title is a lot more limited - it may break the game mechanics if a sprite is changed such that it no longer maps directly over the original. As Dunny mentioned, even something like smoothing curved edges might start causing collision detection to feel "off" when the underlying game engine is working to a pixel perfect hit with the original sprites. Taking it even further if you changed the graphics enough it might even become necessary to change the underlying imagery so the engine has a better concept of what is going on.

[Joefish]

Why not use the pattern-matching algorithm to search for the sprite data in a memory snapshot and present it for editing, then use a Spectrum-256 like approach to present it in-game?

[hikoki]

[mention]geecab[/mention]

Yes, I definitely think all that should be possible. You have put the idea of recoloring Scuba Dive in my head now Excellent game, and there shouldn't be many sprites to recolor either...

I would enjoy playing your mod for sure. That could be another feature idea, how to give the impression there are a greater variety of sprites just by playing with the search and replace mechanism. There could be randomisation when a match is detected like showing different treasures every time on Scuba Dive.

I don't see much of a problem in getting artifacts as long as they don't happen all the time and appear in a good looking way when possible just as a reminder of the original sprites and engine.

[4thRock]

Why not use the pattern-matching algorithm to search for the sprite data in a memory snapshot and present it for editing, then use a Spectrum-256 like approach to present it in-game?

Yep, second that.

Regarding what type of enhanced graphics are more acceptable, I'd go for what the Spectrum might display if it had hardware sprites.
So two colours (*) per 8x8 pixel block, aligned with the sprite origin. Perhaps 8x1 if we consider a Timex Hicolour mode. No sprite colour clash when placed on screen.

*- Well, if we consider paper as transparent, then it's a single colour per block but I can overlook that. No need to be 100% purist.

[4thRock]

Here's my Xevius test recolour. For some reason it can't replace the bullet, the crosshair or the square scenery block.
I was lazy and didn't rename it, so click on iball2.exe
Feel free to play with this and eventually finish it, for me it was just a test. I included a sprite sheet to make things easier

https://we.tl/t-kZizNQltfI

[ZXDunny]

I actually did something similar.

A long time ago, I wrote a remake of Sinclair BASIC. I run the emulation, and then for the parts I want to replace I hook the ROM code - when the emulator encounters one of these hooks, it hands processing off to the relevant handler, and then returns. The emulated Speccy has literally no idea that anything has happened. That way I was able to remake the BASIC editor, the error handling, get statistics about memory usage, monitor variables and do other stuff too, like a full debugging suite that offered breakpoints and watches and suchlike in real time while the BASIC program was running.

It was quite successful.

This is not really much different, but you're trading ease of use in order to be able to recolour and enhance only a very, very small number of games. There are very few out there that don't use XOR to print their sprites - any game that does that will immediately fail to be recoloured and will drop back to Speccy graphics. That's got to be a jarring experience.

As an example, Manic Miner and JSW both disallow (due to their game engines) sprites from touching anything. That's great, but the player sprite can walk through certain passable walls - as soon as it does, your recolouring goes out the window.

And you'll never be able to enhance any 3D games. Not Mercenary, nor Elite, nor Starion, nor any Freescape title. You can't enhance any Ultimate games. Nor any of the Wally Weeks. None of the Odin games (Nodes springs to mind) can be enhanced by image recognition.

To get around this you absolutely must take a different route. Intercept screen memory writes and identify sprites from there - maybe keep track of them in 16x16 (or whatever size) chunks you can examine at the end of the frame and replace in-order. That would eliminate most of the overlapping issues, but only for games that don't do any double buffering. And it wouldn't solve 3D games.

Go the Spec256 route of layering 8 bytes for every byte of RAM. It's pretty simple to do, and an editor that's well written can take the heartache out of making the games for the users. That's certainly better than any image recognition, but may fail on different coloured sprites - unless you can examine the attribute bytes when it's written. An intelligent system could maintain several "pages" of graphics to choose from based on the ATTR value.

But even that can't match the king of recolouring - code hooking. There's a very good reason I used it myself. You can catch anything - sprites, 3D, you name it. Want the Speccy Outrun port to look amazing? Easy. Want it to run faster, higher frame rate? Hook the interrupt routine, or modify T-State counts for RAM writes. The world's your Oyster.

But of course, this method is really, really hard for your average user. You could churn out your own datafiles for games, or you could go for say, a Python scripting interface and open up an API for some of the more savvy users to start converting games for everyone to play.

I know if I was doing this (and I may yet - I have plans to go back to Speccy emulation hopefully soon) I'd go for the latter. Hell, I even have a BASIC interpreter core that is as fast as Python but looks surprisingly like Sinclair BASIC which could be used for the API.

So which do you want? Easy for users to create their own, or high quality recolouring?

[rg_software]

Why not use the pattern-matching algorithm to search for the sprite data in a memory snapshot and present it for editing, then use a Spectrum-256 like approach to present it in-game?

I am not really familiar the details of with Spec256 approach. I mean, I know that it emulates a sort of "shadow graphics processor", but I neither know the details nor can estimate how hard is to to something similar.

Furthermore, I am not entirely sure that the sprites are stored in the memory in the same format for all games. We know exactly that when a certain picture is displayed on the screen, it has a predefined pixel structure. When it sits in memory, it can be literally anything stored in the way the authors wanted. For some games with large sprites (like Popeye) I guess there are maybe no complete sprites at all, just separate sprite pieces later combined into a character on the fly. So in order to recolor such sprites one would need to guess these pieces and process them separately.

[rg_software]

So which do you want? Easy for users to create their own, or high quality recolouring?

Well, you know, I'd say that "easy for users". When I duscussed my approach many years ago with Remakes.org people, some responses were "I do remakes because I like coding, that's it". Most Speccy games aren't rocket science, and it is not extraordinarily difficult to do a decent remake with "additional sauce" like scrolling or music or whatever capability you miss in the original (say, free camera mode in 3D games). Even the best recoloring ever can't get over these limitations.

Yes, you are of course correct when you speak about limitations. I wish we had an option to make recoloring both easy for users and reliable. If we are dealing with really advanced users who can dig deeply into game logic, they may simply opt for their own remake.

[geecab]

[mention]rg_software[/mention]
Great to hear you are getting your interest back into the project FYI. I am working on Rebelstar2 (One of my absolute favorite games of all time) at the moment and it works perfectly with your emulator. Loads of sprite recognitions happening fast as I can use the 8x8 'block' searching option in settings.txt, I'll post screenshots later. This means that Chaos/LazerSquad/SpaceCrusade (Excellent turn-based games that have a big fan base) are all good-to-go with the sprite recognition approach!

Yes, you are of course correct when you speak about limitations. I wish we had an option to make recoloring both easy for users and reliable. If we are dealing with really advanced users who can dig deeply into game logic, they may simply opt for their own remake.

Exactly. I feel the pure sprite recognition approach would always be a good place to start for users at any level (novice/or advanced). You might get to the end of your remake that uses only sprite recognition and realize that is all you need (Rebelstar2 for example), or maybe you do see occasional artifacts but decide that you can live with them. However, if you absolutely can not deal with 'what happens when that sprite goes there and touches that' situation, your sprite recognition work is not in vain, you have all your bitmaps drawn to help with your advanced solution.

[Ralf]

I am not really familiar the details of with Spec256 approach. I mean, I know that it emulates a sort of "shadow graphics processor", but I neither know the details nor can estimate how hard is to to something similar.

Furthermore, I am not entirely sure that the sprites are stored in the memory in the same format for all games. We know exactly that when a certain picture is displayed on the screen, it has a predefined pixel structure. When it sits in memory, it can be literally anything stored in the way the authors wanted. For some games with large sprites (like Popeye) I guess there are maybe no complete sprites at all, just separate sprite pieces later combined into a character on the fly. So in order to recolor such sprites one would need to guess these pieces and process them separately.

It looks like Spec256 website still exists You can read there a little about the used approach:
http://www.emulatronia.com/emusdaqui/sp ... ex-eng.htm

It was a closed project where authors modified themselves a few games but there wasn't open code or any editor available for everybody so he could recolour any game. So we may not know the details but they describe the general idea:

The idea is: in one hand we´ve got the speccy emulator with its memory zone, and in the other we emulate a Z80 which works with 64 bit registers instead of 8 bit, and with a memory map with positions of 64 bit instead of 8.

Each time a z80 instruction is emulated, the same instruction is simulated with data always supposed to be graphics. I´ve named this parallel processor Z80_GFX. Z80_GFX modifies its memory zone accordig to the instructions and doesn´t do anything with Z80´s memory zone. What have we archieved? A faithful emulation of Z80 and a memory zone from which we can obtain 256 colour graphics.

Each time they probably also used some custom hacks tailored for each game.

So that's Spec256. And we have also EmuZWin. I believe it uses exactly the same approach, just it enables the user to recolour any game himself.
And you are right, most time you work on chunks of graphics and not complete sprites. And you have to find these chunks yourself in a mess of pixels And it requires skill+intuition as these data may be stored in different format and you must reorganize the display to get it in the friendliest way. And yes, it's a pain

I believe you cannot avoid it, but you can improve the process with a good, friendly tool

[Ralf]

And for anybody interested, there is a recently active thread on a Russian forum about 256 colour games

Here's the English automatic translation:
https://translate.google.pl/translate?s ... ec256.html

You may find there quite a lot of recently recoloured games (with EmuZWin) and among others you may find there my almost completed (but having problems of course ) recoloured version of Renegade which I didn't announce anywhere else:

[ZXDunny]

The idea is: in one hand we´ve got the speccy emulator with its memory zone, and in the other we emulate a Z80 which works with 64 bit registers instead of 8 bit, and with a memory map with positions of 64 bit instead of 8.

Each time a z80 instruction is emulated, the same instruction is simulated with data always supposed to be graphics. I´ve named this parallel processor Z80_GFX. Z80_GFX modifies its memory zone accordig to the instructions and doesn´t do anything with Z80´s memory zone. What have we archieved? A faithful emulation of Z80 and a memory zone from which we can obtain 256 colour graphics.

Each time they probably also used some custom hacks tailored for each game.

No hacks needed for this particular approach. The 64bit z80 they talk about is misleading. You don't need to do that.

You have 8 "pages" for each RAM page. Each acts as a bitplane - the Speccy RAM contains the low bit of each bit that's read, and when you read a byte from RAM you read 8 bytes, one from each page and store them in 8 registers per speccy register. When performing memory writes, you do the same thing in reverse.

When it comes time to build the display for each frame, you take one bit for each pixel from those 8 banks of RAM and build a 256 colour pixel from those.

That's it - no hacks needed. The extra colours come from a tool which identifies sprites and graphics in RAM and allows you to colour them, eventually spitting out a "snapshot" that contains all 8 pages specially constructed from those 256 colour graphics.

You're right that it's all about the tool - if it's easy for the user to find and modify the graphics then the engine will take care of the rest.

It has limitations. Can't work on tape images. Can't recolour blank backgrounds (those need to be hacked in). Can't do 3D games.

But it's really the simplest one you can do.

[hikoki]

That Renegade remake looks good!!

About these tools, I don't know if it'd be possible a sort of rzx viewer where the user can record,pause,resume a gameplay. One could pause it and start clicking over the screen to assist the algorithm engine to build its configuration parameters about collision bounds, glitches, wrong matches, etc.

[ZXDunny]

Training a DeepMind instance to recognise sprites in a Speccy snapshot would be very, very doable. In fact, it would be an excellent solution to the problem where actually using your eyes to find them is just too hard for everyone.