ZX Recoloring project

Share graphical tips, notes and queries related to our favourite screen layout and its editors.
User avatar
4thRock
Manic Miner
Posts: 415
Joined: Thu Nov 09, 2017 9:35 am
Location: Portugal

Re: ZX Recoloring project

Post by 4thRock »

That Stormbringer recolor project is interesting.
Only gave it a quick look but having the graphics as .bmp is a great idea.

I'd limit myself to a simple graphic replacement with Spectrum "compatible" graphics (original resolution, only one colour per 8x8 or at most 8x1).
That's something that could have happened back in the day if the Spectrum had some sprite support.

Must find some time to try it with Bomb Jack and Golden Axe.
geecab
Drutt
Posts: 32
Joined: Sun Jun 23, 2019 12:44 pm

Re: ZX Recoloring project

Post by geecab »

4thRock wrote: Wed Jul 10, 2019 6:57 pm I'd limit myself to a simple graphic replacement with Spectrum "compatible" graphics (original resolution, only one colour per 8x8 or at most 8x1).
That's something that could have happened back in the day if the Spectrum had some sprite support.
The good things is that as the replacement sprites are bitmaps, you can mod someones else's bitmaps pretty easily in any paint editor and get the look you are after :)

>>Must find some time to try it with Bomb Jack and Golden Axe
Sounds good!
geecab
Drutt
Posts: 32
Joined: Sun Jun 23, 2019 12:44 pm

Re: ZX Recoloring project

Post by geecab »

Hi rg_software!

I've been adding the remaining graphics to Aquaplane (Thought I'd try and get one game completely finished). Unfortunately, I think I've reached the maximum number of sprite recognition jobs that my PC can handle (Which seems to be about 20 'pixel' recognition jobs in total).

I've had a few thoughts on how to overcome this. Please don't think I expect any of it to be implemented (Its sounds like a scary load of work to me now I've wrote it all down :p). I'm just suggesting stuff really and parts of it might be worth considering as and when you are in the code again :)

- In settings.txt, the "SearchArea=Y:n|X:n|H:n|W:n" parameter. Searches for a sprite in a specified area (Where n is pixels. Height (H) and Width (W) sets the size of the search area. The top left of the search area is positioned at Y and X). If omitted, the entire screen is searched.

- In settings.txt, the "MaxInstances=n" parameter. Stops searching for a sprite after 'n' instances of that sprite have been recognized. If omitted, search for as many instances of the sprite as possible.

- In settings.txt, the "Group=n" parameter. Specifies a group of sprites that depict the same object/character (For example, a sequence of sprites depicting a person running). When one of these sprites is recognized, no need to search for any of the other sprites in that group.

- Use multiple CPU cores to perform recognition jobs simultaneously.


Example of how settings.txt might look:-

; Search for the Sun, there'll only be one and it'll be located at the top right of the screen...
0 pixel zx_sun.bmp pc_sun.bmp SearchArea=Y:0|X:200|H:50|W:56 MaxInstances=1

; Below the Sun are some clouds that move horizontally across the screen, there are only ever 3 clouds shown at once...
0 pixel zx_cloud.bmp pc_cloud.bmp SearchArea=Y:50|X:0|H:50|W:256 MaxInstances=3

; Search for the game's hero. There'll only be one of him, but he's depicted by a number of sprites...
0 pixel zx_myhero1.bmp pc_myhero1.bmp SearchArea=Y:100|X:0|H:92|W:256 MaxInstances=1 Group=1
0 pixel zx_myhero2.bmp pc_myhero2.bmp Group=1
0 pixel zx_myhero3.bmp pc_myhero3.bmp Group=1
0 pixel zx_myhero4.bmp pc_myhero4.bmp Group=1


Something else I was thinking about was a dependency setting. For instance, the recognition of sprite A, B & C to occur only while sprite D is recognized. Or perhaps recognition of sprite A, B & C to start only when Sprite E is recognized, and to finish only when Sprite F is recognized. Just trying to think of ways to stop searching for everything all of the time. To search for certain set of sprites that only get displayed when the player reaches a particular level/stage/screen in the game.

Sorry for waffling on. Hope this helps :)
User avatar
ZXDunny
Manic Miner
Posts: 498
Joined: Tue Nov 14, 2017 3:45 pm

Re: ZX Recoloring project

Post by ZXDunny »

Holy crap dude, you're doing this by image recognition? While that's an impressive task, there really are much, much easier ways to do this :)

Are you monitoring an emulator screen display, or do you have your own emulator that this engine is tacked onto?
User avatar
4thRock
Manic Miner
Posts: 415
Joined: Thu Nov 09, 2017 9:35 am
Location: Portugal

Re: ZX Recoloring project

Post by 4thRock »

BombJack didn't work, it just replaced the entire background.

So I tried Xevious and it works :D
Image

The new ship sprite only has added colour. This allows it to downgrade nicely to the original when over a background:
Image
geecab
Drutt
Posts: 32
Joined: Sun Jun 23, 2019 12:44 pm

Re: ZX Recoloring project

Post by geecab »

Nice job 4thRock! That looks great :)
User avatar
4thRock
Manic Miner
Posts: 415
Joined: Thu Nov 09, 2017 9:35 am
Location: Portugal

Re: ZX Recoloring project

Post by 4thRock »

Thanks! Recoloured an enemy sprite and it's the same thing.
Mostly works but sometimes it defaults to the the original. It doesn't affect gameplay much but it's a bit strange ;)
Image
geecab
Drutt
Posts: 32
Joined: Sun Jun 23, 2019 12:44 pm

Re: ZX Recoloring project

Post by geecab »

ZXDunny wrote: Wed Jul 10, 2019 10:54 pm Holy crap dude, you're doing this by image recognition? While that's an impressive task, there really are much, much easier ways to do this :)
Are you monitoring an emulator screen display, or do you have your own emulator that this engine is tacked onto?
Yes that's right, it uses image recognition. Have a read though the thread posts to find out more, but to summarize, rg_software has modified the UnrealSpeccy emulator to do this. Its still at a early stage of development, but I really like the idea :)
geecab
Drutt
Posts: 32
Joined: Sun Jun 23, 2019 12:44 pm

Re: ZX Recoloring project

Post by geecab »

4thRock wrote: Wed Jul 10, 2019 11:39 pm Thanks! Recoloured an enemy sprite and it's the same thing.
Mostly works but sometimes it defaults to the the original. It doesn't affect gameplay much but it's a bit strange ;)
Nice! Its difficult to say why its defaulting to the original without seeing it running for myself (Maybe upload the project and I can take a look?). My guess is that it might the same issue as described in This post
User avatar
ZXDunny
Manic Miner
Posts: 498
Joined: Tue Nov 14, 2017 3:45 pm

Re: ZX Recoloring project

Post by ZXDunny »

geecab wrote: Wed Jul 10, 2019 11:45 pm
ZXDunny wrote: Wed Jul 10, 2019 10:54 pm Holy crap dude, you're doing this by image recognition? While that's an impressive task, there really are much, much easier ways to do this :)
Are you monitoring an emulator screen display, or do you have your own emulator that this engine is tacked onto?
Yes that's right, it uses image recognition. Have a read though the thread posts to find out more, but to summarize, rg_software has modified the UnrealSpeccy emulator to do this. Its still at a early stage of development, but I really like the idea :)
Yeah, that's why I'm amazed. He's hit his limit at 20 sprites - or his PC grinds to a crawl presumably. Unless this is being run on a 486, then a much better strategy is needed. And oh my, there are so many.
geecab
Drutt
Posts: 32
Joined: Sun Jun 23, 2019 12:44 pm

Re: ZX Recoloring project

Post by geecab »

ZXDunny wrote: Thu Jul 11, 2019 8:29 am Yeah, that's why I'm amazed. He's hit his limit at 20 sprites - or his PC grinds to a crawl presumably. Unless this is being run on a 486, then a much better strategy is needed. And oh my, there are so many.
Remember that this is an Alpha build of software, still very much a work in progress / proof of concept. The recognition code is not optimized in any way, it searches for everything, everywhere, for every frame, so its expected to be processor hungry. A few posts ago I mentioned several obvious ways I could think of to boost performance, no doubt the developer has his own ideas too so I am not too concerned about high CPU usage at this stage :)
User avatar
ZXDunny
Manic Miner
Posts: 498
Joined: Tue Nov 14, 2017 3:45 pm

Re: ZX Recoloring project

Post by ZXDunny »

As has been pointed out, the problem is that nobody uses the tool, so... nobody is inclined to use the tool.

Graphics detection is pretty easy to do, especially as you have (in the worst case) only 6kb of image data to search. And if it's written well, you only have to search it once per frame. As the emulator builds the frame byte by byte, that portion (the screen rendering) is the perfect place to either directly replace the graphics or to buffer a list of changes that need to be made once frame generation is complete. Honestly, if I can detect 50 sprites per frame (of 16x16 pixels) in interpreted PC BASIC then a compiled solution should be able to detect thousands per frame.

As you're testing screen bytes for bit patterns (which is blisteringly fast on any PC hardware) you can then check the attribute at the end of the frame to catch those "same sprite, different colour" issues that apparently plagued Spec256 (only they didn't, really).

To be perfectly honest, I think that the best method would be a blend of Spec256's implementation alongside this sprite-detection - would solve all the issues, both the shortcomings of this and that of Spec256. Generic sprites and background items could be immediately drawn as 256 colour items, and anything procedurally generated or colour-changing could be flagged as such as part of the detection algorithm.

One thing though - don't increase resolution - Spectrum games very rarely look better in higher res, as the i-ball example plainly shows, and with a lot of games using pixel-perfect collision detection you're going to muck that up as smoothed curves generally use more pixels.
User avatar
4thRock
Manic Miner
Posts: 415
Joined: Thu Nov 09, 2017 9:35 am
Location: Portugal

Re: ZX Recoloring project

Post by 4thRock »

The problem with recognition on a running game screen is sprite overlap and false positives.

I think a better solution would be to use pattern recognition trick to find/replace the sprites in memory or even better, on a tape file.
That would give you two options:
» Change the original pixel data. Only original resolution and no color. No overlap issues and total compatibility. You'd get a new TAP/SNA.
» Watching the original sprite memory positions and feeding extra graphic data when accessed (like Spec256 I think). Needs a special emulator but has no limitations.
User avatar
ZXDunny
Manic Miner
Posts: 498
Joined: Tue Nov 14, 2017 3:45 pm

Re: ZX Recoloring project

Post by ZXDunny »

4thRock wrote: Thu Jul 11, 2019 3:30 pm The problem with recognition on a running game screen is sprite overlap and false positives.

I think a better solution would be to use pattern recognition trick to find/replace the sprites in memory or even better, on a tape file.
That would give you two options:
» Change the original pixel data. Only original resolution and no color. No overlap issues and total compatibility. You'd get a new TAP/SNA.
» Watching the original sprite memory positions and feeding extra graphic data when accessed (like Spec256 I think). Needs a special emulator but has no limitations.
The second one has legs, IMO. Sprite overlap won't be a problem for a Spec256 implementation, as you're reading/writing 256 colour byte strips directly to the 256 colour screen. As for detection instead, you could take it one step further - hook the z80 code that draws the sprite. It's that simple - you just hijack it and draw your own sprite to the display instead.

"But that would mean we need to be able to detect the game!" You all cry. Well yes, but your current implementation of auto-identifying screen graphics already does that - your replacement graphics won't work for any other game, will they?

Just insert an $ED00 instruction at the sprite draw routine and have your emulator pause emulation while you inspect what's being drawn, render your own sprite and then issue a RET to go back to the emulation.

DId I mention that there were many solutions to this that are superior to image recognition?
User avatar
4thRock
Manic Miner
Posts: 415
Joined: Thu Nov 09, 2017 9:35 am
Location: Portugal

Re: ZX Recoloring project

Post by 4thRock »

Image recognition solves the problem of matching sprites with memory content.
That makes it a "universal" solution. In theory all we need is a sprite sheet, and then the software would map the original memory positions for all the graphics.
This would allow for the Spec256 approach on a user friendly way ;)
User friendly is important (that's why previous efforts have failed, too hard to work with).
User avatar
Joefish
Rick Dangerous
Posts: 2041
Joined: Tue Nov 14, 2017 10:26 am

Re: ZX Recoloring project

Post by Joefish »

ZXDunny wrote: Sat Jul 06, 2019 10:59 pm You need an emulator that reads and writes memory using shadow RAM images.

When a memory read is performed, you read from three extra copies of the 64k that's paged in - and store them in shadow registers. When writing, you either write just the 8 bits you read from main memory (in the case of a general memory write), unless you're writing to the screen in which case you extract 8 bits for PAPER and 8 bits for INK and combine those to a screen that can handle 256 colours. If you have 7 extra banks of RAM, you can easily make graphics that are 256 colours per pixel.

Of course, you can't load games from tape this way, but snapshots are really useful. All you need then is an editor that allows you to identify graphical objects in memory and pre-load those regions with 256 colour representations of their 1s and 0s.

This has already been done, btw, many years ago. But it never gained any traction at all, despite being very simple for the average user to mess around with. You don't need to do image recognition, no need to trap screen display routines.
I quite liked the 256 colour project, but I think there were a few flaws that needed to be overcome. It didn't get updated much. And I've probably forgotten or got most of this wrong...

The added-in backgrounds just looked awful and should have been dropped.

The SNA editor wasn't very sophisticated, and I can't exactly remember but most 256 colour palettes I see are a mess. It may have been better to pick a smaller pallette, or do something like 16 shades of 16 common colours, rather than try and use some 'web-safe' jumble where it's impossible to find the colour you want.

It would also be much more interesting if part of the palette was set aside for e.g. 8 shades of the PAPER colour of a character cell and 8 shades of the INK colour, maybe hue-shifted or tinted by the BRIGHT bit. Then when you used the INK-related colours in a sprite, they would change depending on the colour that sprite was drawn in. So in Manic Miner you could draw a penquin in cyan with an orange beak and a white belly, but the main sprite colour would swap from cyan to yellow automatically for the other penguin.

The sprite editor would also need some sort of intermediate file format where you can designate the sprites and how they're stored in memory. Once that was defined for a game, more people could then get into editing its graphics.
geecab
Drutt
Posts: 32
Joined: Sun Jun 23, 2019 12:44 pm

Re: ZX Recoloring project

Post by geecab »

I have used Emuzwin in spec256 mode a lot quite recently. It is very good, though it doesn't work for every game you throw at it, there are also some general bugs/usability/stability issues with it too (Which I mention in this post). I really like the sound of ZXDummy's solution, and the tool that 4thRock mentions (Software that would map the original memory positions for all the graphics from a tape image). Unfortunately, these things don't exist yet, would be nice though as I'd happily use them :)

Its been interesting listening to opinions regarding the best way to add color to Spectrum games and what does and doesn't look correct (Amount of colors and resolution). It sounds like that would make a good debate thread on its own (I.e. "What colors and resolution should spectrum remakes have?" thread). They way I see it though, there is no right or wrong, Sinclair never released a Spectrum 48K/128K with improved graphics, so with everyone 'imagining what might have been' you are likely to get a difference of opinion. Purists can make a remake that might have been achievable in the 80ties given the hardware available if that's what they desire, but that doesn't mean everyone else must be constrained to do the same. Some people might want to make a completely modern looking remake. I was thinking that I might digitize a few of my friends, and use the tool to replace the graphics on WayOfTheExplodingFist. I thought that'll be quite a fun project, I don't need perfect results, and would be very quick & easy to achieve with the sprite recognition tool.


With all the talk of the correct/theoretical ways of doing things, its easy to loose sight of the benefits this tool (a pure sprite recognition solution) has:-

- Easy to understand and you can get something working quickly

- No Z80 machine code literacy required

- Use any paint editor you like

- Could be developed into a general solution for all platforms (Not just the Speccy)


Sure, it has its limitations, but lots can be over come and we haven't scratched the surface:-

- Sprites hiding behind sprites issues
There is always the danger of so little of one sprite showing, that recognition algorithm won't find focus. However, who is to say the recognition algorithm can't be modified one day to make intelligent guesses as to what a partially hidden sprite is based on its historical position in previous frames. Your eye does this, then recognition algorithm could be trained to, maybe ;) .

- Performance issues
Already mentioned a few ways the tool could be optimized, and with PCs getting faster and faster I don't see this as a long term issue.

:)
User avatar
ZXDunny
Manic Miner
Posts: 498
Joined: Tue Nov 14, 2017 3:45 pm

Re: ZX Recoloring project

Post by ZXDunny »

geecab wrote: Fri Jul 12, 2019 1:39 pm II really like the sound of ZXDummy's solution
<splutter>
- Sprites hiding behind sprites issues
There is always the danger of so little of one sprite showing, that recognition algorithm won't find focus. However, who is to say the recognition algorithm can't be modified one day to make intelligent guesses as to what a partially hidden sprite is based on its historical position in previous frames. Your eye does this, then recognition algorithm could be trained to, maybe ;) .
Yeah, we could have an online connection to Google's deepmind.
Ralf
Rick Dangerous
Posts: 2279
Joined: Mon Nov 13, 2017 11:59 am
Location: Poland

Re: ZX Recoloring project

Post by Ralf »

I believe a sprite recognition is generally a wrong approach. In Spectrum games, graphics often overlap and when they overlap often things like XOR are used which alter pixels even more. And even without any XOR - when a player stands in front of a tree then only a part of a tree is visible so how it's going to get recognized? It will mostly fail and not work instead of a few very special cases where nothing touches nothing.

I would say the proper approach was used in EmuZWin - paint directly over bytes in memory. Yes, it often failed too and would need quite a lot of work to make it work better and become more user friendly but that's the way it should be.
geecab
Drutt
Posts: 32
Joined: Sun Jun 23, 2019 12:44 pm

Re: ZX Recoloring project

Post by geecab »

ZXDunny wrote: Fri Jul 12, 2019 2:14 pm <splutter>
Sorry, honest mistake :)
rg_software
Drutt
Posts: 11
Joined: Sun Jul 07, 2019 4:20 am

Re: ZX Recoloring project

Post by rg_software »

Well, I see the discussion goes into the direction of "whether it is right approach or not" :)

I think the problem boils down to the fact that Spectrum doesn't have any "hardware-supported sprites" or anything like that. While for each particular game you can perhaps point out a better solution (let's say that for games like Elite you don't need sprites at all -- we can scale vector graphics wth perfection), in general case I see no real choice here.

First, on performance. Given that Spectrum screen is just 6912 bytes, I think sprite search should not be an issue. Currently I use a naive approach to searching, but I can implement something like Aho-Corasick algorithm, which should make searching of multiple sprites really fast. I can again complain about lack of time to do it, but I see interest now, so I promise to reconsider my schedule :)

Next, on false positives, etc, etc. Yes, it's all tough. Also, think about cases when sprites overlap and you need to decide which one goes above the other (say, in games like Fairlight). I am aware that we can find a lot of potential issues. But I really see no other "magic" approach that would solve these issues. Every game is different, sprite handling is different and so on. If I could only be sure that somewhere in the memory there is a map of sprites that can be substituted... Alas, I don't think that's the case.

But, anyway, instead of expounding my theories I'd perhaps better focus on the code... Maybe within the next 2 weeks or so I'll try to show something new. Thanks for your interest, really, I am happy to be here.
AndyC
Dynamite Dan
Posts: 1387
Joined: Mon Nov 13, 2017 5:12 am

Re: ZX Recoloring project

Post by AndyC »

In terms of "what resolution should games use" I think it's important to distinguish between real remakes and just coloured in spectrum titles. A genuine remake of the game has a low of flexibility to change how things look and adjust collision detection, movement etc to compensate. Recolouring a spectrum title is a lot more limited - it may break the game mechanics if a sprite is changed such that it no longer maps directly over the original. As Dunny mentioned, even something like smoothing curved edges might start causing collision detection to feel "off" when the underlying game engine is working to a pixel perfect hit with the original sprites. Taking it even further if you changed the graphics enough it might even become necessary to change the underlying imagery so the engine has a better concept of what is going on.
User avatar
Joefish
Rick Dangerous
Posts: 2041
Joined: Tue Nov 14, 2017 10:26 am

Re: ZX Recoloring project

Post by Joefish »

Why not use the pattern-matching algorithm to search for the sprite data in a memory snapshot and present it for editing, then use a Spectrum-256 like approach to present it in-game?
hikoki
Manic Miner
Posts: 576
Joined: Thu Nov 16, 2017 10:54 am

Re: ZX Recoloring project

Post by hikoki »

[mention]geecab[/mention]
Yes, I definitely think all that should be possible. You have put the idea of recoloring Scuba Dive in my head now :) Excellent game, and there shouldn't be many sprites to recolor either...
I would enjoy playing your mod for sure. That could be another feature idea, how to give the impression there are a greater variety of sprites just by playing with the search and replace mechanism. There could be randomisation when a match is detected like showing different treasures every time on Scuba Dive.

I don't see much of a problem in getting artifacts as long as they don't happen all the time and appear in a good looking way when possible just as a reminder of the original sprites and engine.
User avatar
4thRock
Manic Miner
Posts: 415
Joined: Thu Nov 09, 2017 9:35 am
Location: Portugal

Re: ZX Recoloring project

Post by 4thRock »

Joefish wrote: Fri Jul 12, 2019 6:38 pm Why not use the pattern-matching algorithm to search for the sprite data in a memory snapshot and present it for editing, then use a Spectrum-256 like approach to present it in-game?
Yep, second that.

Regarding what type of enhanced graphics are more acceptable, I'd go for what the Spectrum might display if it had hardware sprites.
So two colours (*) per 8x8 pixel block, aligned with the sprite origin. Perhaps 8x1 if we consider a Timex Hicolour mode. No sprite colour clash when placed on screen.

*- Well, if we consider paper as transparent, then it's a single colour per block ;) but I can overlook that. No need to be 100% purist.
Post Reply