SAM Coupé screenshot format

This is the place for general discussion and updates about the ZXDB Database. This forum is not specific to Spectrum Computing.

Moderator: druellan

User avatar
moroz1999
Manic Miner
Posts: 329
Joined: Fri Mar 30, 2018 9:22 pm

SAM Coupé screenshot format

Post by moroz1999 »

SAM Coupé basically supports 4 video modes. I've only supported Mode 1 and Mode 4 in zx-image library, but I'm willing to support all of them.

Mode 1: ZX-Compatible 6912 SCR.
Mode 2: 256*192 Hardware multicolor 8*1 of some kind, similar to Timex. This requires a bit of original docs investigation.
Mode 3: 512*192 hires mode with 4 colors. Extension ss3, size 24617 (?)
Mode 4: 256*192 hires mode with 16 colors. Extension ss4, size 24617. This one consists of pixel data (24576 bytes, each byte contains 2 4bit color palette index) 16 bytes of something and 25 bytes of a palette data. I don't know why this size was chosen, I have to look into the official docs again.
User avatar
moroz1999
Manic Miner
Posts: 329
Joined: Fri Mar 30, 2018 9:22 pm

Re: SAM Coupé screenshot format

Post by moroz1999 »

Some quotations:
https://sam.speccy.cz/systech/sam-coupe ... n_v3-0.pdf
CLUT IN MODE 4 & MODE 3

MODE 4
In screen mode 4, 4 bits are used to address the CLUT for the colour of a pixel. Within a byte, the most significant nibble refers to the first pixel and the least significant nibble refers to the second pixel.

MODE 3
The situation is similar in screen mode 3, however this time there are only 2 bits per pixel which address the CLUT. Out of an 8-bit data byte, the first two most significant bits are used as the address for the first pixel. Normally, only 4 of the sixteen possible registers would be available. This is overcome by using an extra two bits from the high memory page register (HMPR-251 dec). Bit 5 of HMPR is used to access BCD 4 of the colour look up address and bit 6 of HMPR is used to access BCD 8 of the colour look-up address. In this way, we can still access the 16 colours specified in the colour look-up table whilst having high resolution graphics.
I've used this manual to make a parser for SS4 format, and it worked well with whatever BMP2SCR generated, so I'll try to make the same for SS3.
User avatar
Stefan
Manic Miner
Posts: 793
Joined: Mon Nov 13, 2017 9:51 pm
Location: Belgium
Contact:

Re: SAM Coupé screenshot format

Post by Stefan »

Why use a special custom format and not just png?

With spectrum screens / mode 1 and mode 2 you can argue that you want a format that preserves hidden data beneath the attributes and flash, for mode 3 and 4 I do not see the point.

If it is going to be custom then I would certainly include border colour and line interrupt palette changes.
User avatar
moroz1999
Manic Miner
Posts: 329
Joined: Fri Mar 30, 2018 9:22 pm

Re: SAM Coupé screenshot format

Post by moroz1999 »

1. PNG cannot be regenerated if we would like to correct some errors. Like, for example, if we correct the palette one day (happened twice on ZX-Art)
2. PNG cannot guarantee 100% accurate info preserving, and we are actually doing the digital art preservation in most cases.
3. PNG cannot be viewed on a real machine without backwards conversion. Suddenly, this is actual: ZX-Art is being used from at least two advanced ZX Spectrum clones with ethernet support, and I hope that it would also be available on Next.

By the way, how is SAM's palette calculated? I just realized that it's possible that we have to apply the SRGB correction to the palette values.
User avatar
moroz1999
Manic Miner
Posts: 329
Joined: Fri Mar 30, 2018 9:22 pm

Re: SAM Coupé screenshot format

Post by moroz1999 »

The biggest problem is a lack of screenshots saving in original format in SimCoupe. Saving dumps manually is complicated.
User avatar
moroz1999
Manic Miner
Posts: 329
Joined: Fri Mar 30, 2018 9:22 pm

Re: SAM Coupé screenshot format

Post by moroz1999 »

Stefan wrote: Sun Oct 06, 2019 6:13 am If it is going to be custom then I would certainly include border colour and line interrupt palette changes.
Basically I agree with this. If anybody suggests the new format for storing border colour and interrupt palette changes, then I will support it in zx-image library as well.
User avatar
Stefan
Manic Miner
Posts: 793
Joined: Mon Nov 13, 2017 9:51 pm
Location: Belgium
Contact:

Re: SAM Coupé screenshot format

Post by Stefan »

moroz1999 wrote: Sun Oct 06, 2019 1:15 pm 1. PNG cannot be regenerated if we would like to correct some errors. Like, for example, if we correct the palette one day (happened twice on ZX-Art)
Good point.
moroz1999 wrote: Sun Oct 06, 2019 1:15 pm 2. PNG cannot guarantee 100% accurate info preserving, and we are actually doing the digital art preservation in most cases.
ok.
moroz1999 wrote: Sun Oct 06, 2019 1:15 pm 3. PNG cannot be viewed on a real machine without backwards conversion. Suddenly, this is actual: ZX-Art is being used from at least two advanced ZX Spectrum clones with ethernet support, and I hope that it would also be available on Next.
Another good point - although I do expect 'advanced' clones to be able to view png files.
moroz1999 wrote: Sun Oct 06, 2019 1:15 pm By the way, how is SAM's palette calculated? I just realized that it's possible that we have to apply the SRGB correction to the palette values.
I do not have a technical reference, but I would assume that the simcoupe calculation is as good as any, see const COLOUR* GetPalette ()

Code: Select all

const COLOUR* GetPalette ()
{
    static COLOUR asPalette[N_PALETTE_COLOURS];

    // Look-up table for an even intensity spread, used to map SAM colours to RGB
    static const BYTE abIntensities[] = { 0x00, 0x24, 0x49, 0x6d, 0x92, 0xb6, 0xdb, 0xff };

    // Build the full palette: SAM's 128 colours and the extra colours for the GUI
    for (int i = 0; i < N_PALETTE_COLOURS ; i++)
    {
        // Convert from SAM palette position to 8-bit RGB
        BYTE bRed   = abIntensities[(i&0x02)     | ((i&0x20) >> 3) | ((i&0x08) >> 3)];
        BYTE bGreen = abIntensities[(i&0x04) >> 1| ((i&0x40) >> 4) | ((i&0x08) >> 3)];
        BYTE bBlue  = abIntensities[(i&0x01) << 1| ((i&0x10) >> 2) | ((i&0x08) >> 3)];

        // If greyscale is enabled, convert the colour a suitable intensity grey
        if (GetOption(greyscale))
        {
            BYTE bGrey = static_cast<BYTE>(0.299 * bRed + 0.587 * bGreen + 0.114 * bBlue + 0.5);
            bRed = bGreen = bBlue = bGrey;
        }

        // Store the calculated values for the entry
        asPalette[i].bRed = bRed;
        asPalette[i].bGreen = bGreen;
        asPalette[i].bBlue = bBlue;
    }

    // Return the freshly prepared palette
    return asPalette;
}
User avatar
moroz1999
Manic Miner
Posts: 329
Joined: Fri Mar 30, 2018 9:22 pm

Re: SAM Coupé screenshot format

Post by moroz1999 »

https://github.com/moroz1999/zx-image - Sam Coupe mode 3 is supported now.
I've found an issue with BMP2SCR, it seems like it saves SS3 with a wrong order of colors in CLUT. Color 1 and color 2 from range [0..3] are swapped.
Image
User avatar
8BitAG
Dynamite Dan
Posts: 1487
Joined: Sun Dec 17, 2017 9:25 pm
Contact:

Re: SAM Coupé screenshot format

Post by 8BitAG »

If I remember correctly, hybrid screen modes were possible... with splits between say mode 4 for graphics and mode 3 for text... And you could overcome the 16 colour limit of mode 4 by using line interrupts to get all 128 colours on screen at once.

(All of that was beyond me at the time... Although I think some of the text adventures might've used a hybrid mode 4/mode 3 screen...)
8-bit Text Adventure Gamer - games - research.
User avatar
Stefan
Manic Miner
Posts: 793
Joined: Mon Nov 13, 2017 9:51 pm
Location: Belgium
Contact:

Re: SAM Coupé screenshot format

Post by Stefan »

8BitAG wrote: Sun Oct 06, 2019 3:13 pm If I remember correctly, hybrid screen modes were possible... with splits between say mode 4 for graphics and mode 3 for text... And you could overcome the 16 colour limit of mode 4 by using line interrupts to get all 128 colours on screen at once.

(All of that was beyond me at the time... Although I think some of the text adventures might've used a hybrid mode 4/mode 3 screen...)
Which is in a way similar to the spectrum multi-colour stuff - which is basically timex mode x / sam mode 2 - I think a special mlt format or so was created for them.

But yes, any format would need to be able hold the maximum available screen, so taking into account the ability to easily change screen modes and palettes on a line interrupt, the maximum screen (without border area), would be 512 wide x 192 high x 128 colours - and that is not even dealing with screen swappers that are flipping palettes every frame to give a visual (flickery) effect of even more colours. This effect was recently used in the high colour demo

[media]https://www.youtube.com/watch?v=4Ma3X3Y ... tu.be&t=95[/media]
User avatar
moroz1999
Manic Miner
Posts: 329
Joined: Fri Mar 30, 2018 9:22 pm

Re: SAM Coupé screenshot format

Post by moroz1999 »

Stefan wrote: Sun Oct 06, 2019 2:57 pm I do not have a technical reference, but I would assume that the simcoupe calculation is as good as any, see const COLOUR* GetPalette ()
The problem is that in sRGB "intensities" are not linear, so abIntensities array should be gamma-corrected. I will make a bit of calculations and propose corrected values.
RGB in Sam Coupe was linear, but modern devices expect gamma-corrected values, so direct linear non-corrected values are displayed a bit wrong as a result.
User avatar
4thRock
Manic Miner
Posts: 415
Joined: Thu Nov 09, 2017 9:35 am
Location: Portugal

Re: SAM Coupé screenshot format

Post by 4thRock »

Moroz, it would be interesting if ZX-Image also outputted a standard "PAL" image at 768x576.
This would be better for (de)interlaced and high-res modes, and also would allow for a uniform border display.

Also, you already have setSize(2) but that give us 320x240, not a TV resolution. I think 384x288 would be better.

Just some suggestions! ;)
User avatar
moroz1999
Manic Miner
Posts: 329
Joined: Fri Mar 30, 2018 9:22 pm

Re: SAM Coupé screenshot format

Post by moroz1999 »

That's an interesting idea really. But what should I do with resolutions like 512*384?
User avatar
4thRock
Manic Miner
Posts: 415
Joined: Thu Nov 09, 2017 9:35 am
Location: Portugal

Re: SAM Coupé screenshot format

Post by 4thRock »

I guess for 512*384 you won't do any scaling, since it already has twice the resolution than 256*192.

You will need to handle interlace for modes with more than 288 vertical scanlines.
My suggestion are 4 options:
- frame blending @25hz (gigascreen)
- interlace @50hz (two flickering fields)
- interlace with alternate lines @50hz (alternate even/odd lines to simulate CRT scanning)
- deinterlace @25hz (one field to odd lines, the next one to even)

Think of this just like the Amiga modes (check how WinUAE works ;) ), only with a smaller active area (and the border instead of overscan).
obo
Drutt
Posts: 31
Joined: Sat Sep 01, 2018 11:08 am
Location: Nottingham, UK
Contact:

Re: SAM Coupé screenshot format

Post by obo »

moroz1999 wrote: Sun Oct 06, 2019 4:31 pm The problem is that in sRGB "intensities" are not linear, so abIntensities array should be gamma-corrected. I will make a bit of calculations and propose corrected values.
Did you decide on appropriate values for this for your viewer?

I've been experimenting with this area in SimCoupe, along with other display-related changes. If I apply a 2.8 gamma to the original colours, and use an sRGB surface for the final display render (for automatic inverse 2.2 gamma), I get something that looks a bit better to my untrained eyes. The dark colours on the SAM boot screen are a little darker, and the colour 8 background used during Sphera gameplay looks nearer black as it should be. With bilinear filtering enabled there is no longer a dark seam between the magenta and green on the boot screen, due to the corrected linear blending. Even the filtered white text looks bloomier, presumably because the softened grey edges remain brighter.
The biggest problem is a lack of screenshots saving in original format in SimCoupe.
SimCoupe could probably scrape the SCREEN$-style details from the system variables to save. I didn't realise there were any "standard" SAM screenshot formats in use, beyond SCREEN$ with trailing metadata. I think people previously saved to disk and use Edwin Blink's Diskimage Manager to export as BMP, or something similar. Is the requirement of the image format to handle static images and gigascreen animations? Or do you want to include raster effects too? How about border raster effects?

I still favour PNG for static images and GIF for animated, both of which retain the separate palettes that can be updated if without re-generating the image. The compression of the latter formats is probably the biggest barrier to easily displaying them on original hardware. Other than that, with a little pre-processing the shamview viewer could probably be adapted to display most images with raster effects too, as well as 2 and 3 frame gigascreen images.
User avatar
moroz1999
Manic Miner
Posts: 329
Joined: Fri Mar 30, 2018 9:22 pm

Re: SAM Coupé screenshot format

Post by moroz1999 »

obo wrote: Wed Jan 29, 2020 8:50 pm Did you decide on appropriate values for this for your viewer?
It was easy to convert linear values to SRGB gamma-corrected ones. I didn't put this to ZX-Art yet, because there is a problem. Some of the best works for Sam Coupe has been made using a wrong palette, so they look a bit awful after gamma correction. However, the converted images look a lot better with gamma correction, they are not dark anymore. I don't know what to do with this. May be I will make some sort of setting so every author would be able to decide which pallete to use.
obo wrote: Wed Jan 29, 2020 8:50 pm I've been experimenting with this area in SimCoupe, along with other display-related changes. If I apply a 2.8 gamma to the original colours, and use an sRGB surface for the final display render (for automatic inverse 2.2 gamma), I get something that looks a bit better to my untrained eyes. The dark colours on the SAM boot screen are a little darker, and the colour 8 background used during Sphera gameplay looks nearer black as it should be. With bilinear filtering enabled there is no longer a dark seam between the magenta and green on the boot screen, due to the corrected linear blending. Even the filtered white text looks bloomier, presumably because the softened grey edges remain brighter.
There is actually no need to apply 2.8 gamma. SRGB correction is easy:
https://docs. google.com/spreadsheets/d/1TqRxixaD0JSzrV4FdLxBAFomDkxgT1dOy0HgHvScK94/edit
Remove the space in a link.

Bilinear filtering often ignores SRGB because of complex calculations, so it usually makes really inaccurate result for pixel art. I suggest turning it off at least for calibration if you want to see the real colors.

obo wrote: Wed Jan 29, 2020 8:50 pm SimCoupe could probably scrape the SCREEN$-style details from the system variables to save. I didn't realise there were any "standard" SAM screenshot formats in use, beyond SCREEN$ with trailing metadata. I think people previously saved to disk and use Edwin Blink's Diskimage Manager to export as BMP, or something similar. Is the requirement of the image format to handle static images and gigascreen animations? Or do you want to include raster effects too? How about border raster effects?
I didn't want to re-invent the wheel, so I just used the formats I've found information about in Internet. One of the sources for me was BMP2SCR utility which already has a support for some formats, so I had a fine source of materials to test anything.
I'm not currently planning supporting any extra formats for Sam Coupe (having raster effects or gigascreen). I will surely support it if any original art would be available in such new formats.
obo wrote: Wed Jan 29, 2020 8:50 pm I still favour PNG for static images and GIF for animated, both of which retain the separate palettes that can be updated if without re-generating the image. The compression of the latter formats is probably the biggest barrier to easily displaying them on original hardware.
My usual arguments are:
1. Art preservation means we have to preserve the original image, not a conversion to any format.
2. Making palettes correction from original memory dump format is a lot easier than from GIF/PNG. A lot easier, it can be done automatically.
3. GIF/PNG are old and ineffective. Modern web uses webp already. I can switch the whole ZX-Art to webp (for the user agents which support it) as soon as I get to generating animated webp instead of GIF files. And if tomorrow a new effective PC format appears, I can use it instead of gif/webp/bmp/whatever because all the original memory dumps (SCR* files) are preserved. Making same thing for gif or png would be a lot more complicated.
4. Working with original memory dumps is a lot easier for ZX. ZX-Art is already used for at least two Internet solutions for real ZX clones. Downloading 6912 and putting it to a right place in memory is a lot easier for z80 than any decoding/decompression. Same goes for music, for software.
obo
Drutt
Posts: 31
Joined: Sat Sep 01, 2018 11:08 am
Location: Nottingham, UK
Contact:

Re: SAM Coupé screenshot format

Post by obo »

moroz1999 wrote: Sun Feb 02, 2020 11:50 am Some of the best works for Sam Coupe has been made using a wrong palette, so they look a bit awful after gamma correction. However, the converted images look a lot better with gamma correction, they are not dark anymore.
With the sRGB conversion I'm finding eveything looks washed out. There's very little contrast between the bright and regular stripes on the boot screen. Colour 8 (set with PALETTE 0,8) appears mid-grey rather than dark grey, and the darkest blue colour (1) also is not dark anymore. It could be that I'm used to the darker colours now, but perhaps it's also that on the original machine with a CRT I would have tweaked brightness and contrast to give a more balanced image. I don't currently have a working SAM and CRT monitor to compare against.

I don't know what to do with this. May be I will make some sort of setting so every author would be able to decide which pallete to use.
I was wondering something similar for my side. I might even be tempted to keep the current RGB default, but allow the option of changing to sRGB.

Bilinear filtering often ignores SRGB because of complex calculations, so it usually makes really inaccurate result for pixel art. I suggest turning it off at least for calibration if you want to see the real colors.
I'm currently using an sRGB palette surface and render target, with the rest of the pipeline remaining linear. That means the blending is correct and doesn't give the dark fringes between green and magenta.

I didn't want to re-invent the wheel, so I just used the formats I've found information about in Internet. One of the sources for me was BMP2SCR utility which already has a support for some formats, so I had a fine source of materials to test anything.
The SS3 and SS4 formats used by BMP2SCR seem tied to the format used by SAM BASIC, which will include basic colour flashing animations. There aren't dedicated formats for modes 1 or 2 that don't have some ZX limitation on palette, so they're not really suited to preservation.

I could support dumping the current display to a file, which will give the correct output for simple static images only. Perhaps a small signature, screen mode, then CLUT, then the raw display memory bytes. Mode 2 will not include the 2K gap between data and attributes. Any use to you?
User avatar
moroz1999
Manic Miner
Posts: 329
Joined: Fri Mar 30, 2018 9:22 pm

Re: SAM Coupé screenshot format

Post by moroz1999 »

As soon as I find some time, I will post some examples. Is there a Sam palette sample in original ss4 form? It would be good to use it for rendering just to see the final values.
obo
Drutt
Posts: 31
Joined: Sat Sep 01, 2018 11:08 am
Location: Nottingham, UK
Contact:

Re: SAM Coupé screenshot format

Post by obo »

Sorry, I hadn't seeen your reply to this until I logged in just now! I don't know why my email notifications aren't working.

It seems like raw display memory dumps with an appended palette are the way to go. The SS3 amd SS4 files will both be 24576 bytes, relying on file extension to determine how an application should process them. SS1 will be 6912+16=6928 bytes, which is the same as the Spectrum but a palette appended. SS2 will be 6144+6144+16=12304 bytes. Does that work for you?

I've also given up on sRGB support in SimCoupe for now as the results I was getting didn't look anything like the RGB output to my Philips CM8833-II CRT monitor. If you have a screenshot viewer that views SS4 screens and can demonstrate better sRGB results I'll revisit it.
obo
Drutt
Posts: 31
Joined: Sat Sep 01, 2018 11:08 am
Location: Nottingham, UK
Contact:

Re: SAM Coupé screenshot format

Post by obo »

[mention]moroz1999[/mention]: I've added experimental SSX screen saving to a new build of SimCoupe. Hopefully this will give the raw output you wanted for image preservation. Files are currently saved to a SimCoupe directory in the user's Documents directory, using an auto-generated name.

The Save SSX option on the Record menu will save a .ssx fle in one of 5 formats:

- MODE 1 = 6144 data + 768 attrs + 16 CLUT = 6928 bytes.
- MODE 2 = 6144 data + 6144 attrs + 16 CLUT = 12304 bytes.
- MODE 3 = 24576 data + 4 CLUT = 24580 bytes.
- MODE 4 = 24576 data + 16 CLUT = 24592 bytes.
- RAW = 512x192 palette indices = 98304 bytes.

The first 4 are the the display memory exports we've been discussing, which are followed by the CLUT indices into the 128-colour SAM palette. These are used for true static screen images, where the contents will exactly reproduce what you see on the screen. The only change from what I previously mentioned is that MODE 3 saves only the 4 CLUT entries it needs. These formats are only used if SimCoupe doesn't detect any mid-frame raster effects.

The last format is a raw display representation that is able to preserve raster-level effects. As [mention]PeterJ[/mention] mentioned in another thread, SAM titles often change video settings mid-frame. Changing CLUT colours and/or video mode/page mid-frame gives a viewed image that can no longer be represented by a simple memory dump. That's true even for the striped boot screen, which uses colour switching for the bars!

If SimCoupe detects mid-frame video changes it'll use the final raw format shown above instead of a MODE format. The raw format isn't as simple to display on the the original machine, but I already have a viewer that can show most files. The only cases I know it can't yet handle are those mixing hi-res MODE 3 with other modes, but it should be possible to extend to support that for most real-world images.

The older SS3/SS4 files seemed to be variable in size, depending on what SAM BASIC needed to store for LINE colour changes. I don't know if SS1/SS2 existed at all, but it all seemed to be a bit of a mess. The new SSX files have predictable sizes for each format, with no SAM BASIC legacy data. It's still an experimental format so if you find it doesn't meet your needs, please let me know.
User avatar
PeterJ
Site Admin
Posts: 6855
Joined: Thu Nov 09, 2017 7:19 pm
Location: Surrey, UK

Re: SAM Coupé screenshot format

Post by PeterJ »

Excellent [mention]obo[/mention],

I'm glad I can now switch to your emulator. Thank you for doing this.

I use an outlook.com email address for new message alerts. I know they often go in people's junk folders.
obo
Drutt
Posts: 31
Joined: Sat Sep 01, 2018 11:08 am
Location: Nottingham, UK
Contact:

Re: SAM Coupé screenshot format

Post by obo »

Stefan spotted that I'd attached the wrong binaries to the release, using those from 24th March instead of 24th May. Please re-download if you've grabbed the older files, as only the new version supports SSX.

It looks like my forum notifications are working now too :)
User avatar
moroz1999
Manic Miner
Posts: 329
Joined: Fri Mar 30, 2018 9:22 pm

Re: SAM Coupé screenshot format

Post by moroz1999 »

Thank you! That's a really great news, I will add a support of the format as soon as possible.
User avatar
moroz1999
Manic Miner
Posts: 329
Joined: Fri Mar 30, 2018 9:22 pm

Re: SAM Coupé screenshot format

Post by moroz1999 »

Please check github for an updated version of ZX-Image.
I've added support of mode3 and mode4 SSX files, others are to be implemented soon as well.

When I try to run the python script, I get the following problem:
f:\projects\ssxview-master>python ssxview.py mode2.ssx
File "ssxview.py", line 71
sys.exit(f"Invalid SSX file size ({size} bytes)")
^
SyntaxError: invalid syntax

What am I doing wrong? I've copied mode2.ssx from samples to same dir as python script located.
obo
Drutt
Posts: 31
Joined: Sat Sep 01, 2018 11:08 am
Location: Nottingham, UK
Contact:

Re: SAM Coupé screenshot format

Post by obo »

Thanks for the updated zx-image, which I'll get set up soon.

It looks like it's choking on the f-string formatter in ssxview.py, which was added in Python 3.6. I'd best add that as a requirement :)
Post Reply