Next screenshot format

[Seven.FFF]

NXI format is easily dumpable, but only represents one of the many layers of the Next display. A NXI dump of almost any running game will not give you anything close to a screenshot.

[moroz1999]

NXI format is easily dumpable, but only represents one of the many layers of the Next display. A NXI dump of almost any running game will not give you anything close to a screenshot.

First of all I wanted to ask you about how much games really combine the different modes, but then I realized that really having only a dump of Layer2 memory is not really solving much. There is low-res, and so on.
Actually, nxi support is ok for most of static artwork, but for game screens we either need to make some complex format with all layers and ULANEXT support or stick to some simple format like BMP.

1. As for a complex format: it's not a problem for me to make a parser, but I think we shouldnt run in front of train with this one. We need the support in emulators for it AND we need the support in at least some of Next's software. Otherwise it will not have any usage at all: using complex format is unreal
without emulator's support for screenshots and is pointless without support in viewers on a real hardware.

2. As for BMP: it's simple enough to be parsed on a real hardware, but how would we ensure that its palette is a real one? BMP can technically have more colors than RGB333 supports, and this is bad for preservation.
I have already met this problem: there was a competition for ZX Evolution in BMP format. Turns out that BMP images made for this compo were not using the exact palette of ZX Evolution, so some images looked really differently on a real hardware.
This is one thing I would like to avoid totally.

3. One more question for BMP: how would we be sure that it uses a right gamma correction? If we are outputting #808080 color on Next, would it mean 50% of gray (linear RGB) or a darker color (sRGB on every modern PC)?
This is one question to decide before we make a lot of images in BMP modes and it turn out that they would all be different without common logic and rules for palette.

[Seven.FFF]

I agree with all three points.

[mention]TheSMoG[/mention] has some reference gamma data, and is also the author of some Next gfx tools. I will ask him to look at this thread.

[PeterJ]

For Spectrum Computing we have decided that Next screenshots will use PNG as a temporary solution. These will be exported from CSpect. This will be replaced with a better format as soon as it is available.

[TheSMoG]

A few things (I'll return to this thread later as I've screwed up System/Next v.1.3.2 and I'm rebuilding it now so I don't currently have a lot of time).
As [mention]Seven.FFF[/mention] correctly said the Next's display is a amalgam of many layers. There is one layer however that's undumpable and that's the Sprite System. As a result, an honest to goodness COMPLETE screenshot from a Next is impossible to make as the resulting image is a combination of actual RAM (which is dumpable) and BRAM which is inaccessible from the outside. Moreover you cannot just dump two palettes and expect that you're good to go as there are up to 8 palettes in use constantly depending on the software.
I'll have more info plus a few suggestions a bit later! In the meantime have a look in the description of Layers (Chapter 16 of the Next User Manual) to understand the challenge presented

[TheSMoG]

For Spectrum Computing we have decided that Next screenshots will use PNG as a temporary solution. These will be exported from CSpect. This will be replaced with a better format as soon as it is available.

This is the most logical solution at this point. As mentioned previously ANY screenshot from the Next even if you found a way to composite one (that would include to record the layer priorities, L2 palettes (with priority colours), Layer 1 + palettes, Layer 0 (ULA) plus palettes and finally tilemap) it would lack the Sprite layer and its palettes as its display memory of sorts is located in BRAM. Theoretically a way could be devised using DMA (which also sits inside the FPGA thus it can be made to have access to BRAM which would then composite an image in 3 to 6 banks which could then be in turned stored onto a file and then processed to be displayable elsewhere. But we're way off this possibility (and it may actually never happen).

[moroz1999]

it would lack the Sprite layer and its palettes as its display memory of sorts is located in BRAM.

Since 100% of screenshots would be done in emulator, I don't think that this would become a problem.
But I agree that PNG is much more practical solution for screenshots. For art preservation there is already nxi (there is no art in mixed modes yet as I can see) and for all else png can be easily used.

BTW, the questions regarding sRGB and gamma correction are still actual for emulator writers. And screenshot would also benefit if this is done right.

[PeterJ]

I'm going to start taking screenshots this weekend, so hopefully you will see them in the next update.

[Stefan123]

I also think that 24-bit PNG is the best choice for Spectrum Next screenshots.

The Next outputs 9-bit RGB333 colours so there are 512 possible colours in total. The different screen layers (graphics modes) of the Next can at most (depending on the type of layer) display 256 colours simultaneously from a 256 colour palette populated with 9-bit RGB333 colours. However, the different layers can be displayed simultaneously so we could for example have layer 2 displayed on the top half of the screen and lo-res displayed on the bottom half. If the layer 2 palette would contain the first 256 of the 512 possible colours and the lo-res palette the next 256 colours, we could display all 512 colours simultaneously. The resulting image would in effect be an image with a colour depth of 9 bpp. There are also sprites which have their own 256 colour palette and the Copper which can be used for changing the palettes mid-screen.

Ideally we would like to have an image format for 9 bpp images in order to cover all cases when producing Next screenshots.

Unfortunately, none of the common image file formats support 9 bpp. We could create a new file format for 9 bpp images. However, such images would be very hard to display on the Next if they use more than 256 colours since we have no universal 9 bpp layer to cover all the cases and in general we have no idea of which layers, sprites, palettes and Copper tricks were used when creating the screenshot. Displaying a new 9 bpp image format on a PC or web page would also require a new conversion tool for converting to PNG or similar.

So we have to pick an existing and commonly used format with a higher colour depth than 9 bpp for storing the Next screenshots.

The common image formats of today support at most 1, 2, 4, 8, 16 and 24 bpp (and higher). The next step after the unsupported 9 bpp case is 16 bpp. However, not all paint programs and image viewers support 16 bpp since that is not that so common today. Then we have 24 bpp as the next logical step and it is a widely supported color depth.

Of the loss-less image formats of today, PNG is the most common and web friendly, so then we end up with storing the Next 9 bpp screenshots as 24 bpp PNG images.

We could create a simple tool that analyses a 24-bit PNG image and verifies that no more than 512 colours are used and that the used 24-bit RGBB888 colours are exact representations of 9-bit RGB333 colours.

I like the NXI format very much, it's simple and efficient for layer 2 graphics. But it's not suitable as a universal Next screenshot format since it is limited to 8 bpp and sooner or later there will be Next games and demos that display more than 256 colours simultaneously.

[moroz1999]

What about 16 bit BMP with software-controlled restricted 9bit and restricted image size palette for source images storing?
This can be converted to PNG/GIF/WEBP/whatever on the fly exactly in a same way as we already do for 6912 SCR.
I will support such images easily ins ZX-Image (similar thing is already done for ZX Evolution).

Two answers for me which are missing now:
1. Is such BMP already viewable on Next? Should be easy to implement.

2. Still what about sRGB correction? This question is also actual for NXI images as well. For example %100100100 is transformed to rgb(146, 146, 146) as
far as I see from: http://www.zxspectrumnext.online/#gfx
But what actually happens on Next's screen?
I can see that RGB333 is converted to linear RGB. %100 is 4, so corresponding linear RGB channel value is 4 * (255/7) = 146. This means 4/7 ~= 57% of luminosity. I am now sure that to achieve the same luminosity division into 7 equal sections in sRGB we need to apply gamma correction. Everything is simple in linear RGB without gamma-correction, but it's not that simple for sRGB, where we need additional calculations to be made.
After applying gamma-correction to rgb(146, 146, 146) it would become srgb(199, 199, 199). So this will mean 4/7 = 57% luminosity in sRGB (so on most of modern displays).

The most important question now: is Next making this gamma-correction or it attempts to display linear RGB on modern gamma-corrected displays (vga+hdmi)?
If answer is yes, then we need to apply gamma correction as well. Otherwise we would have darker images than on real hardware.
If answer is no, then we also shouldnt apply gamma correction and should imitate strange behaviour of displaying linear rgb values on non-linear gamma-corrected srgb devices.

This is actually why I'm against simple use of PC formats for retro-images preserving. Its not a problem for screenshot: if screenshot is darker than on real hardware, then it's not a big deal. But it's crucial for storing visual art.

Update: https://www.specnext.com/forum/viewtopic.php?f=6&t=1705 - posted a question to Next's forum.

[moroz1999]

TLDR for my previous post:
1. It is practical to use PNG files for screenshots. Wrong gamma won't be so big problem, most users wont notice it.
2. We can use BMP files for storing original art. 16 bit palette, limited size, only corresponding linear RGB values are allowed in palette, others would be ignored. This way we would be 100% sure we are preserving something. This can be displayed as PNG or even better WebP in browser, we dont have to transfer original BMP files. I will add such format to Zx-Image library.
3. If we find out that sRGB gamma correction is required, we wont have to modify BMP original files, we will have to adjust Zx-Image library only.

[AndyC]

Two answers for me which are missing now:
1. Is such BMP already viewable on Next? Should be easy to implement.

I'm not sure it would be "easy" to implement. You need 9bpp image support to store an arbitrary image the Next can generate, but the Next can't necessarily display an arbitrary 9bpp image. Certain things can only be accomplished by careful timing effects, use of sprites/layers etc. I don't think you're likely to be able to just auto generate code that would display any given image and, in fact, some might be impossible to display (I'm no expert on the Next graphics hardware).

At some point you kind of have to accept that an image format that can be displayed on the Next itself is essentially going to require full emulation to display elsewhere (and won't be suitable for screenshots) or is going to be something you can't easily display on the original hardware. The same thing is already true of most other platforms today, the Speccys fairly limited display hardware has really been the only reason it is practical to define screen formats that work for both scenarios.

[moroz1999]

At some point you kind of have to accept that an image format that can be displayed on the Next itself is essentially going to require full emulation to display elsewhere (and won't be suitable for screenshots) or is going to be something you can't easily display on the original hardware. The same thing is already true of most other platforms today, the Speccys fairly limited display hardware has really been the only reason it is practical to define screen formats that work for both scenarios.

Hmm, seems like I still have some problems with understanding the hardware limitations.
There is NXI format, it stores 256*192 image with 9bit palette (256 two-byte colors in RGB333 format). As far as I understood, it can be displayed on Next without problems.
If we take 16bit BMP as a more supported container and strictly limit the amount of it's colors to 256 with Next's supported palette values only, wouldn't it be actually the same as NXI?

I know that this format doesn't solve all the cases with sprites and mixing layers, but should it be solving them? These cases are mostly screenshots of software, thus they can be resolved by true color PNG, and I doubt someone would want to see them on original hardware with 100% accuracy.
Please correct me if I'm wrong.

[4thRock]

Gamma is an interesting subject, but we should assume sRGB.
At least that's what all devices are expected to display since 1999. Analog PAL also follows that standard.
Basically, if gamma and colorspace are undefined, they should be treated as sRGB.

I'd say that any monitor made in the last 10 years should be reasonably calibrated out of the box.
TVs have good calibration if they have "movie mode" - that one is factory set to sRGB.

[moroz1999]

Basically, if gamma and colorspace are undefined, they should be treated as sRGB.

As for emulation, we cannot just translate all the values into gamma-corrected ones. It all depends on what's real hardware doing. If real hardware outputs linear color values, then we should do the same way to get the same image.

I'd say that any monitor made in the last 10 years should be reasonably calibrated out of the box.
TVs have good calibration if they have "movie mode" - that one is factory set to sRGB.

TVs now can use wider gamut with their own proprietary color management, so it's not that easy nowadays. But sRGB is still the lowest common denominator.

[4thRock]

There is NXI format, it stores 256*192 image with 9bit palette (256 two-byte colors in RGB333 format). As far as I understood, it can be displayed on Next without problems.

If it's something like LOAD "" SCREEN$ then it should be treated the same way.
No need to reinvent the wheel, just support known file formats.

[4thRock]

Basically, if gamma and colorspace are undefined, they should be treated as sRGB.

As for emulation, we cannot just translate all the values into gamma-corrected ones. It all depends on what's real hardware doing. If real hardware outputs linear color values, then we should do the same way to get the same image.

I'd say that any monitor made in the last 10 years should be reasonably calibrated out of the box.
TVs have good calibration if they have "movie mode" - that one is factory set to sRGB.

TVs now can use wider gamut with their own proprietary color management, so it's not that easy nowadays. But sRGB is still the lowest common denominator.

You assume that the Next hardware treats pixel values differently than our PC graphic boards. Why ? Won't the hardware (DACs ?) be more or less the same?

For TVs "movie mode" is calibrated out of the box. You'd be amazed on how consistent it is between brands and models.
Of course, if people view on "sport" or "vivid" then yes, you can expect wild colourspaces

[Stefan123]

Hmm, seems like I still have some problems with understanding the hardware limitations.
There is NXI format, it stores 256*192 image with 9bit palette (256 two-byte colors in RGB333 format). As far as I understood, it can be displayed on Next without problems.
If we take 16bit BMP as a more supported container and strictly limit the amount of it's colors to 256 with Next's supported palette values only, wouldn't it be actually the same as NXI?

I know that this format doesn't solve all the cases with sprites and mixing layers, but should it be solving them? These cases are mostly screenshots of software, thus they can be resolved by true color PNG, and I doubt someone would want to see them on original hardware with 100% accuracy.
Please correct me if I'm wrong.

The layer 2 graphics mode (and the NXI file format) uses 8 bpp. The 8-bit pixel values are indexes into a 256-entries palette. This palette contains 9-bit RGB333 colour values.

This means that layer 2 can only display 256 colours simultaneously (from 512 possible colours). To display more than 256 colours simultaneously you would have to combine layer 2 with other layers/sprites or use Copper tricks to change the palette mid-screen.

So there is no general way of displaying Next screenshots on the Next itself since they are effectively 9 bpp images that could contain up to 512 colours if multiple layers and sprites are used together.

Regarding your question if a 16-bit BMP image with the number of colours strictly limited to 256 and using only Next's supported palette values would represent the same image as NXI? Yes, it would.

[RMartins]

I might be a bit late to this party, but I have a suggestion

PNG is an extensible format, it has chunks and every thing is separated into the proper chunk types.
Any PNG compatible application, must process only the chunks it knows how to handle.

So my suggestion is that we can devise a new chunk, for the problematic RGB333 (should probably be a palette), and include another chunk, that will be an array of palette indexes or similar, basically as similar as possible to what the original hardware does/supports natively.
We define these new chunk types as new PNG chunks types with a specific ID, that only specific software will be able to recognize.

And then we can generate a compatible PNG file, where we will encode the picture data as a regular PNG image (24bit RGB ? it will work as a preview), but we will also include all the extra required "native" data chunks, transforming this into a publishable image (any browser can render it) and also work as a preservation format.

For performance reasons on website, we can easily strip the special chunks from the PNG file, or just save a regular PNG for that.

[Seven.FFF]

So there is no general way of displaying Next screenshots on the Next itself since they are effectively 9 bpp images that could contain up to 512 colours if multiple layers and sprites are used together.

Not only this, but Next screenshots can contain both standard-width and half-width (Timex hires-style) pixels at the same time. At arbitrary positions, because of transparently stacked layers and copper effects.

You can easily design a game whose background and playfield has normal-width pixels, but whose player sprites and HUD has half-width pixels. With 512 colours on screen at once.

Not so easy to re-render on the Next itself from a flat file using a general screenshot algorithm.

You can do this on the Uno too, come to that. I was working on an Uno game that mixed radastan palette switching, normal res and hi res on the same frame. Not with transparency, as the Uno didn't support that at the time, but with raster line interrupts to do the mode switching and palette redefinitions.

[PeterJ]

The first batch of Spectrum Next screens are done. Only a few to start with. Here is an example:

https://spectrumcomputing.co.uk/index.p ... 6&id=35136

[Z80]

If I may add my two cents, there's a need for a small parsing routine to convert any 8 or 9 bits (or potentially more) bitmap picture to nxi.

The nxi format can optionally contain the palette. So in the case of a 9 bits per pixel bitmap picture to be converted into nxi, the colour deph encoding doens"t necessary means there are actually 512 different colors at the same time in the picture, and even bitmats formats encoded in TruColor could potentially be easily converted to an nxi picture that can be displayed on the Next, provided the relevant palette would be filled in.

If you're intending to open a foreign format picture right from the NextOS browser, then the picture would have to be transformed on the fly, so it could stll be an interesting programming challenge for the Next, but even without that, a PC converter outputting an nxi file would be OK.

There's still the need to parse the whole picture first and build up a table of all the different colors encountered in the picture, then build the palette contents of the nxi file.

In fact, the problem is not just with the 256 simulataneous different colors limitation. Maybe that's it if the original picture uses a simple RGB palette with 3 bits per base color, but there's no reason why the conversion would be limitated to this case.

Either there's an even greater color depth in the original picture or there are simply more than 256 simulataneous colors in it, there will be a need to simplify it by assigning it to another color that's close enough to the original but existing in the Next 9 bits base palette (first requirement), while limitating the total palette size to 256 different colors (2nd requirement).

The only difficulty would be the criteria to use. If you start from the top left corner of the picture for instance, and encounter a extreme variety or colors there (because there's some color gradient effect there, which is quite likely to happen if this area depicts the sky), then you could end up filling up your color lookup table quite fast and get limitated once you have to encode the main subject in the center, so just looking for a value that's not already in the table as you browse the picture from left to right and top to bottom might be too simple and lead to problems.

There probably should be a better method, allowing to regroup similar colors in order to use less palette space... Of course, his method could sometimes be enough in some cases and still alow to encode the whole picture without losing any detail becase there aren't actually more than 256 different colors in the original picture, but in some cases, measures should be taken to limitate their numbers using better criteria.

Maybe it can't be easily done in only one pass and a seocnd one would be necessary. Note that this method would allow to convert a picture actually containing a reasonably low number of different colors in one pass if the total ends up lower or equal to 256, and trigger the 2nd pass only if it doesn't. Then the initial lookup table just has to be created with more than 256 values - in fact just initialise it from the actual picture color depth, i.e. 512 values for 9 bits, etc. - then simplify it by assigning a Next palette index and value to each the encountered original color values.

The scond pass would then be able to build the palette part of the .nxi file with existing Next colors for each of the 256 color indexes, and job done.

In fact, that's something I want to try programming directly on the Next, so it would be able to open almost any kind of picture (and save it to a .nxi file). :p

Now there's something I'd like to ask, because when reading the recent descriptions of the Next's graphics mode, I have the feeling they've recently allowed the full 320x256 layer 2 screen to be accessed in bitmap mode without having to use tilemaps, but it's very unclear. Has everyone definitve information about this, please ?