Ah, that amazing metallic sound is one of the many, many 'mind blown' events of my Speccy years.
My reverse engineering tool
Re: My reverse engineering tool
Were you attempting to use 48K snapshots, or 128K snapshots (which don't appear to be supported)?Lethargeek wrote: ↑Fri Jan 03, 2020 2:39 pm - "New game from z80 file" does literally nothing for any new snapshots (checked and working in other emulators) added by me. Even after the restart. I was able to load only the snapshots included with the preview (even those without any other data in the subfolders) but nothing else!
- PROSM
- Manic Miner
- Posts: 476
- Joined: Fri Nov 17, 2017 7:18 pm
- Location: Sunderland, England
- Contact:
Re: My reverse engineering tool
I've been in contact with the creator about this issue, as I experienced it as well, and he replied that the tool does not yet support uncompressed Z80 snapshots. You need to save your snapshots in an emulator that has support for compression.Lethargeek wrote: ↑Fri Jan 03, 2020 2:39 pm - "New game from z80 file" does literally nothing for any new snapshots (checked and working in other emulators) added by me. Even after the restart. I was able to load only the snapshots included with the preview (even those without any other data in the subfolders) but nothing else!
All software to-date
Working on something, as always.
Working on something, as always.
- Lethargeek
- Manic Miner
- Posts: 742
- Joined: Wed Dec 11, 2019 6:47 am
Re: My reverse engineering tool
as i don't have many ready z80 snapshots, just created a 48k one in Spin at it worked; 128k one didn't
anyway, why not starting with sna support first? it is simpler and more suitable for any hacking
anyway, why not starting with sna support first? it is simpler and more suitable for any hacking
-
- Drutt
- Posts: 9
- Joined: Fri Dec 27, 2019 8:53 pm
Re: My reverse engineering tool
Thanks for the feedback. It was very useful and has definately given me some areas to focus on.
I've put some answers in-line.
I've put some answers in-line.
Lethargeek wrote: ↑Fri Jan 03, 2020 2:39 pm Tried it, my initial observations:
- Support for the sna format is a must;
I will put this in at some point. It didn't seem a high priority as most of the games I wanted to reverse engineer were available in .z80 format
- Snapshots to be put into a separate dedicated folder instead of creating a mess in the root folder;
Yes, I'm going to do this as it annoys me too!
- "New game from z80 file" does literally nothing for any new snapshots (checked and working in other emulators) added by me. Even after the restart. I was able to load only the snapshots included with the preview (even those without any other data in the subfolders) but nothing else!
So what this is supposed to do is load the .z80 and internally set up the logic for reverse engineering the game. You can then save out the data by either choosing save or closing the application.
If as you say it is doing nothing then that could be due to a missing feature in the .z80 loader that someone else discovered the other day. This issue has been fixed and I'll release a new version shortly.
- F5 key (i mean pressed on the keyboard, not clicked) doesn't work as Break, only as Continue;
I'll have a look at this, could be a keyboard focus issue.
- Graphics View arrangement is very inconvenient and uncomfortable. The slider and the view move in opposite directions and for some reason it can't go above #BFFF. Also rows of fixed height measured in characters is very bad idea for graphics view. Spectrum sprites are often of arbitrary pixel height, so at the very least the view should be arranged column by column of selected byte width. Even better if it will have the ability to reshuffle byte columns inside a sprite column (and then separately for even/odd pixel rows) and to filter out sprite descriptors breaking this view. Or even a simple scripting to describe sprite formats.
Yes this is still very much WIP. The more games I try to reverse engineer, the more issues I find with it. Initially it was used to find the graphics.
The view goes only goes to #bfff because that's the address range minus how much can be displayed, It'll probably be better if I run off to blank rather than clamp the address range controls - I'll have a look.
The slider is a pain, I'll have another look at that.
I'll change the Y height to be in pixel rows rather than characters (or have a fine control as a lots of games use character heights.
I have been thinking about putting some scripting in but I want to look at a few more games before I formulate a plan. It would be good to know some games that it has difficulty viewing.
(didn't check other features yet)
- Lethargeek
- Manic Miner
- Posts: 742
- Joined: Wed Dec 11, 2019 6:47 am
Re: My reverse engineering tool
Even if you clamp the address range, the last one should have been #С000, not #BFFF. But yeah, better to blank the overflow or wrap around.
As how to arrange the view, first look at "sprite scanner" in Xpeccy (one column of selectable width) or repaint tool in EmuZWin (several such columns as fit in the window). Also it's useful to have possible width of column >32 bytes, as some games might use backbuffers wider than ZX screen.
As for the sprite formats, these are the examples i've seen:
- sprite made on the fly in the buffer out of 8x8 tiles
- sprite and mask separately, row by row
- sprite and mask side by side, row by row
- sprite and mask bytes interleaved in a row
- sprite and mask bytes reshuffled - (rick dangerous)
- sprite and mask are interleaved nibbles (!) - (opera soft games)
- interleaved sprite rows (the next row of specific sprite is its previous row+256)
- compressed mask of lower resolution than its sprite (but same byte width) - (capitan Trueno)
This might be complicated further with sprite descriptors put between the sprite pixel data shifting the beginning of the new sprite. Furthermore, bytes in the each row usually all go left to right but sometimes it is different for odd/even rows. Eg Commando sprites (3 bytes wide) have no mask, but even row bytes go left to right, odd row bytes right to left. I'm yet to see this combined with other methods, but won't be surprised.
As how to arrange the view, first look at "sprite scanner" in Xpeccy (one column of selectable width) or repaint tool in EmuZWin (several such columns as fit in the window). Also it's useful to have possible width of column >32 bytes, as some games might use backbuffers wider than ZX screen.
As for the sprite formats, these are the examples i've seen:
- sprite made on the fly in the buffer out of 8x8 tiles
- sprite and mask separately, row by row
- sprite and mask side by side, row by row
- sprite and mask bytes interleaved in a row
- sprite and mask bytes reshuffled - (rick dangerous)
- sprite and mask are interleaved nibbles (!) - (opera soft games)
- interleaved sprite rows (the next row of specific sprite is its previous row+256)
- compressed mask of lower resolution than its sprite (but same byte width) - (capitan Trueno)
This might be complicated further with sprite descriptors put between the sprite pixel data shifting the beginning of the new sprite. Furthermore, bytes in the each row usually all go left to right but sometimes it is different for odd/even rows. Eg Commando sprites (3 bytes wide) have no mask, but even row bytes go left to right, odd row bytes right to left. I'm yet to see this combined with other methods, but won't be surprised.
-
- Drutt
- Posts: 9
- Joined: Fri Dec 27, 2019 8:53 pm
Re: My reverse engineering tool
Here's an updated version with some improvements to the graphics viewer, games directory & z80 loader fix:
https://www.dropbox.com/sh/74olu8a70l14 ... nZQfa?dl=0
https://www.dropbox.com/sh/74olu8a70l14 ... nZQfa?dl=0
-
- Drutt
- Posts: 9
- Joined: Fri Dec 27, 2019 8:53 pm
Re: My reverse engineering tool
Thanks for the info.Lethargeek wrote: ↑Sat Jan 04, 2020 3:29 pm Even if you clamp the address range, the last one should have been #С000, not #BFFF. But yeah, better to blank the overflow or wrap around.
As how to arrange the view, first look at "sprite scanner" in Xpeccy (one column of selectable width) or repaint tool in EmuZWin (several such columns as fit in the window). Also it's useful to have possible width of column >32 bytes, as some games might use backbuffers wider than ZX screen.
As for the sprite formats, these are the examples i've seen:
- sprite made on the fly in the buffer out of 8x8 tiles
- sprite and mask separately, row by row
- sprite and mask side by side, row by row
- sprite and mask bytes interleaved in a row
- sprite and mask bytes reshuffled - (rick dangerous)
- sprite and mask are interleaved nibbles (!) - (opera soft games)
- interleaved sprite rows (the next row of specific sprite is its previous row+256)
- compressed mask of lower resolution than its sprite (but same byte width) - (capitan Trueno)
This might be complicated further with sprite descriptors put between the sprite pixel data shifting the beginning of the new sprite. Furthermore, bytes in the each row usually all go left to right but sometimes it is different for odd/even rows. Eg Commando sprites (3 bytes wide) have no mask, but even row bytes go left to right, odd row bytes right to left. I'm yet to see this combined with other methods, but won't be surprised.
I got Xpeccy but couldn't find how to access the sprite scanner - there doesn't seem to be any instructions. I also tried EmuZWin, I think its approach of displaying the sprites in columns is much better than my approach so I think i'll adopt it.
I appreciate some game references for the sprite layout examples you gave me (I know you gave a couple) - It would be really useful, a lot of the games I've looked at don't use masks, they use the 'good old XOR method'.
Cheers,
Mark.
- Lethargeek
- Manic Miner
- Posts: 742
- Joined: Wed Dec 11, 2019 6:47 am
Re: My reverse engineering tool
in Xpeccy press Esc to open the debugger window and then look for the small spanner icon in the Disasm section, there will be a pull-down menu
Re: My reverse engineering tool
I'm interested.TheGoodDoktor wrote: ↑Sat Dec 28, 2019 8:30 pm I don't imagine many other people are interested in reverse engineering old games.
Re: My reverse engineering tool
Is it possible to change the type of a text/code/data segment if it's been set incorrectly? After setting memory to the code type, it doesn't seem possible to change it back, and there's no undo feature.
Also, are breakpoints able to be saved along with the project?
Also, are breakpoints able to be saved along with the project?
Re: My reverse engineering tool
Seems, there are some dependecies.TheGoodDoktor wrote: ↑Sat Jan 04, 2020 8:45 pm Here's an updated version with some improvements to the graphics viewer, games directory & z80 loader fix:
https://www.dropbox.com/sh/74olu8a70l14 ... nZQfa?dl=0
Win 7:
On another machine with Win 10 program is running normally.
P.S.
tried to get a bunch of DLLs, put these into program folder
XINPUT1_4.dll
api-ms-win-eventing-classicprovider-l1-1-0.dll
api-ms-win-core-sysinfo-l1-2-1.dll
api-ms-win-core-quirks-l1-1-0.dll
api-ms-win-core-libraryloader-l1-2-0.dll
api-ms-win-core-errorhandling-l1-1-1.dll
api-ms-win-core-synch-l1-2-0.dll
api-ms-win-core-processthreads-l1-1-2.dll
api-ms-win-core-io-l1-1-1.dll
api-ms-win-core-com-l1-1-1.dll
api-ms-win-core-file-l1-2-1.dll
api-ms-win-core-heap-l1-2-0.dll
api-ms-win-core-rtlsupport-l1-2-0.dll
but stuck with
-
- Drutt
- Posts: 9
- Joined: Fri Dec 27, 2019 8:53 pm
Re: My reverse engineering tool
Not sure it's in the last published version but you can use the 'C', 'D' & 'T' keys after selecting the line.djnzx48 wrote: ↑Fri Jan 10, 2020 6:15 am Is it possible to change the type of a text/code/data segment if it's been set incorrectly? After setting memory to the code type, it doesn't seem possible to change it back, and there's no undo feature.
Also, are breakpoints able to be saved along with the project?
I'll be publishing a new version soon.
Re: My reverse engineering tool
I'm not well versed on the Spectrum internals, but I find this kind of tools very valuable, specially visual inspectors that allows everyone to spot curious things.
Suggestion: a memory heatmap mode similar to the ones found on the Spud emulator:
Format 1:
Fromat 2:
Format 3:
Format 4:
It looks pretty nice in motion, you can easily spot the buffers and see them work realtime, but the emulator has a bug that misplaces the red and green overprints
Suggestion: a memory heatmap mode similar to the ones found on the Spud emulator:
Format 1:
Fromat 2:
Format 3:
Format 4:
It looks pretty nice in motion, you can easily spot the buffers and see them work realtime, but the emulator has a bug that misplaces the red and green overprints
Re: My reverse engineering tool
How you achieved to run Spud?
It is always crying about missing ROM file, no matter what combination of ZX model/ROM I choose in options...
P.S.
Ahhh, finally got it running.
Switched to default configuration.
Dont know why, but just extracting from archive and run doesnt't work.
-
- Drutt
- Posts: 9
- Joined: Fri Dec 27, 2019 8:53 pm
Re: My reverse engineering tool
Thanks for all the recent feedback!
I've made several improvements to the graphics viewer:
Column based display
Heat map colourisation
Ability to select location & display info below
I've also improved the memory analysis, it was misses accesses before, it should get everything now.
There's some basic assembler output but it's experimental atm.
I was hoping to add .SNA support but ran out of time.
Any more feedback/suggestions will be gratefully received!
Here's the Dropbox link:
https://www.dropbox.com/sh/74olu8a70l14 ... nZQfa?dl=0
I've made several improvements to the graphics viewer:
Column based display
Heat map colourisation
Ability to select location & display info below
I've also improved the memory analysis, it was misses accesses before, it should get everything now.
There's some basic assembler output but it's experimental atm.
I was hoping to add .SNA support but ran out of time.
Any more feedback/suggestions will be gratefully received!
Here's the Dropbox link:
https://www.dropbox.com/sh/74olu8a70l14 ... nZQfa?dl=0
Re: My reverse engineering tool
Maybe switching to older xinput can solve Win 7 compatibility problem?TheGoodDoktor wrote: ↑Sun Jan 19, 2020 4:52 pm Any more feedback/suggestions will be gratefully received!
https://www.gamedev.net/forums/topic/69 ... l-missing/
Re: My reverse engineering tool
Just wanted to chime in and say I fully support this idea.TheGoodDoktor wrote: ↑Sat Dec 28, 2019 8:30 pm I discovered Skoolkit a few months ago and ultimately I want this tool to generate output for it in some form
A few years ago someone else (apologies, don't remember who, and perhaps it would be impolite to name him, anyway) had the idea of making his Spectrum reverse engineering tool spit out SkoolKit control files, but in the end he gave up because control files are too complex (or something like that).
Now, yes, SkoolKit control files can be very complex, but they can also be very, very simple. So my advice, if you care to take it, would be to start off by generating the simplest possible control files, and then gradually work towards generating more complex ones.
Also, by the way, today is SkoolKit's 10th birthday!
https://skoolkit.ca/posts/2020/01/skoolkit-is-10/
Re: My reverse engineering tool
I noticed that the floating bus doesn't seem to be emulated, so Cobra (for example) freezes as soon the game starts. Are there any plans to add this in?
Re: My reverse engineering tool
Is there any further progress on this very interesting tool?
Re: My reverse engineering tool
Is it abandoned project now?
It had so big potential.
It had so big potential.
Re: My reverse engineering tool
I just found this project but haven't been able to try it because - as is typical in the world of retro - it's a Windows only app and there's no source code available :/
From the shared image and comments this really did look promising. Shame.
EDIT: if you're interested, this tool seems to built on top of Andre Weissflog's "chips" tookbox (https://github.com/floooh/chips) - you can see the speccy debugging UI here: https://floooh.github.io/tiny8bit/zx-ui.html?type=zx48k