My reverse engineering tool

Show us what you're working on, (preferably with screenshots).
User avatar
druellan
Dynamite Dan
Posts: 1466
Joined: Tue Apr 03, 2018 7:19 pm

Re: My reverse engineering tool

Post by druellan »

This is how it looks working and with the read/write colors fixed. Sorry the size:

Image
TheGoodDoktor
Drutt
Posts: 9
Joined: Fri Dec 27, 2019 8:53 pm

Re: My reverse engineering tool

Post by TheGoodDoktor »

Thanks for all the recent feedback!
I've made several improvements to the graphics viewer:

Column based display
Heat map colourisation
Ability to select location & display info below

I've also improved the memory analysis, it was misses accesses before, it should get everything now.
There's some basic assembler output but it's experimental atm.
I was hoping to add .SNA support but ran out of time.
Any more feedback/suggestions will be gratefully received!

Here's the Dropbox link:
https://www.dropbox.com/sh/74olu8a70l14 ... nZQfa?dl=0
User avatar
Bedazzle
Manic Miner
Posts: 303
Joined: Sun Mar 24, 2019 9:03 am

Re: My reverse engineering tool

Post by Bedazzle »

TheGoodDoktor wrote: Sun Jan 19, 2020 4:52 pm Any more feedback/suggestions will be gratefully received!
Maybe switching to older xinput can solve Win 7 compatibility problem?
https://www.gamedev.net/forums/topic/69 ... l-missing/
User avatar
druellan
Dynamite Dan
Posts: 1466
Joined: Tue Apr 03, 2018 7:19 pm

Re: My reverse engineering tool

Post by druellan »

Valley of Rains use attributes to hide unused sprites to speed up the code:

Image

Seems that the "screen" viewmode has no read/write indicators or perhaps I´m missing something?
User avatar
SkoolKid
Manic Miner
Posts: 403
Joined: Wed Nov 15, 2017 3:07 pm

Re: My reverse engineering tool

Post by SkoolKid »

TheGoodDoktor wrote: Sat Dec 28, 2019 8:30 pm I discovered Skoolkit a few months ago and ultimately I want this tool to generate output for it in some form
Just wanted to chime in and say I fully support this idea. :)

A few years ago someone else (apologies, don't remember who, and perhaps it would be impolite to name him, anyway) had the idea of making his Spectrum reverse engineering tool spit out SkoolKit control files, but in the end he gave up because control files are too complex (or something like that).

Now, yes, SkoolKit control files can be very complex, but they can also be very, very simple. So my advice, if you care to take it, would be to start off by generating the simplest possible control files, and then gradually work towards generating more complex ones.

Also, by the way, today is SkoolKit's 10th birthday!

https://skoolkit.ca/posts/2020/01/skoolkit-is-10/
SkoolKit - disassemble a game today
Pyskool - a remake of Skool Daze and Back to Skool
User avatar
Bedazzle
Manic Miner
Posts: 303
Joined: Sun Mar 24, 2019 9:03 am

Re: My reverse engineering tool

Post by Bedazzle »

SkoolKid wrote: Tue Jan 28, 2020 5:00 pm Also, by the way, today is SkoolKit's 10th birthday!
Congratulations!
User avatar
djnzx48
Manic Miner
Posts: 729
Joined: Wed Dec 06, 2017 2:13 am
Location: New Zealand

Re: My reverse engineering tool

Post by djnzx48 »

I noticed that the floating bus doesn't seem to be emulated, so Cobra (for example) freezes as soon the game starts. Are there any plans to add this in?
User avatar
Bedazzle
Manic Miner
Posts: 303
Joined: Sun Mar 24, 2019 9:03 am

Re: My reverse engineering tool

Post by Bedazzle »

Is there any further progress on this very interesting tool?
User avatar
Bedazzle
Manic Miner
Posts: 303
Joined: Sun Mar 24, 2019 9:03 am

Re: My reverse engineering tool

Post by Bedazzle »

Is it abandoned project now? :(
It had so big potential.
mrcook
Drutt
Posts: 46
Joined: Tue Jun 09, 2020 7:31 pm

Re: My reverse engineering tool

Post by mrcook »

Bedazzle wrote: Thu Oct 01, 2020 3:26 pm It had so big potential.
I just found this project but haven't been able to try it because - as is typical in the world of retro - it's a Windows only app and there's no source code available :/

From the shared image and comments this really did look promising. Shame.

EDIT: if you're interested, this tool seems to built on top of Andre Weissflog's "chips" tookbox (https://github.com/floooh/chips) - you can see the speccy debugging UI here: https://floooh.github.io/tiny8bit/zx-ui.html?type=zx48k
User avatar
Bedazzle
Manic Miner
Posts: 303
Joined: Sun Mar 24, 2019 9:03 am

Re: My reverse engineering tool

Post by Bedazzle »

mrcook wrote: Sun Oct 04, 2020 9:04 am this tool seems to built on top of Andre Weissflog's "chips" tookbox (https://github.com/floooh/chips) - you can see the speccy debugging UI here: https://floooh.github.io/tiny8bit/zx-ui.html?type=zx48k
Wow, nice!
User avatar
RMartins
Manic Miner
Posts: 776
Joined: Thu Nov 16, 2017 3:26 pm

Re: My reverse engineering tool

Post by RMartins »

mrcook wrote: Sun Oct 04, 2020 9:04 am ...
you can see the speccy debugging UI here: https://floooh.github.io/tiny8bit/zx-ui.html?type=zx48k
How exactly do we load software into it ?

Drag & drop seems to do something, but all files I have thrown at it, crash the speccy.
The only non obvious part of the UI.
User avatar
RMartins
Manic Miner
Posts: 776
Joined: Thu Nov 16, 2017 3:26 pm

Re: My reverse engineering tool

Post by RMartins »

I noticed it doesn't allow to set a breakpoint below 10AF, which is kind of weird.

EDIT: It seems to accept .z80 drag&dropped into it.
crabfists
Drutt
Posts: 29
Joined: Tue Sep 20, 2022 11:52 am

Re: My reverse engineering tool

Post by crabfists »

The tool is still being worked on and has many new features since the last build. You can download the tool and find out more here:

https://colourclash.co.uk/spectrum-analyser/

You can find a walkthrough video on the site showing you how it works. Please report any problems or feedback.

(It's not my tool. I'm posting this on behalf of @TheGoodDoktor )
crabfists
Drutt
Posts: 29
Joined: Tue Sep 20, 2022 11:52 am

Re: My reverse engineering tool

Post by crabfists »

Forgot to mention, the tool can now import and export to the Skoolkit skool format. It needs some proper documentation on how to do this.

Please note, the skool file import feature is very much WIP. It can produce weird results as not every skool file feature is supported. By all means report issues but please bear this in mind.

I'll give a quick and dirty guide here. To import a skool file you need to do it via the command line. I'll use Spellbound as an example.

1. Create game via "New Game From Snapshot File" in the menu.
2. Choose Spellbound.z80 (or whatever the snapshot is called)
3. Make sure the game is saved via Save Game Data.
4. Close the tool.
5. Open a command line in the directory where SpectrumAnalyser.exe lives.
6. Put your spellbound.skool file in the same directory as the .exe (just to make things easier)
7. Type "SpectrumAnalyser.exe Spellbound spellbound.skool" (without quotes) (note: the game name is case sensitive)
8. The tool should open with Spellbound loaded and the skool file imported.
9. Save the game by clicking "File->Save Game Data". This step is really important, otherwise the tool won't save your disassembly.
9. Verify it was imported successfully by opening up the Debug Log window. Click Windows->DebugLog.

You only need to do the above steps once.

You should see something like this:

Code: Select all

[Info] Importing skool file 'spellbound.skool'
[Warning] Item at $FFF4 was set to code: 
[Warning] Code item removed and replace as data
[Info] Saving skoolinfo file 'OutputSkoolKit/Spellbound.skoolinfo'
[Info] Imported skool file 'spellbound.skool' successfully for 'Spellbound'.
[Debug] Disassembly range $5b00-$ffff. 884 locations saved to skoolinfo file
It's worth pointing out it is recommended to start from scratch when importing a skool file. If you have previously worked on the game and added comments, labels etc the import process could overwrite your existing data. As a precaution, the tool will save a backup of your existing game data via a .bak file.

To export your disassembly as a skool file, you can do this via the "File->Export Skool File." You can choose to export as either decimal or hexadecimal. Note: you don't need to have previously imported a skool file to be able to export one.

The export process outputs a ".skoolinfo" file, which stores information that is not stored in the code analysis data, such as sub block directives etc (Spectrum Analyser itself doesn't have a concept of sub blocks). This file is designed to be hand edited in case the exported skool file is not output correctly. It's a json file.

By default, if no skool file was previously imported, the start and end address of the disassembly will be the entire range of memory $4000-$ffff. To set a start and end address you can modify the "StartAddress" and "EndAddress" entries in the .skoolinfo file.

If no .skoolinfo file exists then you can create one in the OutputSkoolKit folder (you may need to create the directory). Put the following in it and change the StartAddress and EndAddress to the desired address range.

Code: Select all

{
    "EndAddress": "0xffff",
    "Locations": [
    ],
    "StartAddress": "0x4000"
}
Thanks to @SkoolKid for answering all my stupid questions :lol: .
User avatar
druellan
Dynamite Dan
Posts: 1466
Joined: Tue Apr 03, 2018 7:19 pm

Re: My reverse engineering tool

Post by druellan »

Thanks for the color highlight on screen changes, love to see this working. To someone that is not deeply technical, visual aids like those are really welcome.
I can see, for example, that Saboteur keeps refreshing the center of the screen for some reason?

Image

The frame trace tool is also very interesting to play with. I was checking how Commando Tracer refresh the score on each other frame:

Image

Is there a way to get the FPS based on screen diffs? The only emulator I know does something similar is EmuZWin, and it basically shows how many frames the software is actually pushing to the screen and the player.
AndyC
Dynamite Dan
Posts: 1388
Joined: Mon Nov 13, 2017 5:12 am

Re: My reverse engineering tool

Post by AndyC »

The problem with FPS is that it's essentially meaningless on a 8 bit machine, because they generally don't work like that. On more modern machines a whole new frame is prepared and then buffers are swapped to replace what is currently on screen with the new frame in one go, you count those swaps and it gives you FPS.

On most 8-bits, drawing is done directly into the frame buffer but it takes multiple frames to update everything. So the question then becomes how you define FPS in a measurable quantity. If the player character is updated on every frame, but the enemies are updated on alternate frames is it 50FPS or 25FPS?
User avatar
Seven.FFF
Manic Miner
Posts: 736
Joined: Sat Nov 25, 2017 10:50 pm
Location: USA

Re: My reverse engineering tool

Post by Seven.FFF »

You are right about “in a measurable quantity”. Most people would concur that a game where the player moves every frame is probably a 50fps game, and a game where the player is static but the background scrolls around him every three frames is probably a 16.67 fps game, but to determine that in a tool you’re off into the realm of heuristics rather than objective measurements.it’s not really a useful thing to do.
Robin Verhagen-Guest
SevenFFF / Threetwosevensixseven / colonel32
NXtel NXTP ESP Update ESP Reset CSpect Plugins
User avatar
druellan
Dynamite Dan
Posts: 1466
Joined: Tue Apr 03, 2018 7:19 pm

Re: My reverse engineering tool

Post by druellan »

Oh, I agree, but there is no need for a super accurate tool, just measure how many times changes are being pushed to the screen is enough to have a general idea. If the game does some Cobra-trickery to improve the FPS perception, so be it (the frame trace tool showed above can help with that), but the fact that is able to reach 50 fps is indicative of something IMHO.

Said that, the argument I can make against this, is the fact that for the task of reverse engineering is probably useless, but since the tool is already measuring screen changes, I was wondering if it is possible to have this value on screen.
AndyC
Dynamite Dan
Posts: 1388
Joined: Mon Nov 13, 2017 5:12 am

Re: My reverse engineering tool

Post by AndyC »

druellan wrote: Sun Jan 29, 2023 6:28 pm , just measure how many times changes are being pushed to the screen is enough to have a general idea.
That'll be 50FPS in just about every game, even the really, really sluggish ones. Because they're drawing parts of the screen on every frame, even if they spend 20 frames drawing a single in game "frame"
User avatar
druellan
Dynamite Dan
Posts: 1466
Joined: Tue Apr 03, 2018 7:19 pm

Re: My reverse engineering tool

Post by druellan »

AndyC wrote: Sun Jan 29, 2023 6:48 pm That'll be 50FPS in just about every game, even the really, really sluggish ones. Because they're drawing parts of the screen on every frame, even if they spend 20 frames drawing a single in game "frame"
Oh, I disagree. There are a lot of interesting cases out there. Vector games are a good example, racing games, some time ago I started to look into cross-compiled games performance, and I even used it to compare 128k basic games vs 48k basic games.
As you pointed out, there are games that are impossible to measure correctly, but the feature is not pointless IMHO.
User avatar
Kweepa
Manic Miner
Posts: 311
Joined: Sat Feb 03, 2018 6:14 pm
Location: Albuquerque, New Mexico

Re: My reverse engineering tool

Post by Kweepa »

Vector graphics games are no easier to guess the framerate of. You don't know when one frame ends and another starts.
You might be able to use some heuristics to guess where the main loop was and check how frequently that was executed, but most likely you'd have to have the user identify the main loop and mark an instruction as part of it.

This looks like an amazingly useful tool. One thing that might be nice is to capture and display a tree of function calls for a quick overview of the code.
AndyC
Dynamite Dan
Posts: 1388
Joined: Mon Nov 13, 2017 5:12 am

Re: My reverse engineering tool

Post by AndyC »

It might work in some case, if the entire changeable screen area is prepared in an off screen buffer and then copied across to screen (and that copy takes less than a frame) but I suspect you'll find even scrolling games that do that will still write things like status panel updates directly to the display and may do so every few frames which will skew calculations.

You can apply some heuristics (like X% of the screen changed) but you'll end up tweaking that for every game. It's just not really a quantifiable mechanism you can really use to compare games. And if you can't really use it for comparison, then what really is the point?
User avatar
druellan
Dynamite Dan
Posts: 1466
Joined: Tue Apr 03, 2018 7:19 pm

Re: My reverse engineering tool

Post by druellan »

Kweepa wrote: Mon Jan 30, 2023 10:36 am Vector graphics games are no easier to guess the framerate of. You don't know when one frame ends and another starts.
You might be able to use some heuristics to guess where the main loop was and check how frequently that was executed, but most likely you'd have to have the user identify the main loop and mark an instruction as part of it.
EmuZWin manages to give "a number", probably not technically accurate, but is close to the FPS you can see on screen, so can be used for comparisons inside the same tool. The main problem with vector software is the inconsistent framerate, but that is also an interesting value to have, since there are engines that are better with busy screens than others.
Post Reply