Page 1 of 2

Re: My reverse engineering tool

Posted: Fri Jan 10, 2020 6:15 am
by djnzx48
Is it possible to change the type of a text/code/data segment if it's been set incorrectly? After setting memory to the code type, it doesn't seem possible to change it back, and there's no undo feature.

Also, are breakpoints able to be saved along with the project?

Re: My reverse engineering tool

Posted: Sun Jan 12, 2020 9:22 am
by Bedazzle
TheGoodDoktor wrote: Sat Jan 04, 2020 8:45 pm Here's an updated version with some improvements to the graphics viewer, games directory & z80 loader fix:
https://www.dropbox.com/sh/74olu8a70l14 ... nZQfa?dl=0
Seems, there are some dependecies.
Win 7:

Image

On another machine with Win 10 program is running normally.

P.S.
tried to get a bunch of DLLs, put these into program folder
XINPUT1_4.dll
api-ms-win-eventing-classicprovider-l1-1-0.dll
api-ms-win-core-sysinfo-l1-2-1.dll
api-ms-win-core-quirks-l1-1-0.dll
api-ms-win-core-libraryloader-l1-2-0.dll
api-ms-win-core-errorhandling-l1-1-1.dll
api-ms-win-core-synch-l1-2-0.dll
api-ms-win-core-processthreads-l1-1-2.dll
api-ms-win-core-io-l1-1-1.dll
api-ms-win-core-com-l1-1-1.dll
api-ms-win-core-file-l1-2-1.dll
api-ms-win-core-heap-l1-2-0.dll
api-ms-win-core-rtlsupport-l1-2-0.dll

but stuck with

Image

Re: My reverse engineering tool

Posted: Sun Jan 12, 2020 9:47 pm
by TheGoodDoktor
djnzx48 wrote: Fri Jan 10, 2020 6:15 am Is it possible to change the type of a text/code/data segment if it's been set incorrectly? After setting memory to the code type, it doesn't seem possible to change it back, and there's no undo feature.

Also, are breakpoints able to be saved along with the project?
Not sure it's in the last published version but you can use the 'C', 'D' & 'T' keys after selecting the line.
I'll be publishing a new version soon.

Re: My reverse engineering tool

Posted: Thu Jan 16, 2020 9:29 pm
by druellan
I'm not well versed on the Spectrum internals, but I find this kind of tools very valuable, specially visual inspectors that allows everyone to spot curious things.
Suggestion: a memory heatmap mode similar to the ones found on the Spud emulator:

Format 1:
Image

Fromat 2:
Image

Format 3:
Image

Format 4:
Image

It looks pretty nice in motion, you can easily spot the buffers and see them work realtime, but the emulator has a bug that misplaces the red and green overprints

Re: My reverse engineering tool

Posted: Thu Jan 16, 2020 9:42 pm
by Bedazzle
druellan wrote: Thu Jan 16, 2020 9:29 pm Suggestion: a memory heatmap mode similar to the ones found on the Spud emulator:
How you achieved to run Spud?
It is always crying about missing ROM file, no matter what combination of ZX model/ROM I choose in options...

Image


P.S.
Ahhh, finally got it running.
Switched to default configuration.
Dont know why, but just extracting from archive and run doesnt't work. :lol:

Re: My reverse engineering tool

Posted: Fri Jan 17, 2020 1:23 am
by druellan
This is how it looks working and with the read/write colors fixed. Sorry the size:

Image

Re: My reverse engineering tool

Posted: Sun Jan 19, 2020 4:52 pm
by TheGoodDoktor
Thanks for all the recent feedback!
I've made several improvements to the graphics viewer:

Column based display
Heat map colourisation
Ability to select location & display info below

I've also improved the memory analysis, it was misses accesses before, it should get everything now.
There's some basic assembler output but it's experimental atm.
I was hoping to add .SNA support but ran out of time.
Any more feedback/suggestions will be gratefully received!

Here's the Dropbox link:
https://www.dropbox.com/sh/74olu8a70l14 ... nZQfa?dl=0

Re: My reverse engineering tool

Posted: Sun Jan 19, 2020 6:04 pm
by Bedazzle
TheGoodDoktor wrote: Sun Jan 19, 2020 4:52 pm Any more feedback/suggestions will be gratefully received!
Maybe switching to older xinput can solve Win 7 compatibility problem?
https://www.gamedev.net/forums/topic/69 ... l-missing/

Re: My reverse engineering tool

Posted: Tue Jan 21, 2020 2:05 pm
by druellan
Valley of Rains use attributes to hide unused sprites to speed up the code:

Image

Seems that the "screen" viewmode has no read/write indicators or perhaps I´m missing something?

Re: My reverse engineering tool

Posted: Tue Jan 28, 2020 5:00 pm
by SkoolKid
TheGoodDoktor wrote: Sat Dec 28, 2019 8:30 pm I discovered Skoolkit a few months ago and ultimately I want this tool to generate output for it in some form
Just wanted to chime in and say I fully support this idea. :)

A few years ago someone else (apologies, don't remember who, and perhaps it would be impolite to name him, anyway) had the idea of making his Spectrum reverse engineering tool spit out SkoolKit control files, but in the end he gave up because control files are too complex (or something like that).

Now, yes, SkoolKit control files can be very complex, but they can also be very, very simple. So my advice, if you care to take it, would be to start off by generating the simplest possible control files, and then gradually work towards generating more complex ones.

Also, by the way, today is SkoolKit's 10th birthday!

https://skoolkit.ca/posts/2020/01/skoolkit-is-10/

Re: My reverse engineering tool

Posted: Tue Jan 28, 2020 7:52 pm
by Bedazzle
SkoolKid wrote: Tue Jan 28, 2020 5:00 pm Also, by the way, today is SkoolKit's 10th birthday!
Congratulations!

Re: My reverse engineering tool

Posted: Tue Jan 28, 2020 10:34 pm
by djnzx48
I noticed that the floating bus doesn't seem to be emulated, so Cobra (for example) freezes as soon the game starts. Are there any plans to add this in?

Re: My reverse engineering tool

Posted: Wed Jun 03, 2020 7:01 am
by Bedazzle
Is there any further progress on this very interesting tool?

Re: My reverse engineering tool

Posted: Thu Oct 01, 2020 3:26 pm
by Bedazzle
Is it abandoned project now? :(
It had so big potential.

Re: My reverse engineering tool

Posted: Sun Oct 04, 2020 9:04 am
by mrcook
Bedazzle wrote: Thu Oct 01, 2020 3:26 pm It had so big potential.
I just found this project but haven't been able to try it because - as is typical in the world of retro - it's a Windows only app and there's no source code available :/

From the shared image and comments this really did look promising. Shame.

EDIT: if you're interested, this tool seems to built on top of Andre Weissflog's "chips" tookbox (https://github.com/floooh/chips) - you can see the speccy debugging UI here: https://floooh.github.io/tiny8bit/zx-ui.html?type=zx48k

Re: My reverse engineering tool

Posted: Mon Oct 05, 2020 10:49 am
by Bedazzle
mrcook wrote: Sun Oct 04, 2020 9:04 am this tool seems to built on top of Andre Weissflog's "chips" tookbox (https://github.com/floooh/chips) - you can see the speccy debugging UI here: https://floooh.github.io/tiny8bit/zx-ui.html?type=zx48k
Wow, nice!

Re: My reverse engineering tool

Posted: Tue Oct 06, 2020 10:05 pm
by RMartins
mrcook wrote: Sun Oct 04, 2020 9:04 am ...
you can see the speccy debugging UI here: https://floooh.github.io/tiny8bit/zx-ui.html?type=zx48k
How exactly do we load software into it ?

Drag & drop seems to do something, but all files I have thrown at it, crash the speccy.
The only non obvious part of the UI.

Re: My reverse engineering tool

Posted: Tue Oct 06, 2020 10:08 pm
by RMartins
I noticed it doesn't allow to set a breakpoint below 10AF, which is kind of weird.

EDIT: It seems to accept .z80 drag&dropped into it.

Re: My reverse engineering tool

Posted: Thu Jan 05, 2023 11:22 pm
by crabfists
The tool is still being worked on and has many new features since the last build. You can download the tool and find out more here:

https://colourclash.co.uk/spectrum-analyser/

You can find a walkthrough video on the site showing you how it works. Please report any problems or feedback.

(It's not my tool. I'm posting this on behalf of @TheGoodDoktor )

Re: My reverse engineering tool

Posted: Fri Jan 06, 2023 10:05 am
by crabfists
Forgot to mention, the tool can now import and export to the Skoolkit skool format. It needs some proper documentation on how to do this.

Please note, the skool file import feature is very much WIP. It can produce weird results as not every skool file feature is supported. By all means report issues but please bear this in mind.

I'll give a quick and dirty guide here. To import a skool file you need to do it via the command line. I'll use Spellbound as an example.

1. Create game via "New Game From Snapshot File" in the menu.
2. Choose Spellbound.z80 (or whatever the snapshot is called)
3. Make sure the game is saved via Save Game Data.
4. Close the tool.
5. Open a command line in the directory where SpectrumAnalyser.exe lives.
6. Put your spellbound.skool file in the same directory as the .exe (just to make things easier)
7. Type "SpectrumAnalyser.exe Spellbound spellbound.skool" (without quotes) (note: the game name is case sensitive)
8. The tool should open with Spellbound loaded and the skool file imported.
9. Save the game by clicking "File->Save Game Data". This step is really important, otherwise the tool won't save your disassembly.
9. Verify it was imported successfully by opening up the Debug Log window. Click Windows->DebugLog.

You only need to do the above steps once.

You should see something like this:

Code: Select all

[Info] Importing skool file 'spellbound.skool'
[Warning] Item at $FFF4 was set to code: 
[Warning] Code item removed and replace as data
[Info] Saving skoolinfo file 'OutputSkoolKit/Spellbound.skoolinfo'
[Info] Imported skool file 'spellbound.skool' successfully for 'Spellbound'.
[Debug] Disassembly range $5b00-$ffff. 884 locations saved to skoolinfo file
It's worth pointing out it is recommended to start from scratch when importing a skool file. If you have previously worked on the game and added comments, labels etc the import process could overwrite your existing data. As a precaution, the tool will save a backup of your existing game data via a .bak file.

To export your disassembly as a skool file, you can do this via the "File->Export Skool File." You can choose to export as either decimal or hexadecimal. Note: you don't need to have previously imported a skool file to be able to export one.

The export process outputs a ".skoolinfo" file, which stores information that is not stored in the code analysis data, such as sub block directives etc (Spectrum Analyser itself doesn't have a concept of sub blocks). This file is designed to be hand edited in case the exported skool file is not output correctly. It's a json file.

By default, if no skool file was previously imported, the start and end address of the disassembly will be the entire range of memory $4000-$ffff. To set a start and end address you can modify the "StartAddress" and "EndAddress" entries in the .skoolinfo file.

If no .skoolinfo file exists then you can create one in the OutputSkoolKit folder (you may need to create the directory). Put the following in it and change the StartAddress and EndAddress to the desired address range.

Code: Select all

{
    "EndAddress": "0xffff",
    "Locations": [
    ],
    "StartAddress": "0x4000"
}
Thanks to @SkoolKid for answering all my stupid questions :lol: .

Re: My reverse engineering tool

Posted: Sun Jan 29, 2023 2:27 pm
by druellan
Thanks for the color highlight on screen changes, love to see this working. To someone that is not deeply technical, visual aids like those are really welcome.
I can see, for example, that Saboteur keeps refreshing the center of the screen for some reason?

Image

The frame trace tool is also very interesting to play with. I was checking how Commando Tracer refresh the score on each other frame:

Image

Is there a way to get the FPS based on screen diffs? The only emulator I know does something similar is EmuZWin, and it basically shows how many frames the software is actually pushing to the screen and the player.

Re: My reverse engineering tool

Posted: Sun Jan 29, 2023 2:41 pm
by AndyC
The problem with FPS is that it's essentially meaningless on a 8 bit machine, because they generally don't work like that. On more modern machines a whole new frame is prepared and then buffers are swapped to replace what is currently on screen with the new frame in one go, you count those swaps and it gives you FPS.

On most 8-bits, drawing is done directly into the frame buffer but it takes multiple frames to update everything. So the question then becomes how you define FPS in a measurable quantity. If the player character is updated on every frame, but the enemies are updated on alternate frames is it 50FPS or 25FPS?

Re: My reverse engineering tool

Posted: Sun Jan 29, 2023 3:23 pm
by Seven.FFF
You are right about “in a measurable quantity”. Most people would concur that a game where the player moves every frame is probably a 50fps game, and a game where the player is static but the background scrolls around him every three frames is probably a 16.67 fps game, but to determine that in a tool you’re off into the realm of heuristics rather than objective measurements.it’s not really a useful thing to do.

Re: My reverse engineering tool

Posted: Sun Jan 29, 2023 6:28 pm
by druellan
Oh, I agree, but there is no need for a super accurate tool, just measure how many times changes are being pushed to the screen is enough to have a general idea. If the game does some Cobra-trickery to improve the FPS perception, so be it (the frame trace tool showed above can help with that), but the fact that is able to reach 50 fps is indicative of something IMHO.

Said that, the argument I can make against this, is the fact that for the task of reverse engineering is probably useless, but since the tool is already measuring screen changes, I was wondering if it is possible to have this value on screen.

Re: My reverse engineering tool

Posted: Sun Jan 29, 2023 6:48 pm
by AndyC
druellan wrote: Sun Jan 29, 2023 6:28 pm , just measure how many times changes are being pushed to the screen is enough to have a general idea.
That'll be 50FPS in just about every game, even the really, really sluggish ones. Because they're drawing parts of the screen on every frame, even if they spend 20 frames drawing a single in game "frame"