SkoolKit: ctl file or skool file?
SkoolKit: ctl file or skool file?
This poll is inspired by [mention]cmal[/mention], who asked the million dollar question that polarises the SkoolKit-using community[1]: control file or skool file?
Back when I released SkoolKit 1.0, I expected users (if there were any) to use a control file in the initial stages of developing a disassembly, and then mostly work on the skool file. But I get the impression that reverse engineers these days generally prefer working on the control file exclusively. What about you?
[1] Maybe.
Back when I released SkoolKit 1.0, I expected users (if there were any) to use a control file in the initial stages of developing a disassembly, and then mostly work on the skool file. But I get the impression that reverse engineers these days generally prefer working on the control file exclusively. What about you?
[1] Maybe.
Re: SkoolKit: ctl file or skool file?
I only use the control file, but probably only because I've been using Skoolkit for so long and that was just how it "was" (or at least, how I assumed it to be). I always start from scratch too, with a blank file ~ not for any particular reason, that's just how I like to do it. One discovery at a time!
Re: SkoolKit: ctl file or skool file?
Never touched the .skool file, always just worked on the .ctl file.
Then occasionally fire a sna2skool and a skool2html and refresh the local html page to see if it looks as I expect...
Then occasionally fire a sna2skool and a skool2html and refresh the local html page to see if it looks as I expect...
My Speccy site: thirdharmoniser.com
Re: SkoolKit: ctl file or skool file?
Yup, I only edit the ctl file. I treat it like the "source code" of the disassembly and build everything else from scripts.
Re: SkoolKit: ctl file or skool file?
Thanks for posting the poll, [mention]SkoolKid[/mention] . Interesting result. So I'm clearly doing something wrong. I voted ".skool" file because that's what I've been doing so far.
It leads to the question:
If you edit the control file, how do you insert annotations that are adjacent to the disassembly commands?
My control file looks like this:
c 24092
c 24113
b 24134
c 24162
b 24205
t 24212
b 24215
t 24238
Skool file looks like this. It has the disassembly so I can add annotations:
; Used by the routines at #R24251, #R24276, #R24328 and #R24472.
c24113 LD HL,(24090) ; Restore A and Flags register
24116 PUSH HL ;
24117 POP AF ;
24118 LD IX,(24088) ; Restore IX
24122 LD HL,(24086) ; Restore HL
24125 LD DE,(24084) ; Restore DE
24129 LD BC,(24082) ; Restore BC
24133 RET ;
(anyone like to try guess what game this is?? )
Perhaps I have to run this step once, is that right?
"use skool2ctl.py to convert a disassembly back into a control file (with annotations retained)"
...then going forward, update the control file.
It leads to the question:
If you edit the control file, how do you insert annotations that are adjacent to the disassembly commands?
My control file looks like this:
c 24092
c 24113
b 24134
c 24162
b 24205
t 24212
b 24215
t 24238
Skool file looks like this. It has the disassembly so I can add annotations:
; Used by the routines at #R24251, #R24276, #R24328 and #R24472.
c24113 LD HL,(24090) ; Restore A and Flags register
24116 PUSH HL ;
24117 POP AF ;
24118 LD IX,(24088) ; Restore IX
24122 LD HL,(24086) ; Restore HL
24125 LD DE,(24084) ; Restore DE
24129 LD BC,(24082) ; Restore BC
24133 RET ;
(anyone like to try guess what game this is?? )
Perhaps I have to run this step once, is that right?
"use skool2ctl.py to convert a disassembly back into a control file (with annotations retained)"
...then going forward, update the control file.
Re: SkoolKit: ctl file or skool file?
For what it's worth, I don't think there's anything wrong with editing and adding annotations to the skool file. That's what I've done for the majority of the disassemblies I've worked on. I like it because the code is right there in the text file, which makes it easy to annotate specific instructions as you go through a routine. If you're comfortable with skool file syntax, working on the skool file is a good option.
I suppose one advantage of working on the control file, though, is that control file syntax is a bit simpler: you don't have to worry about where your curly braces, semicolons and comment continuation lines are going, or how to format an entry header correctly.
For that you need to use sub-block syntax, which means the capital letter counterparts ('B', 'C', 'T' etc.) of the block directives ('b', 'c', 't' etc.).cmal wrote: It leads to the question:
If you edit the control file, how do you insert annotations that are adjacent to the disassembly commands?
And if you're wondering how sub-block syntax works, one way to figure that out is to follow your own advice:
Although I usually prefer to work on skool files, I will still use skool2ctl.py to generate a (temporary) control file if I want to split up or join entries, or reorganise a data block (e.g. change the number of bytes per line) - because that's much easier to do in a control file.cmal wrote: Perhaps I have to run this step once, is that right?
"use skool2ctl.py to convert a disassembly back into a control file (with annotations retained)"
...then going forward, update the control file.
Re: SkoolKit: ctl file or skool file?
Sorry for offtopic, but...
Is it possible to "redirect" output to several files? For example, I do not like one long and huge file (asm or skool)?
Is it possible to "redirect" output to several files? For example, I do not like one long and huge file (asm or skool)?
Re: SkoolKit: ctl file or skool file?
No, skool2asm.py and sna2skool.py just write to standard output, and there's no support for splitting up the output and sending it to more than one file.
Multiple invocations of skool2asm.py and sna2skool.py with carefully chosen --start and --end options might be a workaround, though.
Also, sna2skool.py does support reading multiple control files, so if you prefer control files to skool files, that is one way to split up your sources.
And obviously, skool2html.py already writes output to multiple files, because it would make no sense not to.
Re: SkoolKit: ctl file or skool file?
I did some (incomplete) disassemblies using the Skoolkit, and at some point I wonder if we have some modern IDE support (like syntax highlighting) for Skoolkit file formats.
Found SkoolkitZ80 for Sublime Text 3 editor - see https://github.com/mrcook/SkoolkitZ80 - that's for .skool files.
But, I am a .ctl file guy, so I made attempt to create a syntax highlighting plugin for VS Code: https://github.com/nzeemin/skoolkit-ctl-syntax
To use it with VS Code, git clone the repo or download it, and put in separate folder at %USERPROFILE%\.vscode\extensions
Hope it will be useful for someone other than me
Found SkoolkitZ80 for Sublime Text 3 editor - see https://github.com/mrcook/SkoolkitZ80 - that's for .skool files.
But, I am a .ctl file guy, so I made attempt to create a syntax highlighting plugin for VS Code: https://github.com/nzeemin/skoolkit-ctl-syntax
To use it with VS Code, git clone the repo or download it, and put in separate folder at %USERPROFILE%\.vscode\extensions
Hope it will be useful for someone other than me
Re: SkoolKit: ctl file or skool file?
I'm a CTL guy too. I'm going to try your new plugin for myself tonight. Thanks.nzeemin wrote: ↑Thu Jun 09, 2022 10:27 pm I did some (incomplete) disassemblies using the Skoolkit, and at some point I wonder if we have some modern IDE support (like syntax highlighting) for Skoolkit file formats.
Found SkoolkitZ80 for Sublime Text 3 editor - see https://github.com/mrcook/SkoolkitZ80 - that's for .skool files.
But, I am a .ctl file guy, so I made attempt to create a syntax highlighting plugin for VS Code: https://github.com/nzeemin/skoolkit-ctl-syntax
To use it with VS Code, git clone the repo or download it, and put in separate folder at %USERPROFILE%\.vscode\extensions
Hope it will be useful for someone other than me
I code to live (Job) and I live to code (Retro). One of them has to give!
Re: SkoolKit: ctl file or skool file?
Hmmm, I might be missing something...
I started my first SkoolKit disassembly this week, following the getting started guide. I created a CTL file as per the Hungry Horace example, then another based on a FUSE profiling run, built the Skool from them, then never touched them again.
The documentation seems to imply that once you have your Skool file, that's where you work from going onward. I didn't understand the bit where it explained why you might want to go back to the CTL file, so I left that until I needed it. So far I haven't needed it. Now I'm wondering why.
I started my first SkoolKit disassembly this week, following the getting started guide. I created a CTL file as per the Hungry Horace example, then another based on a FUSE profiling run, built the Skool from them, then never touched them again.
The documentation seems to imply that once you have your Skool file, that's where you work from going onward. I didn't understand the bit where it explained why you might want to go back to the CTL file, so I left that until I needed it. So far I haven't needed it. Now I'm wondering why.
Derek Fountain, author of the ZX Spectrum C Programmer's Getting Started Guide and various open source games, hardware and other projects, including an IF1 and ZX Microdrive emulator.
Re: SkoolKit: ctl file or skool file?
It's up to the individual!
I see the ctl file as the "source" for mine, and keep my changes in version control, ignoring .skool as a generated output file that's just a stepping stone to html or asm. Scripts build everything else for me
It could just be that I'm lazy and don't want to do a lot of typing and text formatting
I see the ctl file as the "source" for mine, and keep my changes in version control, ignoring .skool as a generated output file that's just a stepping stone to html or asm. Scripts build everything else for me
It could just be that I'm lazy and don't want to do a lot of typing and text formatting
Re: SkoolKit: ctl file or skool file?
You're not missing anything. That Disassembly DIY documentation was written a long time ago, back when I expected users to start with a control file and eventually move to editing the skool file. But it seems most people prefer to stick with the control file and never touch the skool file. Which is fine, and means I should probably update the documentation to reflect that preference.dfzx wrote: ↑Fri Jun 10, 2022 10:57 amI started my first SkoolKit disassembly this week, following the getting started guide. I created a CTL file as per the Hungry Horace example, then another based on a FUSE profiling run, built the Skool from them, then never touched them again.
The documentation seems to imply that once you have your Skool file, that's where you work from going onward. I didn't understand the bit where it explained why you might want to go back to the CTL file, so I left that until I needed it. So far I haven't needed it. Now I'm wondering why.
Re: SkoolKit: ctl file or skool file?
Or document both process workflows.SkoolKid wrote: ↑Sat Jun 11, 2022 11:25 pm You're not missing anything. That Disassembly DIY documentation was written a long time ago, back when I expected users to start with a control file and eventually move to editing the skool file. But it seems most people prefer to stick with the control file and never touch the skool file. Which is fine, and means I should probably update the documentation to reflect that preference.
This way one can choose one or the other, without the documentation forcing one or the other.
Note that I'm assuming that they are 100% equivalent, which might not be true.
It also seems that if they are equivalent, there is duplicated functionality between the Ctl and Skool file.
Re: SkoolKit: ctl file or skool file?
I plan to update the documentation for the next release of SkoolKit (8.7).
There's really not much difference between the workflows, though. If you stick with a control file, you need to run sna2skool.py before running skool2html.py (or any other skool2* command). Whereas if you work on the skool file directly, you don't need to do that (because the skool file's always already up to date).
The skool file is derived from the control file (and a snapshot), but I don't think that makes them equivalent in any meaningful sense. That would be a bit like saying an object file produced by a compiler is equivalent to the source code that was used to build it.Note that I'm assuming that they are 100% equivalent, which might not be true.
It also seems that if they are equivalent, there is duplicated functionality between the Ctl and Skool file.
Re: SkoolKit: ctl file or skool file?
I wanted to work on the Skool file, but the thing which caught me out was the difficulty in converting from one type of content to another. For example, when I had this as a product of the Fuse profiler and sna2ctl.py:
Code: Select all
24477 DEFM "+-e3"
The only obvious way to make such conversions is to go from skool back to ctl using the skool2ctl.py utility, tweak the ctl file, then go back from ctl to skool again. But it's a fussy process which involves an intermediate ctl file which I wasn't sure whether I should be keeping. It also loses information from the skool file: mostly formatting of my comments IIRC, but there was enough lost that as I got towards the end of my project that I stopped doing it.
I'm not sure what the answer is. It seems many people just stick with working on the ctl file, but I never really got the hang of it.
Derek Fountain, author of the ZX Spectrum C Programmer's Getting Started Guide and various open source games, hardware and other projects, including an IF1 and ZX Microdrive emulator.
Re: SkoolKit: ctl file or skool file?
If you're not sure how to modify a skool file, but you know how to make the corresponding modification to a control file (e.g. to replace a DEFM with a DEFB, as in your example), then using skool2ctl.py to go from skool to ctl and then back again is definitely the best way to go.dfzx wrote: ↑Sat Aug 13, 2022 8:40 pm The only obvious way to make such conversions is to go from skool back to ctl using the skool2ctl.py utility, tweak the ctl file, then go back from ctl to skool again. But it's a fussy process which involves an intermediate ctl file which I wasn't sure whether I should be keeping. It also loses information from the skool file: mostly formatting of my comments IIRC, but there was enough lost that as I got towards the end of my project that I stopped doing it.
And there's no need to build a complete control file and then rebuild the entire skool file: skool2ctl.py has --start and --end options that enable you to carve out just the portion you want to modify. Then you can paste the resulting skool file snippet into your working skool file, instead of replacing the whole thing. The only caveat with that approach is the the --start address has to be the start address of a routine or data block.
Re: SkoolKit: ctl file or skool file?
The CTL file looks like a blueprint of where code and data are supposed to be.
However, this is the first thing we NEED to do, which is most probably not the best scenario when we don't know anything about the code or data.
This would be a lot easier, if this representation of code/data (AKA CTL), was referenced but not replaced by SKOOL file.
And I say this, because disassembling something, is a process of discovery.
What this means, is that we should be able to go forward and back easily, which currently unfortunately is not that easy.
The DEFM is one of the cases, but there are others.
Blocks of that that now became variables
Blocks of code that became data.
Blocks of data that became code.
There is also the problem of bytes that are code and data, for example, when using self modifying code.
All in all, I see the CTL as a base reference to build a kind of node graph, that should be used by SKOOL file, but not to be replaced or embed its definition into the skool file. Because if the data is embedded, then we have issues to convert back and forth.
However, this is the first thing we NEED to do, which is most probably not the best scenario when we don't know anything about the code or data.
This would be a lot easier, if this representation of code/data (AKA CTL), was referenced but not replaced by SKOOL file.
And I say this, because disassembling something, is a process of discovery.
What this means, is that we should be able to go forward and back easily, which currently unfortunately is not that easy.
The DEFM is one of the cases, but there are others.
Blocks of that that now became variables
Blocks of code that became data.
Blocks of data that became code.
There is also the problem of bytes that are code and data, for example, when using self modifying code.
All in all, I see the CTL as a base reference to build a kind of node graph, that should be used by SKOOL file, but not to be replaced or embed its definition into the skool file. Because if the data is embedded, then we have issues to convert back and forth.
Re: SkoolKit: ctl file or skool file?
This really needs some kind of editor, that would allows us to edit each block, brake it apart, join blocks, etc...
Because when we do these operations, we need to make decisions on how to merge or split and to keep or not the existing labels/comments/descriptions, etc ...
Because some have references to stuff that has changed.
Basically, this is similar to a resolve of merge conflicts in a Source Control system.
Because when we do these operations, we need to make decisions on how to merge or split and to keep or not the existing labels/comments/descriptions, etc ...
Because some have references to stuff that has changed.
Basically, this is similar to a resolve of merge conflicts in a Source Control system.
Re: SkoolKit: ctl file or skool file?
By the way, using SkoolKit 8.6, I seem to have found a bug
This Macro doesn't produce the expected result (the output of the second third of the screen
It comes close, but has a few problems:
- Some of the pixels and full lines, are missing
- Some colors Also seem off (but could be due to missing pixels).
Also tried to get the first third, but it has the same issues with pixel data.
same problem for third, third.
This Macro doesn't produce the expected result (the output of the second third of the screen
Code: Select all
#SCR1,0,8,32,8(secondThird.png)
- Some of the pixels and full lines, are missing
- Some colors Also seem off (but could be due to missing pixels).
Also tried to get the first third, but it has the same issues with pixel data.
Code: Select all
#SCR1,0,0,32,8(firstThird.png)
Code: Select all
#SCR1,0,16,32,8(thirdThird.png)
Re: SkoolKit: ctl file or skool file?
Found what appears to be another bug.
Maybe I should be reporting these elsewhere, but I'm not sure exactly were, so please enlighten me so that I can report them on the proper place.
So when processing "Formula One.TZX", the tapInfo.py correctly reports all the blocks.
As defined as supported by documentation, all data blocks are of type 0x10, except the initial description block (0x30).
However, when running tap2sna.py, it generates a very small (1279 bytes) but incomplete file.
NOTE: I did unzip the file to avoid any possible issues while unzipping. But the result is exactly the same.
Maybe I should be reporting these elsewhere, but I'm not sure exactly were, so please enlighten me so that I can report them on the proper place.
So when processing "Formula One.TZX", the tapInfo.py correctly reports all the blocks.
Code: Select all
> tapinfo.py "Formula One.tzx"
Version: 1.10
1: Text description (0x30)
Text: Created with Ramsoft MakeTZX
2: Standard speed data (0x10)
Type: Header block
Bytes: FORMULA 1
CODE: 23296,1000
Length: 19
Data: 0, 3, 70, 79, 82, 77, 85 ... 232, 3, 0, 91, 0, 128, 76
3: Standard speed data (0x10)
Type: Data block
Length: 1002
Data: 255, 62, 1, 50, 9, 92, 50 ... 0, 0, 0, 0, 0, 0, 49
4: Standard speed data (0x10)
Type: Data block
Length: 7202
Data: 255, 0, 8, 8, 8, 8, 8 ... 0, 0, 0, 0, 0, 0, 90
5: Standard speed data (0x10)
Type: Data block
Length: 38936
Data: 255, 205, 154, 233, 0, 0, 0 ... 96, 96, 96, 96, 96, 96, 129
However, when running tap2sna.py, it generates a very small (1279 bytes) but incomplete file.
Code: Select all
> tap2sna.py "Formula One.tzx" "F1.z80"
Re: SkoolKit: ctl file or skool file?
Another issue that I have, is that we start with a CTL file, then lets suppose we edit the SKOOL file (either file will do).RMartins wrote: ↑Sun Aug 14, 2022 1:32 am The CTL file looks like a blueprint of where code and data are supposed to be.
However, this is the first thing we NEED to do, which is most probably not the best scenario when we don't know anything about the code or data.
...
All in all, I see the CTL as a base reference to build a kind of node graph, that should be used by SKOOL file, but not to be replaced or embed its definition into the skool file. Because if the data is embedded, then we have issues to convert back and forth.
So far so good, but now, we have another run of the game or we found an RZX, that we want to use to generate a new CTL file.
Even if we keep the SKOOL file to retain all the descriptions/editing documentation work, how are we going to merge this ?
Splitting it using start/end sections, generating a new distinct SKOOL file and editing/merging in the previous SKOOL file, all by hand ?
Re: SkoolKit: ctl file or skool file?
The place to report bugs is on GitHub.
That's expected, because the TAP file contains headerless blocks. By default, tap2sna.py ignores headerless blocks because it cannot know where they should be loaded.However, when running tap2sna.py, it generates a very small (1279 bytes) but incomplete file.Code: Select all
> tap2sna.py "Formula One.tzx" "F1.z80"
To make tap2sna.py load a headerless block at a specific address, use the '--ram load' option. Run
Code: Select all
tap2sna.py --ram help
Re: SkoolKit: ctl file or skool file?
Can you share the skool file that you're using? I've just tried those #SCR macros on a loading screen of Hungry Horace, and they worked fine.RMartins wrote: ↑Sun Aug 14, 2022 2:21 am By the way, using SkoolKit 8.6, I seem to have found a bug
This Macro doesn't produce the expected result (the output of the second third of the screenIt comes close, but has a few problems:Code: Select all
#SCR1,0,8,32,8(secondThird.png)
- Some of the pixels and full lines, are missing
- Some colors Also seem off (but could be due to missing pixels).
[snip]
Re: SkoolKit: ctl file or skool file?
I'm not really sure what to say about all this, other than it sounds as if you're working on the skool file far too soon. It's a good idea to work on the control file exclusively in the beginning until you think you've identified all (or almost all) of the code, data and text. Then you can either carry on with the control file (as most SkoolKit users seem to prefer), or switch to editing the skool file, with the understanding that some manual conversion of code to data or vice versa might occasionally be necessary.RMartins wrote: ↑Sun Aug 14, 2022 11:44 am Another issue that I have, is that we start with a CTL file, then lets suppose we edit the SKOOL file (either file will do).
So far so good, but now, we have another run of the game or we found an RZX, that we want to use to generate a new CTL file.
Even if we keep the SKOOL file to retain all the descriptions/editing documentation work, how are we going to merge this ?
Splitting it using start/end sections, generating a new distinct SKOOL file and editing/merging in the previous SKOOL file, all by hand ?
As I've said before, that's how I've done most of my disassemblies. I like editing the skool file directly because the instructions are right there, so it's easier to write annotations "in place". But that approach does require a decent familiarity with the skool file format, and perhaps that's why most people stick with the control file and its somewhat simpler syntax.