SkoolKit: ctl file or skool file?

The place for codemasters or beginners to talk about programming any language for the Spectrum.

When developing a disassembly using SkoolKit, which source file do you work on most?

I mostly edit and add annotations to the control (ctl) file
6
40%
I mostly edit and add annotations to the skool file
5
33%
I'm an equal opportunities disassembly developer and I will edit either file as the need arises
3
20%
What's SkoolKit?
1
7%
 
Total votes: 15

User avatar
RMartins
Manic Miner
Posts: 776
Joined: Thu Nov 16, 2017 3:26 pm

Re: SkoolKit: ctl file or skool file?

Post by RMartins »

SkoolKid wrote: Sun Aug 14, 2022 1:00 pm The place to report bugs is on GitHub.
OK, Thanks.
SkoolKid wrote: Sun Aug 14, 2022 1:00 pm That's expected, because the TAP file contains headerless blocks. By default, tap2sna.py ignores headerless blocks because it cannot know where they should be loaded.

To make tap2sna.py load a headerless block at a specific address, use the '--ram load' option. Run

Code: Select all

tap2sna.py --ram help
for more info.
OK, Thanks for the clarification.

But a Warning or status message, indicating that it skipped a block, would be nice.
User avatar
RMartins
Manic Miner
Posts: 776
Joined: Thu Nov 16, 2017 3:26 pm

Re: SkoolKit: ctl file or skool file?

Post by RMartins »

SkoolKid wrote: Sun Aug 14, 2022 1:09 pm Can you share the skool file that you're using? I've just tried those #SCR macros on a loading screen of Hungry Horace, and they worked fine.
Image
You need a more detailed image to notice the missing bits.

I noticed it using the Formula One Z80 file.

Image
But you can use something even more detailed, like Cybernoid II the Revenge.

But if you really need the SKOOL file, send me a message.
Last edited by RMartins on Sun Aug 14, 2022 3:46 pm, edited 1 time in total.
User avatar
RMartins
Manic Miner
Posts: 776
Joined: Thu Nov 16, 2017 3:26 pm

Re: SkoolKit: ctl file or skool file?

Post by RMartins »

SkoolKid wrote: Sun Aug 14, 2022 1:21 pm I'm not really sure what to say about all this, other than it sounds as if you're working on the skool file far too soon. It's a good idea to work on the control file exclusively in the beginning until you think you've identified all (or almost all) of the code, data and text. Then you can either carry on with the control file (as most SkoolKit users seem to prefer), or switch to editing the skool file, with the understanding that some manual conversion of code to data or vice versa might occasionally be necessary.

As I've said before, that's how I've done most of my disassemblies. I like editing the skool file directly because the instructions are right there, so it's easier to write annotations "in place". But that approach does require a decent familiarity with the skool file format, and perhaps that's why most people stick with the control file and its somewhat simpler syntax.
I have been editing the SKOOL file, but the most important issue I see is that no matter what, this is and always will be an iterative process.
Which means that we need to go back and forth.

I also noticed that if I use any curly braces to group description comments ( "{", "}" ), everytime I run skool2ctl I loose the formating by lines, since it assumes that this is continuous text, which I understand the idea, but in practice is not very useful. since we are editing by line and we loose that.
It's probably because in HTML, a single TD with "rowspan" is being used and hence everything will be concatenated implicitly.

Here is a sample extract from a git diff, to explain it in detail.

Code: Select all

- $9EDE PUSH HL       ; Save    screen pixel addr
-*$9EDF LD A,(DE)     ; { get data from data addr
- $9EE0 LD C,A        ; Save data in C
- $9EE1 LD A,($969F)  ; variable b?
- $9EE4 CP $00        ; compute flags for decision
- $9EE6 LD A,C        ; Restore data in A
- $9EE7 JR Z,$9EEC    ; Skip shift_or operation if #R$969F == 0
- $9EE9 SRL A         ; Shift pixel data right
- $9EEB OR C          ; OR original data with shifted data
-*$9EEC LD (HL),A     ; Draw on screen
- $9EED INC DE        ; Advance data addr
- $9EEE INC H         ; Advance pixel line addr
+ $9EDE PUSH HL       ; Save    screen pixel addr
+*$9EDF LD A,(DE)     ; {get data from data addr Save data in C variable b?
+ $9EE0 LD C,A        ; compute flags for decision Restore data in A Skip
+ $9EE1 LD A,($969F)  ; shift_or operation if #R$969F == 0 Shift pixel data
+ $9EE4 CP $00        ; right OR original data with shifted data Draw on screen
+ $9EE6 LD A,C        ; Advance data addr Advance pixel line addr
+ $9EE7 JR Z,$9EEC    ;
+ $9EE9 SRL A         ;
+ $9EEB OR C          ;
+*$9EEC LD (HL),A     ;
+ $9EED INC DE        ;
+ $9EEE INC H         ;
I use this to group the code sections in small logic groups, but If I make specific notes on specific lines, that formatting per line gets lost.

Code: Select all

c$96C9 PUSH AF       ; {Save Registers
 $96CA PUSH BC       ;
 $96CB PUSH DE       ;
 $96CC PUSH HL       ; }
 $96CD LD HL,$6F95   ; Load #R$6F95 table
 $96D0 LD B,$08      ; Constructor Name length
 $96D2 INC C         ;
 $96D3 SRL C         ;
 $96D5 INC C         ;
 $96D6 CALL $9712    ;
 $96D9 LD A,$00      ;
 $96DB LD C,$08      ;
 $96DD CALL $9728    ;
 $96E0 POP HL        ; {Restore Registers
 $96E1 POP DE        ;
 $96E2 POP BC        ;
 $96E3 POP AF        ; }
 $96E4 RET           ;
But OK, it's a feature that once I know where it can be applied, I can use only where it fits the situation.
User avatar
SkoolKid
Manic Miner
Posts: 407
Joined: Wed Nov 15, 2017 3:07 pm

Re: SkoolKit: ctl file or skool file?

Post by SkoolKid »

RMartins wrote: Sun Aug 14, 2022 3:30 pm But a Warning or status message, indicating that it skipped a block, would be nice.
Good idea. I'll add that to my TODO list for the next version (currently in development).

I'm still not sure what you mean with the #SCR macro. Can you post an image produced by the #SCR macro, and explain how it's broken?

As for braces in comments, yes, you have to be careful with those, because (as you've noticed) the skool file uses them to apply comments to groups of instructions as opposed to individual instructions.

Can you give an explicit example of how you lose the formatting when you run skool2ctl.py? I don't understand the example you've given, and I want to make sure it's not a bug.
SkoolKit - disassemble a game today
Pyskool - a remake of Skool Daze and Back to Skool
User avatar
RMartins
Manic Miner
Posts: 776
Joined: Thu Nov 16, 2017 3:26 pm

Re: SkoolKit: ctl file or skool file?

Post by RMartins »

SkoolKid wrote: Sun Aug 14, 2022 4:24 pm ...

I'm still not sure what you mean with the #SCR macro. Can you post an image produced by the #SCR macro, and explain how it's broken?
Basically, there is a bunch of pixel bytes that are set as if they are #00 or garbled data, in some specific locations.

But I have just checked and the current skool file version, seems to be generating the correct output.
So I probably did something strange before, that triggered something unexpected.

I'll revert back to one of the previous versions and see if I can get it again, and I'll report back.
SkoolKid wrote: Sun Aug 14, 2022 4:24 pm As for braces in comments, yes, you have to be careful with those, because (as you've noticed) the skool file uses them to apply comments to groups of instructions as opposed to individual instructions.

Can you give an explicit example of how you lose the formatting when you run skool2ctl.py? I don't understand the example you've given, and I want to make sure it's not a bug.
If you take a peek at the git diff above, it's simple to see, the lines that start with "-" have been removed, and the lines with "+ have been added new.
But summing it all up the issue is that before I had:

Code: Select all

; { get data from data addr
; Save data in C
; variable b?
; compute flags for decision
; Restore data in A
; Skip shift_or operation if #R$969F == 0
; Shift pixel data right
; OR original data with shifted data
; Draw on screen
; Advance data addr
; Advance pixel line addr }
And now I get the descriptions like if it was a single line, that was later wrapped to some specific pre-determined width:

Code: Select all

; {get data from data addr Save data in C variable b?
; compute flags for decision Restore data in A Skip
; shift_or operation if #R$969F == 0 Shift pixel data
; right OR original data with shifted data Draw on screen
; Advance data addr Advance pixel line addr}
But according to the documentation and expected HTML output, this seems to be correct behaviour.

I also noticed that I loose every whitespace I place after the openning curly brace "{". And that wasn't expected, but can be a side effect of the wrapping being done.

NOTE: Html preserves spaces, except if they are more than one in a row, or if we use the special PRE tag, which keeps everything (not the case here).
User avatar
SkoolKid
Manic Miner
Posts: 407
Joined: Wed Nov 15, 2017 3:07 pm

Re: SkoolKit: ctl file or skool file?

Post by SkoolKid »

RMartins wrote: Sun Aug 14, 2022 4:39 pm But according to the documentation and expected HTML output, this seems to be correct behaviour.
Yes, that looks to me like curly braces doing what they're supposed to do. What were you trying to achieve with those braces originally?

By the way, I'm not sure if this will help at all, but skool2ctl.py has a --keep-lines option that preserves line breaks in comments. So when you run sna2skool.py, the comments will not be re-wrapped.
SkoolKit - disassemble a game today
Pyskool - a remake of Skool Daze and Back to Skool
User avatar
RMartins
Manic Miner
Posts: 776
Joined: Thu Nov 16, 2017 3:26 pm

Re: SkoolKit: ctl file or skool file?

Post by RMartins »

Let's suppose that I have this extract in the Skool file

Code: Select all

 $F5E6 DEFB $FF,$FF,$FF,$FF,$FF,$FF,$FF,$FF ;
 $F5EE DEFB $FF,$FF,$FF,$FF,$FF,$FF,$FF,$FF ;
 $F5F6 DEFB $FF,$FF,$FF,$FF,$FF,$FF,$FF,$FF ; }
; Pit Garage SCR attributes data
@label=PIT_GARAGE_ATTR
 $F5FE DEFB $78,$79,$7A,$79,$79,$79,$78,$47
 $F606 DEFB $47,$47,$47,$47,$47,$47,$47,$47
 $F60E DEFB $47,$47,$47,$47,$47,$47,$4F,$4F
Since I want both the Pixels and Attributes to be on the same HTML page, I don't add a breaking line between the 2 sections.

However, by looking at a generated ASM listing, only the values that have an associated page are provided with an appropriate link.

Code: Select all

DrawPitGarage 	B54A 	LD A,($83A5) 	
		B54D 	CP $00 	
		B54F 	JR Z,DrawPitGarage_0 	Skip next block if variable_0_8bit is ZERO
		B551 	LD A,($6C14) 	
		B554 	INC A 	
		B555 	CALL $A718 	
		B558 	CALL $A691 	
		B55B 	CALL $A65B 	
DrawPitGarage_0	B55E 	LD HL,$4000 	Clear entire screen pixels to $00
		B561 	LD DE,$4001
		B564 	XOR A
		B565 	LD BC,$17FF
		B568 	LD (HL),A
		B569 	LDIR
		B56B 	LD HL,$5800 	Clear entire screen attributes to (White PAPER, Black INK) = $38
		B56E 	LD DE,$5801
		B571 	LD BC,$02FF
		B574 	LD (HL),$38
		B576 	LDIR
		B578 	LD HL,$EDFE 	Set pixel data to PIT_GARAGE_PX
	 	B57B 	LD DE,$4000 	Screen Pixel addr
		B57E 	LD BC,$0800 	Copy a full screen third
		B581 	LDIR 	
All the CALLs have an HTML link on the address being called ($A718, $A691, $A65B), which works great.

But even though the variable ($EDFE) I defined as PIT_GARAGE_PX is on the start of a page and has a label defined (PIT_GARAGE_PX), the asm instructions where the variable is used, don't get a proper link on the variable address.

However, using the reference #R$EDFE in the comments, correctly names de variable.

What am I missing ?

Also, to better support references into labels that are within an ASM page, like "DrawPitGarage_0" (code) or "PIT_GARAGE_PX" (data), it would be nice to have support for HTML links with anchors (like: ".../page.html#label") to the inner labels, defined or contained within a specific page.

NOTE: Assuming this is not supported yet, since I haven't seen it anywhere yet.

This is specially useful, for functions that have several entry points, but which the global function we want it to be on a single page, with all the several entry points it might have, while still being able to highlight all those calls with the proper label name and pointing to the proper HTML page and anchor.
Last edited by RMartins on Sun Aug 14, 2022 7:22 pm, edited 7 times in total.
User avatar
RMartins
Manic Miner
Posts: 776
Joined: Thu Nov 16, 2017 3:26 pm

Re: SkoolKit: ctl file or skool file?

Post by RMartins »

SkoolKid wrote: Sun Aug 14, 2022 5:04 pm By the way, I'm not sure if this will help at all, but skool2ctl.py has a --keep-lines option that preserves line breaks in comments. So when you run sna2skool.py, the comments will not be re-wrapped.
Great!

I'll have to try that.
Thanks.
User avatar
SkoolKid
Manic Miner
Posts: 407
Joined: Wed Nov 15, 2017 3:07 pm

Re: SkoolKit: ctl file or skool file?

Post by SkoolKid »

RMartins wrote: Sun Aug 14, 2022 5:32 pm All the CALLs have an HTML link on the address being called ($A718, $A691, $A65B), which works great.

But even though the variable ($EDFE) I defined as PIT_GARAGE_PX is on the start of a page and has a label defined (PIT_GARAGE_PX), the asm instructions where the variable is used, don't get a proper link on the variable address.

However, using the reference #R$EDFE in the comments, correctly names de variable.
If you're referring to the fact that the operands of LD instructions are neither hyperlinked nor labelled, that is the expected default behaviour. You can add LD instructions to the list of those that are hyperlinked by setting the LinkOperands parameter in the [Game] section of the ref file, like this:

Code: Select all

LinkOperands=CALL,DEFW,DJNZ,JP,JR,LD
The reason for leaving LD off this list by default is that it can create hyperlinks where they are inappropriate (e.g. when the operand of a LD instruction is a pure data value that is coincidentally equal to the address of a labelled instruction). You can use the @keep directive to remove such inappropriate hyperlinks, but that's another story.
Also, to better support references into labels that are within an ASM page, like "DrawPitGarage_0" (code) or "PIT_GARAGE_PX" (data), it would be nice to have support for HTML links with anchors (like: ".../page.html#label") to the inner labels, defined or contained within a specific page.

NOTE: Assuming this is not supported yet, since I haven't seen it anywhere yet.
The #R macro already supports linking to entry points or labelled instructions within a routine, so I'm not sure what you're getting at here. Are you saying you'd prefer to have anchor names that match the labels instead of (or in addition to) anchor names that match the instruction addresses? If so, you could do that by modifying the asm HTML template. Let me know if you want any help with that.
SkoolKit - disassemble a game today
Pyskool - a remake of Skool Daze and Back to Skool
User avatar
RMartins
Manic Miner
Posts: 776
Joined: Thu Nov 16, 2017 3:26 pm

Re: SkoolKit: ctl file or skool file?

Post by RMartins »

SkoolKid wrote: Sun Aug 14, 2022 8:04 pm If you're referring to the fact that the operands of LD instructions are neither hyperlinked nor labelled, that is the expected default behaviour.
Yep that's exactly what I was reporting, since in another thread about disassembly of Chaos, I saw the LD instructions with hyperlinks.
SkoolKid wrote: Sun Aug 14, 2022 8:04 pm You can add LD instructions to the list of those that are hyperlinked by setting the LinkOperands parameter in the [Game] section of the ref file, like this:

Code: Select all

LinkOperands=CALL,DEFW,DJNZ,JP,JR,LD
I new I had to be missing something.
SkoolKid wrote: Sun Aug 14, 2022 8:04 pm The reason for leaving LD off this list by default is that it can create hyperlinks where they are inappropriate (e.g. when the operand of a LD instruction is a pure data value that is coincidentally equal to the address of a labelled instruction). You can use the @keep directive to remove such inappropriate hyperlinks, but that's another story.
Shouldn't they only create a link, if there is an actual label defined with that value ?

I hope that only 16 bit LD instructions support linking (after activating the configuration you mention above), since 8 bits wouldn't make sense, because it must always be an address to reference memory, hence 16 bit is required.
SkoolKid wrote: Sun Aug 14, 2022 8:04 pm The #R macro already supports linking to entry points or labelled instructions within a routine, so I'm not sure what you're getting at here. Are you saying you'd prefer to have anchor names that match the labels instead of (or in addition to) anchor names that match the instruction addresses? If so, you could do that by modifying the asm HTML template. Let me know if you want any help with that.
Anything that defines a memory location is a potential target for linking (with anchor or not).

Labels are obvious memory location definitions, hence it should be possible to create hyperlinks when a label is present, either to a page or page Anchor.
Because NOT all labels are routines, they can be data too, which is also useful to click through.

For example, if I have a sprite, with Pixel Data and Attr data, these can both be on the same HTML page, but one may want to inspect its details coming from an ASM page, which will include the actual data and eventual images or other representation that we might have defined.

Practical example

Code: Select all

 	B578 	LD HL,$EDFE 	Set pixel data to PIT_GARAGE_PX
	B57B 	LD DE,$4000 	Screen Pixel addr
	B57E 	LD BC,$0800 	Copy a full screen third
	B581 	LDIR 	
	B583 	LD HL,$F5FE 	Set Attribute data to PIT_GARAGE_ATTR
	B586 	LD DE,$5800 	Screen Attr addr
	B589 	LD BC,$0100 	copy a full screen third of attributes
	B58C 	LDIR
Once LD linking is enabled, It's preferable to see this instead:

Code: Select all

	B578 	LD HL,PIT_GARAGE_PX
	B57B 	LD DE,SCRN
...
	B583 	LD HL,PIT_GARAGE_ATTR
	B586 	LD DE,ATTR
...

User avatar
SkoolKid
Manic Miner
Posts: 407
Joined: Wed Nov 15, 2017 3:07 pm

Re: SkoolKit: ctl file or skool file?

Post by SkoolKid »

RMartins wrote: Mon Aug 15, 2022 4:34 am Shouldn't they only create a link, if there is an actual label defined with that value ?
Suppose you have a label defined at $8000 (for example). And then in a routine somewhere there is a bit of code that performs a delay:

Code: Select all

$9000 LD HL,$8000
$9003 DEC HL
$9004 LD A,H
$9005 OR L
$9006 JR NZ,$9003
You probably don't want the operand of the 'LD HL,$8000' instruction hyperlinked to the instruction at address $8000, because $8000 is a pure data value that is just coincidentally equal to the labelled address.
I hope that only 16 bit LD instructions support linking (after activating the configuration you mention above), since 8 bits wouldn't make sense, because it must always be an address to reference memory, hence 16 bit is required.
That is correct.
Once LD linking is enabled, It's preferable to see this instead:

Code: Select all

	B578 	LD HL,PIT_GARAGE_PX
	B57B 	LD DE,SCRN
...
	B583 	LD HL,PIT_GARAGE_ATTR
	B586 	LD DE,ATTR
...

You should see this after adding LD to the LinkOperands parameter. Is it not working?
SkoolKit - disassemble a game today
Pyskool - a remake of Skool Daze and Back to Skool
User avatar
RMartins
Manic Miner
Posts: 776
Joined: Thu Nov 16, 2017 3:26 pm

Re: SkoolKit: ctl file or skool file?

Post by RMartins »

RMartins wrote: Sun Aug 14, 2022 4:39 pm Basically, there is a bunch of pixel bytes that are set as if they are #00 or garbled data, in some specific locations.

But I have just checked and the current skool file version, seems to be generating the correct output.
So I probably did something strange before, that triggered something unexpected.

I'll revert back to one of the previous versions and see if I can get it again, and I'll report back.
OK, I think I know what was happening when the image was missing pixels.

Basically, I still had a bunch of badly setup bytes, that were still defined as code, but were in fact data.
Since the Skool file doesn't have access to the snapshot data anymore and only uses the data available in the skool file itself, and since the default settings for the @assemble directive equals:

Code: Select all

@assemble=1,0
This means that all areas marked erroneously as code, are in fact ZERO, hence why the pixels were missing.

TIP: Skool file really needs a hook/reference into the snapshot file to avoid these unexpected behaviours.
User avatar
SkoolKid
Manic Miner
Posts: 407
Joined: Wed Nov 15, 2017 3:07 pm

Re: SkoolKit: ctl file or skool file?

Post by SkoolKid »

RMartins wrote: Tue Aug 16, 2022 12:32 am TIP: Skool file really needs a hook/reference into the snapshot file to avoid these unexpected behaviours.
Since SkoolKit 1.0, the point of the skool file has been that it will "just work" as the source for skool2asm.py, skool2html.py (or any other skool2* command) without any reference to a particular snapshot, and that is not going to change. Requiring the recipient of a skool file to also have a snapshot that is byte-for-byte identical to the one used by the disassembly developer would be an unnecessary burden.

If your skool file has assembly language instructions instead of DEFB statements in the display file or attribute file, then the #SCR macro producing broken images is exactly the expected behaviour. If the assembly language instructions must be there for some reason, you can use the @assemble directive to convert those assembly language instructions into byte values. But the more sensible option (in almost all cases) is to use the control file to disassemble the display file and attribute file as data instead of code.
SkoolKit - disassemble a game today
Pyskool - a remake of Skool Daze and Back to Skool
User avatar
RMartins
Manic Miner
Posts: 776
Joined: Thu Nov 16, 2017 3:26 pm

Re: SkoolKit: ctl file or skool file?

Post by RMartins »

SkoolKid wrote: Tue Aug 16, 2022 1:01 am Since SkoolKit 1.0, the point of the skool file has been that it will "just work" as the source for skool2asm.py, skool2html.py (or any other skool2* command) without any reference to a particular snapshot, and that is not going to change. Requiring the recipient of a skool file to also have a snapshot that is byte-for-byte identical to the one used by the disassembly developer would be an unnecessary burden.
I agree with you, in the sense when the file is completely disassembled and hence can be used to generate assembly output or html documentation.
But that is the end goal. Meanwhile, while working with focus to arrive at the end goal, the particular nuances of the implementation get in the way of the actual work.

Once we get enough knowledge how it works, since it has quite a steep learning curve, then we can handle all that jazz.

But I would say that a tool that is friendlier for the beginners, without making us loose hours trying to understand why something is not working as we expect it to, even after reading the documentation, would be a nicer goal.

And don't get me wrong, from what I have read and experimented, this is a very powerful tool, very configurable and flexible (using Macros) and something that is very welcome.
But with great power comes some complexity.
SkoolKid wrote: Tue Aug 16, 2022 1:01 am If your skool file has assembly language instructions instead of DEFB statements in the display file or attribute file, then the #SCR macro producing broken images is exactly the expected behaviour. If the assembly language instructions must be there for some reason, you can use the @assemble directive to convert those assembly language instructions into byte values. But the more sensible option (in almost all cases) is to use the control file to disassemble the display file and attribute file as data instead of code.
Actually for Formula One, there are 2 big sections that have entire full screen thirds in memory for the Garage and Race screens.
So I had to make a bit of "magic" to use the SCR in order to read one of those thirds since it was not the first third (SCR expects an entire full screen), which is fine, since the tool actually delivered, once correctly configured.

What keeps hampering my progress, is when I try to make something that doesn't have a perfect code versus data definition, I keep stumbling into unexpected behaviour because of the fact that skool file doesn't know about the real snapshot.

And the real kicker is that the tool doesn't complain, when it's reading snapshot data that DOES NOT EXIST in the skool file. :twisted:
So we are left to wounder, until we understand how the tool is working underneath.
Which takes us back to the steep learning curve.

And from what I have gathered so far it's mostly due to the fact that we need to know how the tool is actually implemented, to use stuff that apparently we shouldn't be required to know how it works underneath.

Resuming, it's not very user friendly to new comers.

However, you have been promptly answering my questions and clarifying my doubts.
I can only sincerely thank you for that. :ugeek:

And for the wonderful tool that you have implemented.
I'm pretty sure that this was and still is an herculean task.
User avatar
SkoolKid
Manic Miner
Posts: 407
Joined: Wed Nov 15, 2017 3:07 pm

Re: SkoolKit: ctl file or skool file?

Post by SkoolKid »

@RMartins, there is one important piece of advice I don't think I've properly spelled out yet, so here goes.

I've got the impression from your posts in this thread, and from the issues you've raised on GitHub, that you are editing the skool file to convert parts of the disassembly between code and/or data and/or text, manually. I highly recommend that you stop doing that, and start editing the control file for that purpose instead. That's what the control file is for.

Manually converting between code, data and text in the skool file is cumbersome, tedious and error-prone. You are not a disassembler. Typos will be made, addresses will be wrong, and these and other mistakes will build up to the point that skool2html.py will just choke on the mess.

Converting between code, data and text in the control file, however, is trivially easy in comparison. In most cases you will just be replacing one letter - c (or C) for code, b (or B) for byte values, t (or T) for text - with another. And sna2skool.py is a disassembler. Let it do the disassembling for you. It also knows the syntax of skool files much better than you do. :)

In short, if you find yourself editing the skool file for any reason other than adding annotations (e.g. routine descriptions, instruction comments), you should pause and consider whether editing the control file would make more sense.
SkoolKit - disassemble a game today
Pyskool - a remake of Skool Daze and Back to Skool
User avatar
RMartins
Manic Miner
Posts: 776
Joined: Thu Nov 16, 2017 3:26 pm

Re: SkoolKit: ctl file or skool file?

Post by RMartins »

SkoolKid wrote: Tue Aug 16, 2022 1:11 pm @RMartins, there is one important piece of advice I don't think I've properly spelled out yet, so here goes.

I've got the impression from your posts in this thread, and from the issues you've raised on GitHub, that you are editing the skool file to convert parts of the disassembly between code and/or data and/or text, manually. I highly recommend that you stop doing that, and start editing the control file for that purpose instead. That's what the control file is for.

Manually converting between code, data and text in the skool file is cumbersome, tedious and error-prone. You are not a disassembler. Typos will be made, addresses will be wrong, and these and other mistakes will build up to the point that skool2html.py will just choke on the mess.
To learn, we need to make mistakes.
SkoolKid wrote: Tue Aug 16, 2022 1:11 pm Converting between code, data and text in the control file, however, is trivially easy in comparison. In most cases you will just be replacing one letter - c (or C) for code, b (or B) for byte values, t (or T) for text - with another. And sna2skool.py is a disassembler. Let it do the disassembling for you. It also knows the syntax of skool files much better than you do. :)

In short, if you find yourself editing the skool file for any reason other than adding annotations (e.g. routine descriptions, instruction comments), you should pause and consider whether editing the control file would make more sense.
That is what I have been doing lately, but only after I got the conversion loop (skool2ctl.py <-> sna2Skool.py) working, without major differences.

Using git diff after doing a conversion, was fundamental to catch weird problems in the file definitions.

So, now, when I need to change the type of a memory area e loop back to the CTL, make my changes, and loop forward to the skool file again.
dfzx
Manic Miner
Posts: 683
Joined: Mon Nov 13, 2017 6:55 pm
Location: New Forest, UK
Contact:

Re: SkoolKit: ctl file or skool file?

Post by dfzx »

RMartins wrote: Tue Aug 16, 2022 11:02 pm That is what I have been doing lately, but only after I got the conversion loop (skool2ctl.py <-> sna2Skool.py) working, without major differences.
That's how I ended up. I wrote Makefile rules to do the conversion. I think it would benefit from a "perfect conversion" flag where it preserved the text, whitespace included, to ensure absolutely nothing gets lost. It might not be practical though, I'm not sure.
RMartins wrote: Tue Aug 16, 2022 11:02 pm Using git diff after doing a conversion, was fundamental to catch weird problems in the file definitions.
Ha! I also did exactly that on many occasions!
RMartins wrote: Tue Aug 16, 2022 11:02 pm So, now, when I need to change the type of a memory area e loop back to the CTL, make my changes, and loop forward to the skool file again.
We clearly took the same path and ended up in the same place. :)
Derek Fountain, author of the ZX Spectrum C Programmer's Getting Started Guide and various open source games, hardware and other projects, including an IF1 and ZX Microdrive emulator.
dfzx
Manic Miner
Posts: 683
Joined: Mon Nov 13, 2017 6:55 pm
Location: New Forest, UK
Contact:

Re: SkoolKit: ctl file or skool file?

Post by dfzx »

Something else I could throw in here is how handy a specific error/warning would be if the conversion tools found a line starting with a colon or a period. (I assume there's no valid use case for those?) Creating such a typo is quite easy, and extremely hard to spot among a sea of semicolons! There were a few occasions where I spent ages staring at my skool file unable to work out why it was failing on a nearby line...
Derek Fountain, author of the ZX Spectrum C Programmer's Getting Started Guide and various open source games, hardware and other projects, including an IF1 and ZX Microdrive emulator.
User avatar
SkoolKid
Manic Miner
Posts: 407
Joined: Wed Nov 15, 2017 3:07 pm

Re: SkoolKit: ctl file or skool file?

Post by SkoolKid »

dfzx wrote: Wed Aug 17, 2022 10:45 am I think it would benefit from a "perfect conversion" flag where it preserved the text, whitespace included, to ensure absolutely nothing gets lost. It might not be practical though, I'm not sure.
It's not practical any more, but you probably would have enjoyed skool file templates, which I removed in SkoolKit 8.0!

The only reason I kept skool file templates around for so long is that they could preserve certain ASM directives (specifically, ASM block directives) that control files couldn't. That is still the case, but in the 7.x series I worked on making sure that ASM block directives are redundant, which in turn made skool file templates redundant.

While a control file will not preserve the exact formatting of a skool file, it should preserve the essence of the skool file. That is, the output of skool2asm.py or skool2html.py should not change after putting your skool file on a round trip through skool2ctl.py. Exact preservation of whitespace, or the alignment of semicolons, is not required for that, so it's best to let go of that idea. I actually use the skool2ctl.py round trip technique to tidy up the formatting in my skool files and make it consistent when preparing a release.
SkoolKit - disassemble a game today
Pyskool - a remake of Skool Daze and Back to Skool
User avatar
SkoolKid
Manic Miner
Posts: 407
Joined: Wed Nov 15, 2017 3:07 pm

Re: SkoolKit: ctl file or skool file?

Post by SkoolKid »

dfzx wrote: Wed Aug 17, 2022 10:49 am Something else I could throw in here is how handy a specific error/warning would be if the conversion tools found a line starting with a colon or a period. (I assume there's no valid use case for those?) Creating such a typo is quite easy, and extremely hard to spot among a sea of semicolons! There were a few occasions where I spent ages staring at my skool file unable to work out why it was failing on a nearby line...
IOW (well, two): Syntax highlighting! (Don't try to edit a skool file, ref file, or control file without it.)

The skool parser (used by both skool2asm.py and skool2html.py) just ignores any line that starts with an invalid character (e.g. a colon or full stop). You can (ab)use this "feature" to place "invisible comments" in the skool file, but I wouldn't officially recommend it. :)

Perhaps a "pedantic" mode is in order, in which the skool parser brooks no such shenanigans and immediately aborts with an error message. I'll put that on my TODO list for future consideration.
SkoolKit - disassemble a game today
Pyskool - a remake of Skool Daze and Back to Skool
User avatar
Morkin
Bugaboo
Posts: 3277
Joined: Mon Nov 13, 2017 8:50 am
Location: Bristol, UK

Re: SkoolKit: ctl file or skool file?

Post by Morkin »

SkoolKid wrote: Wed Aug 17, 2022 1:53 pm Syntax highlighting! (Don't try to edit a skool file, ref file, or control file without it.)
Wait, you guys are using text highlighters?

I was using Notepad++ with no highlighting or text colours whatsoever :cry:
My Speccy site: thirdharmoniser.com
dfzx
Manic Miner
Posts: 683
Joined: Mon Nov 13, 2017 6:55 pm
Location: New Forest, UK
Contact:

Re: SkoolKit: ctl file or skool file?

Post by dfzx »

Morkin wrote: Wed Aug 17, 2022 2:59 pm Wait, you guys are using text highlighters?

I was using Notepad++ with no highlighting or text colours whatsoever :cry:
I wasn't using a syntax highlighter either! I use XEmacs on Linux! :)
Derek Fountain, author of the ZX Spectrum C Programmer's Getting Started Guide and various open source games, hardware and other projects, including an IF1 and ZX Microdrive emulator.
User avatar
SkoolKid
Manic Miner
Posts: 407
Joined: Wed Nov 15, 2017 3:07 pm

Re: SkoolKit: ctl file or skool file?

Post by SkoolKid »

dfzx wrote: Wed Aug 17, 2022 3:03 pm I wasn't using a syntax highlighter either! I use XEmacs on Linux! :)
I use original (GNU) emacs on Linux. Is XEmacs compatible? If so, you can try copies of my skool-mode.el and ref-mode.el files.

(Turns out I don't have a ctl-mode.el file. Maybe control files really are just too simple to need one. :P )
SkoolKit - disassemble a game today
Pyskool - a remake of Skool Daze and Back to Skool
dfzx
Manic Miner
Posts: 683
Joined: Mon Nov 13, 2017 6:55 pm
Location: New Forest, UK
Contact:

Re: SkoolKit: ctl file or skool file?

Post by dfzx »

SkoolKid wrote: Wed Aug 17, 2022 3:09 pm I use original (GNU) emacs on Linux. Is XEmacs compatible? If so, you can try copies of my skool-mode.el and ref-mode.el files.
Have you got a link to the EL files? They don't appear to be in the github repo.
Derek Fountain, author of the ZX Spectrum C Programmer's Getting Started Guide and various open source games, hardware and other projects, including an IF1 and ZX Microdrive emulator.
User avatar
SkoolKid
Manic Miner
Posts: 407
Joined: Wed Nov 15, 2017 3:07 pm

Re: SkoolKit: ctl file or skool file?

Post by SkoolKid »

dfzx wrote: Wed Aug 17, 2022 4:24 pm Have you got a link to the EL files? They don't appear to be in the github repo.
Try this (limited time only):

https://skoolkit.ca/skoolkit-emacs.zip
SkoolKit - disassemble a game today
Pyskool - a remake of Skool Daze and Back to Skool
Post Reply