ChatGPT

[patters]

I decided to try ChatGPT this week for some work coding tasks and found it to be useful only for providing the outline of an approach. The code it wrote was pretty awful. In most cases it produced non-working code, and while correcting errors would randomly forget some functional requirements.

As I grew increasingly skeptical of its ability I decided to ask it about Sinclair BASIC:




It tried to show off, offering me something I didn't ask for, and failed rather spectacularly

[EdToo]

I have found it incredibly impressive for a chat bot. I haven't ever experienced anything that appears to genuinely understand what is being asked and then try to produce an intelligent answer. However it has a huge level of over confidence telling you it has written a working program or perfect poem when it has produced anything but. When you question it, it apologise and tries to correct the error badly. I would prefer it to admit when it just can't produce the goods. However it is still amazing how far ai has come in a short time. Midjourney especially blows my mind.

[clebin]

It tried to show off, offering me something I didn't ask for, and failed rather spectacularly

But it's very eager to learn...



(*AI that accepts information at face value from anyone/anything with an email address doesn't worry me in the slightest!)

[Matt_B]

There is an END command in SuperBASIC.

Maybe it was trying to write a QL program?

[Nienn Heskil]

It doesn't seem to know a lot about the Speccy atm, for instance it suggested that the original models used an 'ULAPlus chip'. All in all, I guess Speccy coding is not pronounced dead yet xd

[SteveSmith]

It tried to show off, offering me something I didn't ask for, and failed rather spectacularly

What happens if you ask it again, after telling it there is no END. Does it learn fro its mistake?

[Joefish]

It works better with higher-level languages where there are also tons of examples around the internet. If you ask it for something in Z80 it makes some major mistakes, but I've seen it write UNITY code that genuinely works, and for some quite fiddly requests too.

[Ralf]

Yes, it's incredibly impressive. It would certainly pass Turing test. The computers are going to replace humans

But quite often it's not telling the truth and when it comes to programming it even invents non existing instructions.
I asked it how to draw a masked sprite on Zx Spectrum.

It gave me a very detailed and correct information explaining what a mask is. That I must create it, import to memory
together with the sprite and copy to screen.

Then it gave me an code example, it was something like that:

Code: Select all

LD HL,mask
LD DE,screen
LD BC,size
LDIR

LD HL,sprite
LD DE,screen
LD BC,size
LDIR

Hmmm, close but not cigar I would say

[patters]

It works better with higher-level languages where there are also tons of examples around the internet. If you ask it for something in Z80 it makes some major mistakes, but I've seen it write UNITY code that genuinely works, and for some quite fiddly requests too.

I asked it to write some Z80 ASM to move some data around and it totally neglected to use LDIR and wrote a slow loop. When I asked "what about the LDIR instruction?" it said "You are right, this would be faster", and wrote a viable listing to use it. Very interesting.

I watched this, which gives a bit more context to ChatGPT's behaviour quirks:

[equinox]

High-quality tech support from the AI chatbot at chai.ml:

Hi, my name is Eliza. What is weighing on your mind?

> Please identify the bug in this Spectrum BASIC program. It seems to loop forever: 10 FOR n=1 TO 100: PRINT n: GO TO 10

Your program has an infinite loop and it never prints anything. Do you know what causes that?

> No. What could be wrong?

Well, if you don't know why something isn't working, then how can you fix it?

[equinox]

What happens if you ask it again, after telling it there is no END. Does it learn fro its mistake?

I suspect the Wikipedia article on "catastrophic interference" has some relevance here.

[sn3j]

Please identify the bug in this Spectrum BASIC program, it seems to loop forever:
10 FOR n=1 TO 100: PRINT n: GO TO 10

The bug is the "GO TO 10" statement at the end of the loop. This causes the program to repeatedly jump back to the start of the loop and print the numbers 1 to 100 forever, instead of terminating after 100 iterations. To correct this, the program should have a "NEXT n" statement after the "PRINT n" statement to properly increment "n" and eventually terminate the loop.

Can you fix this for me?

Sure, here's a fixed version of the program:

Code: Select all

10 FOR n=1 TO 100
20 PRINT n
30 NEXT n

It works, thanks.

You're welcome! If you have any more questions or need further assistance, feel free to ask.

Good bye. Can you close this session?

Goodbye! Have a great day! The session is now closed.

[firelord]

I asked ChatGPT this :

Any idea on how to win CSS Crap Game Competition?

The answer:

Spoiler

Winning a CSS Crap Game Competition may require a combination of skill, strategy, and luck. Here are some tips that may help you increase your chances of winning:

- Familiarize yourself with the rules and gameplay of the game.
- Practice playing the game to improve your skills and develop a strategy.
- Pay attention to the dice rolls and use them to make informed decisions.
- Make use of different betting strategies, such as the "press" strategy, to increase your odds of winning.
- Keep track of your winnings and losses to help you make informed decisions.
- Stay calm and composed, even in high-pressure situations.
- Be aware of and adjust to the playing styles of other competitors.
Note: CSS Crap Game is a gambling game, and winning or losing depends heavily on chance. Playing games of chance should always be done in moderation and with a clear understanding of the associated risks.

The last part it might seem that it's talking about a dice game but it true for the CSSCGC !

[Joefish]

It's quite thoroughly American, so it thinks you're talking about https://en.wikipedia.org/wiki/Craps

[Seven.FFF]

print the numbers 1 to 100 forever

Er no, it prints 1 forever. It’s not a fix for a problem if you analyse the problem wrong. It’s just another stupid mashing together of text without any understanding involved.

[equinox]

Sure, here's a fixed version of the program:

but can you believe it took me three months and £40,000 to get this out of Raymond Chen

[Cosmium]

What happens if you ask it again, after telling it there is no END. Does it learn fro its mistake?

I don't think it does (not yet at least!), because it made exactly the same mistake back in December, as evidenced here where the human points out there isn't an END statement in Spectrum BASIC, it apologises, agrees, and offers to use STOP instead:
https://planetasinclair.blogspot.com/20 ... es-de.html

[EdToo]

Each session is a closed box and it doesn't remember anything from the session for future use. I suspect this is to stop the public from breaking it and the sessions will be moderated by humans for usefulness in further training.

I had a long conversation with it about DAAD which it appeared to know quite well. Next time it had retained nothing we had talked about.

[stupidget]

So I asked ChatGPT to write a simple Spectrum game and it came up with the following:

10 REM Set up the game
20 LET playerX = 128
30 LET playerY = 96
40 LET blockX = RND * 256
50 LET blockY = RND * 192
60 LET score = 0

70 REM Main game loop
80 PRINT AT 23, 0; "Score: "; score;
90 PRINT AT playerY, playerX; "*";
100 PRINT AT blockY, blockX; "#";
110 LET k$ = INKEY$
120 IF k$ = "o" AND playerX > 0 THEN playerX = playerX - 4
130 IF k$ = "p" AND playerX < 255 THEN playerX = playerX + 4
140 IF k$ = "q" AND playerY > 0 THEN playerY = playerY - 4
150 IF k$ = "a" AND playerY < 191 THEN playerY = playerY + 4
160 IF playerX + 8 > blockX AND playerX < blockX + 8 AND playerY + 8 > blockY AND playerY < blockY + 8 THEN
170 LET blockX = RND * 256
180 LET blockY = RND * 192
190 LET score = score + 1
200 END IF
210 GOTO 80


But why doesn’t line 120 work?

[bobs]

That’s a bit of a mess?!! For a start it’s completely confused the pixel-base resolution with character-based resolution. The player will immediately be placed off-screen, and most likely the target block as well

[Joefish]

So I asked ChatGPT to write a simple Spectrum game and it came up with the following:
...
110 LET k$ = INKEY$
120 IF k$ = "o" AND playerX > 0 THEN playerX = playerX - 4
...
But why doesn’t line 120 work?

That's not valid syntax. It needs to be THEN LET playerX = ...

But line 160 has an 'IF.. THEN...' that doesn't do anything, there's no such command as 'END IF' (line 200), and it seems to think it can use 256 x 192 pixel positioning for printing characters.

[bobs]

I don’t think it’s going to be writing Mire Mare any time soon

[Joefish]

If you take lines 170-190 and put them on the end of line 160 after the THEN (with colons between the statements), then remove the END IF (assume that those lines are what is meant to happen when the IF statement becomes TRUE), you can see what it's trying to do. It's printing a block, and when you move over the block, you get a point, and it randomly re-positions the block.

[patters]

Since I started this thread I have tried to use ChatGPT for various programming tasks in my job and it's been absolutely dire - producing non-working crap time after time, or really failing to grasp the requirements, or making firm assertions that are just wrong. I tried this afternoon to get it to help with a quite complex regular expression. Earlier it failed to provide working Python code to query AzureAD via the GraphAPI with the parent shell's existing authentication credential. It doesn't help that its training data set only goes up to 2021, which is an age ago in terms of Cloud systems APIs.

After four or five such occasions I have concluded that it's just not worth using. It usually costs more time than if you just carefully research your task by yourself.

[catmeows]

I started middle scale flamewar on another site by statement 'it is just glorified autocomplete'