ZX Spectrum code on The IT Crowd DVD ?

Y'know, other stuff, Sinclair related.
User avatar
flatduckrecords
Manic Miner
Posts: 787
Joined: Thu May 07, 2020 11:47 am
Location: Oban, Scotland
Contact:

Re: ZX Spectrum code on The IT Crowd DVD ?

Post by flatduckrecords »

MatGubbins wrote: Wed Nov 02, 2022 12:00 am Here's all 6 of the files processed in the same style as before.
Thanks! Quite mesmerising to watch…

Here's a better annotation of the tape-reader source code, now that I've read the Microhobby article, in case anyone else is curious:

Code: Select all

		ORG 40000

		DI
		LD HL,#53F	;return address as used by normal SAVE/LOAD routines
		PUSH HL
		LD DE,#4000	;DE=destination for the stored bytes 
				;(in this case, screen display file)
Check_Br:	LD A,#7F
		IN A,(#FE)	;Read keyboard & mic port $7FFE (bit 0 is SPACE key)
		RRA		;Rotate the 0th bit into the CARRY flag
		RET NC		;RETurn if break was pressed
New_byte:	LD L,1		;L=%00000001 (marker to track 8x left rotations)
		LD BC,#B201	;B=$B2 (timing constant), C=$01(border colour and edge type) for LD-EDGE-2
Get_bit:	CALL #5E3	;LD-EDGE-2
		JR NC,Check_Br	;CARRY=0 if no pulse or BREAK
		LD A,#CB	;inter-pulse time to determine 0 or 1
		CP B		;B=how long it took to find edges
				;if that time was exceeded, the pulse was
				;a "1" otherwise it was a "0". Result in CARRY
		RL L		;Rotate CARRY value onto bottom of L and
				;move 'marker' up.
		LD B,#B0	;Time constant for next bit
		JR NC,Get_bit	;CARRY will be 1 when we've rotated L eight times
				;and we've gathered a full byte. If zero we need
				;to fetch another bit and try again. When CARRY
				;is 1 we continue on to Store_byte.
Store_byte:	LD A,L		;A=byte to be stored
		LD (DE),A	;put byte in display file
		INC DE		;move to next display file address
		JR New_byte	;gather next byte
		
And here's an English translation of the article in case that's useful as well.
User avatar
PROSM
Manic Miner
Posts: 476
Joined: Fri Nov 17, 2017 7:18 pm
Location: Sunderland, England
Contact:

Re: ZX Spectrum code on The IT Crowd DVD ?

Post by PROSM »

zxbruno wrote: Tue Nov 01, 2022 3:02 pm If anyone has most of the existing Spectrum software collection on their hard drive, would it be possible to look for this block on code on that hard drive? Someone a few months ago said it would be easy to do if we were looking for part of a loading screen.
That would probably be the easiest and most sensible thing to do. You'd need to write a program which, given a string of bits, would parse through every tape file in a directory and search for matches. It would need to search against 8 shifted variations of the input (as we could be starting at any of the 8 bits in the byte).

I'll have a go at the weekend, should be fun! :dance
All software to-date
Working on something, as always.
User avatar
R-Tape
Site Admin
Posts: 6409
Joined: Thu Nov 09, 2017 11:46 am

Re: ZX Spectrum code on The IT Crowd DVD ?

Post by R-Tape »

Image
PROSM wrote: Wed Nov 02, 2022 3:27 pm I'll have a go at the weekend, should be fun! :dance
Ah!
User avatar
Morkin
Bugaboo
Posts: 3277
Joined: Mon Nov 13, 2017 8:50 am
Location: Bristol, UK

Re: ZX Spectrum code on The IT Crowd DVD ?

Post by Morkin »

Image
My Speccy site: thirdharmoniser.com
Dandu
Drutt
Posts: 5
Joined: Sun Oct 30, 2022 2:05 pm

Re: ZX Spectrum code on The IT Crowd DVD ?

Post by Dandu »

Hi. I have edited the audio file and removed the jingle.

http://down.dandu.be/ep00b.wav

I know it's very short, the video is the entire sample on the DVD.
User avatar
PROSM
Manic Miner
Posts: 476
Joined: Fri Nov 17, 2017 7:18 pm
Location: Sunderland, England
Contact:

Re: ZX Spectrum code on The IT Crowd DVD ?

Post by PROSM »

Dandu wrote: Wed Nov 02, 2022 6:05 pm Hi. I have edited the audio file and removed the jingle.

http://down.dandu.be/ep00b.wav

I know it's very short, the video is the entire sample on the DVD.
Nice :) Is this a rip from the original DVD or from the copy on YouTube?
All software to-date
Working on something, as always.
Dandu
Drutt
Posts: 5
Joined: Sun Oct 30, 2022 2:05 pm

Re: ZX Spectrum code on The IT Crowd DVD ?

Post by Dandu »

From the DVD.

I have extracted the vidéo and the WAV from the DVD.
User avatar
PROSM
Manic Miner
Posts: 476
Joined: Fri Nov 17, 2017 7:18 pm
Location: Sunderland, England
Contact:

Re: ZX Spectrum code on The IT Crowd DVD ?

Post by PROSM »

Dandu wrote: Wed Nov 02, 2022 6:30 pm From the DVD.

I have extracted the vidéo and the WAV from the DVD.
Brilliant - fewer compression artefacts should give this a higher chance of working properly :D

EDIT: Just spotted you had a link to a WAV in the original post... I really should read things more thoroughly :oops:
All software to-date
Working on something, as always.
User avatar
+3code
Manic Miner
Posts: 433
Joined: Sat Mar 19, 2022 7:40 am

Re: ZX Spectrum code on The IT Crowd DVD ?

Post by +3code »

I can imagine the author of the intro reading this thread and laughing, "oh, man I only used a bird!"
User avatar
Morkin
Bugaboo
Posts: 3277
Joined: Mon Nov 13, 2017 8:50 am
Location: Bristol, UK

Re: ZX Spectrum code on The IT Crowd DVD ?

Post by Morkin »

I'm guessing the original recording was done secretly to recruit for MI6.

...If someone finds the answer/solution, enjoy your new career, pop back to post occasionally..!
My Speccy site: thirdharmoniser.com
Dandu
Drutt
Posts: 5
Joined: Sun Oct 30, 2022 2:05 pm

Re: ZX Spectrum code on The IT Crowd DVD ?

Post by Dandu »

Actually, there is many easter eggs and hideent things on the DVD from The IT Crowd.

With the season 1, there is subtitles encoded with base64 or ROT13. On the S2 DVD, there is a program for a 8 bit computer (not the ZX Spectrum, the BBC Micro B) hidden into the subtitles, for example.

This is the reason why i have tried to decode the sound : many things on the DVD are very complicated and real things for geek.
Stu
Manic Miner
Posts: 243
Joined: Wed May 20, 2020 2:59 pm

Re: ZX Spectrum code on The IT Crowd DVD ?

Post by Stu »

Sokurah wrote: Tue Nov 01, 2022 10:43 pm Too bad that Graham Linehan was banned from Twitter - we could've asked him if he knew.
Does anyone know if he's on Instagram or Facebook?
I'm not sure how much he had to do with the DVD stuff; this fascinating guide to the S2 DVD Easter eggs compo shouts out Tim Browse, Sean Sollé and Jim Lynn for their work in devising it. The trio give a fairly high-level overview of the contest in this video which I think is an extra from the S3 DVD. In other words, I think the standard consultant nerds were given free reign to put things on the DVDs, so they may be a better source of info than Glinner.
User avatar
PROSM
Manic Miner
Posts: 476
Joined: Fri Nov 17, 2017 7:18 pm
Location: Sunderland, England
Contact:

Re: ZX Spectrum code on The IT Crowd DVD ?

Post by PROSM »

I know I'm quite late in reporting back on this - in truth I stalled very quickly when I realised that it was going to be tricky to actually extract something from the audio on the disc. The issue is that, through the various filters the loading noise has had applied to it, and then the subsequent audio compression when it was mastered onto the DVD, the edges of the signal have become very muddy. It's pretty much useless to try and load this directly into a Spectrum, it has to be cleaned up first.

Of course, the major problem with processing this signal is that the 0s and 1s are of different length in the Spectrum's tape encoding scheme - if you misidentify a bit, you end up shifting the entire stream after that point one bit left or right.

I had a go at writing a script to clean the audio yesterday: the general idea is that it uses a Schmitt trigger to more reliably detect the edges of the signal. This worked alright for periods of solid tone (i.e. runs of all 1s or 0s), but started coming apart when the 1s and 0s were placed together. I still got odd peaks of signal, and the input looked pretty thin - the duty cycle wasn't a proper 50%, but instead something like 20%.

In an attempt to ameliorate this, I wrote a second pass which would look for spikes in the output and make them the same amplitude as the surrounding parts. After this, there is a processing step in which periods of low and high signal are grouped together into cycles - the overall length of the cycles is then used to determine whether the bit is a 0 or a 1.

Image

(Audacity waveforms - top: the original signal, middle: Schmitt trigger only, bottom: Schmitt trigger and second pass)

However, this still isn't enough to get intelligible data. Some of the cycles are of an odd length, not quite 0 or 1, which means that there could be parts of the signal mistakenly being labelled as spikes upstream. Searching the TOSEC archive for short snippets of the raw data has given me no results so far. Knowing that the audio sounds like a screen, I've also tried loading the data into the Spectrum's display file to see if I can recognise any patterns in it - again, this has been a fruitless endeavour.

Image

I will leave my Python code here in case anyone else wants to have a go at it - bear in mind that this was written "off the cuff" and isn't structured at all well.

Code: Select all

import wave
import numpy as np

def getWave(filename):
    # taken from https://stackoverflow.com/a/62298670 (revision 21/06/2020)
    # Read file to get buffer                                                                                               
    ifile = wave.open(filename)
    samples = ifile.getnframes()
    audio = ifile.readframes(samples)

    # Convert buffer to float32 using NumPy                                                                                 
    audio_as_np_int16 = np.frombuffer(audio, dtype=np.int16)
    audio_as_np_float32 = audio_as_np_int16.astype(np.float32)

    # Normalise float32 array so that values are between -1.0 and +1.0                                                      
    max_int16 = 2**15
    audio_normalised = audio_as_np_float32 / max_int16

    return audio_normalised

def maxMinPeaks(data, leftb, rightb):
    lb = leftb
    if (lb < 0):
        lb = 0

    rb = rightb
    if (rb >= len(data)):
        rb = len(data) - 1

    return (data[lb:rb].max(), data[lb:rb].min())

def schmitt(cur, in2, ub, lb):
    if (cur == -1):
        if (in2 > ub):
            return 1
        else:
            return -1
    else:
        if (in2 < lb):
            return -1
        else:
            return 1


audioFreq = 48000
cpuFreq = 3.54 * 10**6 # The source sounds closer in pitch to the 128K models
tstate0 = 855
tstate1 = 1710

pulse0 = (tstate0 / cpuFreq) * audioFreq
pulse1 = (tstate1 / cpuFreq) * audioFreq

# Get time mid-way between 0 and 1 cycle
mid = ((pulse0 * 2) + (pulse1 * 2)) / 2

level = -1

waveData = getWave("ep00b.wav")
print(len(waveData))

maxData = np.full_like(waveData, 1)
minData = np.full_like(waveData, 1)
binData = []

waveOut = bytearray()

# Get max/min peaks within given window for the adaptive Schmitt trigger
for i in range(len(maxData)):
    maxVal, minVal = maxMinPeaks(waveData, i - 40, i + 20)
    maxData[i] = maxVal
    minData[i] = minVal

run = 0

# Adaptive Schmitt trigger - process wave file into 1-bit run-length encoded file
for i in range(len(waveData)):
    newLevel = schmitt(level, waveData[i], maxData[i]*0.1625, minData[i]*0.1325)
    if (newLevel != level):
        binData.append((level, run))
        level = newLevel
        run = 0

    run = run + 1

# Now we do a second pass over the 1-bit quantised file, fixing up any little spikes here and there
curLRun = 0
curTRun = curLRun
curLev = -1
conData = []

print(len(binData))

# The idea - group together low and high pulse in one
for lev, run in binData:
    if (run < 6):
        curLRun = curLRun + run
        continue
    
    if (lev != curLev):
        curLev = lev
        if (lev == 1):
            curTRun = curLRun
            curLRun = 0
        else:
            conData.append(curTRun + curLRun)
            curTRun = 0
            curLRun = 0
    curLRun = curLRun + run

bits = []

# Now, we write out a cleaned version of our wave for analysis in an audio editor
# (in 8-bit unsigned PCM)
# We also check the time-length of these runs and output a 0 or 1 bit accordingly
for run in conData:
    cycle = run // 2
    for i in range(cycle):
        waveOut.extend(b'\x10')
    for i in range(run - cycle):
        waveOut.extend(b'\xf0')

    if (run < mid):
        bits.append(0)
    else:
        bits.append(1)

waveOutFile = open("out.pcm", mode="wb")
waveOutFile.write(bytes(waveOut))
waveOutFile.close()

# Now write out 8 shifted variations of the data to disk
for i in range(8):
    rawData = bytearray()
    rawDataFile = open("raw"+str(i)+".bin", mode="wb")
    bit = int(2**i)
    byte = 0
    for b in bits:
        if (b == 1):
            byte = byte ^ bit

        bit = bit // 2
        if (bit < 1):
            bit = 128
            rawData.extend(bytes([byte]))
            byte = 0

    #print(rawData) # might be easier to copy data from the printed version if you're using grep
    rawDataFile.write(bytes(rawData))
    rawDataFile.close()

All software to-date
Working on something, as always.
dfzx
Manic Miner
Posts: 683
Joined: Mon Nov 13, 2017 6:55 pm
Location: New Forest, UK
Contact:

Re: ZX Spectrum code on The IT Crowd DVD ?

Post by dfzx »

PROSM wrote: Tue Jan 30, 2024 11:53 am again, this has been a fruitless endeavour.
Nevertheless, I am in utter awe of your audio signal preprocessing abilities...
Derek Fountain, author of the ZX Spectrum C Programmer's Getting Started Guide and various open source games, hardware and other projects, including an IF1 and ZX Microdrive emulator.
User avatar
R-Tape
Site Admin
Posts: 6409
Joined: Thu Nov 09, 2017 11:46 am

Re: ZX Spectrum code on The IT Crowd DVD ?

Post by R-Tape »

Likewise - I'm impressed too. If I knew better, I'd say the problem was the Schmitt trigger or something.

Summing up the from thread, this is surely a snippet of bona-fide loading speccy noise, probably from a classic era program or something they wrote themselves. It sounds like screen, graphics or level data, not program code.
Stu wrote: Thu Nov 03, 2022 12:20 am Tim Browse, Sean Sollé and Jim Lynn for their work in devising it.
Is anyone in touch with any of the IT crowd easter egg crew? Or anyone involved? Or maybe you're reading? I really want to know now!
Post Reply