Page 1 of 1

Using SD Cards directly in Fuse on Linux

Posted: Sat Nov 18, 2017 4:12 am
by lister_of_smeg
DISCLAIMER: I accept no responsibilty for any data loss you may incur by following this guide. YOU DO SO AT YOUR OWN RISK.

OK, now that is out of the way...

The following guide explains how you can use to Linux device mapper to 'glue' a modified HDF file header to the front of an SD card and create a device file which can be seen and used by Fuse just like any other HDF file. Sorry, I'm afraid it's not very straight-forward and rather long winded, but it does work. I was using Arch Linux and the fuse-emulator 1.4.0-1 package for AUR, but I can't see any obvious reason why this guide shouldn't work on any modern distro, or with any recent version of Fuse.
Most of the commands in this guide use standard tools which are common to most distros. You will however, need a hex editor and you may also need to download and compile fuse-utils (unless it's provided in your distro's repos) for the 'createhdf' utility.


Creating a tweaked HDF header

First, create a HDF file using createhdf from fuse-utils:-

For this guide, I'm creating a 504Mb HDF file. Make a note of the CHS values you use here as you'll need them later.

Code: Select all

# createhdf 1024 16 63 temp.hdf

Next, dump the first 1024 bytes to a separate file. This file needs to be a multiple of 512 bytes, as later on it will be used to create a loopback device.

Code: Select all

# dd if=temp.hdf of=hdfheader.bin count=2

You'll then need to open this file in a hex editor and alter the 2 bytes at offset 0x09 from '16 02' to '00 04'. This modification to the header will tell Fuse that the disk data starts at an offset of 1024 bytes as opposed to the regular 534.


Preparing the SD card

WARNING: This next part will destroy any data currently on your card, so back up any data which you care about before proceeding.

Insert a card equal in size to or larger than the HDF file you created. You'll need to find out what the device file of the card is. In my case, it's '/dev/sdc'. BE VERY CAREFUL here, as selecting the wrong device can potentially lead to the loss of ALL DATA on that device. YOU HAVE BEEN WARNED!


First, create a loopback device for the SD card. This is done primarily to restrict the FAT filesystem to the size indicated in the HDF header. You'll need the calculate the 'sizelimit' parameter using the CHS values you noted earlier. So, in my example - 1024*16*63*512 = 528482304.

Code: Select all

# losetup --sizelimit 528482304 /dev/loop0 /dev/sdX

Next, create a FAT filesystem on /dev/loop0. Set the volume label of your choice...

Code: Select all

# mkfs.vfat -F 16 -n <label> /dev/loop0
...and remove /dev/loop0. It's not needed beyond this point.

Code: Select all

# losetup -d /dev/loop0

Now, mount the SD card using the method of your choice. In most cases the easiest way will be to eject and reinsert the card, and let it automount. Copy your ESXDOS/FATware/etc. files to the volume. At this point it's probably a good idea to pop your SD card in your Spectrum and test it works correctly.

Once you have verified your SD card is working, insert it back into your computer and dismount it from the command line. DO NOT use a file manager to do this as it will also eject the media from the device, which you don't want to do.

Code: Select all

# umount /dev/sdX

Creating a device file using the device-mapper

Start by creating a loopback device for the HDF header:-

Code: Select all

# losetup /dev/loop0 hdfheader.bin
Now, it's time to 'glue' the header to the SD card using the device mapper. First, create a file (I called mine 'map.txt') containing the the following two lines:-

Code: Select all

0 2 linear /dev/loop0 0
2 1032192 linear /dev/sdX 0
Note the second value on the second line is the value calculated earlier divided by 512. Also, you may want to reference your SD card using one of the symlinks created in /dev/disk e.g. /dev/disk/by-label/<label>.


Then, create a device file using dmsetup. Choose a suitable filename, but I'd recommend adding the '.hdf' extension to make the file easier to find in Fuse.

Code: Select all

# cat map.txt | dmsetup create <filename>.hdf

If all has gone to plan, you will now have a symlink '/dev/mapper/<filename>.hdf'. Lastly, you need to give your user access to the file:-

Code: Select all

# chown <username> /dev/mapper/<filename>.hdf

Fire up Fuse, and you should now be able to use this HDF 'file' as you would any other!


Cleaning up

To remove the mapped device, use the following command:-

Code: Select all

# dmsetup remove <filename>.hdf
And to remove the header loop device:-

Code: Select all

# losetup -d /dev/loop0

At this point you can also make a shell script to automate things. For example, something like...

Code: Select all

#!/bin/bash

sudo losetup /dev/loop0 hdfheader.bin
cat map.txt | sudo dmsetup create <filename>.hdf
sudo chown <username> /dev/mapper/<filename>.hdf
fuse
sleep 1
sudo dmsetup remove <filename>.hdf
sudo losetup -d /dev/loop0

Just a couple of notes to finish on:-

1. It's probably not a good idea to have your SD card mounted whilst using it in Fuse.
2. Ditto for removing and reinsert your SD card whilst the mapped device file exists.


Oh, and one last thing...

Good Luck and Have Fun!

Re: Using SD Cards directly in Fuse on Linux

Posted: Sat Nov 18, 2017 8:33 am
by dfzx
Um, I hate to sound like a total ignoramus, but can you explain a bit what this is all about? :oops:

What's HDF and what do I gain from getting Fuse to, er, do stuff with it?

Re: Using SD Cards directly in Fuse on Linux

Posted: Sat Nov 18, 2017 12:32 pm
by lister_of_smeg
dfzx wrote: Sat Nov 18, 2017 8:33 am Um, I hate to sound like a total ignoramus, but can you explain a bit what this is all about? :oops:

What's HDF and what do I gain from getting Fuse to, er, do stuff with it?
Happy to explain. :)

The guide is of use for folk who have mass storage interfaces (i.e. DivIDE, DivMMC, ZXMMC) for their Spectrums. It's a method which allows you to use the mass storage device from your Spectrum directly in Fuse (i.e without having to dump the data to an image file first). I used an SD card and the FAT filesystem in my guide, but you should be able to use other storage devices such as hard disks or CF cards just as easily, or use the IDEDOS filesystem as used by the +3e ROMs or ResiDOS (although I haven't tried these scenarios yet).

HDF is a hard disk image format which I believe originated in the RealSpectrum emulator and is also used by Fuse (amongst other emulators).

http://ramsoft.bbk.org.omegahg.com/hdfform.html


I would say being able to do this would be of most use to developers, as it allows them to develop and test software using Fuse (or any other emulator for *nix which emulates mass storage utilising the HDF format), then quickly test on real hardware simply by swapping the SD card (or hard disk, etc) between their PC and Spectrum.

Hope this helps.