Using SD Cards directly in Fuse on Linux
Posted: Sat Nov 18, 2017 4:12 am
DISCLAIMER: I accept no responsibilty for any data loss you may incur by following this guide. YOU DO SO AT YOUR OWN RISK.
OK, now that is out of the way...
The following guide explains how you can use to Linux device mapper to 'glue' a modified HDF file header to the front of an SD card and create a device file which can be seen and used by Fuse just like any other HDF file. Sorry, I'm afraid it's not very straight-forward and rather long winded, but it does work. I was using Arch Linux and the fuse-emulator 1.4.0-1 package for AUR, but I can't see any obvious reason why this guide shouldn't work on any modern distro, or with any recent version of Fuse.
Most of the commands in this guide use standard tools which are common to most distros. You will however, need a hex editor and you may also need to download and compile fuse-utils (unless it's provided in your distro's repos) for the 'createhdf' utility.
Creating a tweaked HDF header
First, create a HDF file using createhdf from fuse-utils:-
For this guide, I'm creating a 504Mb HDF file. Make a note of the CHS values you use here as you'll need them later.
Next, dump the first 1024 bytes to a separate file. This file needs to be a multiple of 512 bytes, as later on it will be used to create a loopback device.
You'll then need to open this file in a hex editor and alter the 2 bytes at offset 0x09 from '16 02' to '00 04'. This modification to the header will tell Fuse that the disk data starts at an offset of 1024 bytes as opposed to the regular 534.
Preparing the SD card
WARNING: This next part will destroy any data currently on your card, so back up any data which you care about before proceeding.
Insert a card equal in size to or larger than the HDF file you created. You'll need to find out what the device file of the card is. In my case, it's '/dev/sdc'. BE VERY CAREFUL here, as selecting the wrong device can potentially lead to the loss of ALL DATA on that device. YOU HAVE BEEN WARNED!
First, create a loopback device for the SD card. This is done primarily to restrict the FAT filesystem to the size indicated in the HDF header. You'll need the calculate the 'sizelimit' parameter using the CHS values you noted earlier. So, in my example - 1024*16*63*512 = 528482304.
Next, create a FAT filesystem on /dev/loop0. Set the volume label of your choice...
...and remove /dev/loop0. It's not needed beyond this point.
Now, mount the SD card using the method of your choice. In most cases the easiest way will be to eject and reinsert the card, and let it automount. Copy your ESXDOS/FATware/etc. files to the volume. At this point it's probably a good idea to pop your SD card in your Spectrum and test it works correctly.
Once you have verified your SD card is working, insert it back into your computer and dismount it from the command line. DO NOT use a file manager to do this as it will also eject the media from the device, which you don't want to do.
Creating a device file using the device-mapper
Start by creating a loopback device for the HDF header:-
Now, it's time to 'glue' the header to the SD card using the device mapper. First, create a file (I called mine 'map.txt') containing the the following two lines:-
Note the second value on the second line is the value calculated earlier divided by 512. Also, you may want to reference your SD card using one of the symlinks created in /dev/disk e.g. /dev/disk/by-label/<label>.
Then, create a device file using dmsetup. Choose a suitable filename, but I'd recommend adding the '.hdf' extension to make the file easier to find in Fuse.
If all has gone to plan, you will now have a symlink '/dev/mapper/<filename>.hdf'. Lastly, you need to give your user access to the file:-
Fire up Fuse, and you should now be able to use this HDF 'file' as you would any other!
Cleaning up
To remove the mapped device, use the following command:-
And to remove the header loop device:-
At this point you can also make a shell script to automate things. For example, something like...
Just a couple of notes to finish on:-
1. It's probably not a good idea to have your SD card mounted whilst using it in Fuse.
2. Ditto for removing and reinsert your SD card whilst the mapped device file exists.
Oh, and one last thing...
Good Luck and Have Fun!
OK, now that is out of the way...
The following guide explains how you can use to Linux device mapper to 'glue' a modified HDF file header to the front of an SD card and create a device file which can be seen and used by Fuse just like any other HDF file. Sorry, I'm afraid it's not very straight-forward and rather long winded, but it does work. I was using Arch Linux and the fuse-emulator 1.4.0-1 package for AUR, but I can't see any obvious reason why this guide shouldn't work on any modern distro, or with any recent version of Fuse.
Most of the commands in this guide use standard tools which are common to most distros. You will however, need a hex editor and you may also need to download and compile fuse-utils (unless it's provided in your distro's repos) for the 'createhdf' utility.
Creating a tweaked HDF header
First, create a HDF file using createhdf from fuse-utils:-
For this guide, I'm creating a 504Mb HDF file. Make a note of the CHS values you use here as you'll need them later.
Code: Select all
# createhdf 1024 16 63 temp.hdf
Next, dump the first 1024 bytes to a separate file. This file needs to be a multiple of 512 bytes, as later on it will be used to create a loopback device.
Code: Select all
# dd if=temp.hdf of=hdfheader.bin count=2
You'll then need to open this file in a hex editor and alter the 2 bytes at offset 0x09 from '16 02' to '00 04'. This modification to the header will tell Fuse that the disk data starts at an offset of 1024 bytes as opposed to the regular 534.
Preparing the SD card
WARNING: This next part will destroy any data currently on your card, so back up any data which you care about before proceeding.
Insert a card equal in size to or larger than the HDF file you created. You'll need to find out what the device file of the card is. In my case, it's '/dev/sdc'. BE VERY CAREFUL here, as selecting the wrong device can potentially lead to the loss of ALL DATA on that device. YOU HAVE BEEN WARNED!
First, create a loopback device for the SD card. This is done primarily to restrict the FAT filesystem to the size indicated in the HDF header. You'll need the calculate the 'sizelimit' parameter using the CHS values you noted earlier. So, in my example - 1024*16*63*512 = 528482304.
Code: Select all
# losetup --sizelimit 528482304 /dev/loop0 /dev/sdX
Next, create a FAT filesystem on /dev/loop0. Set the volume label of your choice...
Code: Select all
# mkfs.vfat -F 16 -n <label> /dev/loop0
Code: Select all
# losetup -d /dev/loop0
Now, mount the SD card using the method of your choice. In most cases the easiest way will be to eject and reinsert the card, and let it automount. Copy your ESXDOS/FATware/etc. files to the volume. At this point it's probably a good idea to pop your SD card in your Spectrum and test it works correctly.
Once you have verified your SD card is working, insert it back into your computer and dismount it from the command line. DO NOT use a file manager to do this as it will also eject the media from the device, which you don't want to do.
Code: Select all
# umount /dev/sdX
Creating a device file using the device-mapper
Start by creating a loopback device for the HDF header:-
Code: Select all
# losetup /dev/loop0 hdfheader.bin
Code: Select all
0 2 linear /dev/loop0 0
2 1032192 linear /dev/sdX 0
Then, create a device file using dmsetup. Choose a suitable filename, but I'd recommend adding the '.hdf' extension to make the file easier to find in Fuse.
Code: Select all
# cat map.txt | dmsetup create <filename>.hdf
If all has gone to plan, you will now have a symlink '/dev/mapper/<filename>.hdf'. Lastly, you need to give your user access to the file:-
Code: Select all
# chown <username> /dev/mapper/<filename>.hdf
Fire up Fuse, and you should now be able to use this HDF 'file' as you would any other!
Cleaning up
To remove the mapped device, use the following command:-
Code: Select all
# dmsetup remove <filename>.hdf
Code: Select all
# losetup -d /dev/loop0
At this point you can also make a shell script to automate things. For example, something like...
Code: Select all
#!/bin/bash
sudo losetup /dev/loop0 hdfheader.bin
cat map.txt | sudo dmsetup create <filename>.hdf
sudo chown <username> /dev/mapper/<filename>.hdf
fuse
sleep 1
sudo dmsetup remove <filename>.hdf
sudo losetup -d /dev/loop0
Just a couple of notes to finish on:-
1. It's probably not a good idea to have your SD card mounted whilst using it in Fuse.
2. Ditto for removing and reinsert your SD card whilst the mapped device file exists.
Oh, and one last thing...
Good Luck and Have Fun!