Apple ][ Floppy archiving

8 posts / 0 new
Last post
Offline
Last seen: 13 hours 47 min ago
Joined: Apr 5 2024 - 20:16
Posts: 20
Apple ][ Floppy archiving

New on the NorseWanderer blog: how do you rescue a 40-year-old Apple ][ floppy collection when one capture isn't enough?

 

The answer turned out to be multiple captures, a majority-consensus pass to vote out the bad bits, and a custom repair tool that fills in the gaps. This is the first in a series of about 12 posts on the tool I created and the process I used.

 

Read it here: https://norsewanderer.com/blog/BlogDSK1.html

 

And if you want to see what all this recovery work brought back to life, the BBS is live at norsewanderer.com. Come on in and say hello.

 

Prepare ye to plunder!

 

Michael Heilemann - The Norseman/The Viking

Offline
Last seen: 13 hours 47 min ago
Joined: Apr 5 2024 - 20:16
Posts: 20
Part 2 is up: Talking to the Apple ][e (seriALL + ADTPro 2.1 wor

Part 2 of the disk revival series is up. It covers the hardware chain I use to get bits off the disks and onto a PC, plus a few things I couldn't find documented anywhere else.

A working Practical Peripherals seriALL + ADTPro 2.1 configuration. I searched forums and wikis before setting this up and couldn't find a single report of a seriALL working with ADTPro, so here's what works for me:

  • seriALL in Slot 2, jumper on position 5, DIP switches and full settings in the post
  • PC side must stay at 115200 baud. Dropping it to 19200 to match the card's self-reported rate would not connect at all
  • No IN#2 / Ctrl-A handoff sequence needed like on an SSC. Boot ADTPro, pick Serial, and it connects
  • Quick sanity check without ADTPro: PR#2, then CTRL-A V prints the card's configuration to the screen. I also got ADTPro working with switches set differently from the by-the-book config, so a successful connection doesn't prove the switches are right

Bootstrapping ADTPro over the cassette port. My Floppy Emu went back for warranty replacement mid-project, and I'd never made a physical ADTPro boot disk. The audio bootstrap (two audio cables, a USB sound card, speaker volume around 60%) sends ProDOS and the client into a bare machine, then you format a disk and pull down the ADTPro image. It was a lot less painful than I remembered.

Other things covered:

  • Unenhanced ][e with a genuine Disk II and a Cumana clone on separate controllers, and why two physically different drives make cross-checking meaningful
  • FTDI USB-serial adapter and null modem cable, and why I'd avoid cheap PL2303 clones for archival work
  • Drive prep before touching the collection: head and rail cleaning, speed check, and a round-trip test of a known-good DOS 3.3 System Master to make sure the chain is clean before any real disk goes in

Full post: https://norsewanderer.com/blog/BlogDSK2.html

Offline
Last seen: 13 hours 47 min ago
Joined: Apr 5 2024 - 20:16
Posts: 20
Part 3 is up

Part 3 of 12 is up. This one is about capture, and the short version is that a single ADT Pro pass is one sample. A marginal sector can read fine now and come back as garbage thirty seconds later, and the drive never tells you it was iffy. It just hands back 256 bytes and moves on.

So nothing gets captured once. Any disk that isn’t a known-good copy gets at least five passes, and anything that throws an error or reads slowly gets seven or nine. An odd number matters, because the consensus repair later has to vote between reads and an odd count avoids ties.

The order I settled on is four piles:

Standard captures on the Cumana, the drive that came with the system. About 30 seconds a disk, so five passes across a whole stack is cheap.

  • Anything that errored goes to the Disk II, still standard mode. Two reads off different drives don’t share the same alignment, dust or belt tension, so agreement between them means more than a same-drive re-read.
  • Still unresolved goes to nibble mode on both drives. It takes a little over 2 minutes a disk, so I keep this pile small.
  • Still unreadable goes to flux hardware like an Applesauce, KryoFlux or Greaseweazle. That wasn’t in the budget, so I stop at pile 3 for now.
  • The latest full run covered 183 distinct disks, and 61 of them were rebuilt by voting between several reads. Part 7 has the numbers on what those extra reads were worth.

Two practical notes for anyone using batch mode. Name each capture after the physical disk label so you can find the disk again when a problem turns up later. And don’t rename or reshuffle files mid-session, because the verifier uses ADT Pro’s four-digit counter and the file times to work out which captures are re-reads of the same disk. Also, the Virtual.po files the virtual drive server drops in its folder aren’t captures, and the program deletes them from a collection it scans.

https://www.norsewanderer.com

Prepare ye to plunder!

 

Offline
Last seen: 3 days 15 min ago
Joined: Apr 26 2016 - 08:36
Posts: 884
Seems to me you could easily

Seems to me you could easily do multiple reads on a single floppy disk using several disk drives if necessary and then assemble your image file from a composite of several image files using AppleSauce's (freely available) software.  Doesn't need to be a flux image - even a set of .dsk images will do. 

However, unless we're talking about really poor condition diskettes which may not be readable at all in some cases, an ADTPro pass may capture the entire image correctly on a drive that is compassionately aligned to the disk being read.

Or diskettes with mould...that would need to be washed off before attempting to read it.

 

Offline
Last seen: 13 hours 47 min ago
Joined: Apr 5 2024 - 20:16
Posts: 20
Yes, that is exactly what it does...
baldrick wrote:

Seems to me you could easily do multiple reads on a single floppy disk using several disk drives if necessary and then assemble your image file from a composite of several image files using AppleSauce's (freely available) software.  Doesn't need to be a flux image - even a set of .dsk images will do. 

However, unless we're talking about really poor condition diskettes which may

 

Yes, that is how it works, I take the reads and then do checks on which sectors dont match and if they are not empty do a byte for byte best of and write that out.

Offline
Last seen: 13 hours 47 min ago
Joined: Apr 5 2024 - 20:16
Posts: 20
 Part 4 of 12 is up. This one

 

Part 4 of 12 is up. This one is about reading a DOS 3.3 disk the way DOS itself reads it: start at the VTOC on track 17, follow the same pointers DOS follows, and don't trust a pointer until it has been checked against the disk's real geometry.

What the validator checks, in order: Boot record on track 0, sector 0. It counts how many of the nine boot sectors actually have content, which tells a stock DOS disk from a data disk or a custom or protected loader.

The VTOC header. It checks for 256 bytes per sector and 122 track/sector pairs per list. A bad header makes a disk SUSPECT, not BAD, because a non-standard DOS is a real possibility.

The catalog chain. Every step is bounds-checked, and every sector already visited is remembered, so a loop gets reported instead of hanging the reader.

Every file's T/S list, followed to the very end for every file, not a sample. That catches off-disk pointers, loops, truncated files, and any sector claimed by two files at once.

Every problem goes into one list that becomes one of three verdicts: OK, SUSPECT or BAD. If at least two catalog entries and more than 15% of them fail, the disk is labeled DOS-3.3-shaped with a non-standard catalog, so protected games don't get written off as corrupt.

One thing I found along the way: Apple's DOS Manual (Appendix C) and Beneath Apple DOS disagree about what VTOC bytes $30 to $33 hold. I checked the disks in my collection, and the book is right. They hold the last track allocated and the direction, not a bitmap mask as you may think.

Next up is the check that catches the most real damage of any in the series: cross-referencing every sector the walk found against DOS's own free-sector bitmap.

Read the full blog post here: https://norsewanderer.com/blog/BlogDSK4.html

https://www.norsewanderer.com

Prepare ye to plunder!

Michael Heilemann - The Norseman / The Viking

Offline
Last seen: 13 hours 47 min ago
Joined: Apr 5 2024 - 20:16
Posts: 20
Part 5 of 12 is up. This one

Part 5 of 12 is up. This one is about the check that catches the most real damage in the whole series: the allocation bitmap cross-check.

DOS 3.3 keeps a second, independent record of which sectors are free, the bitmap in the VTOC. It is written separately from the catalog and the track/sector lists, so it is a second opinion on the same disk. When the two disagree, something is really wrong, and the T/S-list walk alone can't see it. The classic case is a file whose pointer lands on a sector the bitmap says is free. Every other check can still pass, because it is a valid track, a valid sector and readable data.

The post covers the bitmap layout (a set bit means free, and the two bytes per track are in the order you wouldn't guess), how the validator builds an ownership map of every sector and compares it against the bitmap in both directions, and how two files claiming the same sector falls out of the same map for free.

It also covers the VTOC rebuild, which fixes the ledger on a repaired copy. It refuses to run unless every file's chain read cleanly, it never frees DOS's own tracks or anything on track 17, and your original is never touched. Some copy protection hides data in sectors marked allocated that belong to no file, so there is a switch to turn the rebuild off.

The series so far:Part 1: https://norsewanderer.com/blog/BlogDSK1.html - Archiving, Validating and Repairing Apple ][ Digital DisksPart 2: https://norsewanderer.com/blog/BlogDSK2.html - Talking to the Apple II: seriALL, a Floppy Emu, and Two DrivesPart 3: https://norsewanderer.com/blog/BlogDSK3.html - The Calibrated Capture: Why Every Disk Gets Read More Than OncePart 4: https://norsewanderer.com/blog/BlogDSK4.html - Reading DOS 3.3 the Way DOS DoesPart 5: https://norsewanderer.com/blog/BlogDSK5.html - The Bitmap Doesn't Lie

https://www.norsewanderer.com

Prepare ye to plunder!

Michael Heilemann - The Norseman / The Viking

 

Offline
Last seen: 13 hours 47 min ago
Joined: Apr 5 2024 - 20:16
Posts: 20
Part 6 of 12 is up. This one

Part 6 of 12 is up. This one is about ProDOS, and the short version is that the DOS 3.3 checks can't just be pointed at it, but the principle carries over: what the files claim about themselves has to agree with what the volume says is allocated.

ProDOS thinks in 512-byte blocks, not 256-byte sectors. On a 140K image each block is two sectors, and in a DOS-order image the pair isn't adjacent, so the program carries a small lookup table. Block 19, for example, is track 2 (19 // 8) with a remainder of 3, which lands on sectors 0x9 and 0x8.

Three things in the post that are easy to get backwards:

Index blocks split their pointers. The low bytes of all 256 pointers sit in the first half of the block and the high bytes in the second half. It looks odd, but it was done for the 6502, whose index registers can only reach 256 bytes at a time.

In the volume bitmap, a set bit means free (same as DOS 3.3), but bit 7 is the lowest-numbered block. That's the reverse of the DOS 3.3 numbering, so the code can't be copied across.

Stray bits past the end of the volume get flagged. Garbage in the unused tail of the bitmap block once made CATALOG report 681 blocks free on a 144-block disk.

Then the ownership check: the program maps every block that a file, directory or the bitmap itself owns, and compares that against the bitmap in both directions. A block a file owns that the bitmap calls free is a hard signal, and it's one of the things that makes a disk BAD. A block marked allocated that nothing owns is only noted, since it's usually lost space from an interrupted write.

Part 7 is the first repair engine: when you've read the same disk five times and the reads disagree, how do you decide what the disk says?

https://norsewanderer.com/blog/BlogDSK6.html

Prepare ye to plunder!

Michael Heilemann The Norseman/The Viking

Log in or register to post comments