Archiving and saving all my disks

10 posts / 0 new
Last post
Offline
Last seen: 1 year 1 month ago
Joined: May 27 2009 - 01:37
Posts: 1002
Archiving and saving all my disks

The short version:
I'm looking for ideas and suggestions on how to best save/transfer all my disks (as a final Apple II project) to the PC. There's lots to be saved and it is my intent to trickle them down to asimov for all to enjoy. Been prattling on about this for a long while. And if I don't start this in the next couple of days I'll never get it done. So I've made a few practice runs.

Handling standard 35-track disks isn't a problem ADTPro whips them by in a minute each. But it doesn't handle 36-track disks, of which I have many because back in the day we were desperate for storage space, and every sector counted. And this is where I'm getting hung up. Besides ADTpro not handling these, Copy II+ and other standard utilities don't really do it either. These aren't protected disks, they're just disks with crap we accumulated over the years.

I'm not interested in preserving copy-protected disks, most of them except for a few are already on asimov. I'm just interested in saving all the stuff from the BBS era and all programming tricks and experiments. The goal here is to preserve the information and maybe even prepare compilation disks, as a lot of it is scattered all over the place.

So I'm open to suggestions how to deal with those 36-track bastardized disks.
And how to develop a good workflow that will minimize wear and tear on my original hardware. 4000 disks is a lot. Let's discuss!

The longer version:
Back in the day we would do many tricks and hacks to gain as much performance as possible, storage capacity is no exception. We'd do things like deleting DOS, shortening the VTOC, and adding a 36th track.

Back then it was pretty cool. We got a whole extra track's worth of data. We weren't thinking about the future or saving this stuff. And it's become a hassle. A hassle because not all modern tools handle these modded disks.

I'm at the point in my massive archiving project where I've encountered several hundred 36 track disks prepared by something called AAJS Disk Optimizer (and other pirate tools of the day). This essentially eliminated DOS and added a 36th track. The elimination of DOS is not a problem. But the addition of the 36th track is. While manipulating them on real hardware is not a problem - transferring them in a timely manner and quick manner to the PC looks like it WILL be. AAJS Disk Optimizer wasn’t the only tool that could do this. There were a couple of others IIRC.

ADTPro won't work, it only captures $0-22, not grabbing $23. Same thing with Copy II+ series, only works with $0-22 when it comes to pushing files around and isn't used for transferring anyways.

What a pain in the ass!! I don't even remember what DOS I used to even read/write to those 36 track disks. Not every disk has files crammed into that 36th track.

I'm open to suggestions as to how to handle these disks, and discovering some utilities & methods that will help out in "restoring" them to standard .DSK format. What DOS do I need? What copy programs. All that good stuff

Yup, we were all young and stupid and back then and were not thinking 50 years into the future!

I'm also open to suggestions as how to best handle imaging 4000 disks. I don't think I want to put wear and tear on my good drives for that amount of reading + the fact that these are old disks and I know that MEMOREX disks will be shedding their oxide quite easily. So I'm only going to have 2 or 3 read tries at best before they blow up on me. I've already bought a couple back from the brink! A lot of it is from the hacking days, lots of text files, lots of BBS material, different versions of stuff already there. Really, a good variety, and insight into the local BBS scene. And other not-seen-before things. It'll be trickled down to Asimov as I recover it and verify it.

This FAQ entry describes the exact "problem" I'm facing today.

---
001- How many tracks can I use on a 5.25" diskette?
From: Rubywand
001- How many tracks can I use on a 5.25" diskette? So far,
I've heard 35, 36, and 40. What's the actual number?
The standard number of tracks on a 5.25" diskette is set by DOS 3.3
and ProDOS at 35, numbered 0-34 ($00-$22 in hexadecimal).
The original Disk ][ drive can usually handle 36 tracks with no
problem. Newer 5.25" drives can handle 40 tracks.
Various modified versions of DOS 3.3 allow using 36 tracks and a
few allow using 40 tracks. These mods, especially the 36-track versions,
were fairly popular before the advent of 3.5" diskettes when an extra
track made a noticeable difference in capacity. However, unless the extra
capacity is vital for some specific application, it is best to stick
with 35 tracks in order to retain full compatibility with disk utilities
(such as Copy II Plus) and other wares.
---

It was only mid-way into my Apple II computing "Career" that I (we) stopped playing with space-increasing tricks. And then shortly thereafter I got into the PC, where storage capacity was never a problem again. Shit, today they just demonstrated a 1TB SD card!

I'm not necessarily interested in preserving these as an exact disk image. But more so for the information. Text files, utilities, BBS installations.. Making it easy accessible in emulators, and easy-to-recreate standard disk images.. That sort of thing. If TEXTFILE#1 is at the beginning of the disk or at the end of the disk is pretty much irrelevant, as long as it's ON the disk - because most of these disks were made by fellow hackers and stuff.

In fact, it might be best to make compilations, like for example, a single disk full of all the Dalton Disk Disintegrators. I’ve got something like 17 distinct versions of that. And 6 versions of Com-Ware II, you get the idea.

Back in the day we didn’t organize everything efficiently or comprehensively. We just focused on getting it. Hoarding it.

I also wouldn't mind doing this entirely away from my real Apple II hardware, something like a "disposable" drive attached to a PC or something. Something to keep the wear and tear down. Suggestions please!

But most importantly of all I want this to be a fun, final, Apple II project. And not turn it into a miserable drudgery. It's been a long time coming and time to get it done.

speedyG's picture
Offline
Last seen: 4 years 10 months ago
Joined: Nov 16 2011 - 07:45
Posts: 2493
Re: Archiving and saving all my disks

Hello Keatah,

i fear that i have in general bad news.....
in fact the only safe way to handle that 36 and 40 track disks is using FID.....
and it's recommended to boot that disks with extra tracks, because if you don't boot them
the system won't recognize that extra tracks....

and the only configuration to be recommended will be using SCSI-disks besides.....

I'll explain the reason:
like mentioned that extra tracks are accessable only because some bytes have been altered in DOS
and only if you boot that disks the computer will get that altered bytes in memory....
and the only storage system that you may use - even if you didn't boot from that system - will be SCSI...

so the recommended setup will be:
Slot 6 Disk 1 free for booting the altered disks with different altered bytes ( will work with 35, 36 and 40 tracks )
Slot 6 Disk 2 Utility disk ( Homebrew) with SCSI Utilities and FID
Slot 5 or slot 7 SCSI interface with disks and sorted volumes ( Volume for text files Volume for programs etc )

then booting each disk in slot 6, d1

starting SCSI from S6,d2 and specific FID

shoveling S6,d1 to S5, vol. xy

repeating this till SCSI disk is nearly full....

Then shut down and add to system CFFA in alternate slot 7 or 5 still keeping SCSI in the system.

Restart system from CFFA and then also SCSI volumes are accessable and you may shovel the stuff from SCSI
to the CF-Card.

After the stuff has been relocated from SCSI Volumes to CFFA Volumes you can
access all that stuff with cardreader at MS-DOS computer with Ciderpress and
Emulationpprogramms and Conversionprograms.

That permits you to bundle the stuff and prepare for transfer to asimov
and choose if you want to transmit diskimages or harddisk images.

Facing 4000 disks will be a long and stony roadpath..... long struggle through death valley.....

reminder: some disks that are related to specific hardware ( with drivers or programs for special cards )
are in general distributed at normal 35 track disks... that disks should be transfered with normal
ADT to become DSK-images....

If you are on the SCSI road and have a CD Drive at a MAC you could instead shovel the stuff to a Mac
and make a CD from that disks... but it should be ISO 9660 !
( That restricts length of filenames as well as kind of filenames and depth of directories )

i don't even start thinking about that task at the moment about my disks....
3000 disks normal ( 35 track )
1000 disks extended with 540 kB ( 160 tracks )
and nearly 1500 disks at 3,5 format ( 800 kB )
600 Disks CPM
150 Disks UCSD....
That's stuff for a complete winter season...

good luck
speedyG

Offline
Last seen: 1 year 1 month ago
Joined: May 27 2009 - 01:37
Posts: 1002
Re: Archiving and saving all my disks

Yes. I was afraid FID would be necessary. Such a tedious thing.. I don't want to set up anything SCSI right now either.

We even deleted DOS off of the disks to "recover" even more space up front from tracks 0, 1, 2. So I'm not sure what DOS to use??

Offline
Last seen: 1 year 1 month ago
Joined: May 27 2009 - 01:37
Posts: 1002
Re: Archiving and saving all my disks

IIRC VTOC offset $34 contains the number of tracks per disk. I remember this because $34 and 0-34. Or 35-tracks! I'll need to check that when I get back in town. And I think I now suddenly recall there is a version of Copy II Plus that recognizes, understands, and works with 36-track disks.

And if I'm not totally mistaken. You only need a modified DOS to INIT the 36 or 40 track disk. Since the data about how many tracks the disk has is stored in VTOC and read by DOS, you don't need a special DOS to read the modded disks.

DOS will happily step the stepper motor to its physical limit, within reason and internal calculating/counting ability, which is 36 for the early shugart mech, and 40 for the later half-height mechs.

Sheesh! All this trouble for a lousy 16-sector increase in storage space! But in the BBS days that sometimes meant the difference between a 2-drive rig and a 3-drive or ramdisk rig!

speedyG's picture
Offline
Last seen: 4 years 10 months ago
Joined: Nov 16 2011 - 07:45
Posts: 2493
Re: Archiving and saving all my disks

And I think I now suddenly recall there is a version of Copy II Plus that recognizes, understands, and works with 36-track disks.

Most copyprograms permitted access to various amount of tracks....
In Nibble / Byte mode no problem.... just enter amount of tracks as parameter....

the problem is in filemode.....

the most programs just read the VTOC and display the entries of the directory and don't decode the
number of tracks..... but chance is high that vers. 9.0 can handle it...

Offline
Last seen: 6 years 6 months ago
Joined: Sep 23 2016 - 17:58
Posts: 10
Re: Archiving and saving all my disks

I realise you mention that you are only after the content (ie. the files), but it might be worthwhile for you to consider imaging the disks anyway. There are some products available that allow you to use standard PC floppy drives to image disks. The one I am familiar with is the Kryoflux, which you can think of as something like a USB floppy interface, but there are others out there as well.

This would offer you a few advantages:

  • Ability to capture those failing disks while you still can
  • Use of modern hardware/software (except for the disk drive itself)
  • Zero wear and tear on your period Apple II equipment
  • "Superior" solution (capture the actual disk flux and not just the files)

There may be some disadvantages:

  • Expense (you need to purchase the controller and suitable disk drive), but way cheaper than a commercial or specialist service (especially for the quantity you have)
  • You almost certainly used non-genuine double sided disks and need to purchase a flippy modified drive (sometimes available in the Kryoflux online store) or find someone with experience who can mod them for you
  • Do 36-track disk images work in emulators or CiderPress?

I feel such a solution would nicely address your concerns about "disposable" hardware aspects. It may still leave you with a problem on decoding and actually retrieving information from the resulting disk images, but I don't have any experience with 36-track Apple II disks and can't give any advice in this regards.

If emulators/CiderPress can't read 36-tracks, then in the worst case the Kryoflux captures something that you might like to think of as "raw" dumps of each track. So it would at least be possible to manually/artificially craft a disk image in which you swap tracks around (eg. Swap track 36 with track 0) - so you end up with disk A (tracks 0-35) and disk B (track 36 at track 0). This would all be a very manual process but would allow you to at least access that data from a standard emulator (leaving you to manually piece/recreate whatever files happen to be present in that track) - it would take a long time and a lot of patience, but is possible if you know enough about DOS.However it may send one insane in the process.

In addition to capturing the "raw" tracks, the Kryoflux can also be told to dump the standard disk image at the same time. This would possibly be necessary for "reliable" disks in your case as it allows the Kryoflux to determine whether the track it read was indeed a valid Apple II track and triggers it to retry the track if it detects any errors. However if your media is falling apart, you may opt to skip that process and simply capture a 1-pass "raw" dump and then use the Kryoflux software to export the Apple image from the raw files after the event. You would get any errors reported in the image and can then decide an whether (if there are only a couple of errors) if it is worthwhile to attempt to re-image that floppy with the Apple II export on, or in the case of very many errors whether to consider the floppy too far gone.

I recommend purchasing a few different types of drives - in my experience some do better than others for different floppies.

Offline
Last seen: 1 year 1 month ago
Joined: May 27 2009 - 01:37
Posts: 1002
Re: Archiving and saving all my disks

Virtual II on a mac should read 36-track disks just fine. Applewin and Ciderpress do not - and both are in need of modernization, but they will have to suffice as I'm familiar with their capabilities and limitations and can spot problems when something isn't quite right.

I think that while Kryoflux would work here I also think it is too much and doesn't address the "36-track disk -> FTP ASMIOV" problem, which is making it accessible for everyone.

So far the only way I can imagine doing these 36-track disks is to copy some of the files off them and image the disks with conventional tools like ADTPro/CFFA3000/COPYII+. It is important to keep in mind some of these are haphazard compilations and the information contained in them is the important thing, not the disk format or things like that.

At least that's how I see it. The information needs to be in a user accessible format. And that means the likes of FTP ASIMOV, Ciderpress, Applewin, standard DSK format, and other lowest common denominators. It must be able to be migrated as time goes on too. So working in any kind of kryoflux / Software Preservation Society context and format is way way overkill. No one is going to know what to with IPF files. Whereas everyone except the newest of newbies knows about DSK and how work with them. And they have the hardware too.

I also want to keep this simple and versatile with a flair of old-school for my personal enjoyment.

While I was excited about the whole kryoflux idea some odd 10 years ago.. I'm finding the SPS philosophy rather elitist now. Call me jaded, whatever.. I don't go for high-art, electronic, canvas, software, or otherwise. I'm not turned on by the snobby museum types. Sorry. Standard DSK images are the order of the day.

Many of these text files and utilities and thousands of doc files and Applesoft/Integer files and other various things were gathered from users groups (including our own). And I ran into a large lot of disks that I don't even know what's on them yet. They need to be organized and at least loosely grouped. And then there's never before dumped stuff.

Like for example some of my earlier things I developed are co-habiting on the same disk as hi-res pictures and infantile text files consisting of swear words. A little bit of sorting and curating is in order I think. I intend to work in the Windows/Ciderpress/Applewin/CopyIIplus once the imaging is done. I don't intend to delete the images. Only make additional ones of select material. So nothing will be lost.

I also don't give a rat's ass whether [<--MUFFLER & CAR TRICKS-->] is stored on tracks 7, 8, and 9 as opposed to tracks 17, 18, and 19 and on the same disk with HAYES HACKAMATIC and CATSEND 3.0. About 40% of the material comes from BBS'ing and we all know there was no set order or rules and regulations specifying how it should be stored. What *IS* important is that any multi-part files and things be kept together and tested before posting.

I don't really care much about copy-protected original disks. Many have been cracked and re-cracked over the years and are all online. Any efforts I put forth there will be in duplicate, in triplicate, if not more!

I also don't agree with SPS' philosophy. As original hardware falls out functionality those IPF images aren't going to be that important. They won't work on all emulators. And there's no telling *IF* new highly accurate emulators will be developed to work with IPF. But right now we have emulators working great with what's available. Just about any game can be played. Any text file be read. Any disk utility and operation be conducted. All from standard DSK/PO formats along with a little help from NIB.

You're right, the cost of a kryoflux setup is too much. There are other projects I want to see through. And funds will be going there. So I just may get a couple of drives from ebay and consider them consumable. There'd be little wear on the computer itself, and I can use an external keyboard.

I don't agree that capturing flux transitions is a superior solution. Maybe if you want to preserve the copy protection and quarter tracks and all that. Preserve the difficulty of access.. Uhmkay.. But that's like preserving the locks on a treasure chest and having to dig through a box of keys. I think it's sufficient to document copy protection behavior as a curiosity for Sunday morning reading. In real life use it's a hindrance.

Offline
Last seen: 1 year 1 month ago
Joined: May 27 2009 - 01:37
Posts: 1002
Re: Archiving and saving all my disks

I now located a hack/mod version of copy II+ that works with a DOS 3.3 and large (1500) sector DOS 3.3 volumes on a hard drive. Cool! The steaming pile of disks I need to get under control is providing the very tools which could be useful.

This is good because I can retain the long descriptive filenames when pushing stuff around.

Offline
Last seen: 1 year 1 month ago
Joined: May 27 2009 - 01:37
Posts: 1002
Re: Archiving and saving all my disks

Thanks to all the ideas and commentary provided and a couple of trial runs with random disks I now have a system in place and ready to go!

Offline
Last seen: 6 years 6 months ago
Joined: Sep 23 2016 - 17:58
Posts: 10
Re: Archiving and saving all my disks

I understand your concerns (and knew my suggestion was overkill); I also had those same sorts of concerns some 5+ years ago. In this time, I have come to accept (and maybe understand) the vault that is SPS.

I was also in the same sort of situation where I just wanted to get DSK dumps from a relatively small collection of (non copy protected) Apple II floppies and no longer had access to or space for period Apple II equipment. At that stage, the CatWeasel was well on the way out and my casual research found only a few similar products. One of them (sorry I don't remember the name) would have done you very nicely; it was very cheap compared to Kryoflux and/but only supported 5.25 devices. In the end, I chose the more expensive Kryoflux solution for the more active community, 5.25/3.5 support and for the software support of the Apple II disk format - capture of flux transitions was merely an archival-format bonus for me.

I agree that the Kryoflux is by no means an inexpensive undertaking, but ironically for me it was more cost effective and reliable than acquiring and shipping period Apple II equipment (or getting someone else to do the imaging for me). I also got the added educational benefit from doing it all myself as well.

I was by no means suggesting distribution as IPS images, rather, using IPS as an interim step in your workflow. I could be wrong, but I'm not aware of any Apple II emulator that supports IPS (or any other flux transition format).

However, I am glad you have found something that works with your setup and budget. May we all be rewarded with whatever gems you recover.

Log in or register to post comments