In article , Andy McFadden wrote: > In comp.sys.apple2 Paul Schlyter wrote: >>> CiderPress looks for DOS, ProDOS, Pascal, CP/M, and RDOS filesystems in >>> "DOS", "ProDOS", and "physical" sector ordering every time you open a >>> disk image. > >> Then how do you determine the sector interleaving on a blank Apple CP/M >> disk images (no files ever written to the disk, no CP/M system on the >> system tracks)? Such a disk will have all bytes in each and every >> sector set to hex E5. Perhaps in such a case you just assume DOS 3 >> sector order, since that's what works best with those Apple II emulators >> which also emulates a Z80 ? > > I scan the volume directory and look for weirdness. If the disk were > completely empty there'd be nothing to scan and I'd get the wrong answer, > but since CP treats CP/M as read-only it's not like you'll miss out on > anything. :-) > I would have preferred to examine blocks in the boot area (the first 3 > tracks), but it looks like AE and Microsoft had different implementations. The structure and contents of the CP/M system tracks are indeed completely implementation specific. It doesn't even have to be 3 tracks: om the classic CP/M disk format (8" SSSD) the system tracks were only 2 tracks; otoh each track on 8" SSSD disks contained more than each track on an Apple II 5.25" disk. CP/M Plus (CP/M 3) decreased the number of system tracks by one and instead put the CCP (Console Command Processor) in the file system area. MS-DOS carried this process even further, and decreased the "boot area" to one single sector, keeping everything else in the file area (BIOS, OS, CCP) although with some limitations (these files had to reside at the very beginning of the disk and had to be unfragmented). > The saving grace with CP/M is that it does allocations in 1K blocks, That is configurable in the DPB (Disk Parameter Block); see the "CP/M Alteration Guide". > it appears to start allocating from the front, and it doesn't fill the > entire "catalog track". There is no "catalog track" on CP/M..... Above the BIOS level, the file system area (i.e. everything except the system tracks) on the disk are regarded just as a data area of some specific size. CP/M doesn't care where a new tracks start; that's a business only for the BIOS. The directory always resides at the start of the file system area, and then it contains a specific number of directory entries (64 entries on an Apple CP/M disk, but again that can be altered bu changing the DPB appropriately). The DPB does not reside on the disk though, but in memory at a specific place (there's apointer there among the pointers at the start of the CP/M BIOS). Each disk volume has its own DPB, thus different volumes can be disks with different characteristics (you could have one 8" floppy, one 5.25" floppy, one 3.5" floppy and one harddisk for instance). The CP/M DPB can descripe disks up to 8 MBytes large. On CP/M disks with a bootable system the DPB of course resides on the floppy -- somewhere on the system tracks, in an implementation specific way. But if the disk is not a bootable "system disk", then the disk itself contains no info whatsoever of the specifics of the disk format. > This means that the first file you create will > have data on the catalog track itself. So if you load the entire catalog > into memory, scan it, and find anything other than valid entries and 0xe5 > values, you know you have the order wrong. (If the files stored in the > catalog track happen to be filled with 0xe5 or look like valid catalog > entries, then you're in trouble. In that case, having the correct label > on the disk image is essential, or you're going to have to use "query > image format" to override the automatic detection.) > > Overall the CP/M code in CiderPress is the weakest of the filesystems, > since nobody ever wrote a "Beneath Apple CP/M". There was the official "CP/M Alteration Guide" by Digital Research, who gave enough BIOS details to let anyone implement their own BIOS. However, as opposed to Apple DOS, CP/M ran on a wide variety of hardware platforms, each having its own BIOS. And there was actually a book giving details of the Apple SoftCard CP/M BIOS --- don't remember the title, but important details from it have been summarized in: http://home.tiscali.se/pausch/apple2/CPM.ref.txt http://home.tiscali.se/pausch/apple2/Apple.CPM.ref.txt http://www.stjarnhimlen.se/apple2/CPM.ref.txt http://www.stjarnhimlen.se/apple2/Apple.CPM.ref.txt which also gives details from the standard CP/M documentation. I also describe the 640K diskette format I implemented for my own Apple II in the early 80'ies, using 80-track double-sided diskette drives from Teac. I compare that disk format with the Apple CP/M standard 13-sector and 16-sector formats, as well as the CP/M standard 8" SSSD format. So, yes, there have been plenty of docs about the Apple II CP/M format, although it wasn't that wellknown in large parts of the Apple II community. > (Oddly enough, there is the equivalent of a "Beneath SSI RDOS".) > There are some bit fields in the catalog entries that I still > don't understand. Consult the CP/M Alteration Guide in the CP/M manual: http://www.cpm.z80.de/manuals/archive/index.htm http://www.cpm.z80.de/manuals/archive/cpm22htm/index.htm (the latter is the CP/M manuals online in HTML format), or my summaries: http://home.tiscali.se/pausch/apple2/CPM.ref.txt http://home.tiscali.se/pausch/apple2/Apple.CPM.ref.txt http://www.stjarnhimlen.se/apple2/CPM.ref.txt http://www.stjarnhimlen.se/apple2/Apple.CPM.ref.txt > Fortunately I was able to find some large text files and compare > the output on an Apple II to the output from CP to verify that > all was well. -- ---------------------------------------------------------------- Paul Schlyter, Grev Turegatan 40, SE-114 38 Stockholm, SWEDEN e-mail: pausch at stockholm dot bostream dot se WWW: http://www.stjarnhimlen.se/ http://home.tiscali.se/pausch/ In article , Andy McFadden wrote: > Paul Schlyter wrote: >> The structure and contents of the CP/M system tracks are indeed >> completely implementation specific. It doesn't even have to be 3 >> tracks: om the classic CP/M disk format (8" SSSD) the system tracks >> were only 2 tracks; otoh each track on 8" SSSD disks contained more >> than each track on an Apple II 5.25" disk. > [... lots of stuff removed ...] > > So it sounds like a program that processes disk images needs a set of > DPBs, from which it chooses one based on the physical characteristics of > the drive. For the Apple II world there are separate DPBs for 13-sector > and 16-sector disks. Yes. But in practice you can forget about the 13-sector CP/M disks because they were rarely used. Early versions of the Z80 Softcard came with a 13-sector floppy (I still have it), but all subsequent Apple II software I ever saw for sale came on only 16-sector disks. >> Above the BIOS level, the file system area (i.e. everything except >> the system tracks) on the disk are regarded just as a data area of >> some specific size. CP/M doesn't care where a new tracks start; that's >> a business only for the BIOS. > > CiderPress has to be the BIOS as well as the operating system, which is > part of what makes all this so much fun. :-) Otoh in a disk image there are no tracks and sectors either, only a data area of some specific size. >>> (Oddly enough, there is the equivalent of a "Beneath SSI RDOS".) >>> There are some bit fields in the catalog entries that I still >>> don't understand. >> >> Consult the CP/M Alteration Guide in the CP/M manual: >> >> http://www.cpm.z80.de/manuals/archive/index.htm >> http://www.cpm.z80.de/manuals/archive/cpm22htm/index.htm > > This one I have seen, but I couldn't find details of filesystem structure > or directory layout in it. It has command references, boot loaders, and > what amounts to an RWTS system, but none of that helps much. OK -- a directory entry has the following structure: 1 byte User number, or E5h if it's a free entry 11 bytes Filename + extension: 8+3 characters 2 bytes Extent number of this entry 2 bytes Number of 128-byte records used in last alloc. block 16 bytes Allocation map for this directory entry A few comments: Extent number: large files in CP/M receives several directory entries, called "extents" - as many as needed to store the full allocation map for the file. On Apple CP/M (with 1K allocation blocks and singe bytes in the allocation maps), each directory extent is enough to define 16 Kbytes if the file. Thus, any file larger than 16K will get more than one extent, any file larger than 32K more than two extents etc. If 2K allocation blocks had been used instead, each allocation map would define 32K of the file, etc. The extent number defines the order between the different directory entries with the same name (e.g. MBASIC.COM on an Apple CP/M master diskette). Number of 128-byte records used in last allocation blocks: the CP/M file system defines the file length only in units of 128-byte "records" or "logical sectors". Since an allocation block usually contains several 128-byte records, this field defines how many records are in use in the last block allocated to that file. Allocation map of directory entry: single bytes of the disk has 256 allocation blocks or less, double bytes (little endian order) if the disk has more than 256 allocation blocks. In the latter case, each directory extent will only be able to map 8 allocation blocks to the file. Note that CP/M allows spares files -- these are marked by 00 in the corresponding position in the allocation map and since allocation block 00 always contains the directory, it cannot be a valid file allocation. Interestingly, the CP/M utility PIP fails to properly copy a sparse file -- only the data up to the first "hole" in the allocation map is copied. >> http://home.tiscali.se/pausch/apple2/CPM.ref.txt >> http://home.tiscali.se/pausch/apple2/Apple.CPM.ref.txt > > The DPB info is useful. Now, if somebody has a .NIB of a 13-sector CP/M > disk, I can try to make that work... Don't bother.... the 13-sector CP/M disks were almost never used. Instead, try to make CiderPress able to also transfer files to CP/M disk images - that would be immensely more useful. -- ---------------------------------------------------------------- Paul Schlyter, Grev Turegatan 40, SE-114 38 Stockholm, SWEDEN e-mail: pausch at stockholm dot bostream dot se WWW: http://www.stjarnhimlen.se/ http://home.tiscali.se/pausch/ Paul Schlyter wrote: > OK -- a directory entry has the following structure: > > 1 byte User number, or E5h if it's a free entry > 11 bytes Filename + extension: 8+3 characters > 2 bytes Extent number of this entry > 2 bytes Number of 128-byte records used in last alloc. block > 16 bytes Allocation map for this directory entry [...] > Number of 128-byte records used in last allocation blocks: the > CP/M file system defines the file length only in units of > 128-byte "records" or "logical sectors". Since an allocation > block usually contains several 128-byte records, this field > defines how many records are in use in the last block allocated > to that file. The part I couldn't figure out is the part you labeled "number of 128-byte records used". The values I'm finding don't quite match up with what you show. Here's a dump of the first 512 bytes of a typical directory: 000: 00 46 4f 52 4d 41 54 20 20 c3 4f 4d 00 00 00 12 .FORMAT COM.... 010: 02 03 04 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 020: 00 43 4f 50 59 20 20 20 20 c3 4f 4d 00 00 00 08 .COPY COM.... 030: 05 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 040: 00 4d 42 41 53 49 43 20 20 c3 4f 4d 00 00 00 80 .MBASIC COM.... 050: 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13 14 15 ................ 060: 00 4d 42 41 53 49 43 20 20 c3 4f 4d 01 00 00 40 .MBASIC COM...@ 070: 16 17 18 19 1a 1b 1c 1d 00 00 00 00 00 00 00 00 ................ 080: 00 47 42 41 53 49 43 20 20 c3 4f 4d 00 00 00 80 .GBASIC COM.... 090: 1f 20 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d 2e . !"#$%&'()*+,-. 0a0: 00 47 42 41 53 49 43 20 20 c3 4f 4d 01 00 00 48 .GBASIC COM...H 0b0: 2f 30 31 32 33 34 35 36 37 00 00 00 00 00 00 00 /01234567....... 0c0: 00 43 4f 4e 46 49 47 49 4f 42 41 53 00 00 00 3a .CONFIGIOBAS...: 0d0: 1e 38 39 3a 3b 3c 3d 3e 00 00 00 00 00 00 00 00 .89:;<=>........ 0e0: 00 50 49 50 20 20 20 20 20 c3 4f 4d 00 00 00 3a .PIP COM...: 0f0: 40 41 42 43 44 45 46 47 00 00 00 00 00 00 00 00 @ABCDEFG........ 100: 00 53 54 41 54 20 20 20 20 c3 4f 4d 00 00 00 30 .STAT COM...0 110: 48 49 4a 4b 4c 4d 00 00 00 00 00 00 00 00 00 00 HIJKLM.......... 120: 00 45 44 20 20 20 20 20 20 c3 4f 4d 00 00 00 34 .ED COM...4 130: 4e 4f 50 51 52 53 54 00 00 00 00 00 00 00 00 00 NOPQRST......... 140: 00 41 53 4d 20 20 20 20 20 c3 4f 4d 00 00 00 40 .ASM COM...@ 150: 55 56 57 58 59 5a 5b 5c 00 00 00 00 00 00 00 00 UVWXYZ[\........ 160: 00 44 44 54 20 20 20 20 20 c3 4f 4d 00 00 00 28 .DDT COM...( 170: 5d 5e 5f 60 61 00 00 00 00 00 00 00 00 00 00 00 ]^_`a........... 180: 00 4c 4f 41 44 20 20 20 20 c3 4f 4d 00 00 00 0e .LOAD COM.... 190: 62 63 00 00 00 00 00 00 00 00 00 00 00 00 00 00 bc.............. 1a0: 00 52 57 31 33 20 20 20 20 c3 4f 4d 00 00 00 12 .RW13 COM.... 1b0: 64 65 66 00 00 00 00 00 00 00 00 00 00 00 00 00 def............. 1c0: 00 41 50 44 4f 53 20 20 20 c3 4f 4d 00 00 00 0d .APDOS COM.... 1d0: 67 68 00 00 00 00 00 00 00 00 00 00 00 00 00 00 gh.............. 1e0: 00 53 55 42 4d 49 54 20 20 c3 4f 4d 00 00 00 0a .SUBMIT COM.... 1f0: 3f 69 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ?i.............. The lazilong posting (elsewhere in this thread) showed: typedef struct { Byte userNumber; Cpm_FileName name; Cpm_FileType fileType; Byte extentNum; RboShort system; Byte numRecords; Cpm_BlockSpec extent[Cpm_kBlocksPerExtent]; } Cpm_DirEntry; In which case "system" is pretty much always zero, and "numRecords" varies from 0x0a to ox80, suggesting that "numRecords" is actually the total number of 128-byte records in this extent of the file, rather than the number used in the last allocation block. -- Send mail to fadden@fadden.com (Andy McFadden) - http://www.fadden.com/ CD-Recordable FAQ - http://www.cdrfaq.org/ CiderPress Apple II archive utility for Windows - http://www.faddensoft.com/ Fight Internet Spam - http://spam.abuse.net/spam/ & http://spamcop.net/ In article , Andy McFadden wrote: > The part I couldn't figure out is the part you labeled "number of > 128-byte records used". The values I'm finding don't quite match up > with what you show. OK, it appears the book I looked this up in (Andy Johnson-Laird's "The Programmer's CP/M Handbook") wasn't quite accurate here. That book probably defines the contents of the FCB instead (which is more or less a mirror of the first 16 bytes of the directory entry; however there are some differences). FCB = File Control Block, a data area the application provides to CP/M when dealing with files: you fill in the filename when opening the file, and give a pointer to the FCB to CP/M, and then CP/M uses that area as a "scratchpad" for its file operations. This is vaguely corresponding to the "buffers" in Apple DOS, with the difference that the application, not the OS, provides space for these buffers (a separate buffer must also be provided for the data read from and written to the files). This had the disadvantage of more trouble for the application, but also had the advantage that there was no limit of concurrently open files in CP/M, since the application could provide as many FCB's and buffers as it needed. All within the 64K limit of RAM for the application plus CP/M itself, of course..... I did a Google search for "CP/M directory entry", and found a site with info which agrees with your findings: http://www.seasip.demon.co.uk/Cpm/format22.html Describes the format of CP/M 2.2 disks http://www.seasip.demon.co.uk/Cpm/formats.html Has links to similar descriptions of CP/M 1.4, CP/M 3 and CP/M 4. Ver 1.4 and 4 were never used on the Apple II, and ver 3 was rarely used (by "The CP/M Card", which had its own 64K RAM _and_ used Apple's RAM, which that card could access from the Z80 as well, by bank switching. CP/M 3 could use more than 64K of RAM, through bank switching, and put the CCP, BDOS and part of BIOS in the other memory bank if available). http://www.seasip.demon.co.uk/Cpm/ Look under "Technical information" here..... On http://www.seasip.demon.co.uk/Cpm/format22.html you can find this info (note: exm=0 on all Apple II CP/M floppies, including my own 80-track double-sided 640K Apple II CP/M floppies): -------------------------------------------------------------------- CP/M 2.2 directory The CP/M 2.2 directory has only one type of entry: UU F1 F2 F3 F4 F5 F6 F7 F8 T1 T2 T3 EX S1 S2 RC .FILENAMETYP.... AL AL AL AL AL AL AL AL AL AL AL AL AL AL AL AL ................ UU = User number. 0-15 (on some systems, 0-31). The user number allows multiple files of the same name to coexist on the disc. User number = 0E5h => File deleted Fn - filename Tn - filetype. The characters used for these are 7-bit ASCII. The top bit of T1 (often referred to as T1') is set if the file is read-only. T2' is set if the file is a system file (this corresponds to "hidden" on other systems). EX = Extent counter, low byte - takes values from 0-31 S2 = Extent counter, high byte. An extent is the portion of a file controlled by one directory entry. If a file takes up more blocks than can be listed in one directory entry, it is given multiple entries, distinguished by their EX and S2 bytes. The formula is: Entry number = ((32*S2)+EX) / (exm+1) where exm is the extent mask value from the Disc Parameter Block. (my note: on Apple II, exm=0) S1 - reserved, set to 0. RC - Number of records (1 record=128 bytes) used in this extent, low byte. The total number of records used in this extent is (EX & exm) * 128 + RC (my note: since exm=0 on Apple II, this will always be equal to RC. exm will be different from 0 only on large disks such as harddisks) If RC is 80h, this extent is full and there may be another one on the disc. File lengths are only saved to the nearest 128 bytes. AL - Allocation. Each AL is the number of a block on the disc. If an AL number is zero, that section of the file has no storage allocated to it (ie it does not exist). For example, a 3k file might have allocation 5,6,8,0,0.... - the first 1k is in block 5, the second in block 6, the third in block 8. AL numbers can either be 8-bit (if there are fewer than 256 blocks on the disc) or 16-bit (stored low byte first). -- ---------------------------------------------------------------- Paul Schlyter, Grev Turegatan 40, SE-114 38 Stockholm, SWEDEN e-mail: pausch at stockholm dot bostream dot se WWW: http://www.stjarnhimlen.se/ http://home.tiscali.se/pausch/