Msg: 5925 *Conference*

08-07-95 12:10:50

From: RON WIESEN

To : COMET _

Subj: REPLY TO MSG #5900 (UNDELETE -- COLD START RECOVERY)

I've read "UNDELETE -- COLS START RECOVERY" in the MSG thread: 5174, 5186,
5222, 5446, and 6013.  Two things bother me and they may be related.
 
1.  Context of cold-start is unclear.  In MSG 5174, phrase "picture loading...
files, doing a cold-boot," and phrase "you walk over and cold-start..." lead me
to think that the cold-starts you are concerned with are intentionally induced
by the operator.  Are you speaking about cold-starts that are a direct result
of operator intention and action and thus no corruption exists in memory?
 
2.  Logic seems flawed.  In MSG 5174, phrase "...  only reason the files don't
show...  MUST be that the directory is munged." But in MSG 6013, phrase "if one
merely had a copy of the file directory.  [Assuming that the copy wasn't
munged,...]" implies that whatever afflicted the file directory DOES NOT
AFFLICT A COPY OF IT.  Much of the rest of MSG 6013 pertains to schemes to
install a directory copy mechanism and where to maintain the copy.  This is
fine technically and I have some suggested methods for it, but it seems doomed
to become "another munged file directory" which solves nothing.
 
At this time I'll offer a few facts and ideas which assume the context that the
cold-start is unintended.
 
1.  Fact -- Where the Operating System (OS) finds corruption it cold starts.
The OS can not discriminate between a corrupt file directory versus corrupt
file construction, so cold-start is the only course it can take.
 
2.  Fact -- Generally, the OS finds corruption after it has had time to spread
beyond its inception.  As people, we then are generally faced with multiple
"munges" rather than just an isolated "munge".
 
3.  Fact -- As you know, the 1st byte of each 11-byte directory slot identifies
the file type and the 2nd and 3rd bytes are a low/high pair which is the
File_Address to a given file in RAM.  Cold-start replaces all file types with
the "unused" value of zero.
 
4.  Fact -- Files are located in RAM, one adjacent to the next, in progressive
order by type: BASIC, ASCII and Machine Language (M/L).  This dictates the
progression of file recovery: BASIC first, then ASCII, and M/L last.  Any
"recovery shuffles" must be reflected, as relative compensation, in the
File_Address of all remaining candidate slots of the file directory.
 
5.  Idea -- Use a statistical recovery program which the operator guides based
on his knowledge of his files.  N-DKTR.BA is an example but it relates to
directory and file recovery for a RAMPAC.  Screen view of ASCII files is
helpful.
 
6.  Idea -- In addition to determining the end of a BASIC file by searching for
a zero triplet (00H, 00H, 00H) file end mark, confirm its line chain in
relative, absolute, and logical terms and assign increasing statistic
probability of "correctness" as each term is confirmed.
 
Each line has a 4-byte header and a zero byte line end mark:
 
| NEXT_LINE_@ | THIS_LINE_# | ......... |   LINE END  |
|low/high pair|low/high pair|           |      00     |
 
Immediately following the last line, a NEXT_LINE_@ of zero (ROM address) is
found.  So with the line end mark of the last line followed by the low/high
byte pair for a NEXT_LINE_@ of zero, the end of a BASIC file has a zero
triplet.  In all other cases the high byte of NEXT_LINE_@ must be non zero (RAM
address), so even where the byte pair of THIS_LINE_# is zero (line number 0) a
zero triplet is precluded.  This explains the fact that a zero triplet
unconditionally identifies the end of a BASIC file.
 
In the relative view of a line chain, the difference between the NEXT_LINE_@ of
"adjacent" lines is an absolute constant regardless of file locale in RAM.
This absolute difference persists as the OS shuffles the BASIC file locale
upward or downward in RAM where prior lines are inserted/deleted/edited or
BASIC files in a lower locale are created/deleted/edited.
 
In the absolute view of a line chain, the OS applies relative compensation to
the NEXT_LINE_@ of all lines (including line chains in other BASIC files) in
RAM locales above an inserted/deleted/edited line during such a shuffle.  When
the shuffle is concluded, the NEXT_LINE_@
of all of the lines agree with the absolute locale of their respective BASIC
files in RAM.
 
In the logical view of a line chain, the OS sorts lines based on THIS_LINE_#
where a new line is inserted in a BASIC file.  When the sort is concluded, the
THIS_LINE_# of all lines are in ascending line number order.
 
7.  Idea -- M/L file locales are top-most in RAM so M/L files are the files
that are most exposed to the effect of faulty shuffles of the BASIC, ASCII, and
lower locale M/L files.  Only the "locale correctness", not "content
correctness" can be statistically ascertained.  M/L file "correctness" should
be only confirmed via the operator.  Listing of TOP, END (i.e., TOP+LEN-1), and
EXE is helpful.
 
The File_Address values within the file directory and the FILE LENGTH values
within the 6-byte headers of M/L files are the only means to evaluate "locale
correctness" for a series of M/L files.  A review of how the OS handles M/L
files follows.
 
Each M/L file has a 6-byte header which is appended by the object code image:
 
| TOP ADDRESS | FILE LENGTH | EXE ADDRESS |  object code image  |
|low/high pair|low/high pair|low/high pair| absolute references |
 
As you know, the 2nd and 3rd bytes of each 11-byte directory slot are a 
low/high pair which is the File_Address to a given file in RAM.  Where file
type is M/L (per the 1st byte of a file slot), this address is to the 6-byte
header.  Note that not all M/L files are programs.  Where the M/L file is a
program, the appendage is a .COmmand object code image of the program.
 
All memory references within the object code image are absolute.  Any reference
to a point internal to the program is absolute and it is based on the ORGinate
in the source program; TOP ADDRESS in the object is the same as ORGinate of the
source.  Where a M/L file is LOADMed, RUNMed, or invoked from the Main menu,
the OS copies the object code image into HIMEM protected RAM.  Where a M/L file
is a program and it is RUNMed or invoked from the Main menu, the OS also passes
program control execution to the .COmmand code at the EXE ADDRESS.
 
M/L file recovery means finding at least one series of one or more FILE LENGTHs
in RAM where each FILE LENGTH agrees with the difference between a pair of
"locale adjacent" File_Addresses in the directory slots.
 
8.  Idea -- This bears only on "copy of the file directory" in MSG 6013.
Ignore the 5 built-in ROM applications, the option ROM, the unnamed BASIC file,
and the PASTE file.  This reduces from 27 the number of candidate file names to
19.  These 19 slots are contiguous in the file directory.
 
Rather than "save" all 11 bytes of these slots, save no more than the first 3
bytes (type and File_Address).  For 3 bytes per slot, this requires 57 bytes.
Type is the first byte, and File_Address is the second and third bytes.  The
remaining 8 bytes are the file name and name extension.  Based on "saved
before" and after content of the File_Addresses (I still am unclear how the
"saved" copy is immune to the cold-start affliction), I suppose the point
(i.e., file directory slot) of any interrupted shuffle can be determined.
 
For minimal storage requirement, save only bits 7, 6, 5, and 4 of the first
byte (i.e., type) of these 19 slots.  Packing them at 2 slots per byte, this
requires 10 (9 and 1/2) bytes.
 
9.  Idea -- This bears on "piece of RAM...  to store this" in MSG 6013.  For
the 10 byte minimum storage, there are several choices:
 
 A. within 18-byte area at FA8E
 B. within 95-byte area at F8C1
 C. 10-byte end of 64-byte RS-232 buffer (ends at FF85H)
 D. 10-byte end of 64-byte Typeahead buffer (ends at FFEAH)
 
Choice A seems best to me.  I don't know what the OS uses this area for but it
appears to survive a cold-start.
 
Choice B also appears to survive a cold-start.  Given its large size, I suspect
it may conflict with some operation such as file buffering.
 
Choices C and D can conflict with operator usage.  I'm not sure they survive a
cold-start, so they may not be viable.
 
For the 57 byte storage requirement, choice B is the only choice.
 
If choice A and choice B are both viable, there is a tidy arrangement that
dovetails with "M/L to copy the directory...  the entry invisible" in MSG 6013.
Assuming one particular file directory slot (e.g., last one) is either ignored
or dedicated to some special purpose, then only 18 RAM file slots need be
saved.  Choice A is where the file type info (1st byte of 11-byte slot) is
saved.  No packing is used, so bits 3 to 0 of each byte (file type) may be
transient flags the OS uses during shuffles and thus may prove helpful.  Choice
B is where the File_Address (2nd and 3rd bytes) is saved followed by a copy of
3 critical RAM address pointers (low/high byte pairs) from the OS, plus a 
53-byte block of recover/save code.  Although file name and name extension info
is not saved, names A through R can be arbitrarily assigned to the 18 slots at
recovery and the name extensions .BA, .DO, and .CO can be presumed per the file
type.  See detail below.
 
18-byte cell (18 by 1) FA8E file type, RAM file slot #1
 .
 . FA9F file type, RAM file slot #18.
 
 
95-byte cell (18 by 2, 3 by 2, plus 53) F8C1/F8C2 File_Address, RAM file slot
#1
 .
 . F8E3/F8E4 File_Address, RAM file slot #18
 
F8E5/F8E6 address to first BASIC file F8E7/F8E8 address to first ASCII file 
F8E9/F8EA address to first M/L file
 
F8EB to F91F recover/save code.
 
Recovery is invoked from BASIC by CALL-1813.  Code at F8EB (-1813 or 63723)
recovers files.  First it assigns file names A through R to the 18 files slots.
Based on file type, it assigns the name extensions
.BA, .DO, and .CO to the slots.  The addresses to the first BASIC, ASCII and
M/L files assist in the progressive file recovery that I