RAM File Handling

Origin

This information comes from the NEC 8201A technical reference. The original is in broken english, and has some photocopy issues. I will fix what I can.

Overview

In this chapter, the techniques for managing RAM files is described. The main purpose is to create or delete a RAM file for RAM and Option ROM resident applications. If there is a violation of the rules of RAM file handling, the file you made, and possibly all files in the RAM may be lost upon entry to standard file management routines in the main ROM.

There are many useful routines the programmer may leverage to avoid breaking the file management rules. Unfortunately, heavily leveraging file management routines from the main ROM by a program operating from an Option ROM will incur a performance penalty. If you want to manipulate the RAM filesystem without the Main ROM, please make sure to follow the rules laid out in this chapter. Also refer to the sections on "Bookkeeping" and "Directory Structure."

Checklist for RAM File Handling

Make sure that there is enough free space

When a new file is opened, or new data is appended and inserted, investigate whether there are enough free bytes available. In particular, the free area required simply to create an empty file is sometimes overlooked. At least one byte is required to create a DO file. Six bytes are required for a CO files. Refer to "What is a RAM File" and the following sections.

You can find where the free space is in the figure "Bookkeeping Area." The difference between the pointer "STREND" and the value in the stack pointer indicates the free size. But don't forget that some area will be used for the stack operation in that free area. For instance, the make-room routine used in BASIC and TEXT recognizes that the current free space is less 120 bytes than that difference. In other words, 120 bytes is always maintained for the stack area when new data is stored. Refer to "MAKHOL" in "Useful Routines for RAM File Handling In the Main ROM."

Register the file name correctly

The contents of the directory is described in "Directory construction." Everyone seems to remember to register the filename in it. But often programmers will forget to set up the directory flag byte and the starting address of the file. If you don't set the directory flag, the file might be deleted by Menu or another operation. If you write a bad starting address in the address field, the implied (by pointer order) link between the directory entries and the corresponding files will be lost. The result of breaking that implied link is that the user cannot select a file in Menu mode, or the laptop hangs or crashes. Refer to "Directory Construction" and the following sections to avoid this outcome.

Maintain the order of the files

In order to maintain the order of the files, the starting address of the new file must be selected carefully. For a new DO file, we have to set ASCTAB - 1 as the starting address of that new file in the directory area. And for a new BA file, you have to register the ASCTAB - 1 in the "non-registered" file's directory area and insert double NUL there. That new BA file will be created at ASCTAB - 1 and will have the starting address, ASCTAB - 2. In making both a new DO file and a new BA file, LNKFIL should be executed before end of its process. Refer to "Useful Routines for RAM File Handling in the Main ROM" to understand the role and usage of LNKFIL.

Make and shrink a hole safely

The calculation of the free space is very important. And you have to maintain the stack area when you make room. One more important issue in the management of the pointers: The reason why many programs, Menu, BASIC, TEXT and so on can use the same RAM area safely is that they adjust the pointers for RAM every time they change the RAM configuration. For example, when BASIC deletes a BASIC program file, it changes many pointers: STREND, ARYTAB, VARTAB, BINTAB and ASCTAB. And, it turns off the "in-use" flag and the end of the operation to free the entry. Refer to MAKHOL and MASDEL in "Useful Routines for RAM File Handling in the Main ROM."

Note that MAKHOL and MASDEL seem to have been created with DO files in mind. For BA and CO files there is a small amount of file section pointer fixup work to do across the call.

Insert the mandatory bytes in the file

When you open a DO file, you have to enter one byte of data at least. The data is Control-Z ($1A). This character marks the end of the file. A BA file must have a proper "next line" link structure, and have two NUL bytes at the end of the file. A CO file must at least have a 3 word (6 byte) header. Refer to "What is a RAM File" for more details.

Fix up starting addresses in the directory

When you change the RAM configuration, you have to care not only for the section pointers, but also the starting address for each file in the directory. Given the order of the sections, if you add a DO file, directory entries for all files above it, including DO, BA, and CO must have their start pointer corrected. Refer to "LNKFIL" in the "Useful Routines for RAM File Handling in the Main ROM."

Do not embed forbidden characters in DO files

Never store $00, or $08 characters in a DO file. Never store a $1A (Control-Z) character in a DO file except as the end of a file marker. Refer to the section "DO File"

How to make a new file

How to register a new file name

When creating a new file, the first thing you should do is register the file in the user's directory area. The user's directory area starts at USRDIR. Last byte of the directory area is a $FF byte, called the "Directory Terminator."

An in-use slot has a type byte greater than or equal to $80, that is, the high bit set. This makes it easy to identify a free slot. Refer to the sample program shown later.

It is important to ensure a file you wish to create does not already exist. Two files with the same name can cause serious problems. So, check for collisions, and offer the user the choice of deleting the old file or aborting the operation.

If you find a free slot, register a proper flags byte, a start address, and the filename. Refer to "Directory Construction."

How to make a DO file

If you have already registered the filename and directory flag at the slot in the directory area, the remaining information to store is the start address of the DO file. If you didn't read "How To Register A New File Name" and you have not set the filename and directory flag yet, read that section and fill those in first.

Usually the DO file is created just above the value of ASCTAB. ASCTAB holds a pointer to the start of the DO file section. Refer to the figure in "What is a RAM file." If you go with the standard rule which Menu, BASIC and others in the Main ROM use, you can store the result of [ASCTAB] - 1 as the starting address of the new file. Then the registration of the new DO file is completed. The reason why we have to use ASCTAB - 1 instead of ASCTAB is to maintain the order of the files.

The LNKFIL routine corrects the directory start pointers after changes to the RAM filesystem. It does this by walking all files in memory, pairing each subsequent file with the unprocessed entry in the directory having the lowest start pointer. If the temporary start address you choose for your new DO file is identical to an existing entry, the assignment of directory entry to file is ambiguous. If you just used [ASCTAB] as the address instead of [ASCTAB] - 1, that is exactly what would happen. Refer to "LNKFIL" in "Useful routine for RAM file handling in the Main ROM."

But there are still two more steps. First, you must insert the eof-of-file byte ($1A). The other is, you must make room for the data to be placed in the file (and correct the directory afterward).

There is no DO file whose size is zero, since every DO file must have a $1A terminator.

In order to make room for the new file, a convenient routine is in the Main ROM. Its name is MAKHOL, MAKe HOLe. This routine inserts spaces from the specified point and whose size is contains in the BC register. Refer to "MAKHOL" in "Useful routines for RAM file handling in the Main ROM." The concept of the MAKHOL operation is shown briefly in that section.

If there is no free area in RAM, and you cannot insert a $1A, you must clear the directory entry "in-use" flag.

Finally, to fixup the starting address in the directory area, the LNKFIL routine is available in the Main ROM. The flow diagram of that routine is shown in the "Useful routines for RAM file handling in the Main ROM." You can get information there to write your own LNKFIL routine as well.

If you succeed in inserting the $1A, and in setting a starting address, you can now save data to the new file using MAKHOL, and correct the directory table using LNKFIL. Refer to another section to determine how to Append, Insert, and Delete data. The sample program in the following section will show you how to make a new file and save data.

Summary:

1. Find a free slot in the directory. Give up if there are no free slots, or the file exists and it is not OK to delete it.
2. Register the filename and directory flag in the free slot.
3. Get [ASCTAB] - 1 and save it in the address field of the slot.
4. Make a one-byte hole at [ASCTAB]; free the directory entry and abort if cannot make a one-byte hole.
5. Set $1A in the one-byte hole.
6. Fixup the directory entry starting addresses using LNKFIL
7. Done.

How to make a BA file

There is a difference between how to make DO file and how to make a BASIC file. There is no differnce in the registration of the filename and the directory flag. The first difference is that you have to end the BASIC file with two NUL (0) characters instead of Control-Z in DO files. In order to understand what double NUL means, you have to be familiar with the function of the link pointer in Microsoft BASIC. But the basic concept of RAM file handling is exactly the same as for a DO file. That is, you register the file in the directory and make room for the file data.

The second difference is the new BA file is created just above the BA files which are already stored. In other words, the new BA ifle is inserted just below the lowest DO file. Refer to the section, "What is a RAM file?"

I will assume that if you are looking to create BASIC files in memory, that you already know how to build a tokenized BASIC program in RAM. The details of that are out of scope for this article.

How to create a new BA file:

1. Lookup the filename in the directory table. If found, delete the file or abort the operation.
2. Allocate a slot in the directory table. If no free slots exist, abort the operation. Otherwise, setup the directory flag and copy in the filename.
3. Copy ASCTAB -1 into NULDIR, the unsaved BASIC program's directory slot. Make a 2-byte hole and store the double NUL into the unsaved BASIC program.
4. Make a hole as large as possible at ASCTAB - 1.
5. If the size of the hole is too small for the new BA file, delete the hole, free the directory entry, and abort the operation.
6. Copy the BASIC program into the hole. Don't forget to ensure double NULs at the end of the program.
7. Fill in the start pointer into the directory entry. Normally, the address used is one byte less than the starting of address of the unsaved BASIC program.
8. Reduce the size of the hole to the minimum size possible if you overshot the size.
9. Adjust the section pointers ASCTAB, BINTAB, VARTAB, ARYTAB, and STREND. Fixup the starting addresses of all files in the directory. Refer to LNKFIL.
10. Done.

How to make a CO file

The difference between DO and CO files is that CO files have a 3 word (6 byte) header, rather than a terminating $1A. So when wedging open a spot for the CO data, ensure you have room for 6 bytes of overhead.

Heading of a CO file:

START ADDRESS	2 bytes
LENGTH	2 bytes
EXECUTION ADDRESS	2 bytes

Upon creation of a CO file, make sure that you correct the VARTAB, ARYTAB and STREND pointer values.

The CO file is usually made just under the address pointed to by VARTAB. So, the starting address of the other files need not be changed after saving a new CO file. But, I recommend to run LNKFIL after saving a new CO file for safety.

To make a CO file:

1. Check free space; abort the operation is there is not sufficient space for your program plus 6 byte header.
2. Lookup the filename in the directory table. If found, delete the file or abort the operation.
3. Allocate a slot in the directory table. If no free slots exist, abort the operation. Otherwise, setup the directory flag and copy in the filename.
4. Wedge open a hole at the address pointer to by VARTAB and store the file header and program there. You have to preserve and restore BINTAB across the MAKHOL operation if you use MAKHOL.
5. Adjust VARTAB, ARYTAB and STREND. Fix up the starting address of all files in the directory "for safety." "If you use LNKFIL for adjustment of the start addresses, you have to care about BINTAB as you do with MAKHOL."
6. Done.

How to delete a file

To delete a file from RAM, you must release its directory entry by clearing the high bit of the directory entry flag byte. In addition, you must adjust the file section pointers. Finally, you must fixup the specified starting addresses of all files in the directory.

How to delete a DO file

First, lookup the directory entry for the file to be deleted. Check the file to ensure it is not the file currently opened in BASIC. Do not delete that file.

Steps to delete a DO file:

1. Lookup the directory entry with matching filename.
2. Abort the operation if the directory flags indicate that this file is opened by BASIC.
3. Get the starting address of the file.
4. Determine the size of the file by searching for the $1A character.
5. Adjust the file section pointers to remove the hole. MASDEL will do it automatically. MASDEL adjusts the pointers BINTAB, VARTAB, ARYTAB, and STREND automatically.
6. Revise the starting address of other files. LNKFIL will do this.
7. Free the directory table entry
8. Done.

How to delete a BA file

When you do not have a BASIC program open, there are few differences between killing a DO file and a BA file. The main difference is in determining the size of the file. For a BA file, you must follow the "next line" linked list to the terminating double-NUL. If you can utilize the Main ROM, you can call the CHEAD routine. CHEAD searches for the end of a BA file. MASDEL can be used to delete the data by revising the file section pointers. However, you must give consideration to the TXTTAB pointer. If you delete a BA file which is located "under" the file pointed to by TXTTAB, you must adjust TXTTAB. This case occurs when TXTTAB points to the second BA file, and then you delete the first BA file. Finally, you must fix up the file start pointers in the directory area. LNKFIL will do this.

NOTE: MASDEL does not adjust ASCTAB. When a BA file is killed, ASCTAB must be adjusted. So, after calling MASDEL, adjust ASCTAB. Refer to the sample program in the following section. Also, "How to make a BA file" should yield some clues.

Another difference is that there is a limitation in deleting the current BASIC (open/loaded or running) program. This is important to consider if you are writing a machine language subroutine that is available to a BASIC program.

NOTE: TXTTAB has no meaning when the system is not in BASIC mode.

Refer to "What is a RAM file" and "Bookkeeping area" to understand the issues around BA files and TXTTAB.

To delete a BA file:

1. Search for the filename in the directory.
2. Check the directory flag and if the file is not BA, abort the operation.
3. Get the starting address of the file from the directory entry.
4. Compare the starting address to TXTTAB. If they are identical, you must not delete the file. Otherwise, just remember which is higher, the starting address or TXTTAB.
5. Determine the file size by searching for the end-of-file using CHEAD
6. Adjust the file section pointers and block move the file sections areas to remove the file bytes. MASDEL will update the pointers BINTAB, VARTAB, ARYTAB, and STREND automatically. It will not adjust ASCTAB, but note that it returns the negative length of the file in BC. This can be used to adjust ASCTAB.
7. Adjust ASCTAB.
8. Revise the starting address of other files using LNKFIL.
9. Recover the comparison result between the start address and TXTTAB. If TXTTAB is greater than the start address, adjust it.
10. Release the directory entry.
11. Done.

To delete a CO file

You don't have to care about where you are in now like killing a BA file or killing a DO file. You may delete any CO file you want to delete, even if you are executing that CO file. The CO file is copied to the specified area when the program is invoked from Menu or BASIC mode. So, the "CO" program can delete the "CO file of itself, and can recover some memory.

Unfortunately, you cannot simply use MASDEL to shrink the hole made by killing the CO file, since MASDEL changes the pointer BINTAB. So if you wish to use MASDEL, you must save BINTAB and restore it after calling MASDEL.

To delete a CO file:

1. Search for the file you want to delete by name
2. Save the starting address from the directory entry
3. Calculate the size of the CO file. The size of the CO file is the contents of the header length field, plus the 6 header bytes themselves.
4. Set the starting address and length for MASDEL.
5. Save BINTAB
6. Call MASDEL
7. Restore BINTAB
8. Release the directory entry
9. Done.

How to append or insert data into a DO file

Appending data to a DO file is easy. First, get the starting address of the DO file, and walk the file contents to find the $1A. Then use MAKHOL to open up space. When done, revise the directory using LNKFIL.

Appending data to a DO file:

1. Lookup the file by name in the directory
2. Check the file type and status in the entry flags.
3. Get the starting address from the entry
4. Search for the insertion point (for append, it's the position of $1A)
5. Use MAKHOL to open up space
6. Store data in the open space
7. Shrink the hole if you overshot the size using MASDEL.
8. Revise the directory table (starting addresses).
9. Done.

How to delete data from DO file

To delete data from the DO file, simply use MASDEL, followed by LNKFIL to delete the data.

Useful routines for RAM file handling in the Main ROM

There are several useful routines in the Main ROM for RAM file handling. Of course, you have accept the overhead of bank switching to utilize these routines from Option ROM space.

The routines to be presented are:

• MAKHOL: Make room for data entry by block moves and adjustment of section pointers
• LNKFIL: Correct the start addresses in the directory area
• MASDEL: Remove bytes added by calling MAKHOL.
• CHEAD: Walk a BA file to find the end.

MAKHOL

Make a hole

ADDRESS $6B6D

ENTRY:

[HL] points where you want to wedge open a hole

[BC] size of the hole to create

EXIT:

[HL] and [BC] are preserved

Carry is set if insufficient memory

In order to know the free area's size, STREND is the best pointer. The value of STREND and your file's size should be less than [SP] - 120. The 120 bytes are reserved for the stack. If there is enough room, MAKHOL shifts all data between the specified address and STREND. If not MAKHOL returns with carry set. The MAKHOL operation is detailed below:

1. Return with carry set (out of memory) if STREND + hole size < SP - minimum stack size (120 bytes)
2. Move the data between the specified address and STREND.
3. Adjust the pointers ASCTAB, BINTAB, VARTAB, ARYTAB and STREND.
4. Return

It is unnecessary to care about the pointers unless you make your own MAKHOL routine. The MAKHOL in Main ROM manages the pointers automatically. But it does not revise the starting addresses in the directory fields. For this, use LNKFIL.

NOTE: When you make a hole at ASCTAB to create a new DO file, you have to adjust the pointers BINTAB, VARTAB, and ARYTAB. ASCTAB must be modified only when you make a hole at ASCTAB to register a new BA file.

Obviously, calling MAKHOL too often results in excessive overhead. It is preferable to call MAKHOL with a large number in the BC register, and shrink the file to minimum size later using MASDEL.

The NEC 8201 documentation states that BINTAB must be preserved and restored across MAKHOL. The implication is that MAKHOL adjusts BINTAB by the size of the hole being created, which is actually incorrect for holes created at VARTAB.

LNKFIL

Fix up the directory start pointers

ADDRESS $2146

ENTRY:

NONE

EXIT:

NONE

All registers may be altered.

This routine fixes up all pointers from directory table entries to the start of the associated file. To avoid overhead of constantly updating these pointers, many operations defer calling LNKFIL to the end of the operation. It is the programmer's responsibility to ensure that LNKFIL is called when required. For instance, when a file is deleted, all link pointers should be fixed up before performing further I/O.

1. Mark the all valid directory flag (turn 0 bit of all valid directory flag)
2. Get the lowest file address
3. Get the lowest link pointer in the valid file's directory
4. Save the link pointer
5. Search the lowest link pointer in the marked files in directory area
6. Save the saved link pointer at this marked filesm link pointer field
7. Clear the mark from the directory flag of that file (turn off bit 0)
8. Get next lowest file address from the bottom of RAM
9. Go back to step 5 unless the mark been removed from all directory flags?
10. Return

When the top address of the next file is searched, the pointers ASCTAB and BINTAB are useful to know what kind of file is currently being searched.

MASDEL

Delete BC bytes from HL

ADDRESS: $6B9F

ENTRY:

[HL] pointer of the hole should be squeezed

[BC] size of the hole.

EXIT:

[HL] preserved

[HL] preserved

This routine performs the reverse operation of MAKHOL. The data above the HL + BC is moved up. And the pointers, BINTAB, VARTAB, ARYTAB are modified. If you use this routine for shrinking a hole of BA file, you must adjust ASCTAB with the negated [BC] after exiting this routine, since MASDEL does not correct ASCTAB.

Also, you can adjust the TXTTAB by using this negated BC counter if necessary. You have to adjust TXTTAB when you remove a BA file which is located at a lower address than that pointed to by TXTTAB.

If you want to utilize this routine for CO file, you must correct BINTAB after calling MASDEL. MASDEL was designed for deleting bytes from a DO file, so it adjusts BINTAB down by the deletion size. But when deleting a CO file, BINTAB shouldn't change. To deal with this, you need to save and restore BINTAB across calls to MASDEL, or add back in the number of bytes you are deleting to BINTAB.

CHEAD

Search for the end of a BASIC program

ADDRESS: $05F4 (I think)

ENTRY:

[HL] start of the BASIC file

EXIT:

[HL] contains the address of the last byte of the file + 1.

All registers and flags may be modified

The main purpose of CHEAD is to fixup the "next line" linked list pointers of a BASIC program. CHEAD goes through program storage for a given file and fixes up all the links. The end of each line is found by searching for the single zero at the end. The double-NUL link is used to detect the end of the program. So, upon return, HL contains the starting address of the subsequent file.

Example Programs

The example programs are not presented until such time as they can be corrected and tested. As presented in the technical reference, they are pretty buggy.