	    -----------------------------------
	      The Sather and pSather Runtimes
	    -----------------------------------

There are seven important interfaces to the Sather compiler which are
external to the compiler, that is, not written in Sather language
itself.  They are:

    1. The compiler command line interface and module file syntax.

    2. The Platform dependent information

    3. The configuration file syntax.  The configuration file is
       created at installation of the compiler and read when the
       compiler starts.

    4. Compiler generated macros.  These macros are used by the runtime
       and runtime kernel to select code appropriate to the specific
       compilation environment.

    5. The general runtime.  This is a set of C macros and functions.
       The general runtime is the target language of the compiler.

    6. The runtime kernel.  This is a set of C macros, used by the
       general runtime.  Porting to another platform may be achieved by
       rewriting the runtime kernel, although good performance may
       require platform-specific changes to other levels.  The general
       runtime is written using the runtime kernel as the machine
       abstraction.

    7. The Library organization and the creation of some classes
       via special preprocessing commands out of other classes

The first is described in the 'sacomp' man page.  This document specifies
the other ones in the following sections.



The Platforms
-----------------------------------------------------------------

The Sather language by itself is platform independent, but the runtime used
to run sather programs is not. The garbage collector, special system
classes (Sockets, UNIX calls, TCL/TK interface, the pSather extensions)
depend on the particular platform and not all of them are available on all
platforms.

Each platform has its own directory in the System/Platforms directory of
the sather distribution. Each platform directory must include at least the
following four files, but these files may be emtpy

	CONFIG:		Entries in this CONFIG file overwrite the the 
			entries in the CONFIG file stored in 
			System/Common/CONFIG.

	Platform.module:This module file is used as if it would have been
			added to the command line of the sather compiler.

	header.h:	This file is included at the top of every C file
			generated by the Sather compiler.
	
	Makefile:	Only used when installing the compiler. It must
			have three rules:

			all:	executed when installing a compiler that
				supports this platform. This may be used to
				precompile libraries, like for example the
				garbage collector.

			clean:	used to clean the installation

			test:	used to test this platform

As mentioned before there is another common platform with the same
structure, which is stored in System/Common. The CONFIG file there is
read in any case, but the CONFIG file in the Platform directory
will overwrite any element defined here.


The Configuration File
-----------------------------------------------------------------

Shipped with the compiler is a prototype configuration file that
contains cpp directives.  At installation, this file is transformed by
`cpp' into the configuration file that will be read by each run of the
compiler.  This file contains special per-installation information that
the compiler needs, such as the interface to the C compiler, make
utility, and linking environment.  It also provides definitions for
builtin routines in the Sather environment.  Together with the C files
making up the Sather runtime system, this file contains everything the
compiler needs in order to produce an executable.  (There is
additionally a file of forbidden identifiers in the generated
C code.  These are kept in a separate file because of the size.)

As in Sather and module files, comments begin with two dashes and
continue to a newline.

The whole file consists of definitions of the form

    definition 	=> UID: expr {, expr} ;
    expr        => arg | ID arg {arg}
    arg 	=> { INT | STRING | ID }
    UID 	=> U_LETTER { U_LETTER | DIGIT | _ }
    U_LETTER 	=> any uppercase letter
    ID  	=> LETTER  { LETTER | DIGIT | _ }
    LETTER 	=> any letter
    INT 	=> { + | - } DIGIT { DIGIT }
    STRING 	=> any string that is acceptable in a Sather program.

Some definitions are used to pass global information to the compiler,
while others inform the compiler about special properties of the
builtin routines.

Global compiler information
---------------------------

This part of the config files defines how to construct and call the
makefile, any options that have to be passed and any necessary
libraries.  Each of these definitions takes one string as an argument.

    UID			Interpretation
    ---                 --------------
    VERSION		version of the compiler

    PLATFORMS		Names of Platforms supported by that particular
			installation.

    DEFAULT_PLATFORM	Platform to use if the user does not specify a
			platform.

    THREADS		true if this platform supports threads

    DISTRIBUTED		true if this platform supports several clusters

    TRACE		true if this patform creates trace fiels (what ever
			this means)

    LIBRARY		The first string is an environment variable. If
			this variable exists, it must point to a module file
			then the compiler will use the this module file as
			the sather library. Otherwise it will use the
			second string as the library (this string is
			interpreted relative to SATHER_HOME)

    C_COMPILER          The string preceding C compiler options in the
			generated Makefile.

    CC_OPTIONS		List of options passed to the C compiler.

    LINK_OPTIONS        A list of libraries needed to link a sather
			program.

    MAKE_COMMAND        The command to invoke the (possibly parallel)
			make utility.

    OBJECT_EXT          The file extension that defines an object file.

    LIB_EXT             The file extension that defines a library file.

    SHELL_SEP           String used to separate commands for the
			standard shell.

    EXEC_OPTION         String to be inserted before the executable on
			the compiler command line when linking a
			program.

    NULL_SEGFAULTS      Either "true" or "false".  Set this to true if
			reads or writes to *NULL will cause a
			segfault.  If this is true, inline checks
			against void can be omitted.  Otherwise set it
			to "false".  On most systems the implication is
			that without checking on, the location of the
			void error won't be known unless running under
			gdb.  However, the inline checks bloat code
			quite a bit.

    SEPARATE_POINTERS   Either "true" or "false".  If true, the
			compiler must seggregate pointers into a struct
			for inspection/modification by the GC, and all
			pointer arguments must be passed via reference
			into this struct.  This may have an impact on
			performance, because these values cannot then
			be placed in registers across routine calls.


The following definitions deal with flags that have to be passed to C
or the make utility depending on the flags given to the Sather
compiler. Each definition has two strings, the first is used if the
given flag is NOT used, the other one if the flag is used.

    CC_DEBUG_FLAG:	  triggered by `-debug'.  Example:
			  DEBUG_FLAG:	"-O", "-g";

    CC_O_FLAG:	  	  triggered by `-O' and `-fast'.  Example:
			  O_FLAG:	"", "-O2";

    MAKE_VERBOSE_FLAG:    triggered by `-verbose'.  This flag is passed to make,
			  not to the compiler.  Example:
			  VERBOSE_FLAG:	"-s", "";

    CC_PROLIX_FLAG:  	  triggered by `-prolix'.  This flag is passed to make,
			  not to the C compiler.  Example:
			  PROLIX_FLAG:	"", "";

The following definitions are used for pSather and inform the compiler
of special properties of the target machine:

    POLLING             Defines an estimate of how often a special
			macro (`POLL') should be executed. It takes an
			INT argument that defines after how many
			lines/instructions of C code a poll call should
			be inserted. 0 means don't poll at all.

    ATOMIC_CLASSES      A list of value class names (as strings) for
			which the system guarantees that an assignment
			is atomic.  Classes which are not declared
			atomic is this way will require special locking
			to guarantee atomicity.  Pointer assignment is
			always assumed to be atomic.

    BUILTIN_CLASSES     A list of class names (as strings) for which
			layouts should not be generated by the
			compiler.  In addition to these classes, there
			is special handling by the compiler for STR,
			classes which include AREF or AVAL, and types
			which subtype from $LOCK.  In general, the
			compiler assumes that for all builtin classes
			there is a C typepdef with the same name

    REFERNCE_FREE	A list of builtin class names (as strings) for which
			it is known that they contain no references to
			other objects, examples are INT, FLT, ...
			
Declaring builtin methods
-------------------------

Some of the methods in the base Sather libraries have special handling
by the compiler, usually for efficiency.  Such builtin routines are
defined in the ordinary class text as follows:

    plus(a:SOME_CLASS):SOME_CLASS is builtin SOMECLASS_PLUS; end;

`builtin' is considered a keyword by the ICSI 1.1 compiler, but is
not part of the Sather 1.1 language specification.

Pre and post conditions may appear in the method signature.  `builtin'
must be the only statement in the body.   The identifier following the
`builtin' keyword is associated with a specific definition in the
compiler configuration file.  For example, the above function
definition would result in using the definition provided in the
configuration file for SOMECLASS_PLUS.  The definition informs the
compiler of how to generate C code, as well as any special properties
the method may have.

Many of the keywords used in a builtin method definition take strings
as arguments.  These strings are subject to the following macro
expansions:

  	@0  		expands to the type of self
	@n 		expands to the type of the argument `n'
	@r 		expands to the return type of the function
	@Pn 		expands to the type of the nth type parameter of
			    @self (@P1 is the _second_ parameter)
	@.attrib_name   expands to the type of the attribute name of @0
	@(x)		expands to (@x) iff @x is a reference type, otherwise
			    it expands to an empty string, useful for casting
  	$n 		expands to the nth argument of the function
	$0 		expands to the self argument
	$.atrrib_name   expands to the attribute attrib_name of @0
	$$              expands to a unique string associated with
			    each call, suitable as an identifier in C

In the case of out/inout arguments, the C expression replacing the `$'
syntax will be a local variable that the address can be taken of.  The
following macros expand to `0' if the type is a reference or abstract
type, '1' if it is a value type that can be assigned normaly (that is,
the assignment is guaranteed to be atomic), or '2' if the assignment is
not atomic (in this case you need to use the macros VASS_xx (see
pSather.h))

	%0              expands to the storage type of self
	%n              expands to the storage type of the argument `n'
	%r              expands to the storage return type of the function 
	%.attrib_name   expands to the storage type of the attribute attrib_name
	%Pn             expands to the storage type of the nth type
			    parameter of @self

The following macros are used to find out if a given argument is near
or not. They exapnd to 1 if the argument is near, to zero otherwise.
It will expand to 1 for all types that are immutable, and also if
the value is known to be void.

	^0		expands to 1 if self is known to be near
	^n		expands to 1 if the nth argument is known to be near

A builtin function definition accepts the following keywords:

    volatile    declares that the method may have side-effects
		and hence must not be moved or eliminated by the
		optimizer.

    raises   	List of exception types that may be raised by this function.
		Each exception type is passed as one string argument to
		raise, as in 

		    raises "STR","INT","@self" 

    raises_any  This may be used in place of raises to indicate that
		any type may be raised.

    reads       List of kinds of state that this function may have a
		data dependence on.  Each attribute or shared is passed
		in as a string argument of the form
		"CLASS::attr_name".  A special form exists for array
		elements, in which case the string looks like
		"CLASS::[]".  Example:

             	    reads "@0::size2","@0::[]"

    reads_any   This may be used in place of reads to indicate that
		the method should be assumed to be data dependent on
		all kinds of state.

    writes      List of kinds of state that may be modified by this
		function.  It has the same format as reads.

    writes_any  This may be used in place of writes to indicate that
		the method should be assumed to modify all kinds of state.

    no_pre 	Informs the compiler that it should ignore any 
		pre condition, as the inlined C code will test them.

    no_post	Same as pre, but for post conditions.

    fragile     Informs the compiler that the method might produce a
		fatal error if used with unsuitable arguments (ie.
		divide by zero).

    arith       Informs the compiler that this method may fail with a
		fatal error, but only if compiled with arithmetic
		checking (ie. integer plus).

    declare     List of strings that should be printed at the top of
		each file that uses this builtin method.  This is handy
		to include the correct header file or to define macros
		or structures.  Each string is printed on its own line,
		and duplicate strings are emitted only once.

    var         The strings defined here may be used to declare
		variables to be used by the implementation of the
		method. For iters you must make sure that all variable
		names start with $$. In this case $$ will be replaced
		by frame->uniq_string (remember that local variables
		of iters are stored on the heap).

    exec        Takes as arguments strings representing the C code to
		be emitted in place of a call to this method.  Each
		string is emitted on it's own line, and any that begin
		with a '#' character start in the first column.  If the
		function returns a value, the compiler assumes that the
		last string returns it.  Such a return string must
		evaluate to an addressible local variable.

    import 	This indicates the method has an implicit import (pSather).

    export     	This indicates the method has an implicit export (pSather).

    blocks	This indicates the method may block (pSather).

A builtin iter definition accepts all the keywords a builtin function
definition accepts, with the exception of the exec call.  Instead it
offers these keywords, which may be used similar to the way `exec' is
used, but placing code at different points throughout the calling loop.

    temp        For iterators, variables declared with var persist
		throughout the iter state; they are part of the iter
		frame.  If variables don't have to persist from one
		yield to the next, they can be more cheaply allocated
		with temp.  Variables declared here may not be
		initialized in init; they must be used exclusively in
		iter and their name may not start with $$.

    init	Here one can initialize all variables before the loop.

    iter        These strings are executed directly before the iter
		value is used.  If the iter returns a value, the
		compiler assumes that the last string in iter returns
		it.  Such as return string must evaluate to be an
		addressible local variable.  If the compiler finds the
		string @@ it replaces it with a command to end the
		loop.

Example (the definition of the upto! iter):

INT_UPTO:	var "INT $$;",
                init "$$=$0-1;",
		iter "if($$==$1) @@; else $$++;","$$";

The compiler can perform special optimizations on iters which have the
property that they use a single integer induction variable which is
initialized to zero, incremented by one each time through the loop, and
terminated at a single precomputed maximum value.  Most array iterators
have this property.  In such iters, the special value "$#" indicates
the index value.  (The actual variable may be shared by the
implementation of more than one iter in the loop.)  The following
keyword may be used for such iters:

    break       Defines the C expression which evaluates to the maximal
		index value for this iter.  The expression must be
		side-effect free and hoistable.

Example:

AREF_AELT:      break "$0->asize",
		temp "@r $$;",
		iter "$$=$0->arr_part[$#];" "$$";

Note that for pSather, you may also use any of the above macros, but
prefixing them with a 'f_'. In this case, the compiler will use those
definitions, unless all the objects mentioned in reads/writes are known
to be near at that point.


Compiler Generated Macros
-----------------------------------------------------------------

The general runtime interface may be divided into those macros which
the compiler must define and are used by the runtime, and those that
the runtime defines and may be used in code emitted by the compiler.

The code created by the compiler is partitioned into different files
depending on size and class, in an attempt to allow incremental
compilation.  There is a general header file which is included by
every C file, and specific header information which may differ from
file to file.  There is also a special "runtime.c" file.

These macros are use in the general header and are the same for all
files.  They must be emitted by the compiler if appropriate.

    DETERMINISTIC	- Defined by "-deterministic" flag
    DEBUG		- Defined by "-debug" flag
    DESTROY_CHK		- Defined by "-destroy_chk" flag
    NULL_SEGFAULTS	- Defined by NULL_SEGFAULTS flag in configuration file
    STATS		- Defined by "-stats" flag

    PSATHER		- Defined by "-psather" flag
    PSATHER_CHK  	- Defined by "-psather_chk" flag
    PSATHER_STATS      	- Defined by "-psather_stats" flag
    PSATHER_TRACE	- Defined by "-psather_trace" flag
    CLUSTERS		- Defined by "-psather_clusters" option
    PSATHER_DIST	- Defined if clusters>1

    VERSION		- Compiler version (ie. "1.0.5")

These macros are defined on a per-file basis by the compiler if
appropriate.

    PRE_CHK		- These are turned on by compiler options,
    POST_CHK              discussed in the `sacomp' man page.
    INVARIANT_CHK
    ASSERT_CHK
    ARITH_CHK
    BOUNDS_CHK
    VOID_CHK
    WHEN_CHK
    RETURN_CHK

    RUNTIME		- Defined only during compilation of runtime.c

These C routines are emitted by the compiler because they require type
information that is only available at code generation time.

    STR gen_SYS_str_for_type(INT); - Implements SYS::str_for_type.

    BOOL gen_SYS_ob_eq(OB,OB);     - Implements SYS::ob_eq.



The Serial Runtime Kernel
-----------------------------------------------------------------

The general runtime is written in such a way that porting the Sather
environment requires only the runtime kernel to be ported to a new
platform; the general runtime is written portably in terms of the
kernel runtime.  However, there may be #ifdef-ed platform specific code
at the other levels for obtaining efficiency.

All macros in both the kernel and general runtime expect local
variables as arguments.  It is not legal to nest these macros, for a
variety of reasons.  Arguments to these macros must always be local
variables.  The result of this constraint is that the output of the
Sather compiler is similar to an infinite register instruction set
architecture.

The most basic part of the kernel defines the mapping between built-in
Sather types and C types.  Some of these are fixed; for example,
strings have to be a specific form because string constants are emitted
by the compiler.  By convention, FOO is a pointer to a struct of type
FOO_struct, and if BAR is a value type, the boxed form is BAR_boxed.

    BOOL	- Basic Sather typedefs.
    INT
    CHAR
    FLT
    FLTD
    STR

    OB_HEADER	- First field of every reference object must define this.  

Some constants:

    SUINT_MAX	- Maximum unsigned INT
    SINT_MAX	- Maximum signed INT
    SINT_MIN	- Minimum signed INT
    FALSE	- BOOL::false
    TRUE	- BOOL::true

These macros are used to declare single precision functions portably.
The first letter is return type, (Float, Bool, Int or Void) and the
last letters are argument types:

    FLT_F	
    FLT_F_I
    FLT_F_F
    FLT_F_FF
    FLT_F_FI
    FLT_F_IF
    FLT_B_F
    FLT_I_F
    FLT_V_FP

This macro is used in error checking:

    FATAL(msg)	- Emit runtime message and die gracefully.



The General Serial Runtime
-----------------------------------------------------------------

The general runtime consists of macros which operate on local
structures and a superset which is used for distributed pSather.  In
the single-cluster, nondistributed case, C types are declared for each
Sather type.  In the distributed case, two C types are declared: one
for the local case, and one which is a far pointer, which by convention
begins with `F_'.  The macros for operating on near structures are
presented first.

These macros are used for manipulating objects:

    RATTR(local,x,y)	- set local to field y of x
    WATTR(x,y,local)	- set field y of x to local
    RARR(local,x,y) 	- set local to element y of x
    WARR(x,y,local) 	- set element y of x to local
    TAG(x)          	- return tag this object created with (type INT)
    ASIZE(x)        	- return asize this object created with (type INT)

    SYSTP(x)        	- SYS::tp:INT
    SYSID(x)        	- SYS::id:INT
    SYSSTRFORTP(x)      - SYS::str_for_tp:STR
    SYSOBEQ(x,y)        - SYS::ob_eq:BOOL
    SYSEXTOBFOR(x)      - SYS::ext_ob_for:EXT_OB

(UNRESOLVED) There should something like

    ACOPY(dest_arr,dest_beg,num,src_arr,src_beg)

for doing optimizations on acopy.

These macros 

    RT_START		- Called before execution begins.
    RT_STOP		- Called after execution completes

Macros related to exceptions.

    RAISE($OB)		- `raise' statement
    EXCEPTION		- `exception' expression

A protect statement should be emitted:

    PROTECT_BEGIN
	<code>
    PRTOECT_WHEN 
	<switch on EXCEPTION type>
    PROTECT_END

When exiting a protect statement by returning out of it or breaking out
of a loop by an iter, it is necessary to restore state of the protect
stack.  (This state also is used with lock statements in pSather.)  The
following macros allow popping up N levels of the stack.  In the common
case, POP_EXCEPTION1 can be used for n=1.

    POP_EXCEPTION1
    POP_EXCEPTION(n)

Memory allocation and garbage collection
----------------------------------------
(* WARNING: THIS SECTION IS OBSOLETE AND OLD AND NEEDS TO
   BE COMPLETELY REWRITTEN *)

In order to do non-conservative, type-aware garbage collection, the
compiler must handle pointers with special macros that allow special
handling.  For example, if a write barrier is used, then all
assignments to pointers must have an associated macro which implements
this.

Furthermore, in order to guarantee reachability analysis, all pointers
from the stack to the heap must be identifiable.  This is done by
placing all nonredundant pointers in a single struct and identifying
the position of this struct to the garbage collector.  Pointers passed
as arguments must be passed by reference to this structure, which
guarantees that they may be identified and modified on the fly.  An
important C-compiler specific optimization will be to allow and
identify pointer values in registers, but the portable compiler can't
do that.

Some GC implementations are not copying and may be conservative.  In
such cases, the compiler configuration file may set SEPARATE_POINTERS
to "false" and generate the more efficient register-amenable code.

All routines with pointer locals call:

    PROLOGUE(s)	- local struct containing all pointer locals
    EPILOGUE

These frames may be iterated over by the garbage collector:

    FIRSTROOT	- Called to begin, returns pointer to pointer on stack
    NEXTROOT	- Pointer to next pointer, or NULL when no more

The PWATTR, and PWARR macros are special forms to be used when pointers
are stored into heap objects.  PASS is used when assigning to a local
or global that is not in heap.  Unlike other macros, `dest' and `src'
need not be locals.

    PASS(dest,src)  - Assign pointer to a local or global

Allocation is done with these macros which return a heap object:

    ALLOC(type, tag)
    ARR_ALLOC(type, tag, asize)

These macros handle destroying and invoking the garbage collector:

    SYS_DESTROY(ob)   - SYS::destroy(ob)
    SYS_COLLECT_NOW   - SYS::collect_now

Every object in the runtime heap has a type descriptor tag.  To allow
garbage collection to apply to the general runtime interface, internal
data structures of the runtime are marked with type tags as well.  The
compiler emits only positive tags, and the runtime must use negatives.

The compiler emits a routine:

    gc_follow(ob)

which sees to it that the runtime-provided macro

    FOLLOW(ob->field)

is called for every word, but only for positive tags.  For negative tags,

    runtime_follow(ob)

will be called, and the runtime must arrange to call

    FOLLOW(ob->field)

for every word in the runtime structure containing a pointer.



The Single cluster pSather Kernel
-----------------------------------------------------------------

Functionality in the SYS class:

    DEFER		- SYS::defer
    IMPORT		- SYS::import 
    EXPORT		- SYS::export

The compiler configuration file indicates to the compiler if it is
necessary to emit periodic polling instructions (for example, to test
for received messages or to allow thread preemption.)

    POLL	- Inserted by compiler such that it is executed often

    Creating threads?
    Creating semaphores/mutex?



The General Single Cluster pSather Runtime
-----------------------------------------------------------------
(* WARNING: THIS SECTION IS OBSOLETE AND OLD AND NEEDS TO
   BE COMPLETELY REWRITTEN *)

pSather defines all assignments to be atomic.  Some types will have
this property by the characteristics of the hardware, while others will
have to be explicitly protected.  Notice that there are essentially two
different mechanisms impacting atomicity.  One is when it is possible
for thread pre-emption to occur.  The other is the placement and size
of structures with respect to cache lines which are atomic in a
write-back multiprocessor cache system.  The compiler configuration
file indicates which value types do not need explicit protection.  (It
is assumed that pointer assignments are always atomic, so this isn't
necessary for reference types.)

    VASS(dest,src) 	- Assign for value types which aren't atomic.
			  Arguments must be addressible.

An attach `:-' is implemented with the following macro.  `location'
is always `0' when there is a single cluster.

    ATTACH(func_ptr, self, single argument, gate, location)

typedefs for pSather:


The disjunctive lock
--------------------

Disjuctive locks are implemented in two steps.  First, the lock statement
is declared, including the number of branches and the number of locks
per branch.  Each branch is then declared one at a time.  After all
branches are declared, the actual lock begins.

Declaring the lock:

    DECLARE_LOCK(number_of_branches,max_number_of_locks_per_branch)
    DECLARE_LOCK_ELSE(number_of_branches,max_number_of_locks_per_branch)

Declaring the branches:

    The branches have to be defined in the textual order that they
    occur in the pSather program.  Each branch is defined as follow:

    ADD_BRANCHn(condition,lck1, ...lckn)
        n:              the number of locks in this branch
        condition:      the branch will be ignored if condition is false

Selecting the correct branch and executing it:

    SELECT_LOCK
    BRANCH(1)
	<code_for_branch_one>
    BRANCH(n)
	<code_for_branch_n>
    BRANCH_ELSE
	<code_for_the_else_branch>
    LOCK_END

Example:

    lock 
       when lck1,lck2 then <psather_code1>
       when lck3 then <psather_code2>
       else <psather_code3>
    end;

should be translated to

        DECLARE_LOCK(2,2,LOCK_WITH_ELSE) 
        ADD_BRANCH2(TRUE,lck1,lck2);
        ADD_BRANCH1(TRUE,lck3);
        SELECT_LOCK
        BRANCH(1)
             <psather_code1>
        BRANCH(2)
             <psather_code2>
        BRANCH_ELSE
             <psather_code3>
        LOCK_END

Unlocking may be done with this macro.  The runtime detects if the
unlocked lock is not presently locked.

    UNLOCK(lock)



Runtime for distributed pSather
-----------------------------------------------------------------

(* WARNING: THIS SECTION IS OBSOLETE AND OLD AND NEEDS TO
   BE COMPLETELY REWRITTEN *)

In distributed pSather, most types have two representative C types
associated with them.  A Sather type `FOO' will have the usual near
representation, and a far represention (i.e., a struct with a node ID
and pointer to a FOO) that is named by convention `F_FOO'.  In the
distributed case, there are additional macros used for operating on
these far pointers.

Near object pointers are all castable to `OB'.  Far object pointers
are castable to `FOB':

    FOB    		-- typedef for generalized pointer which may be far

These block until they have read the value in the local:

    F_RATTR(local,x,T,y)-- local := remote read of ((T)x)->y
    F_RARR(local,x,T,i)	-- local := remote read of ((T)x)->arr_part[i]

These block until remote write is acknowledged:

    F_WATTR(x,T,y,local) -- remote write ((T)x)->y=local
    F_WARR(x,T,i,local)  -- remote write ((T)x)->arr_part[i]=local

There is a far pointer equivalent to the macro PASS:

    FPASS(dest,src)  - Assign far pointer to a local or global

In addition to the above synchronous reads and writes, there are also
split-phase asynchronous reads and writes.  These use a `wait counter'
for extremely lightweight synchronization.

Wait counters are a relaxed, lightweight form of semaphore.  The
implementation is free to use these in any way it wishes; it is a valid
implementation as long as it waits at least as long as it would have if
each counter were distinct.  (This permits implementations with only a
single wait counter, or none at all, reverting the implementation to be
purely synchronous.)

    WAITCTR	- typedef for a wait counter
    WCLEAR(ctr)	- set a wait counter to zero; must be called
		    before use
    WAIT(ctr)	- wait for counter to be zero

The following macros are asynchronous (but it is a valid although poor
implementation to have them simply block and ignore wait counters).
They obey the import/export consistency model.  They are only useful
when used with the appropriate IMPORT or EXPORT macro.

    AF_RATTR(local,x,T,y)
    AF_RARR(local,x,T,i)
    AF_WATTR(x,T,y,local)
    AF_WARR(x,T,i,local)

These have an explicit wait counter, independent of import and export:

    AFW_RATTR(local,x,T,y,wait)
    AFW_RARR(local,x,T,i,wait)
    AFW_WATTR(x,T,y,local,wait)
    AFW_WARR(x,T,i,local,wait)

Typical usage:

    INT result;
    WAIT_CTR w;
    WCLEAR(w);
    AFW_RATTR(result,fob,FOO,field,w);
    ...
    WAIT(w);
    /* use `result'. */

The rest of the runtime must "do the right thing" with wait counters;
for example, if an exception is raised all pending wait counters should
be waited on first.  If the compiler declares its own wait counters, it
must also declare the code to wait for them.  The runtime does not know
which wait counters are in use (apart of any special ones used together
with import/export).

The following macros have special versions that must be used when the
local variable is a value type requiring explicit protection (VASS):

    Macro	Protected version
    ----------- -----------------
    F_RATTR 	FV_RATTR
    F_RARR 	FV_RARR
    F_WATTR 	FV_WATTR
    F_WARR 	FV_WARR
    AF_RATTR 	AFV_RATTR
    AF_RARR 	AFV_RARR
    AF_WATTR 	AFV_WATTR
    AF_WARR 	AFV_WARR
    AFW_RATTR 	AFWV_RATTR
    AFW_RARR 	AFWV_RARR
    AFW_WATTR 	AFWV_WATTR
    AFW_WARR 	AFWV_WARR

Bulk transfers can take place between array objects:

    BULK_XFER(type,fob,idx,fob2,idx2,length,waitctr)

There are also macros for other object manipulations:

    FTAG(fob)          	- return tag this object created with
    FASIZE(fob)        	- return asize this object created with
    FSYSOBEQ(fob,fob)  	- SYS::ob_eq
    FNULL		- NULL for far types
    FVOID(fob)		- void() for far pointers
    FDESTROY(fob)	- SYS::destroy

Location is handled with the following macros:

    ANY			- pSather "any"
    HERE		
    WHERE(fob)
    MAKENEAR(fob)  	- Returns near pointer
    MAKEFAR(ob)    	- Returns far pointer
    NEAR(fob)
    FAR(fob)

The `attach' macro, defined above, allowed a location arg.  This
macro allows synchronous execution of a method on a remote node:

    REMOTE(func_ptr, self, single argument, location)

There are also the variants that may be faster if the compiler
can prove that a call doesn't block or can't raise an exception:

    REMOTE_NB(...)	-- Known to not block
    REMOTE_NR(...)	-- Known to not raise exceptions
    REMOTE_NBR(...)	-- Known to not block or raise exceptions

Finally, these macros return a near pointer to spread memory:

    SPREAD_ALLOC(type, tag)
    SPREAD_ARR_ALLOC(type, tag, asize)




THE LIBRARY ORGHANIZATION
-------------------------

The Sather distribution has three different library hierarchies:

1. general Sather Library in $SATHER_HOME/Library/Library.module
2. general pSather Library in $SATHER_HOME/pLibrary/Library.module
3. system dependent Libraries $SATHER_HOME/System/Library

If the compiler is unpacked only the general pSather and the system
dependent libraries exists. The general Sather library is created during
installation with the use of the PP preprocessor (see Doc/PP.description
for a detailed description of the preprocessor).
