SRE2003: An Internet Interface for OS/2

Table of Contents

1. Introduction
   • The basic disclaimer
   • Additional documentation
2. Installation
3. Starting SRE2003
   • Using SRE Watch
4. Configuring SRE2003
   1. Description of SRE2003 configuation parameters.
   2. Detailed description of SRE2003 parameters
5. Using SRE2003
   1. An outline of how SRE2003 processes a request
   2. Arguments sent to the filter
   3. The SRE2003 functions
6. Appendices
   1. Appendix 1: Basic copyright and it's never our fault disclaimer
   2. Appendix 2: Using SRE2003's request cache
   3. Appendix 3: Running SRE2003 in loopback mode
   4. Appendix 4: Using a fast filter
   5. Appendix 5: Tracking client status
   6. Appendix 6: The HOSTINFO.CFG file
   7. Appendix 7: A short glossary
   8. Appendix 8: A list of SRE2003 features

SRE2003 is designed for small to intermediate sized sites. It's primary strengths are flexibility and ease of customization. Speed, resource use, security, and scalability are not the primary goals. That said, these are operational concerns; and quite a bit can be done to enhance performance along any of these dimensions.

In many ways, SRE2003 is an outgrowth of GoServe, which was designed and created by Mike Cowlishaw. In fact, it is the end of support for GoServe that motivated the creation of SRE2003.

In comparison with GoServe, the primary advantage of SRE2003 is increased support for http/1.1 functions, including a greatly improved cache. Just as important, SRE2003 can be readily modified (it's written in REXX) -- we are always interested in suggestions and ideas!

Are you are upgrading from GoServe/SRE-http? You can find a number of hints and suggestions in UPGRADER.TXT.

Of course, there is a disclaimer ..

Before choosing & using SRE2003, you must read the disclaimer (in appendix 1). Basically, SRE2003 is gnu style freeware; and is to be used at your own risk. We do try to create a fairly secure product, and we try to quickly fix any problems (or potential problems). However, we can NOT guarantee that SRE2003 has no security holes or is free from other potentials for failure.

More help can be found in ...

SRE2PRC.HTM contains detailed descriptions of SRE2003 functions. SREAPI.TXT contains further details on working with SRE2003 SRE_DMN.TXT discusses how you to user SRE's functions that create and manipulate daemons, flags, and queues. SRE_TASK.HTM discusses SRE2003's built in task manager. SRE_DCSH.TXT discusses SRE2003's built in database facility. IM_USE.HTM discusses how to use and create SRE2003 Instance Manipulation modules .

1. UnZIP SRE2003.zip to an empty temporary directory
2. open up an OS/2 prompt, and CD to this directory
3. run INSTALL.CMD
   For example, assuming you unzipped SRE2003.zip to E:\TEMP:
      E:\TEMP>install
4. You will be asked to enter a working directory -- this is where SRE2003 will be installed.

Notes

• If you are using Object Rexx (under OS/2 4.0), you will not be able to use SRE2003.

• If you are using Warp 4.0 (or Warp Server 4.0), you should be using the latest fix packs for TCP/IP (INETVER should report 4.02t or above).
  You can get the latest TCP/IP fix-packs from:
  ftp://ftp.software.ibm.com/ps/products/mpts/fixes/english-us/
  or you can try
  http://duanec.indelible-blue.com/fixes/latestwarp4.html

• Other then adding a few simple entries to OS2.INI (entries only used for informational purposes) SRE2003 does not modify any system files. To remove SRE2003, just delete the SRE2003 working directory (that you chose in step 4 above).

• SRE2003 uses several DLLS, which are stored in x:\SRE2003\DLL.
  Since older versions of OS/2 (such as 3.0 and early revisions of 4.0 ) do not support on-the-fly modification of the LIBPATH, if you are using an older version (or if you plan to use a multitude of CGI-BIN scripts), you may want to move these DLL files into your LIBPATH; say, to x:\OS2\DLL, where x: is the system drive.
  Alternatively, you can add x:\sre2003\DLL to the LIBPATH line in your CONFIG.SYS file (where x:\sre2003 is your SRE2003 directory).

• By default, the SIMPLE.RXX filter (that comes with SRE2003) is used.

• The installation process creates several subdirectories (under the working directory).
  BAK : Backup of prior SRE2003 versions (used by INSTALL program)
  BIN : Programs and modules used by SRE2003
  CFG : Configuration files
  DLL : Dynamic link libraries
  DOC : Documentation files and manuals
  LOG : Log files
  TEMP : Temporary files created by SRE2003
  SIMPLE : The "SIMPLE" filter and related files

• It is possible to simultaneously run more then one copy of SRE2003, with each copy handling a different port (say, port 80 and port 8080). To do this, we recommend having totally seperate subdirectories for each version (for each port).

  To use a different port, all you need to change is the PORT parameter in SRE2003.CFG.

3) Starting SRE2003

1. Open an OS/2 box
2. CD to the working directory
3. Run SRE2003
   For example, assuming the working directory is D:\SERVER:
         D:\SERVER>SRE2003
4. SRE2003 will start, and after some initializations a text-based status screen will be displayed.

Notes

• From the SRE2003 status screen, you can enter several options:

  Esc	Shutdown SRE2003 After hitting Esc, it may take a few seconds to shutdown. In some cases, hitting Ctrl-C may be faster. However, it's cleaner to shut down using ESC (various resources will be properly freed up). You can use the DUAL_KEY parameter (in SRE2003.CMD) to change what keystroke is used to Shutdown SRE2003.
  R	Refresh the screen, and reset/re-read SRE2003 parameters. Refresh is useful if error messages, or other unusual conditions (such as your filter using SAY) causes information to be written to the screen in an uncontrolled fashion. reset/re-read is useful if you manually (with a text editor) changed SRE2003 (or filter-specific) configuration files.
  M	Modify parameters. This will let you modify the dynamically-modifiable parameters. You can suppress this option by setting the NO_MODIFY_ONTHEFLY option.
  S	Snipe(kill) connections. Use this to list, and selectively close, currently active connections. When in snipe mode, no more connections will be accepted (though currently open connections will continue to be active). You can use the DUAL_KEY parameter (in SRE2003.CMD) to change what keystroke is used to Snipe

  
  
• After shutting down SRE2003, you can immediately restart it (by re-running SRE2003.CMD). However, we recommend restarting SRE2003 from a freshly opened OS/2 session (and closing the session SRE2003 had been running in).

• If you do not like the colors used in the status screen, you can modify them -- see the user configurable parameters section at the top of SRE_STAT.RXX (in the BIN subdirectory).

• To see SRE2003 start up options, run SRE2003 ?

• SRE2003 uses PMPRINTF to display status and error messages. To view PMPRINTF output, you can use PMPRINTF.EXE, which can be found at http://www2.hursley.ibm.com/goserve/

• SRE2003 writes error messages to an ERROR.LOG file (in the LOG subdirectory).

• Technical Note: When you Refresh (by hitting the R key in the status screen), SRE2003 sets the FILTER_RESET SRE2003 flag. Filters (such as SREhttp/2) can monitor this flag, and reset their parameters when it is turned on.

SREWATCH will start SRE2003, and will monitor its status. If SRE Watch detects that SRE2003 has crashed (or is otherwise hopelessly hung), it will attempt to kill, and then restart, SRE2003. In fact, you can even tell SRE Watch to reboot the machine when things get bad enough.

To run SRE Watch, run the SREWATCH.CMD program (located in the SRE2003 working directory) instead of SRE2003.CMD. You might need to set a few parameters (in SREWATCH.CMD) before the first time you run it, but most people can use it as delivered.

• For details on using and configuring SREWATCH, read SREWATCH.CMD with any text viewer (or editor). Or, from an OS/2 prompt, run SREWATCH.CMD with a ?
  For example: D:\SRE2003>srewatch ?

• The use of SREWATCH is highly recommended for those running unattended servers.

  For example: run SREWATCH.CMD from your STARTUP.CMD file, and enable the SRE Watch "reboot if necessary" option,

1. SRE2003.CFG: Server parameters
2. HOSTINFO.CFG: Host definitions
3. BADIPS.IN: Unallowed IP addresses.

SRE2003.CFG is a text file that you can modify using any text editor (such as EPM). You can also modify SRE2003 by entering, from an OS/2 prompt:
      SRE2003 -m
You will be presented, in a sequential fashion, with a list of the most important SRE2003 parameters, and their current values. You can change, or retain these values; you can also dislay some rudimentary on-line help.
Alternatively, if you enter
      SRE2003 -mall
you will be presented with a list of all the parameters contained in SRE2003.CFG, many of which you'll never need to change.

Note that several SRE2003 filters (such as sreLite2 and SREhttp/2) come with configuration utilties that provide an on-line method of modifying SRE2003.CFG (the server parameters file), HOSTINFO.CFG (the host definition file), and BADIPS.IN (the unallowed IP addresses file).

Notes

• The following parameters can be reset on the fly (from the SRE2003 status screen, hit M).

  DATADIR	LIMITBODY	LIMITCLIENTS
  LIMITHEADER	LIMITTIMETOTAL	LIMITTIMEWAIT
  LIMITTIMEINACTIVE	SERVER_SOFTWARE	SERVER_SOFTWARE_SHORT

4a) Detailed description of SRE2003 parameters

AUDITFILE

Where to write audit items (as specified by AUDIT_ITEMS) to. Should be a relative filename; relative to the SRE2003 working directory. By default, it's LOG\AUDIT.LOG.

Technical Note	The max_filesize user configurable parameter (in the SRE2003\BIN\SRE_AUDI.RXX directory) controls the maximum size (in bytes) of the audit file and the cache-hits file. When this size is exceed, the audit (or cache) file is renamed, moved to the sre2003\log\arc subdirectory, and a new one is started.

AUDIT_DELAY

How many minutes to wait when writing to audit file. This is a "lazy write" parameter -- SRE2003 will store this many minutes worth of AUDIT output, and write it to the audit file all at once.

You can use fractional values. For example:
    AUDIT_DELAY=0.25
means "write to audit file every 15 seconds"

Notes:

• this delay is approximate -- the actual delay will depend on system activity
• REQ_CACHE_RECORD is used to set the delay when writing to the "cache hit" file

AUDIT_ITEMS

Specifiy what additional events to write to audit file. Should contain a space delimited list of options. Supported options are: CLIENT, MESSAGE, REQUEST, SENT, RESET, DIAG, and DIAG2.
   Example: audit_items=CLIENT SENT

These options yields the following information.

CLIENT

Client and server info. For example:

    C : 1A 00:25:07: 131.10.51.21 80 1 15.12.112.3 1104
where:
          C :    -- this is the "client audit item"
           1A    -- transaction and request number
     00:25:07    -- time of connection
  31.10.51.21    -- ip address of server
           80    -- port of server
  15.12.112.3    -- ip address of client
         1104    -- port of the client

MESSAGE

Messages written using the SRE_WRITE_MESSAGE function. SRE_WRITE_MESSAGE is used to write text to the message area of the SRE2003 status screen. For example:

   M  : 82/21/SRE2003 7 Sep 2002 09:00:51 : FILE  send_file problem: error missing response file=F:\wwwbig\GERBILS\describe.txt

DIAG

Diagnostic messages. A variety of diagnostic messages. For example:

   D : 3A 00:33:50: REQFIELD = If-match If-Unmodified-Since If-None-Match
   D : 3A 00:33:50: FILE = CHUNK NOWAIT ERASE type text/plain NOCACHE Name G:\SRENEW\TEMP\_17111__.TGZ
      

DIAG2

A superset of DIAG -- you should not use both DIAG and DIAG2. The following are examples of information produced by DIAG2, but not DIAG.

  D : 3A 00:33:50: Header = HEADER NOAUTO
  D : 3A 00:33:50: Header = Header Add Date: Sat, 25 Mar 2000 05:33:49 GMT
  D : 1A 00:25:06: ReqHdr = Referer:do_get@localhost
  D : 1A 00:25:06: ReqHdr = If-none-match: "dir.d","dir.doc"

REQUEST

The request string. For example:       R : 1A 00:25:06: GET /samples/daemons.doc HTTP/1.1

RESET

A reset has occurred. For example:
      RESET : 21:34:52 5 Jul 2001

SENT

Summary of bytes sent & recieved. For example:
      S : 1A 00:25:07: 158, 200 8992
Where:
158 : Bytes recieved
200 : response code
8992 : Bytes sent

Note that you can use the SRE_AUDIT function to write custom messages to the AUDIT file.

DATADIR

The default "data" directory. This is the "root" directory of you web site -- it's where resources corresponding to selectors of the form "/afile.ext" are located.

Note that the filter does NOT have to use this directory!

Notes:

• SRE_DATADIR('default') can be used, by filters, to obtain this value. Note that SRE_DATADIR() will return either a request-specific data directory, a host-specific data directory, or this (the default) data directory.
• the DATADIR parameter directory applies to "default" (non host-specific) requests. The data directory for host specific are defined with the SRE_HOSTINFO procedure (or with the INITHOST.RXX initialization procedure).
• For security reasons, we strongly recommend that the SRE2003 working directory not be in or under the datadir directory.
• The SRE2003\DATA directory is not the same thing as the DATADIR. The DATA directory is where SRE2003-specific data (such as counter files) are kept. The DATADIR is the default root of your web resources.

DEFAULT_HOSTS

A space delimited list of host names that are implicitily assigned to the default host. For example, they will have no host nickname (or a host nickname of DEFAULT), and they will use the default data directory.

Furthermore:

If a host is not:

• the canonical host, or
• listed in DEFAULT_HOSTS, or
• defined in HOSTINFO.CFG,

  then

• a 400 Bad Request response is returned to the client.

Notes:

The canonical host is defined by TCP/IP calls, or by the SERVERNAME_USE parameter. It is always accepted. Thus, DEFAULT_HOST augments the canonical host name, and complements HOSTINFO.CFG (or some other, filter-specific, host definition file).

In this context, host names are synonymous with domain names

The host (to whom this request is directed) is determined by (in order of precedence):

• A host in the selector. For example, a request line of:
    GET http://foo.bar.net/hello.html HTTP/1.1
• A Host: request header. For example:
     Host: foo.bar.net
• If neither of the above, then use the canonical host

You can use wildcarded host names. In particular, the installation default, *, means any host will be accepted.

Examples. If the host is not the canonical host, and it's not defined in HOSTINFO.CFG (or in a filter specific host definition file)...

If DEFAULT_HOSTS is	then the following hosts will be served (using default parameters)
*	All hosts wil be served
0	No additional hosts will be served
foo.bar.net	foo.bar.net will be served.
foo.bar.* goo.com	goo.com, and any host name that starts with foo.bar., will be served

DEFAULT_SOCKET_TIMEOUT

Time to wait on a socket, in seconds. If a read or write action on a TCP/IP socket takes longer then DEFAULT_SOCKET_TIMEOUT to do anything, then a timeout will result (the connection will be closed).

Notes:

• The LIMITTIMEINACTIVE and LIMITTIMETOTAL timeouts refer to total connection time, assuming the connection is still healthy. DEFAULT_SOCKET_TIMEOUT refers to how long one should wait before concluding that the "connection is dead".

ENTITY_HEADERS_IGNORE

This option controls which entity-headers SRE2003 should not consider when auto-creating an ETAG (via the AUTO_ETAG option of SRE_COMMAND). It should either be empty, or contain a case insensitive, space delimited list of entity-headers.

Basically, the rules for etags are (as defined in RFC2616):

An etag value should be unique for all entities derived from a request-uri.

Entities are defined as having an entity-body and entity-headers

Thus, if the entity-body, or an entity-header, changes its value (again, for a given request-uri), then the etag should change.

Unfortunately, this can mean that essentially inconsequential changes in entity headers require a new etag (such as changes in file creation dates, or dropping a Content-Length header if chunked transfer encoding is used). The required changes in the etag can diminish the effectiveness of caches and proxy servers, as well as diminish the ability of SRE2003 to use conditional (304) responses to requesst that contain If-None-Match request headers.

Since RFC2616 is silent on this issue, SRE2003 uses the ENTITY_HEADERS_IGNORE parameter to select entity-headers to not consider when automatically creating an etag. When you select what to include in this list, you should consider whether it matters that a client may recieve an entity with headers that may not be completely up-to-date, versus the probability that an inconsequential change in an entity-header will unnecessarily suppress caching and conditional responses.

In general, we recommend including the following entity-headers: CONTENT-LENGTH, CONTENT-MD5, EXPIRES, and LAST-MODIFIED.

However, if you have resources subject to unexpected changes in freshness (that is, resources whose expiration times may depend on circumstances), then EXPIRES and LAST-MODIFIED probably should not be included (at the cost of reducing SRE2003's ability to use 304 responses).

Notes:

• The ENTITY_HEADERS_IGNORE_206 parameter (in SRE_TRAN.RXX) specifies additional entity headers to ignore on Range (206) responses.
• In no case will general-headers and response-headers (as defined in RFC2616) be considered when automatically creating an etag.
  
• As defined by RFC2616 ...
  General-Headers: Cache-Control Connection Date Pragma Trailer Transfer-Encoding Upgrade Via Warning
  Response-Headers: Accept-Ranges Age Etag Location Proxy-authenticate Retry-After Server Vary WWW-Authenticate
  all other headers in the response are considered to be entity-headers.
  Some examples of entity-headers include: Allow Content-Encoding Content-Language Content-Length Content-Location Content-MD5 Content-Range Content-Type Expires Last-Modified

ERRORFILE

The file where errors are recorded to. By default, this is log/errors.log

EXIT_PROC

File, or a comma delimited list of files, containing exit procedures. These exit procedures are called just before SRE2003 shuts down.

• If you do not specify any full path, a file is assumed to be relative to the SRE2003 \BIN directory.
• If you specify a relative path, a file is assumed to be relative to the main SRE2003 directory (the directory containing SRE2003.CMD).
• Or, specify a 0 if you do not have an exit procedures.

Examples:

1.     EXIT_PROC= PEND1.RXX
   The PEND1.RXX procedure (say, in x:\SRE2003\BIN\PEND1.RXX) will be called.
2.     EXIT_PROC= EXIT1.RXX , SRELITE2\LITEEXIT.RXX
   First the EXIT1.RXX procedure (say, in x:\SRE2003\BIN\EXIT1.RXX) will be called. Then, LITEEXIT.RXX will be called (say, in x:\SRE2003\SRELITE2\LITEEXIT.RXX).
3.     EXIT_PROC=0
   No exit procedure

Notes:

• Each EXIT_PROC procedure is called with the http port, the servername, and the directory containing the filter. Thus, exit procedures should start with something like:

  parse arg PORT,SERVERNAME,FILTER_DIR

• EXIT_PROC procedures can return anything (the returned value is ignored).
• Possible uses for exit procedures include cleaning up daemons and flags that are created by a filter.

FILTER

Name of the filter program. Should be a filename relative to the SRE2003 working directory. Example: FILTER = SREhttp2.RXX

FILTER_VERSION

Descriptive name of the filter program. Should be a short phrase that describes this filter.

Example: FILTER_Version= Simple Filter version 1.12d

The SRE_SERVER('F') function returns the value of the FILTER_VERSION parameter.

FAST_FILTER

Name of "fast-filter" program. If specified, the FAST_FILTER is called first. If the fast-filter does not return a response to the client, the "normal" filter is then called. For more details on use of a "fast filter", see Appendix 4.
Examples:
    FAST_FILTER = FASTFILT.RXX
    FAST_FILTER = 0 -- do not use a fast filter.

IM_TYPES

Define the available Instance Manipulation modules. These modules may be called by SRE2003 after recieving a FILE to send to the client, but before any final encodings. Examples of instance manipulation include range extraction, and delta encoding.

Syntax: IM_TYPES= im_name1 im_file1 , im_name2 im_file2 , ....

where each (case insensitive) im_name1 & im_file1 pair are seperated by a comma:

im_nameN = a short name identifying this module. This name can be used in the IM_DEFAULT configuration parameter, or with the IM option of SRE_COMMAND('FILE ...')

im_fileN = a rexx program that implements this module. This should either be a fully qualified filename, or a filename relative to the SRE2003 directory.

Example:
   IM_TYPES=XRange bin\xrange.rxx , deltas D:\TEST\DELTA\delta.rxx

IM_DEFAULT

The default Instance Manipulation module. This is used if no IM option is specified [in a SRE_COMMAND('FILE ...')].
IM_DEFAULT can be set to:

• 0 = By default, do not attempt an Instance Manipulation.
• RANGE = Extract a range (if the client requested one). This uses SRE2003's built-in range extraction procedures.
• or, one of the id_names specifed in IM_TYPES.

Examples:
  IM_DEFAULT=DELTAS
  IM_DEFAULT=RANGE

INIT_PROC

File, or a comma delimited list of files, containing initialization procedures.
These initialization procedures are called just before SRE2003 starts accepting requests.

• If you do not specify any full path, a file is assumed to be relative to the SRE2003 \BIN directory.
• If you specify a relative path, a file is assumed to be relative to the main SRE2003 directory (the directory containing SRE2003.CMD).
• Or, specify a 0 if you do not have an initialization procedures.
• You can also include a string, after the procedure name, that will be sent to the initialization procedure. If you include a string, it must not contain any commas.

Examples:

1.     INIT_PROC= PSTART1.RXX
   The PSTART1.RXX procedure (say, in x:\SRE2003\BIN\PSTART1.RXX) will be called.
2.     INIT_PROC= INITHOST.RXX , SRELITE2\LITEINIT.RXX
   First the INITHOST.RXX procedure (say, in x:\SRE2003\BIN\INITHOST.RXX) will be called. Then, LITEINIT.RXX will be called (say, in x:\SRE2003\SRELITE2\LITEINIT.RXX).
3.     init_proc=srehttp2\SREH2INI.RXX, chk_init.rex 80 5 10 30 0
   The srehttp\sreh2ini.rexx procedure is called first. Then, CHK_INIT.REX (in SRE2003\BIN) is called, and is sent the string 80 5 10 30 0
4.     INIT_PROC=0
   No initialization procedure

Notes:

• Each INIT_PROC procedure is called with the http port, the servername, the directory containing the filter, and the (optional) argument string. Thus, initialization procedures should start with something like:

  parse arg PORT,SERVERNAME,FILTER_DIR,ARG_LIST

• Each INIT_PROC procedure must return either a
  1 for success, or a
  0 for failure
  If 0 is returned, SRE2003 will immediately shutdown.

• Possible uses for initialization procedures include launching a daemon manager and reading in host definitions.

LIMITBODY

The maximum size of a request body (as may be included in a POST or PUT request). If the request body is greater then this size, a 413 response is returned.

Example: LIMITBODY=150

Notes:

• LIMITBODY can be set on a connection specific basis -- see the description of SRE_COMMAND for the details.

LIMITHEADER

The maximum size of request headers, in Kbytes. If the request has headers greater then this size, a 413 response is returned.

Note that there are no size restrictions on a single request header -- so long as all request headers (including the request line) are less then LIMITHEADER kbytes.

Example: LIMITHEADER=10

LIMITCLIENTS

Maximum simultaneous connections. Each connection handles a single request at a time (though a "maintained connection" can handle a sequence of requests).

Example: LIMITCLIENTS=15

LIMITTIMETOTAL

Maximum time allowed per request (in seconds). If the total time required to resolve AND transmit a response exceeds LIMITTIMETOTAL, the connection will be closed. No error message is sent to the client -- the connection is just closed.

Example: LIMITTIMETOTAL=160

Notes:

• LIMITTIMETOTAL must be greater then LIMITTIMEINACTIVE.
• LIMITTIMETOTAL can be set on a connection specific basis --
• see the description of SRE_COMMAND for the details.
• LIMITTIMETOTAL refers to a particular request, not to all the requests that may be handled over a single maintained connection.

LIMITTIMEWAIT

Maximum time to wait on a maintained connection (in seconds). This is the number of seconds that SRE2003 will wait for a new request to arrive. This wait occurs after a prior request has been resolved.

Example: LIMITTIMEWAIT=10

Notes:

• LIMITTIMEWAIT can be set on a connection specific basis -- see the description of SRE_COMMAND for the details.

LIMITTIMEINACTIVE

Maximum intra-request idle time (in seconds). If LIMITTIMEINACTIVE seconds have passed without any bytes being transmitted (or recieved), SRE2003 will close the connection.

Example: LIMITTIMEINACTIVE=20

Notes:

• LIMITTIMEINACTIVE must be less then LIMITTIMETOTAL
• You can set LIMITTIMEINACTIVE on a request specific basis -- see the description of SRE_COMMAND for the details.
• LIMITTIMEINACTIVE should be greater then DEFAULT_SOCKET_TIMEOUT. Note that DEFAULT_SOCKET_TIMEOUT is typically used to detect a bad connection (it's the maximum wait time allowed within a sequence of socket calls). LIMITTIMEINACTIVE is the maximum time SRE2003 will wait for the filter to "do something".

MAXCOUNTERS

Maximum number of "counter" variables. Counter variables are used to maintain counts. Currently, SRE2003 maintains the following counts: BYTESREADTOTAL, BYTESSENTTOTAL, TRANSACTIONS, REQUESTS, LIMITS, ERRORS, BROKEN, and CACHEHITS.

Thus, MAXCOUNTERS should be at least 10.

Examples: MAXCOUNTERS=30

Notes:

• Each counter variable requires about 30 bytes of storage
• You can set, augment, and read counter variables with the SRE_COUNTVARS function.

MAXCLIENTNAMES

Size (number of entries) in "client name" cache. The client name cache is used to lookup an IP name given an IP numeric address (IP requests contain the IP address of the client, but not the IP name). Since this may require a DNS lookup, SRE2003 can maintain a cache of the most recently used names (thereby avoiding a potentially costly call to your DNS server). This is most useful when clients tend to revisit your site several times in a short time periods (say, to fetch embedded images).

Example: MAXCLIENTNAMES=250

Notes:

• each entry in the clientname cache requires about 100 bytes
• the clientname cache is dynamic. Once it is filled, a LRU algorithim is used to remove old entries (so that new entries can be added).
• To disable the "client name cache", set MAXCLIENTNAMES=0
• the SRE_CACHE function can be used to set and view entries in the clientnames cache.

MAXSTATVALS

Size (number of entries) to use to compute statistics SRE2003 can compute several sets of statistics, including average, minimum, variance, and maximum. To do this, a set of the "most recent" values is required. MAXSTATVALS sets how large this set should be.

Currently, SRE2003 maintains statistics on the following measures:
    TOTAL_TIME,START_TRANSMIT_TIME, BYTES.
SRE2003 also retains the most recent values of CLIENTADDR (dotted numeric address) and REQUEST (the request line).

In other words, MAXSTATVALS sets the number of "measures of the most recent responses" to retain.

Example: MAXSTATVALS=100

Notes:

• To set and view "statistics", you can use the SRE_LIST function.

MAXMD5S

Size (number of entries) of MD5 cache. Http/1.1 allows the server to attach a Content-MD5 response header, which contains the md5 hash of the response body. Clients can use this value to detect transmission problems, tampering, or other errors.

Since computation of an MD5 is mildly CPU intensive (especially for long files), SRE2003 maintains an "MD5" cache. The MD5 cache is used for "permanent" files -- when SRE2003 is asked to compute an MD5 for such a file, SRE2003 can first check it's cache to see if such a value has already been computed.

Examples:
      MAXMD5S= 200
      MAXMD5S=0 -- do NOT maintain an MD5 cache.

Notes:

• If you do NOT intend to include a Content-MD5 response header, you might as well set MAXMD5S=0.
• Each entry in the MD5 cache requires about 125 bytes.
• The SRE_MD5 function will automatically check the SRE_MD5 cache (if it is enabled) before computing an MD5 of a file.

  If you use the SRE_MD5 function with the MD5 cache enabled, all MD5 computations are serialized. This seems to to help system stability. Thus, if you set MAXMD5S=0, then there will be no MD5 cache, and MD5 computation will not be serialized.

• The SRE_CACHE function can be used to view the current contents of the MD5 cache.
• The MD5 cache is dynamic. Once it is filled, a LRU algorithim is used to remove old entries (so that new entries can be added).

NO_MODIFY_ONTHEFLY

Flag to suppress the M modify a few SRE2003.CFG parameters on-the-fly feature.

Examples:
      NO_MODIFY_ONTHEFLY=1 suppress on the fly modification
      NO_MODIFY_ONTHEFLY=0 allow on the fly modification

Notes:

• you can still modify SRE2003.CFG using SRE2003 -m , or edit it directly with a text editor.

NO_TASKS

The NO_TASKS parameter can be used to limit who can launch tasks. The following values of NO_TASKS are recognized.

• 0 = disable the task manager.
• 1 = enable the task manager
• 2 = the task manager is enabled when SRE2003 starts, but is disabled before the first request is received. This is useful if you want to use an initialization procedure to launch a few (perhaps recurring) tasks.

In particular, the use of NO_TASKS=2 may reduce some security concerns due to scripts launching unwanted/resource-wasting tasks (though it doesn't allay it much, since a well-written script can do just about anything).

PORT

Port the server is running on. The standard http port is 80.

Example: PORT=80

REQ_CACHE_ENABLE

Enable the request cache.

REQ_CACHE_ENABLE=nn	enable, with nn the time-to-live of each entry, in days. Thus, REQ_CACHE_ENABLE=1 means "enable the cache, each item has 24 hour lifespan"
REQ_CACHE_ENABLE=0	do NOT attempt to resolve requests from the request cache

PRE_REQUEST

File, or a comma delimited list of files, containing one (or several) pre-request procedures.
These procedures are called before each request is processed (it is called after the client's request had been read, and before checking the request cache). Note that PRE_TRANSACTION procedures are called well before the PRE_REQUEST procedures.

• If you do not specify any path, a file is assumed to be relative to the SRE2003 \BIN directory.
• If you specify a relative path, a file is assumed to be relative to the main SRE2003 directory (the directory containing SRE2003.CMD).
• Or, specify a 0 if you do not have any pre-request procedures.

Examples:

1.     PRE_REQUEST= BALANCER.RXX
   The BADIPS.RXX procedure (say, in x:\SRE2003\BIN\BALANCER.RXX) will be called.
2.     PRE_REQUEST= CHK_PRE.RXX
   The CHK_404.REX procedure (SREhttp/2 "check for a repetitive attack"), in SRE2003\BIN, is called.
3.     PRE_REQUEST= BALANCER.RXX , SRELITE2\PREAUDIT.RXX
   First the BALANCER.RXX procedure (say, in x:\SRE2003\BIN\BALANCER.RXX) will be called. Then, PREAUDIT.RXX will be called (say, in x:\SRE2003\SRELITE2\PREAUDIT.RXX).
4.     PRE_REQUEST=0
   No pre-request procedures are called.

Notes:

• Each PRE_REQUEST procedure is called with the following arguments (these are a subset of the arguments sent to the filter).

  source	port and ip address info:    serveraddr serverport transaction clientaddr clientport
  request_line	The request line (sent by the client):    verb selector protocol
  hostinfo	The host-nickname to whom this request is directed:    host host_nickname data_directory
  socket	The socket used for this request.
  own_id	The transaction's OWN_ID. This can be used by pre-request procedures to talk to other daemons (for example, it be used in the SRE_CACHE procedure).

• Each PRE_REQUEST procedure must return either a
  1 for success -- request processing will continue
  0 for failure -- don't process this request, but wait for further requests (over a possibly maintained connection)
  An http response, structured as:
  response_line '0d0a'x resp_headers '0d0ax' blank_line '0d0a'x a_message
  Where ...

  • response_line is the response line. For example:
    HTTP/1.1 200 ok
  • resp_headers are the response headers.
    If no resp_headers are specified, SRE2003 will add DATE, SERVER, and CONNECTION response headers.
  • a_message is the content body (which can be text or binary).

  Note that blank_line means a blank line. That is, the resp_headers should be a '0d0a'x delimited list of response headers, that end with an empty line.

• Possible uses for pre-request procedures include load balancing, and status monitoring. BALANCE1.RXX, in the BIN subdirectory, contains a simple example of a load-balancer pre-request procedure.

• You can not modify PRE_REQUEST procedures on-the-fly (if you modify a pre-request procedure, it's changes will take place when you restart SRE2003).

• In addition to the information provided in the arguments to the PRE_REQUEST procedures, you can use many of the SRE2003 procedures.For example, you can use...
  SRE_VALUE to obtain system and global variables.
  The string manipulation procedure (such as SRE_REPLACEWILD and SRE_PACK64)
  The time procedures (such as SRE_GMT and SRE_DATESTAMP)
  SRE_EXTRACT_USERNAME

PRE_TRANSACTION

File, or a comma delimited list of files, containing pre-transaction procedures.
These procedures are called before each transaction is processed (it is the first thing that the transaction daemon does).

• If you do not specify any path, a file is assumed to be relative to the SRE2003 \BIN directory.
• If you specify a relative path, a file is assumed to be relative to the main SRE2003 directory (the directory containing SRE2003.CMD).
• Or, specify a 0 if you do not have any pre-transaction procedures.

Examples:

1.     PRE_TRANSACTION= BADIPS.RXX
   The BADIPS.RXX procedure (say, in x:\SRE2003\BIN\BADIPS.RXX) will be called.
2.     PRE_TRANSACTION= BADIPS.RXX , SRELITE2\TOO_BUSY.RXX
   First the BADIPS.RXX procedure (say, in x:\SRE2003\BIN\BADIPS.RXX) will be called. Then, TOO_BUSY.RXX will be called (say, in x:\SRE2003\SRELITE2\TOO_BUSY.RXX).
3.     PRE_TRANSACTION=0
   No pre-transaction procedures are called.

Notes:

• Each PRE_TRANSACTION procedure is called with the client's (numeric) IP Address, and a 1 if VERBOSE is a status-item.
  Thus, initialization procedures should start with something like:

  parse arg client_address,verbose

• Each PRE_TRANSACTION procedure must return either a
  1 for success -- transaction processing will continue
  0 for failure -- immediately close connection
  0 a_message for failure -- return a 503 response, using a_message in a X-Message response header.
  If 0 or 0 a_message is returned, no other PRE_TRANSACTION procedures will be called for this transaction.

• Possible uses for pre-transaction procedures include checking for "wildcarded" BAD ip addresses, or checking to see if the server is currently too busy. BADIPS.RXX, in the BIN subdirectory, contains a simple example of check for bad IP addresses, using wildcards pre-transaction procedure.

• In addition to the information provided in the arguments to the PRE_REQUEST procedures, you can use several of the SRE2003 procedures. For example, you can use...
  SRE_VALUE to obtain system and global variables.
  The string manipulation procedure (such as SRE_REPLACEWILD and SRE_PACK64)
  The time procedures (such as SRE_GMT and SRE_DATESTAMP)
• You can not modify PRE_TRANSACTION procedures on-the-fly (if you modify a pre-transaction procedure, it's changes will take place when you restart SRE2003).

REQ_CACHE_CALL_ANYWAYS

For auditing and other purposes, SRE2003 can call the filter even when the request has been resolved through use of the request cache.

REQ_CACHECALL_ANYWAYS can take the following values:

REQ_CACHECALL_ANYWAYS 0	Do not call the filter after use of the request cache
REQ_CACHECALL_ANYWAYS *	Always call the filter
REQ_CACHECALL_ANYWAYS 1	Always call the filter (same as *)
REQ_CACHECALL_ANYWAYS A_LIST	Call the filter if any of the "abbreviations" in the spaced delimited A_list matches the selector

Example: REQ_CACHECALL_ANYWAYS=/DOCS/ /HTMLS/

Notes:

• See Appendix 2 for a discussion of the SRE2003 request cache.
• See the REQ_CACHE_RECORD parameter for a discussion of alternate means of auditing cache resolved requests.
• See the description of the arguments sent to the filter, and the description of the STILL_ALIVE function, for a discussion of how to detect whether the request cache was used.

REQ_CACHE_HITFILE

File containing record of cache hits. This file is used when REQ_CACHE_RECORD is enabled. It should be a relative filename (relative to the SRE2003 installation directory). You might want to place it in the LOG subdirectory of the working directory.

Example: REQ_CACHE_HITFILE=LOG\CACHEHIT.LOG

Technical Note	The max_filesize user configurable parameter (in the SRE2003\BIN\SRE_AUDI.RXX directory) controls the maximum size (in bytes) of the audit file and the cache-hits file. When this size is exceed, the audit (or cache) file is renamed, moved to the sre2003\log\arc subdirectory, and a new one is started.

REQ_CACHE_RECORD

Record cache hits to the REQ_CACHE_HITFILE file.

As an alternative to "calling the filter anyways", SRE2003 can maintain a simple log of all requests for which a "request cache" response was used. This log file (specified in REQ_CACHE_HITFILE) contains a running sum, by selector, of the 200 and 30x (unmodified) responses).

REQ_CACHE_RECORD can take the following values:

0	Do NOT use the cache-hit file
mmm	Use the cache hit file, and save results very mmm minutes

Example: REQ_CACHE_RECORD = 5

Notes:

• the cache hit file is NOT augmented when you "call the filter anyways"

REQ_CACHE_SIZE

The size of the request cache, in number of entries. Each entry requires about 300 bytes.

Example: REQ_CACHE_SIZE=200

REQ_CACHE_VERIFY

Enable http/1.1 style verification prior to use of the request cache.

REQ_CACHE_VERIFY can take the following values:

0	never verify (always use cache entry if it exists)
1	verify, check Pragma, Cache-Control, If-None-Match, and If-modified request headers. If these are binding (i.e.; if Cache-control: no-cache is specified), then do NOT use the request cache.
2	verify, but ignore Pragma and Cache-Control:no-cache request headers (that is, just check etag and date)

SECURITY_LEVEL

A value that is used by some filters (such as the SIMPLE filter, sreLite2, and SREhttp/2) to determine who is allowed to remotely modify server parameters.

The following values of SECURITY_LEVEL are used by the SIMPLE filter (the simple filter that comes with SRE2003), and by the SREhttp/2 and sreLite2 filters.

value of SECURITY_LEVEL	Interpretation
0	Remote configuration of server parameters is not permitted. Instead, you'll have to hand-edit (using your favorite text editor) the parameter files. In other words, the on-line configuration utilities are totally disabled (assuming you are using SIMPLE, SRElite2, or SREhttp/2).
1	Remote configuration is only allowed from the server's own machine. In other words, it's not really remote -- you have to be running a browser on the same machine that SRE2003 is running (actually, the client ip address must match the server ip address, so a careful spoofer could cause damage). However, this does permit an "on-site webmaster" to use the on-line (html form based) configuration utilities.
otherwise	SIMPLE: Anyone is allowed access sreLite2: By default (if you use the version of SRELITE2.CFG file that comes with sreLite2) remote configuration is granted to users with a SUPERUSER privilege. Note that there is no encryption of the http stream, so this is not a very high level of protection. If you have removed the /SYSTEM/* SUPERUSER SEL_REQUIRES entry, then anyone will be able to use the configuration utilities! SREhtt/2: Only clients with SUPERUSER privileges are granted access. Note that this is hard-coded -- it does not require any entries in the SREhttp/2 ATTRIBS.CFG file(s). Also note that, as with SRElite2, there is no encryption of the http stream, so this is not a very high level of protection.

For other filters, the filter's documentation should describe how SECURITY_LEVEL is used.

SERVERADDR_USE

The dotted numeric IP address that SRE2003 will report to the world. This is not used by the TCP/IP layer, it only effects SRE2003 functions (such as by SRE_EXTRACT("SERVERADDR")).

SERVERADDR_USE should be one of:

• The dotted (numeric) IP address of this site.
• A 0, which means use TCP/IP calls to determine this site's dotted IP address

In general, it is safest to use a value of 0. However, there are cases (such as when your server is behind a NAT-capable router, and has a 192.168.0.nn address) for which the numeric ip address is meaningless. In such cases, set SERVERADDR_USE to the IP address of this site.

Notes:

• To find what TCP/IP reports as this site's IP address, check (with the SRE_VALUE function) the SRESYS variable SERVERADDR_CANONICAL
• If you use SERVERADDR_USE, you should consider setting SERVERADDR_INHOUSE (say, to the server machine's canonical IP address).

Examples:

SERVERADDR_USE=152.22.61.6

SERVERADDR_USE=0

SERVERADDR_INHOUSE

A space delimited list of dotted numeric IP addresses that SRE2003 will treat as being in-house..

SRE2003 does not directly use this parameter.

Filters can use SERVERADDR_INHOUSE to define a set of IP addresses to treat as being in house, in the sense of being trusted (i.e.; behind a firewall).

For example, the SREhttp/2 filter uses SERVERADDR_INHOUSE to control what machines can access configuration utilities (when SECURITY_LEVEL=1).

SERVERADDR_INHOUSE is often used in conjunction with SERVERADDR_USE. For example, if your server is behind a router (say, with an explicitily assigned address of 192.168.0.3), you can:

• Use SERVERADDR_USE to set what IP address SREhttp/2 should report
• Use SERVERADDR_INHOUSE to specify a set of machines from which you can modify SREhttp/2 parameters.

SREhttp/2 note: if you...

1. set SERVERADDR_USE,
2. set SERVERADDR_INHOUSE=0,
3. set SECURITY_LEVEL=1

you may not be able to run configuration utilities from the server (since SREhttp/2 will compare the SERVERADDR_USE address to the 192... IP address).

Example:
   SERVERADDR_INHOUSE=192.168.0.3 192.168.0.5 192.168.0.12

SERVERNAME_USE

The domain name that SRE2003 will report to the world. This is not used by the TCP/IP layer, it only effects SRE2003 functions (such as by SRE_SERVERNAME('default')).

SERVERNAME_USE should be one of:

• A dotted domain name
• A 0, which means use TCP/IP calls to determine this site's domain name

In general, it is safest to use a value of 0. However, there are cases (such as when your server is behind a NAT-capable router, and has a local name) for which the TCP/IP derived domain name is meaningless. In such cases, set SERVERNAME_USE to the "canonical" domain name of this site.

Notes:

• SERVERNAME_USE is only used for the "default" (the canonical) host.
  In particular, it is not used when a Host: header matches one of your host definitions.
• To find what TCP/IP reports as this site's domain name, check (with the SRE_VALUE function) the SRESYS variable SERVERNAME_CANONICAL

Examples:

SERVERNAME_USE=www.foobar.net

SERVERNAME_USE=0

SERVER_SOFTWARE

Reported name of this server software. This variable can be used when reporting the name of the server software; say, in a Server: response header.

Set SERVER_SOFTWARE=0 to use the default name (that is hard coded into SRE2003.CMD).

Example: SERVER_SOFTWARE = SRE2003 for OS/2, ver 1.12

Notes:

• The SRE_SERVER('N') function returns the value of the SERVER_SOFTWARE parameter.

SERVER_SOFTWARE_SHORT

Short reported name of this server software. This is an alternative to SERVER_SOFTWARE.

Example: SERVER_SOFTWARE_SHORT= SRE2003

Notes:

• The SRE_SERVER('H') function returns the value of the SERVER_SOFTWARE_SHORT parameter.

STATUS_ITEMS

Specify what to display on the status screen. SRE2003 can display several types of dynamic information on it's status screen (the status screen is what SRE2003 displays while it is running).

STATUS_ITEMS should be a space delimited list containing one or more of the following tokens:

ERRORS, CONNECTION, CLIENTS, CLOCK, LASTACCEPT, LIMITS, PEAK, STATS, VERBOSE
where:

CLIENTS	report number of currently active connections
CONNECTION	report each connection
CLOCK	display current time
ERRORS	report cumulative number of errors.
LASTACCEPT	display time of most recent connection
LIMITS	report cumulative number of timeouts
PEAK	report peak number of connections
STATS	report summary statistics
VERBOSE	report various other items

STATUS_MESSAGE

A message displayed on the side of the status screen. Only the first 12 characters are used.

TRACKING_ON

Enable SRE2003's "client tracking" facility.
    TRACKING_ON = 0 suppress
    TRACKING_ON = 1 enable

The tracking facility instructs SRE2003 to store "status" and other "tracking" information. You can the retrieve this tracking information, on a client specific basis, by using SRE_TRACKIT.

For more details on tracking, see Appendix 5.

UNALLOWED_FILE

A file name (relative to the SRE2003 working directory). This file can contain a list of UnAllowed numeric IP addresses. All requests from clients with these IP addresses are immediately closed -- no response is given, the socket is shutdown.

If you do NOT need this capability, set UNALLOWED_FILE=0

Examples:
UNALLOWED_FILE=BadIPS.IN
UNALLOWED_FILE=0

The UNALLOWED_FILE should contain one IP address per line. Lines that start with a semi-colon are comments (and are ignored), as are blank lines. Note that IP addresses should be the dotted numeric address (do not use domain names).

Alternatively, you can enter IP addresses that contain a * wildcard character. These entries are used by the BADIPS.RXX PRE_TRANSACTION procedure.
For example:
    125.22.*.1 matches 125.22.0.1 to 125.22.255.1

Examples of UNALLOWED_FILE entries

      98.13.61.22
      98.13.61.23
      ;clients from 98.14.* are not granted access
      98.14.*

Note that you can add and remove entries from the UNALLOWED_FILE at any time -- SRE2003 will check the UNALLOWED_FILE about once a minute.

Before proceeding, consider this short introduction to how SRE2003 works. Abstracting from configuration and startup issues, and ignoring some fancy tricks one can use, the sequence of events leading to resolution of a request is:

1. SRE2003 (the main thread) recieves a TCP/IP request from a client.

   If the client's IP address is listed in the UNALLOWED_FILE, the connection will be immediately closed.

2. SRE2003 launches a daemon (the "transaction daemon"), which reads the request line, request headers, and (possibly) the request body.

   Before doing anything (that is, before reading any bytes from the socket), the transaction-daemon will call PRE_TRANSACTION procedures (if any have been specified).

3. The transaction daemon launches another daemon (the "request daemon"). This request daemon calls the "filter", and provides several arguments that give information about the request -- such as the "selector" and the client IP address.

   Before launching the request daemon ...

   1. the PRE_REQUEST procedures are called (if any have been specified). Depending on the action of the PRE_REQUEST procedure(s), the request daemon may not be invoked.
   2. if the request cache has been enabled, the request-uri is looked for in the request-cache. If found, the file it points to returned to the client (and the request daemon is not invoked).

   If a fast_filter was specified, the request daemon will call it first; and call the regular filter only if the fast-filter was unable to resolve this request.

4. The filter then uses these arguments, along with additional information, to fashion a response. Typically, the response consists of the contents of a file stored on the server's hard disk. However, if may be a 3xx (redirection) or a 4xx (access denied) response.

5. In addition to the arguments sent from SRE2003, the filter can use function calls to request additional information (such as the value of specific request headers) from SRE2003. Operationally, these function calls commumicate with the "transaction daemon".

   In some sense, one can think of SRE2003 as providing an API that a REXX program (a "filter") uses to recieve http requests and to send http responses.

6. When the filter is ready to send a response (to the client), it uses function calls to tell the transaction what the contents of the response should be. The transaction daemon takes care of all the tcp/ip details, and upon completion of the response informs the filter.

   Prior to sending a response to the client, the transaction daemon may also apply instance manipulations to the contents. For example, range extraction is built into SRE2003; and other manipulations (such as delta encoding) can be supported via special IM modules for SRE2003.

7. The filter may then do some auditing or other such tasks.
8. The filter informs the transaction daemon that it is done, and exits (the request daemon immediately ends).
9. The transaction daemon waits to see if another request has been sent (that is, the connection may be a "maintained connection"). If a new request is available, go back to step 3. Otherwise, the daemon records some transaction information, and exits.

Note:

"daemons" are independent threads. Actually, in order to avoid operating system overhead, daemons are "recycled", so that rather then ending, they go into a dormant state and await re-invocation.

In greater detail ...

SOURCE

Contains the following information in a space delimited list

servaddr	ip address of destination server (may vary if your server is handling multiple ip addresses)
port	port that recieved the request
transaction	the transaction number
who	numeric IP address of the client
whoport	port used by the client

You can parse "source" using:
    parse var source myaddr port transaction who whoport .

REQUEST

The request line.
Examples:
    GET /samples/foo.bar HTTP/1.1
    GET /abc/de%28.txt HTTP/1.1

SELECTOR

The "selector" portion of the request line, with URL-encoding performed (%nn converted to their ascii equivalent) / removed.

From the above examples, the selectors are:
    /samples/foo.bar
    /abc/de(.txt

HOSTINFO

Information about the host this request was sent to. This will be three words, seperated by commas:
    host,host_nickname,datadir
where:

host	either the ip name of your site, or the value of a HOST: request header.
host_nickname	the "host-nickname" assigned to this host, or ' ' (if a host-nickname was not defined for this host)
datadir	the "default data directory". If no host_nickname is available, this will be the value specified by the DATADIR parameter (in SRE2003.CFG). Otherwise, it's a host-specific data directory (which may be the same as the "default" data directory).

Examples:
    jones.gonzo.net,,f:\www
    bigtime.circus.org,circus1,g:\webs1\circwww

The first example could be to a site for which no hosts have been defined.

Notes:

• Specification of host-nicknames is done with the SRE_HOSTINFO procedure, (or with the INITHOST.RXX initialization procedure).

ID_INFO

Request and transaction daemon identifier information. Several SRE2003 functions (such as SRE_REQUEST_INFO) will run just-a-bit-faster when you supply them with this parameter.

REQUEST_NUMBER

The request number. This has two components: a numeric "transaction number", followed by an alphabetic "request number". The request number, which starts from A, is the "request within this transaction", it is NOT the total number of requests recieved.

Note that the sequence of request numbers is: A,B,..,J,AA,AB,..,AJ,BA,...

Example: 4A

AUTHORIZATION_HEADER

Since the Authorization: request header is often used, it is provided as an argument (it can also be read using the REQFIELD function).

Example: Basic ZGFuOmRhbg==

IS_ALIVE

Flag indicating whether the request is still alive. The request may already have been satisfied (typically due to a request-cache hit), and the filter may have been "called anyways" -- say, so that the filter can audit the request.

If IS_ALIVE=1, the request has NOT been satisfied.
Otherwise, it has.

• you can obtain the values of these arguments using:

      parse arg source,request,sel,MOREINFO,idInfo,reqnum,authh,isalive

• Upgraders note: the first three arguments are identical to what GoServe sends.

SRE2003 includes a library of procedures that can be used when writing filters, or when writing addons for preexisting filters. Note that filters (such as sreLite2 or SREhttp/2) may have their own set of functions -- in many cases these provide additional capabilities that addon/script creators may find useful.

The following briefly describes a few of the more important SRE2003 functions. You can find more complete descriptions in SRE2PRC.HTM.

SRE_AUDIT	write lines to the AUDIT.LOG file
	Syntax: Call sre_audit(source_name,amessage) Example: call sre_audit('ADDON_1','Addon #1 completed at 10:30 with no errors')
SRE_AUTH_RESPONSE	return an authorization required response
	Syntax: rcode=sre_auth_response(realm_name) Example: rcode=sre_auth_response('The Protected Zone')
SRE_COMMAND	recieve and send information to the client, etc.
	Syntax: STATUS=SRE_COMMAND('ACOMMAND OPTION',amessage) ACOMMAND can be one of: ALIVE AUDIT CONTROL FILE HEADER NODATA READ REPSONSE SEND SET VAR OPTION is an option list that depends on ACOMMAND. amessage is an argument that is required for the VAR and other options Examples: rcode=sre_command('FILE type text/html name D:\WWW\HELLO.HTM') rcode=sre_command('VAR type text/plain',a_text_string) rcode=sre_command('STRING This is a simple string! ') rcode=sre_command('HEADER X-Color: Neon Green ')
SRE_DATADIR	return the default (or host specific) data directory
	Syntax: datadir=SRE_datadir()
SRE_ERROR_RESPONSE	return a server error response
	Syntax: rcode=sre_error_response(type) type can be one of the following http/1.1 error response codes:   403 404 410 413 414 500 501 503 (which respectively mean Forbidden, NotFound, Gone, LargeBody,LargeURI, ServerError, NotImplemented, and Busy) Example: rcode=sre_error_respons(403)
SRE_REQFIELD	returns the value of a request header
	Syntax: avalue=SRE_reqfield(header_name) Example: avalue=SRE_reqfield('User-Agent')
SRE_SERVER	return descriptive name of the server software
	Syntax: sname=SRE_server()
SRE_SERVERNAME	return the domain name of the server
	Syntax: sname=SRE_servername()
SRE_WRITE_ERROR	write a message to the error log and to the status screen
	Syntax: call sre_write_error(error_message) Example: call sre_write_error('Foo1: got a BAR error')
SRE_WRITE_MESSAGE	write a message to the status screen
	Syntax: call sre_write_message(short_message) Example: call sre_write_message(' Hello cruel world? ')

  Copyright 2000-2003 by Daniel Hellerstein.

  Permission to use this program for any purpose is hereby granted
  without fee, provided that the author's name not be used in
  advertising or publicity pertaining to distribution of the software
  without specific written prior permision.

  Use of this product, or portions of this product, is subject to the
  following:
     1)  Portions of the code are adapted from other authors' work
         (these are noted where appropriate); you'll need to contact these
         other authors for appropriate permissions.
     2)  SRE2003 uses several 3rd party dynamic libraries and executables:
         i) Quercus System's REXXLIB procedure library.  The
            license for REXXLIB gives the author the right to distribute
            REXXLIB without charge.  This right may NOT extend to
            redistributors (though as of April 2000 it appears that
            REXXLIB has been released to the public domain).
            Please contact Quercus Systems for details.
         ii) MKEY is freeware, and is part of a larger menuing system.
             For more info, contact the author at m1@uni-one.dl
        iii) FILEREXX is freeware. The author (Jeff Glatt) can be found
             on comp.os.os2.rexx. However, he no longer works with os/2.

     3)  We, the authors of SRE2003 and any potentially affiliated
         institutions, disclaim any and all liability for damages due
         to the use, misuse, or failure of the product or subsets of
         the product.

         *   In particular, SRE2003 and related product are NOT     *
         *   guaranteed to be secure.                               *

         We do design and code our product with careful attention to potential
         security holes, and and we try to quickly fix any problems (or
         potential problems) that may be discovered. However, SRE2003's
         fundamental design precept is "open-source", with security an
         important secondary consideration.

            If you REQUIRE a highly secure web-server, you should
            carefully review and test SRE2003. In other words,
            you may need to choose a different server.

  Furthermore you may also charge a reasonable re-distribution fee for
  SRE2003; with the understanding that this does not remove the
  work from the public domain and that the above provisos remain in effect.

    THIS SOFTWARE PACKAGE IS PROVIDED "AS IS" WITHOUT EXPRESS
    OR IMPLIED WARRANTY.
    THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE PACKAGE,
    INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS.
    IN NO EVENT SHALL THE AUTHOR (Daniel Hellerstein) OR ANY PERSON OR
    INSTITUTION ASSOCIATED WITH THIS PRODUCT BE LIABLE FOR ANY
    SPECIAL,INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER
    RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION
    OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR
    IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE PACKAGE.

   Lastly, if you would like to modify SRE2003, please contact the
   authors for the source code (Daniel Hellerstein, danielh@crosslink.net).

Basically, the request cache matches a selector to a file. If such a match exists (if a cache hit occurs), then SRE2003 will return the file to the client, without "calling the filter". By not calling the filter, hence avoiding request resolution overhead, responses can be sent substantially quicker -- 50% time savings are not uncommon.

Actually, even though the filter is not called, some processing will take place for "request cache hits". In particular:

• If- conditions will be checked
• Range, or other Instance Manipulation modifications can occur
• A few new headers may be computed (such as current date)

The point is that the request cache is not a cache of contents -- it is a cache of what file a selector points to.

Entries are automatically created in the request cache when:

1. SRE2003 used a 'FILE' mode to respond to a prior request for this selector,
2. caching is not explicitily suppressed (on a selector specific basis),
3. the file is not temporary
4. and, of course, request caching is enabled.

For more details on 'FILE' mode, see the description of the FILE option in SRE_COMMAND.

Use of the request cache does have a few drawbacks :

• The request cache requires memory resources
• The request cache should not be used with dynamic resources
• Auditing of responses resolved "from the request cache" may be incomplete.

The following description is taken from the GoServe documentation, with a few minor changes.

1. To the file called 'HOSTS' (no extension) in your \TCPIP\ETC, or in \MPTN\ETC, directory add the line:
      127.0.0.1 loopy
   where loopy is the name by which you want your machine to be known when using the loopback connection. This name can be a single word (e.g., loopy), or an internet-style name (e.g., loopy.my.org).

   It's a good idea to have both formats, on two lines:

               127.0.0.1  loopy.my.org
               127.0.0.1  loopy  

   Notes:

   In general, check the value of the ETC environment variable to find out where the HOST file should be placed.

   If there is no \xxx\ETC\HOSTS file, create one.

   If you are running OS/2 4.5 or above ..
   Instead of edting HOSTS., you can use the TCPCFG configuator --

   1. Run TCPCFG2 at an OS/2 command prompt
   2. Select the Host Names tab
   3. Click on the Hosts tab (at the bottom of the screen)
   4. Add a row. For example:

                 IP address: 127.0.0.1
                 Host Name: loopy.my.org
                 Alias: loopy
                 Comments: this is my loopback entry 

2. Before running SRE2003, execute the command:
      ifconfig lo 127.0.0.1
   This only needs to be run once, so can be run from STARTUP.CMD or from any command referenced in your Startup folder. Note that the second word is the lowercase of the two letters 'L' and 'O'.

   If you are running OS/2 4.5 or above ..
   Instead of the above, you can use the TCP/IP configurator (TCPCFG2):

   On the 'Network' page, click on 'loopback interface',
   then check 'Enable interface' and 'Manually, using',
   then enter '127.0.0.1' as the 'IP address'.
   
   

Even if you are not connected to a network, your browser should not have a proxy or SOCKS server enabled (or it will try and use the network to find it before checking the HOSTS file).

The FAST_FILTER option allows you to do this. If you specify FAST_FILTER, then SRE2003 will:

1. first call the FAST_FILTER.
2. if the fast filter is able to handle the request, SRE2003 exits (or waits for the next request on a maintained connection)
3. otherwise, SRE2003 calls the "normal" filter

Fast-filters are essentially the same as normal filters, they are sent the same arguments, and can access the same parameters and functions. There is one difference -- the fast-filter needs to inform SRE2003 as to whether it was successful or not.

To do this, the fast filter should return either:

• 0 : not successful -- the normal filter should be called
• 1 : success. SRE2003 can close the connection, or wait for next request on a maintained connection

When enabled (with the TRACKING_ON parameter), SRE2003 will maintain
information on the current status of each client. This information is
can be read (or modified) by using the SRE_TRACKIT procedure.

Note that a shared daemon is used to store this information. Therefore, it
may not be completely up to date -- when reading parameters, the actual
parameters are not read; rather, their values as of their most recent
update is used.

Many of the variables that are "tracked" are also available via SRE2003
"EXTRACT" procedure. However, EXTRACT is designed to be used to obtain
values of a client (where a client is synonymous with a connection) "own"
variables.

Basically, SRE_TRACKIT makes it easy to find status information for other
clients.

Currently, the following client-specific parameters are set by SRE2003.

   BYTESSENT_REQ  bytes sent for this request (possibly reset if multiple
                  requests per connection)

   BYTESREAD_REQ  bytes read for this request (possibly reset if multiple
                  requests per connection)
   CLIENTADDR     dotted numeric IP address of the client

   RECIEVE        RECIEVE can take the following values 
                   0 - not currently sending 
                  >0 - amount to be received
   READHEAD       Read header data. READHEAD can take the following
                  values:

                     PENDING Waiting to read
                     ACTIVE  Currently reading header
                     END     Header has been read

   READBODY       Read body data. READBODY can take the following values:
                     0 Not yet read
                     START Currently reading body
                     END Body has been read
   REQUEST        the request (within a transaction, starting with 1)
   SELECTOR       the full request selector (as recieved from the client)

   SEND           Currently "sending" information to client. SEND can take
                  the following values:
                      0   not currently sending
                      >0   sending this many bytes
                  timeout  timeout occurred on most recent send
                   broken  client broke the connection
                   error   error when sending
                     ok    piece succesfully sent

   SENDPIECES     # of pieces sent
                        0     -- send mode not activated
                        START -- send mode activated
                        END   -- send mode completed
                        number -- number of pieces sent

   STATUS         Current action being undertaken.

                  STATUS may take the following values:
                      START
                      ERROR_CONN_CLOSE
                      ERROR_LIMIT
                      WAITING 
                      ERROR_417 (expectation failed)
                      ERROR_TIMEOUT
                      ERROR_NO_HOST (Host: missing from http/1.1 query)
                      REQ_CACHE_USED (request-cache used)
                      PROCESS_xxx (xxx is one of the SRE_COMMAND options)
                      SNIPE   (S "sniping" is currently active)

   TRANSACTION    the transaction

In addition, you can set your own "connection specific" values by using
SRE_TRACKIT.

   ; comment lines start with semi-colon
   ; Entries should be of the form: ip_address , host_nickname , host_datadir
   ; for example:
   ; 127,0.0.1 , local1 , f:\www\h1
   ; localhost , local1 , f:\www\h1
   ; www2.mysite.org , altsite ,  e:\web\altfiles

This comparision almost always uses the Host: request header provided by all modern (post 2.0) browsers. Sometimes, as when you are serving multiple numeric IP addresses, the numeric IP can be used.

If the client-provided host information does NOT match any entry in the HOSTINFO.CFG file (for example, if the HOSTINFO.CFG has no entries), then the request is assumed to be to the "generic" host.

The host_nickname is a shorthand that is often used by SRE2003 filters (such as sreLite2). For example, sreLite2 can place access restrictions on selector, using both the resource name, and the host_nickname.

The host_datadir is the default data directory. Most filter will use this as the "root" of the web tree. That is, the host_datadir is used instead of the DATADIR parameter set in SRE2003.CFG. In other words, the DATADIR parameter is only used for requests to the "generic" host.

Notes:

Daemons (technical note)

In SRE2003, daemons are simply threads that run parallel to the main SRE2003 program. Daemons do almost all of the actual work -- about all the main program (actually, the #1 thread) does is pass information to daemons.

Rougly speaking, there are two kinds of daemons used in SRE2003:

1. Permanent daemons. These are created when SRE2003 starts, and stay active until SRE2003 shuts down. Permanent daemons handle database chores, auditing, and other such tasks.
2. Transient daemons. These are created on as-needed basis, and terminate when their single task is done. Requests from clients are resolved by transient daemons (actually, by semi-transient daemons -- see SREAPI.TXT for the details).

Hosts

It is possible for one server machine to handle requests to multiple IP addresses. This "multiple hosting" (also refered to as multi-homing) can occur either by having the server handle several numeric IP addresses, or by the existence of multiple aliases for a single numeric IP address.

Aliases:
A given IP address can be used by several hosts: including the host associated with the "canonical" name, and hosts associated with one of several aliases. Unfortunately, TCP/IP requests arrive with only the numeric IP address, making it difficult for the server to determine to which host the request is directed to. However, almost all browsers (such as Netscape 2.0 and above) include a HOST: request header containing the host to whom this request is directed. SRE2003 (as an http/1.1 compliant server) can use this request header to determine the proper host.

Host nickname

SRE2003 uses host nicknames as a shorthand for identifying when identifying hosts. For example, in the sreLite2 filter you can define parameters to apply to requests to selected hosts. Furthermore, the use of host nicknames allows several hosts to point to identical resources (given that each host has been assigned the same host nickname).

IDS: daemon and thread (technical note)

The SRE2003 documentation refers to ids in several contexts:

tid	Transaction and request thread ids. These are simply an integer value, indicating the thread-number (within the SRE2003 process). These tids are often used as indices in global data structures, since they are guaranteed to be unique to a thread (hence, to a request and to a transaction).
OWN_ID	Transaction, request, and other daemon-ids. These daemon-ids are multi-word identifers. Not only do they identify a daemon, but they contain crucial information that is used in intra-daemon communication. Each request is given its own own-id. This own_id is often used when requesting information from SRE2003's permanent daemons (such as the SRE2003 cache).
ID_INFO	In many contexts, an id_info argument is used. The id_info contains several pieces of information, such as the request and thread daemon-id. id_info is formatted as a comma delimited string, with each phrase containing one or more words. ID_INFO is used by many SRE2003 procedures (and procedures of SRE2003 filters) to communicate with the transaction-daemon. The transaction-daemon actively maintains information on a request, and does all the work of sending & and recieving information from the client,

Request string

When a client asks for a URI (say, by clicking on a link), a request string (along with request headers and a request body) is sent to the server (at the domain name and port listed in the URI). The request string consists of three tokens: the http method, the location of the resource on the server, and the http protocol.

Examples:

GET /dir1/sample.htm HTTP/1.0 GET /prices?type=mammal&class=retail HTTP/1.0

Selector (also referred to as the request selector )

The selector is the location of the resource on the server. Specifically, it's the slightly modified second token in the request string.

Examples (assuming the above request strings):

dir1/sample.htm prices?type=mammal&class=retail

Note that the "slight modifications" consists of decoding URL-encoded characters (note that the above examples do not include any URL-encoded characters).

In many cases, the connection between the selector and server resource it refers to is simple (such as when the selector is a filename relative to the SRE2003 data directory). In other cases, it is completely virtual (as when the resource consists of the output of a program that's run using information provided by the client).

Technical Note:

RFC2616 (the http/1.1 spec) defines an HTTP request URL as:   http_URL = "http:" "//" host [ ":" port ] [ abs_path [ "?" query ]] . Basically, the selector is the same thing as the abs_path [ "?" query ]

Server Response

When a server returns an http response to a client's http request, it can consist of three components:

1. A response line
2. A set of response headers
3. The message body

The response line (which is always included) contains status information (such as whether the request could not be satisfied, or if authorization is required. The typical response line, for a satisfied request, is:
  HTTP/1.1 200 Ok

The response headers contain meta-information on the response. These includes time and date information, caching information, server identifiers, and cookie definitions.

Response headers are divided into three categories:

General-Headers Response-Headers Entity-headers

The message body is the html-file, image, or other resource the client requested.

URI and URL

Universal Resource Indicators (URI), which is the technical term for the more commonly used Universal Resource Locators (URL), is a scheme for specifying Internet resources using a single line of printable ASCII characters. A URI should contain a protocol, domain name, port number (optional), the location of the resource, and an (optional) option list (following a ?).

Examples:

http://your.server.net/dir1/sample.htm http://pet.store.com/prices?type=mammals&class=retail

• Supports multiple hosts
• Supports maintained connections (multiple requests over a single connection)
• Supports chunking (as a transaction encoding)
• Supports on-the-fly GZIP compresssion (as a transfer encoding)
• Supports automatic generation of ETAG and Content-MD5 response headers.
• Automatic generation of conditional (3xx) returns (which can be suppressed).
• Automatic range extraction (which can be suppressed).
• A variety of optional procedures can be invoked while processing a request:
  1. Pre-transaction procedures (before any processing is done)
  2. Pre-request procedures (after reading the client's message, but before resolving the request)
  3. instance manipulation (after the contents have been determined)
• A task manager, with scheduling options, is built into SRE2003
• A simple database system, with caching features, is built into SRE2003
• Basic statistics include average response time, total number of transactions, and total bytes of throughput.
• Optional tracking of the current state of all requests
• Snipe individual connections without shutting the server down
• Optional auditing of requests, responses, errors, and other events.
• Time and size limits can be set for all requests, and can be changed on a request specific basis
• A proxy-like request cache is built into SRE2003.
• A list of unallowed IP addresses (that are immediately disconnected) can be specified
• Supports multi-part, and multi-piece, responses
• SRE2003 contains a number of functions, including functions to:
  Extract request headers by name
  Easily generate authorization and error responses
  Support basic and digest authentication
  Return a file, or the contents of a variable, to the client
  Wildcard match one string to one (or several) other strings
  Simplify the creation of, and communication between, semi-permanent daemons