![[OPUS]](gif/opus.gif)
A number of enhancements and changes to the OPUS Observation Manager (OMG) and OPUS Process Manager (PMG) have been made over the last six months. Most of these should be transparent to you, and many of the changes will be picked up when you install the new software. However, we wanted to point out that some changes will require action on your part.
However, it is possible that some users may not distinguish between a space and other non-printing characters, like the tab character, when constructing path files. This will lead to a failure to initialize the path file in such cases. To eliminate this possibility, path files should be converted to use '=' as the separator and the code changed as appropriate.
!!!! CHANGES REQUIRED !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
OPUS path files, and the OPUS.ENV file, must now use '=' as the keyword/ value separator instead of space characters.
For example, the g2f_template.path file supplied with the OPUS 1.2 build had lines like:
!---------------------------------------------------------------------------- ! ! Template path for sample pipeline. Fill in user directories to run ! the sample pipeline. ! ! Ver Date Number Author Reason !---------------------------------------------------------------------------- ! 001 06/09/97 33428 MSwam Create a UNIX version ! 002 11/06/97 35572 MSwam Add STAGE_FILE ! !---------------------------------------------------------------------------- ! !for UNIX, directory names MUST end with slash !for ALL, variable names MUST end with colon ! !---------------------------------------------------- ! STAGE_FILE OPUS_DEFINITIONS_DIR:g2f_pipeline.stage !---------------------------------------------------- STAGE_FILE <filespec for pipeline stage file> ! !---------------------------------------------------- ! OPUS_OBSERVATIONS_DIR /home/mswam/opus/sample/obs/ !---------------------------------------------------- OPUS_OBSERVATIONS_DIR <directory for OSF files> ! !---------------------------------------------------- ! gif_data /home/mswam/opus/sample/dat/ !---------------------------------------------------- gif_data <directory for input GIF files for sample pipeline> ! !---------------------------------------------------- ! fits_data /home/mswam/opus/sample/fits/ !---------------------------------------------------- fits_data <directory for output FITS files from sample pipeline>======================================================================= -----> These lines MUST CHANGE to include the '=' delimiter: =======================================================================
!---------------------------------------------------------------------------- ! ! Template path for sample pipeline. Fill in user directories to run ! the sample pipeline. ! ! Ver Date Number Author Reason !---------------------------------------------------------------------------- ! 001 06/09/97 33428 MSwam Create a UNIX version ! 002 11/06/97 35572 MSwam Add STAGE_FILE ! 003 07/24/98 37263 WMiller Add = separator & rename ! !---------------------------------------------------------------------------- ! !for UNIX, directory names MUST end with slash !for ALL, variable names MUST end with colon ! !---------------------------------------------------- ! STAGE_FILE = OPUS_DEFINITIONS_DIR:g2f_pipeline.stage !---------------------------------------------------- STAGE_FILE = <filespec for pipeline stage file> ! !---------------------------------------------------- ! OPUS_OBSERVATIONS_DIR = /home/mswam/opus/sample/obs/ !---------------------------------------------------- OPUS_OBSERVATIONS_DIR = <directory for OSF files> ! !---------------------------------------------------- ! gif_data = /home/mswam/opus/sample/dat/ !---------------------------------------------------- gif_data = <directory for input GIF files for sample pipeline> ! !---------------------------------------------------- ! fits_data = /home/mswam/opus/sample/fits/ !---------------------------------------------------- fits_data = <directory for output FITS files from sample pipeline> !
Note that the OPUS.ENV file is currently a one line file:
! This is the OPUS environment file. ! ! PSF_DIR points to the environment directory. PSF_DIR = OPUS_HOME_DIR:
!!!! CHANGES REQUIRED !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
This REQUIRES that all blackboard directories (i.e. OPUS_OBSERVATIONS_DIR and OPUS_HOME_DIR) MUST contain a subdirectory called "lock/" (UNIX) or "[.lock]" (VMS). This subdirectory must be created before the processes are started on the blackboard. Processes will stick in a "Starting" state on the PMG display if the subdirectory is missing from OPUS_HOME_DIR. Observations will stick in their initial state on the OMG if the subdirectory is missing from OPUS_OBSERVATIONS_DIR (they will never progress beyond the creation of the OSF).
Also, a new logical under VMS is required called OPUS_HOME_DIR_LOCK. It should be defined to point to the lock subdirectory under OPUS_HOME_DIR. For example,
$ define opus_home_dir disk$leo:[leo.mswam.g2f.home] $ define opus_home_dir_lock disk$leo:[leo.mswam.g2f.home.lock]
No such logical is required under UNIX, nor is any OPUS_OBSERVATIONS_DIR_LOCK value required to be defined in the path.
setenv OPUS_SORT_FILES
in either your login shell initialization file (e.g. .cshrc) or in your opus_login.csh. This feature was requested by some sample-pipeline users when trying to adapt OPUS for their own applications.
In addition these new features are now part of the OMG.
The menu of toggle buttons can be torn-off and left "up" for constant display. Also a "FILTERED" indication will appear next to the OMG timestamp if any OSF categories are being filtered out.
A new mode, "All Selected (safe)" will display a single modify window no matter how many OSFs are selected for modification. The status of the FIRST selected OSF is made available for changing. After you make A new mode, "All Selected (safe)" will display a single modify window no matter how many OSFs are selected for modification. The status of the FIRST selected OSF is made available for changing. After you make your OSF changes in the window, the changes are applied to each selected OSF that matches the status of the FIRST selected OSF. Any selected OSFs whose status does NOT match the first selected OSF are not modified, and a warning is written to the OMG log display area. This is "safe" in that if you select an OSF by mistake, it will not be modified if its status is different from the other OSFs selected.
A new mode, "All Select (NO CHECKS)" works much like "All Selected (safe)", except that the OSF changes requested will be applied to all selected OSFs, no matter what their current status is. When using this mode just make sure that you select only the OSFs that you really want to change.
The NO CHECKS feature can be disabled by putting this entry in the omg.dat file:
*AllSelectedNoChecks.sensitive: 0
The keywords appearing in the omg_view.dat files have changed significantly to support the three possible view modes.
Instead of the 2-level search hierarchy provided in the "Primary" and "Alternative" entries in omg_view.dat, there is now support for N-level search hierarchies using "Search1", "Search2", ..."Search N". You can also search several locations at the same search level.
A selection box will be displayed when multiple qualifying files are found, to allow you to select which files to view. The selection window stays "up" until you dismiss it, allowing multiple selections to be made.
Also a new sub-field syntax is available when using the special "<root>" entry in the omg_view.dat file. This is needed for the OTFC (on-the-fly calibration) pipeline, but might find other uses as well. The syntax is <root[x:y]> to use only a subset of characters from the OSF dataset name in creating filemasks and directory specs, where "x" and "y" are integers. (e.g. <root[1:10]> uses the first ten characters of the OSF dataset name).
Because of the replacement of the "Primary" and "Alternative" keywords, and the new "trl", "ascii", and "fits" entries, the old omg_view.dat file WILL NO LONGER WORK. Use the new one as a template if you need an override version.
The source of the information displayed in the new popup menus is the pipeline stage file for the path being monitored. The full title comes from the entries for each stage of the form:
STAGEnn.TITLE = tt
where nn is the stage number and tt is the two character stage title. Four new resources were added for the status values and their descriptions:
STAGEnn.CSTATUS.v = dddddddd STAGEnn.PSTATUS.v = dddddddd STAGEnn.TSTATUS.v = dddddddd STAGEnn.NSTATUS.v = dddddddd
where nn is the stage number, v is the stage value, and dddddddd is the description of that value. For example,
STAGE03.PSTATUS.P = 'Processing observation'
Note that descriptions containing spaces should be quoted with SINGLE QUOTES (as in all OPUS resource files).
The four different resources correspond to the four states that an OSF can be in : C for complete, P for pending, T for trouble, and N for none. Each status value should be defined using one of the four resources depending on how that value should be interpreted for the purposes of tallying the number of OSF's in each state by the OMG. Categorization of stage values must be consistent across all stages (in other words, if 'P' is defined using PSTATUS for one stage, it should not be defined using one of the other resources for other stage definitions).
Each time the OMG refreshes the display, it counts the number of OSF's in each state (except none) by first looking for trouble values, then pending values, and finally complete values in each OSF. The search order is important because once a match is found, that total is incremented and the next OSF is examined. That means the precedence of categories is trouble, holding (ie., HALTed OSFs), pending then complete, so an OSF will be counted towards the total for a category with the highest precedence when it contains status values of several categories.
The user still has the capability of modifying an OSF or range of OSF's using the "Modify Selected" option in the "Manage" menu. These dialogs work the same as before except that the choices available for each stage are determined by the pipeline stage file entries-- a common list of status values for each column no longer exists.
It is important to note that the OMG is a Motif based application, so the graphical user interface conforms to the standards dictated by Motif. This has the following consequences for using these new features:
*PathDialog*ManageButton.set : 0
The default can be changed to 1 (i.e., initially checked) in omg.dat, or it can be overridden on the command line:
%omg -xrm "*PathDialog*ManageButton.set : 1"
Usage: %osf_test -p pathfile
-f filename -t data_id -n dcf_number
-c column -s status -m command -x time
[-pr time status dataset data_id dcf_num
command COLUMN_NAMES]
The pathfile argument is required. All others are optional. Any OSF selection options omitted are replaced with default search characters that will match all OSFs. If the print options (-pr) are omitted, the entire OSF is printed, otherwise the selected field is printed.
Examples:
% osf_test -p otfcal -f u2440101t
Finds OSF under otfcal blackboard matching dataset name "u2440101t".
Prints the entire OSF to the screen.
% osf_test -p otfcal -c CA -s e -pr dataset data_id
Finds OSFs under otfcal blackboard with value 'e' in the CA column.
Prints the dataset and data_id fields of these OSFs, separated by a space,
to the screen.
See the help page for more details.
Also, the osf_update tool no longer requires entry of a DCF_number and data_type. These fields will be left out of the OSF search if they are not supplied by the user.
For example, in the resource file for an external poller where the goal is to delete the trigger file whenever XPOLL receives an exit status of 5, but rename the trigger file to *_BAD on exit status 3, the relevant portion of the resource file might look like:
FILE_RANK = 1 FILE_DIRECTORY1 = OPUS_DEFINITIONS_DIR: FILE_OBJECT1 = *.DAT FILE_PROCESSING = _PROC FILE_ERROR = _BAD FILE_ACTION = '%rm ^f*' ! ^f is replaced with current file name FILE_ACTION_OK = 0 ! rm returns 0 when successful XPOLL_STATE.03 = FILE_ERROR XPOLL_STATE.05 = FILE_ACTION
The value of FILE_ACTION can be a DCL command (preceded by a $), a DCL command procedure (preceded by a @), a Unix command, or a binary executable (the executable must NOT indicate a path; instead, it should be in the path as defined by the Unix PATH environment variable or by the VAXC$PATH logical). If there are spaces in the command string of FILE_ACTION, the entire string must be enclosed in single quotes.
(VMS) - COMMAND can contain a DCL command procedure (preceded by the @ character and including a full path specification), a single DCL command (preceded by the $ character), or a VMS image (only the file name- no path specification; the image must be in the path as specified by the logical VAXC$PATH).
Examples-
(DCL command procedure) COMMAND = @OPUS_DEFINITIONS_DIR:WCS_NIC.COM
(DCL command) COMMAND = '$dir disk$data:[my_data]'
(VMS Image) COMMAND = 'MY_PROGRAM.EXE ARG1 ARG2 ARG3'
(UNIX) - COMMAND can contain any shell command, shell script or binary executable. Shell scripts and executables should NOT include a path- the file must be in the login shell's PATH.
Examples-
COMMAND = 'ls -l /data/my_data'
COMMAND = my_script
COMMAND = 'my_program arg1 arg2 arg3'
*** NOTE: if the value of COMMAND contains spaces, the entire string must be enclosed in single quotes.
This file is used to restrict the number of copies of certain processes that the PMG is allowed to start up. Certain requirements dictate that only one copy of a process be allowed to run, so this file is a flexible method for enforcing that behavior through the PMG.
The syntax of entries is:
PROCESS.PATH.NODE = NUM_COPIES (case insensitive)
where PROCESS = name of pipeline process (same as resource filename)
PATH = name of the path in which the restriction occurs
NODE = name of the machine on which the restriction occurs
NUM_COPIES = number of copies of the process allowed
The "*" wildcard entry is allowed for the PATH and NODE components.
Some examples:
gifin.g2f.orchid = 2 ! maximum of 2 copies of gifin running in path g2f
! on node orchid
dp_new.*.* = 1 ! only one copy of DP_NEW across all paths and nodes
PI_MAIN.*.AREA51 = 1 ! only one copy of PI_MAIN on area51 at a time