Hyrax Data Server Installation and Configuration Guide: Chapter 10—Hyrax Troubleshooting

Hyrax Data Server Installation and Configuration Guide

10. Hyrax Troubleshooting

10.1. Hyrax - Running bescmdln - OPeNDAP Documentation

2019-06-21 :numbered: :toc:

10.1.1. Running bescmdln - Basic Commands

First we will issue some simple commands to make sure that the client is talking to the server. First, start the command-line client:

% bescmdln -h localhost -p 10022

The -h option specifies the machine on which the BES is running. In this case, it’s your local machine. The -p option specifies the port the BES is running on. The default, set in the BES configuration file, is 10022. If you changed this, or if you started the server with the -p option, then you need to use that port number here.

If you just use these options then you will start using the command line version of the client. There are other options, but we’ll start here. From here you should get a prompt. Let’s try a simple command (remember to terminate each command with a semicolon):

BESClient> show status;

You should get a response showing the status of the server:

Listener boot time: MDT Thu Jun  9 14:12:22 2005

Try another one:

BESClient > show help;

This one should show both the BES core commands, DAP commands, and your help information.

If you have installed a data handler, let’s take a look at your data. By executing this request you should see the root node of your data directory.

BESClient > show catalog;

If you can’t see your data, then make sure that the RootDirectory parameters in the BES Configuration file are correct.

BESClient > exit

This one will exit out of interactive mode.

10.1.2. Commands for Hyrax Testing
Poke around in the RootDirectory to see what’s actually visible to the BES

Show the Root Catalog

show catalog;

…will show the contents of "pathname"

For example, show catalog for "/data/nc"; will show all the stuff in the /data/nc directory.

show catalog for "pathname";
Get the BES to return a DAP response object

You need three commands to do this:

Bind the dataset to a container in a catalog

set container in catalog values c,/data/nc/feb.nc;

Make a definition so you can access that container

define d as c;

Request a particular response

get ddx for d;
10.1.3. Command line options

Other command line options available to the bescmdln program:

-u specifies the name of a Unix socket for connecting to the server.
-h specifies the name of a host for TCP/IP connection
-p specifies the port where the server is listening for TCP/IP connection.
-x makes the client execute a command and exit. This flag requires the -f flag.
-f sets the target file name for the return stream from the server.
-i sets the target file name for a sequence of input commands.
-t sets the timeout in seconds and is optional.
-d "cerr|<filename>,<context>" sets the client session for debugging and is optional.
-v forces the client to show the version and exit

Connection Flags: -u or -p -h are required to connect to a server and specify either a Unix socket or a TCP/IP socket.

Input/Output Flags: you can specify that the input is from the command line with the -x flag or that the input must be read from a file with the -i flag. If you specify either -x or -i you must specify the name of a file for the output stream of the server with the -f flag. If neither the -x nor the -i flags are specified then the client goes into interactive mode. To exit out of interactive mode just type 'exit' (without the quotes) at the BESClient> prompt.

For debugging information either specify cerr to have debugging information dump to standard error, or the name of a file. The context option is a comma separated list of debugging context (component debugging). Specify all to get debugging from all components. = How to Debug the BES - OPeNDAP Documentation

10.1.4. Tricks
  • Set the beslistener to run in single, not multiprocess, mode. Do this in the bes.conf file (use the BES.ProcessManagerMethod parameter).
  • Build the bes using developer mode (so it won’t need to be root, among other things). Do this with ./configure --enable-developer
10.1.5. Use the BESDEBUG Macro

Use the macro BESDEBUG defined in BESDebug.h.

Set the macro’s 'context' as "bes" (nominally, or you can make up whatever you want) and then use the "cerr << "text: " << var << endl" style output except that you should leave off the initial "cerr <<" and start with the first argument of the stuff to be output - the marco will take care of getting the output sink and using the output operator.

Example:

#include <BESDebug.h>
...
BESDEBUG( "h4", "File Id:" << _file_id << endl);

Notes:

  1. You’ll need to include the BES_DAP_LIBS when you link an executable or a libtool library and you’ll need BES_CPPFLAGS when you compile (for libdap code)
  2. The trailing semicolon is not needed but including it makes automatic code indent software (eclipse, emacs, …) much happier.
10.1.6. Start the BES with Debuging on

Use the -d option to besctl and give -d one argument, a string, with two parts: "<output sink>,<context>". For example,

besctl start -d "cerr,bes"

would start up beslistener with the bes debug context active and write all the debugging info to cerr, which is standard error. You can provide several contexts. For example, you could say

besctl start -d "./bes.dbg,bes,nc"

This will send debug statements to the file ./bes.dbg for the context bes and nc (netcdf_handler). You can also specify the context all, that will send debugging statements for all context.

The BES has debug statements for bes, ppt and server. Each of the modules that you install will also have debug context. And, you can create your own context when writing your own module. In your Module class you would register your context, so as to be available with the help command, by using the following code:

BESDebug::Register( "<context>" ) ;

Where context is the string that will be used for your module’s debug context. For example, nc for the netcdf_handler.

To see what debug context is available, when you start the BES using besctl, use the help option:

besctl help
BES install directory: /Users/westp/opendap/opendap
BES configuration file: /Users/westp/opendap/opendap/etc/bes/bes.conf
Developer Mode: not testing if BES is run by root
/Users/westp/opendap/opendap/bin/beslistener: -i <INSTALL_DIR> -c <CONFIG> -d <STREAM> 
-h -p <PORT> -s -u <UNIX_SOCKET> -v
-i back-end server installation directory
-c use back-end server configuration file CONFIG
-d set debugging to cerr or <filename>
-h show this help screen and exit
-p set port to PORT
-s specifies a secure server using SLL authentication
-u set unix socket to UNIX_SOCKET
-v echos version and exit
Debug help:
Set on the command line with -d "file_name|cerr,[-]context1,[-]context2,...,[-]contextn"
  context with dash (-) in front will be turned off
Possible context:
  ascii: off
  bes: off
  dap: off
  ff: off
  h4: off
  h5: off
  nc: off
  ppt: off
  server: off
  usage: off
  www: off
USAGE: besctl (help|start|stop|restart|status) [options]
where [options] are passed to besdaemon; see besdaemon -h
10.1.7. Send Commands to the BES

Now run some commands using bescmdln. You should see debugging being output to either cerr, or the file you specified when you started the BES. Here’s an example:

BESClient> set context dap_format to dap2;
BESClient> set container in catalog values c,/data/nc/fnoc1.nc;
BESClient> define d as c;
BESClient> get das for d;
Attributes {
    u {
        String units "meter per second";
        String long_name "Vector wind eastward component";

10.2. BES Client Commands - Introduction

These are the commands that the BES supports. Documented here are the XML versions of the commands that are typed into the bescmdln client. All of these have a non-XML version as well that might be easier to type as the command line. However, if you’re making command files, these are often the easiest to use because the SQL-like syntax of the 'text' commands can be confusing.

If you want to find documentation on the XML document that the BES expects to receive, look at the BES XML Commands documentation. There you’ll see that the commands listed here are generally sent as given to the bescmdln client but embedded in other XML that provides the BES with information such as a request ID and other bookkeeping information.

10.2.1. Current Core Commands Available With BES

NB: The BES supports both XML and a SQL-like syntax. Here we attempt to document both.

  • <showHelp /> or show help;
  • shows this help
  • <showVersion /> or show version;
  • shows the version of OPeNDAP and each data type served by this server
  • <showProcess /> or show process;
  • shows the process number of this application. This command is only available in developer mode.
  • <showConfig /> or show config;
  • shows all key/value pairs defined in the bes configuration file. This command is only available in developer mode.
  • <showStatus /> or show status;
  • shows the status of the server
  • <showContainers /> or show containers;
  • shows all containers currently defined
  • <showDefinitions /> or show definitions;
  • shows all definitions currently defined
  • <showContext /> or show context;
  • shows all context name/value pairs set in the BES
  • <setContainer name="container_name" space="store_name" type="data_type">real_name</setContainer> or set container in catalog values c,/data/nc/fnoc1.nc;
  • defines a symbolic name representing a data container, usually a file, to be used by definitions, described below
  • the space property is the name of the container storage and is optional. Defaults to default volatile storage. Examples might include database storage, volatile storage based on catalog information.
  • real_name is the full qualified location of the data container, for example the full path to a data file.
  • data_type is the type of data that is in the dataset. For netcdf files it is nc. For some container storage the data type is optional, determined by the container storage.
  • <setContext name="context_name">context_value</setContext>
  • set the given context with the given value. No default context are used in the BES
  • <define …>
 <define name="definition_name" space="store_name">
     <container name="container_name">
         <constraint>legal_constraint</constraint>
         <attributes>attribute_list</attributes>
     </container>
     <aggregate handler="someHandler" cmd="someCommand" />
 </define>
  • creates a definition using one or more containers, constraints for each of the containers, attributes to be retrieved from each container, and an aggregation. Constraints, attributes, and aggregation are all optional.
  • There can be more than one container element
  • space is the name of the definition storage. Defaults to volatile storage. Examples might include database storage.
  • <deleteContainer name="container_name" space="store_name" />
  • deletes the specified container from the specified container storage (defaults to volatile storage).
  • <deleteContainers space="store_name" />
  • deletes all of the currently defined containers from the specified container storage (defaults to volatile storage).
  • <deleteDefinition name="definition_name" space="store_name" />
  • deletes the specified definition from the specified container storage (defaults to volatile storage).
  • <deleteDefinitions space="store_name" />
  • deletes all of the currently defined definitions from the specified container storage (defaults to volatile storage).
10.2.2. Added commands for dap enabled servers

If you are serving up OPeNDAP data responses (DAS, DDS, DataDDS) then you will have loaded the dap commands in your configuration file. Here are the available commands in the dap module.

  • <showCatalog node="node_name" /> or show catalog; or show catalog for [node_name];
  • Shows catalog information, including contents if a container. If node is not specified then the root node information is returned. If node is specified then that nodes information is returned.
  • <showInfo node="node_name" />
  • Shows catalog information for just that node, the root node if no node is specified. If the node is a container the contents are not displayed.
  • <get type="das | dds | dods | ddx | dataddx | ascii" definition="def_name" returnAs="returnAs" />
  • dds: request the data descriptor structure. Returned as text.
  • das: request the data attributes. Returned as text.
  • dods: request for the data stream, this output is an octet binary stream made up of two parts and similar to a multipart MIME document (but not a real MPM doc). The first part is the DDS that describes the contents of this response; the separator is the text Data:; and the data make up the third part. The data are represented using XDR-encoded binary values. There is a a one-to-one mapping between variables, name and types in the first part and the binary values in the second part. A library such as libdap can easily decode this response.
  • ddx: request the data attributes and data descriptor structure returned as an xml document
  • dataddx: This is the 'DAP4' counterpart to the dods response, just as the ddx is the DAP4 counterpart to the das and ddsresponses from DAP2. The dataddx response is a true multipart MIME document with the first part a text/xml section that holds the ddx that describes the data in the response and the second part an application/octet-stream section that holds the matching XDR-encoded values.
  • ascii: request the data stream (i.e., dods) and then pass that through a formatter to generate an ASCII representation of the data and return it in a text/plain MIME document.
  • <setContext name="errors">dap2 | xml | html | txt</setContext>
  • set the context 'errors' to dap2 in order to have all exceptions and errors formatted as dap2 error messages in the response.
  • <setContext name="dap_format">dap2</setContext>
  • set the context 'dap_format' to dap2 in order to suppress the addition of an additional structure to the DDS/DDX whose elements are the containers named in the setContainer element.
10.2.3. Using the bescmdln client to test the BES

Here are some tricks/command sequences that are useful when you need to test the BES without using a web browser. This section assumes that the DAP commands have been loaded into the BES. In this section, the examples use the older syntax because it’s a bit more amenable to a command line environment. With the XML syntax, multiple commands cab be grouped together and sent to the BES in one shot.

Find the versions of all the installed and running modules

show version;

Show the status os the BES

show status; Poke around in the RootDirectory to see what’s actually visible to the

BES

show catalog; will show you the root catalog; show catalog for "pathname"; will show the contents of "pathname"(e.g., show catalog for "/data/nc"; will show all the stuff in the /data/nc directory).

Get the BES to return a DAP response object

You need three commands to do this:

bind the dataset to a container in a catalog

set container in catalog values c,/data/nc/feb.nc;

make a definition so you can access that container

define d as c;

a definition with a constraint

define d as c with c.constraint="lat";

request a particular response

get ddx for d;

**Note that there is a set container command but that does not use the default catalog while the command here explicitly binds the dataset to a container in the default catalog (which is called catalog). This pathname is rooted in the directory set using the BES.Catalog.catalog.RootDirectory configuration parameter in the bes.conf file. The 'plain' set container …command uses pathnames rooted in the directory name by the BES.Data.RootDirectory parameter, which is often null for Hyrax installations.

Last Updated: Sep 24, 2019 at 3:54 PM EDT