--- /dev/null
+%
+%% Copyright (c) Members of the EGEE Collaboration. 2004-2010.
+%% See http://www.eu-egee.org/partners for details on the copyright holders.
+%%
+%% Licensed under the Apache License, Version 2.0 (the "License");
+%% you may not use this file except in compliance with the License.
+%% You may obtain a copy of the License at
+%%
+%% http://www.apache.org/licenses/LICENSE-2.0
+%%
+%% Unless required by applicable law or agreed to in writing, software
+%% distributed under the License is distributed on an "AS IS" BASIS,
+%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+%% See the License for the specific language governing permissions and
+%% limitations under the License.
+%
+% System tests
+
+\section{Nagios Probe}
+\label{s:nagios}
+
+There is a Nagios probe to check the service status of an \LB server node. It is distributed from the EMI repository and the name of the package is \texttt{emi-lb-nagios}.
+
+\subsection{Tests Performed}
+Before starting the actual test the probe checks for existence and validity of a proxy certifiacate, and for availability of commands (essential system commands, various \LB Client commands and grid proxy manipulation commands).
+
+The following tests are performed by the probe. Various tests check the working status of various processes running on the \LB server node:
+
+\begin{enumerate}
+\item Register job
+ \begin{itemize}
+ \item \LB server (\texttt{glite-lb-bkserverd})
+ \end{itemize}
+\item Register to receive notifications
+ \begin{itemize}
+ \item \LB server (\texttt{glite-lb-bkserverd})
+ \end{itemize}
+\item Log events resulting in state \emph{cleared}
+ \begin{itemize}
+ \item \LB logger (\texttt{glite-lb-logd})
+ \end{itemize}
+\item Check job state
+ \begin{itemize}
+ \item \LB server (\texttt{glite-lb-bkserverd})
+ \item Interlogger (\texttt{glite-lb-interlogd})
+ \end{itemize}
+\item Receive notifications
+ \begin{itemize}
+ \item Notification interlogger (\texttt{glite-lb-notif-interlogd})
+ \end{itemize}
+\end{enumerate}
+
+The test also tries to drop the test notification and purge the test job to clean up after itself. However, purging the job won't probably be allowed by the \LB server's policy and the test job will remain registered on the \LB server until removed by a regular purge.
+
+\subsection{Return Values}
+Return values follow the Nagios pattern:
+
+\begin{description}
+\item[0] The service is running normally
+\item[1] The service is running but there were warnings
+\item[2] The service status is critical
+\item[3] The service status is unknown, probe could not run
+\end{description}
+
+\subsection{Console Output}
+Text output indicates the results of the probe and gives a more detailed description of failure causes.
+
+The probe can return one of the following:
+
+\begin{tabularx}{\textwidth}{|c|p{5cm}|X|}
+\hline
+\multirow{3}{*}{WARNING} & \emph{Unexpected version output} & The server responded to a query for server version over the WS interface, but the format of the response did not match the expected pattern. \\
+& \emph{Unexpected state of test job} & The state of the test job did not remain unchanged (\texttt{Submitted}) but neither did it reach status \texttt{Cleared} in the alotted time. All deamons seem to work but the processing is slow.\\
+& \emph{Could not drop notification} & The owner should be able to drop their own notification. Failure to do so is unexpected but does not mean that the service is not functioning. \\
+\hline
+\multirow{6}{*}{DOWN} & \emph{Unable to Get Server Version} & The server did not respond to a query for server version over the WS interface. It is probably not running or is unaccessible. \\
+& \emph{Job Registration Failed Locally} & The probe was unable to perform the local side of job registration. This should be rare. \\
+& \emph{\LB Server Not Running} & The probe was unable to register a test job or a test notification. It is probably not running or is unaccessible. \\
+& \emph{Event Delivery Chain (Logger/Interlogger) Not Running} & The server process is running but events are not being delivered by \LB's local logger/interlogger. Check the Logger and the Interlogger. \\
+& \emph{Notification Interlogger Not Running} & Events are being delivered correctly and server responds properly to status queries, but it its not delivering notification messages. The notification interlogger is probably not running.\\
+\hline
+\multirow{5}{*}{UNKNOWN} & \emph{Probe timed out} & The probe was unable to finish before the alotted time. Consider increasing the timeout with \texttt{-t}. The minimum reasonable value is 10\,s. \\
+& \emph{No server specified} & Server address was not specified when running the probe. Give one with \texttt{-H} \\
+& \emph{Probe could not write temporary files} & The temporary directory was not writable. Check the default location or specify a new one with \texttt{-T}.\\
+& \emph{Some commands are not available} & Probe could not run. Some of the required commands are not present on the system. Run probe from command line with \texttt{-v[vv]} and check output. \\
+& \emph{No Credentials} & No proxy certificate was found. Probe could not run. \\
+& \emph{Credentials Expired} & A proxy certificate was found, but expired. Probe could not run. \\
+\hline
+\end{tabularx}
+
+\subsection{Running the Probe}
+
+\subsubsection{Command Line Arguments}
+The probe recognizes the following command line arguments:
+
+\begin{tabularx}{\textwidth}{l l X}
+ \texttt{-h} & \texttt{-{}-help} & Print out simple console help \\
+ \texttt{-v[vv]} & \texttt{-{}-verbose} & Set verbosity level (\texttt{-{}-verbose} denotes a single \texttt{v}). \\
+ \texttt{-H} & \texttt{-{}-hostname} & \LB node address. Environmental variable \texttt{GLITE\_WMS\_QUERY\_SERVER} used if unspecified. \\
+ \texttt{-p} & \texttt{-{}-port} & \LB server port. Other port port numbers (logger, WS interface) are derrived from it. Environmental variable \texttt{GLITE\_WMS\_QUERY\_SERVER} or default port \texttt{9000} used if unspecified. \\
+ \texttt{-t} & \texttt{-{}-timeout} & Timeout in seconds. The minimum reasonable timeout is approx. 10\,s. There is no default, except the internal waiting cycle for notifications, which will time out after approx. 20\,s.\footnote{The probe adjusts the internal waiting cycle to spend a maximum of $\frac{3}{4}$ of the specified timeout interval while waiting for notifications to deliver. It will finish correctly before timing out if undelivered notifications are the only problem.} \\
+ \texttt{-T} & \texttt{-{}-tmpdir} & Directory to store temporary files. By default the probe uses \texttt{/var/lib/grid-monitoring/emi.lb} and falls back to \texttt{/tmp} if the former does not exist or is not writable. \\
+\end{tabularx}
+
+
+\subsubsection{Sample Nagios Service Definition}
+Simple definition to be included in \texttt{/etc/nagios/commands.cfg}:
+
+\begin{verbatim}
+define command{
+ command_name check-lb-server
+ command_line /usr/libexec/grid-monitoring/probes/emi.lb/LB-probe \$HOSTADDRESS$
+ }
+\end{verbatim}
+
+
git://scientific.zcu.cz / jra1mw.git / commitdiff