From e6d72860498ef5e72528ea1df9e3a880c11aef86 Mon Sep 17 00:00:00 2001 From: Miroslav Ruda Date: Mon, 7 Jul 2008 19:43:25 +0000 Subject: [PATCH] User guide - use cases, notification --- org.glite.lb.doc/src/LBUG-Introduction.tex | 38 ++++++++++-- org.glite.lb.doc/src/LBUG-Tools.tex | 29 ++++++--- org.glite.lb.doc/src/LBUG.tex | 4 +- org.glite.lb.doc/src/change_acl.tex | 4 +- org.glite.lb.doc/src/log_usertag.tex | 23 +++----- org.glite.lb.doc/src/logevent.tex | 41 ++++--------- org.glite.lb.doc/src/notify.tex | 94 ++++++++++++++++++++++++++++++ 7 files changed, 172 insertions(+), 61 deletions(-) diff --git a/org.glite.lb.doc/src/LBUG-Introduction.tex b/org.glite.lb.doc/src/LBUG-Introduction.tex index ba47960..d6e439a 100644 --- a/org.glite.lb.doc/src/LBUG-Introduction.tex +++ b/org.glite.lb.doc/src/LBUG-Introduction.tex @@ -579,8 +579,10 @@ the \LB server. However, the user may also store information into the \LB explicitly by logging user events---\emph{tags} (or annotations) of the form ``name = value''. -Authorization information is also manipulated in this way, see -\LB User's Guide for details. +Authorization information is also manipulated in this way. + +Description of tools for event submission and ACL manipulation +can be found in Section \ref{s:lb-tools}. \subsubsection{Retrieving information} @@ -611,11 +613,34 @@ The query language supports common comparison operators, and it allows two-level nesting of conditions (logically \emph{and}-ed and \emph{or}-ed). Our experience shows that it is sufficiently strong to cover most user requirements while being simple enough to keep the query cost reasonable. -Complete reference of the query language can be found in~\LB User's Guide. +Complete reference of the query language can be found in~\LB Developer's Guide +\cite{lbdg}. + +\subsubsection{Notifications} +The other mode of user interactions are the \LB notifications. + +\LB infrastructure enables its users to be notified when something interesting happens on a \LB server +(typically job status change). It enables the user not to poll \LB server periodically to find changed +information, but confortably wait for server to inform the user. + +User registers for notifications via notification client \verb'glite-lb-notify', described in Section +\ref{s:lb-notify}. +Conditions under which the notification is sent can be specified. For example - job XY reaches status +DONE. In \LBold, one or more JOBID's are required in the condition and only a single occurence +of a specific attribute is allowed among ANDed conditions. In \LBnew, more complex conditions are allowed. + +The registration is then delivered to \LB server where it is stored. +At the same time, server generates a unique notification ID for the registration and returns it to the +user. + +The registration exists only for limited amount of time. The validity is returned by \LB server together with +the notification ID when registering. During this period user can attach to server and receive notifications, +change conditions which triger notification, prolong validity of the registration, or remove the registration +from \LB server. -\ludek{%notifikace -The other mode of user interactions are the \LB notifications -(Sect.~\ref{retrieve}).} +While the registration is valid, user is able repeatably connect to \LB server from different places in the +network and continue receiving notifications associated with given notification ID. Notifications generated +during the time when user is connected are stored and waiting when user reconnects. \subsubsection{Caveats} \LB is designed to perform well in the unreliable distributed Grid @@ -870,6 +895,7 @@ This functionality is being replaced by the direct download of all the relevant data from the Job Provenance. \subsubsection{Computing Element reputability rank} +\label{s:ce-rank} Production operation of the EGEE middleware showed that misbehaving computing elements may have significant impact on the overall Grid performance. diff --git a/org.glite.lb.doc/src/LBUG-Tools.tex b/org.glite.lb.doc/src/LBUG-Tools.tex index 5d474d5..2aa42e2 100644 --- a/org.glite.lb.doc/src/LBUG-Tools.tex +++ b/org.glite.lb.doc/src/LBUG-Tools.tex @@ -1,15 +1,30 @@ \section{Tools description} - -\TODO{glite-wms-*} - -\TODO{RPM, environment} +\label{s:lb-tools} In this section we give a description of the tools that are installed -together with the \verb'glite-lb-client'. These are the only \LB\ tools that a -regular grid user may need to use. +together with the \verb'glite-lb-client' RPM. These tools are \LB-specific. +Besides these tools, \verb'glite-wms-job-status' and \verb'glite-wms-job-logging-info' available +in \verb'glite-wms-ui-cli-python' RPM, provide more information about user jobs and corresponding events. + +Behaviour of \verb'glite-lb-*' commands can be changed by setting the following enviroment variables: +\begin{tabularx}{\textwidth}{lX} +GLITE\_WMS\_LOG\_DESTINATION & address of \verb'glite-lb-logd' daemon, in form \verb'hostname:port'\\ +GLITE\_WMS\_LOG\_TIMEOUT & timeout (in seconds) for asynchronous logging\\ +GLITE\_WMS\_LOG\_SYNC\_TIMEOUT & timeout (in seconds) for synchronous logging\\ +GLITE\_WMS\_NOTIF\_SERVER& address of \verb'glite-lb-bkserver' daemon, in form \verb'hostname:port'\\ +GLITE\_WMS\_NOTIF\_TIMEOUT& timeout (in seconds) for notification registration\\ +X509\_USER\_CERT and X509\_USER\_KEY & location of user credentials\\ +\end{tabularx} +For backward compatibility, all \verb'GLITE\_WMS\_*' variables can be prefixed by \verb'EDG\_WL\_' instead. \input{logevent} \input{notify} -\TODO{Maybe add other tools: lite-lb-job-log, glite-lb-job-status, ...? Zlehka.} +\subsection{Other useful tools} +\label{glite-lb-other} + +For debugging purposes, low-level commands for getting \LB job status and job related events are provided in +\verb'examples' directory (\verb'glite-lb-job-status' and \verb'glite-lb-job-log'). The same directory +contains also debugging commands for getting of all user jobs (\verb'glite-lb-user\_jobs') and +CE-reputability rank (see Section \ref{s:ce-rank}, \verb'glite-lb-stats'). diff --git a/org.glite.lb.doc/src/LBUG.tex b/org.glite.lb.doc/src/LBUG.tex index 2bb1bcb..c003ad7 100644 --- a/org.glite.lb.doc/src/LBUG.tex +++ b/org.glite.lb.doc/src/LBUG.tex @@ -30,8 +30,8 @@ use cases. } \newpage \input{LBUG-Tools} -\newpage -\input{LBUG-UseCases} +%\newpage +%\input{LBUG-UseCases} \appendix \newpage diff --git a/org.glite.lb.doc/src/change_acl.tex b/org.glite.lb.doc/src/change_acl.tex index 760b80b..be4bd7a 100644 --- a/org.glite.lb.doc/src/change_acl.tex +++ b/org.glite.lb.doc/src/change_acl.tex @@ -1,5 +1,5 @@ -\subsection{Changing Job Access Control List} -\label{change_acl} +\subsubsection{Example: Changing Job Access Control List} +\label{e:change-acl} \TODO{Review if it is still correct - Dan?} In order to change ACL for a job a special event \verb'ChangeACL' is used. This diff --git a/org.glite.lb.doc/src/log_usertag.tex b/org.glite.lb.doc/src/log_usertag.tex index 588cf60..4333014 100644 --- a/org.glite.lb.doc/src/log_usertag.tex +++ b/org.glite.lb.doc/src/log_usertag.tex @@ -1,9 +1,12 @@ -\subsection{Logging a UserTag event} -\label{log_usertag} +\subsubsection{Example: Logging a UserTag event} +\label{e:usertag} User tag is an arbitrary ``name=value'' pair with which the user can assign additional information to a job. Further on, LB can be queried based also on -values of user tags. % (see Sect.~\ref{tag-query}). +values of user tags. \LB treats all values as strings only, semantic meaning +is left to user application. For internal reasons, all tag names are stored +in lower-case format. Support for case-sensitivenes is planned in future +versions of \LB. In order to add user tag for a job a special event \verb'UserTag' is used. This event can be logged by the job owner using the glite-lb-logevent command (see @@ -24,11 +27,8 @@ glite-lb-logevent -s Application -e UserTag \ where \begin{tabularx}{\textwidth}{lX} -\verb'' & specifies the job to change \\ -\verb'' & specifies the sequence code returned by previous call - of verb 'glite-lb-logevent' \\ -\verb'' & specifies the name of user tag \\ -\verb'' & specifies the value of user tag \\ +\texttt{} & specifies the name of user tag \\ +\texttt{} & specifies the value of user tag \\ \end{tabularx} The user application is always executed from within a JobWrapper script (part @@ -44,9 +44,7 @@ more subsequent calls, the user is responsible for capturing an updated sequence code from the stdout of \verb'glite-lb-logevent' and using it in subsequent calls. The \LB\ design requires the sequence codes in order to be able to sort events correctly while not relying on strictly synchronized -clocks. - -\TODO{What are the possible values/types of the tag? String/Integer/Float/Double?} +clocks. The example bellow is a job consisting of 100 phases. A user tag phase is used to log the phase currently being executed. Subsequently, the user may monitor @@ -66,6 +64,3 @@ execution of the job phases as a part of the job status returned by \LB. # do the actual computation here done \end{verbatim} - -\TODO{Add more examples.} - diff --git a/org.glite.lb.doc/src/logevent.tex b/org.glite.lb.doc/src/logevent.tex index 28d3dcc..7e0040a 100644 --- a/org.glite.lb.doc/src/logevent.tex +++ b/org.glite.lb.doc/src/logevent.tex @@ -8,7 +8,7 @@ should not be used for common event logging because of possibility of confusing \LB\ server job state automaton. The command \verb'glite-lb-logevent' is a complex logging tool and the complete -list of parameters can be obtained using the \verb'-h' option. However, +list of parameters can be obtained using the \verb'-h' option. However, the only legal user usage is for logging \verb'UserTag' and \verb'ChangeACL' events. The following description is therefore concentrating only on options dealing with these two events. @@ -30,40 +30,21 @@ where \texttt{ -e -{}-event} & select event type (see -e help)\\ \end{tabularx} -\medskip +%\medskip Each event specified after \verb'-e' option has different sub-options enabling to set event specific values. -Sub-options usable with \verb'UserTag' event are: - - -\begin{tabularx}{\textwidth}{lX} -\texttt{ -{}-name} & tag name\\ -\texttt{ -{}-value} & tag value\\ -\end{tabularx} - -\medskip - -Sub-options usable with \verb'ChangeACL' event are: - -\begin{tabularx}{\textwidth}{lX} -\texttt{ -{}-operation} & operation requested to perform with ACL (add, remove)\\ -\texttt{ -{}-permission} & ACL permission to change (currently only READ)\\ -\texttt{ -{}-permission\_type} & type of permission requested (0 = 'allow', 1 = 'deny')\\ -\texttt{ -{}-user\_id} & DN or VOMS parameter (in format VO:group)\\ -\texttt{ -{}-user\_id\_type} & type of information given in \verb'user_id' (DN or VOMS)\\ -\end{tabularx} - -\bigskip - -To be able to use this command several environmental variables must be set -properly. User must specify where the event should be sent. This is address and -port of glite-lb-logd daemon. It is done using environmental variable -\verb'EDG_WL_LOG_DESTINATION' in a form \verb'address:port'. +Address of local logger, where event is sent, must be specified by environment +variable \verb'GLITE_WMS_LOG_DESTINATION' in a form \verb'address:port'. Because user is allowed to change ACL or add user tags only for her jobs, paths to valid X509 user credentials has to be set to authorise her. This is done -using two environmental variables \verb'EDG_WL_X509_KEY' and -\verb'EDG_WL_X509_CERT' in a form \verb'path_to_cred'. +using standard X509 environment variables \verb'X509_USER_KEY' and +\verb'X509_USER_CERT'. + +For additional information see also manual page GLITE-LB-LOGEVENT(1). + +\input log_usertag.tex +\input change_acl.tex diff --git a/org.glite.lb.doc/src/notify.tex b/org.glite.lb.doc/src/notify.tex index af3cf2d..6d84f7e 100644 --- a/org.glite.lb.doc/src/notify.tex +++ b/org.glite.lb.doc/src/notify.tex @@ -1,3 +1,97 @@ \subsection{glite-lb-notify} +\label{s:lb-notify} + +\verb'glite-lb-notify' is a fairly simple wrapper on the \LB notification API +(see also \cite{lbdg}). +It allows to create a notification (with a restricted richness of specifying +conditions), bind to it for receiving notifications, and drop it finally. + +\LB notification is a user-initiated trigger at the server. +Whenever a job enters a state matching conditions specified with the notification, +the current state of the job is sent to the notification client. +On the other hand, the notification client is a network listener +which receives server-initiated connections to get the notifications. +Unless \verb'-s' is specified, the notification library creates the listener +socket. + +Within the notification validity, clients can disappear and even migrate. +However, only a single active client for a notification is allowed. + +\LB server and port to contact is specified with GLITE\_WMS\_NOTIF\_SERVER +environment variable. + +\verb'glite-lb-notify' support these actions: +\begin{description} +\item[new] Create new notification registration +\item[bind] Binds an notification registration to a client. +\item[refresh] Enlarge notification registration validity. +\item[receive] Binds to an existing notification registration and listen to server. +\item[drop] Drop the notification registration. +\end{description} + +For action \verb'new', command usage is: + +\begin{verbatim} + glite-lb-notify new [ { -s socket_fd | -a fake_addr } -t requested_validity ] {-j jobid | -o owner | -n network_server | -v virtual_organization } +\end{verbatim} + +For action \verb'bind', command usage is: +\begin{verbatim} +glite-lb-notify bind [ { -s socket_fd | -a fake_addr } -t requested_validity ] notifid +\end{verbatim} + +For action \verb'refresh', command usage is: +\begin{verbatim} +glite-lb-notify refresh [-t requested_validity ] notifid +\end{verbatim} + +For action \verb'receive', command usage is: +\begin{verbatim} +glite-lb-notify receive [ { -s socket_fd | -a fake_addr } ] [-t requested_validity ] [-i timeout] [-f field1,field2,...] [notifid] +\end{verbatim} + +For action \verb'drop', command usage is: +\begin{verbatim} +glite-lb-notify drop notifid +\end{verbatim} + +where + +\begin{tabularx}{\textwidth}{lX} +\texttt{ notifid} & Notification ID \\ +\texttt{ -s socket\_fd} & allows to pass a opened listening socket \\ +\texttt{ -a fake\_addr} & fake the client address \\ +\texttt{ -t requested\_validity} & requested validity of the notification (in seconds) \\ +\texttt{ -j jobid} & job ID to connect notification registration with \\ +\texttt{ -o owner} & match this owner DN \\ +\texttt{ -n network\_server} & match only this network server (WMS entry point) \\ +\texttt{ -v virtual\_organization} & match only jobs of this Virtual Organization \\ +\texttt{ -i timeout} & timeout to receive operation in seconds \\ +\texttt{ -f field1,field2,...} & list of status fields to print (only owner by default) \\ +\end{tabularx} + +For additional information see also manual page GLITE-LB-NOTIFY(1). + +\subsubsection{Example: Registration and waiting for simple notification} +\label{e:notify} + +The two steps bellow describe basic testing procedure of the notification +system by registering a notification on any state change of this job +and waiting for notification. + +Register notification: +\begin{verbatim} + export EDG_WL_NOTIF_SERVER=HOSTNAME:9000 + ./glite-lb-notify new JOBID +\end{verbatim} + +Wait for notification: +\begin{verbatim} + TODO +\end{verbatim} + +When you let notification client running several minutes without any incomming notification, it will finish and remove your registration from the server automatically. + +\subsubsection{Example: Waiting for more notifications on one socket} \TODO{ljocha} -- 1.8.2.3