From: Jan Pospíšil Date: Tue, 22 Jul 2008 09:45:59 +0000 (+0000) Subject: LBUG update: finished several TODOs X-Git-Tag: merge_316_6_after~78 X-Git-Url: http://scientific.zcu.cz/git/?a=commitdiff_plain;h=52bd3b873f3e1527d75fb19cac18a7a57140dd7b;p=jra1mw.git LBUG update: finished several TODOs --- diff --git a/org.glite.lb.doc/src/LBUG-Introduction.tex b/org.glite.lb.doc/src/LBUG-Introduction.tex index 0e2b824..815cbeb 100644 --- a/org.glite.lb.doc/src/LBUG-Introduction.tex +++ b/org.glite.lb.doc/src/LBUG-Introduction.tex @@ -114,11 +114,11 @@ defined an \LB schema with specific \LB event types (see Appendix\ref{a:events}). The schema contains a common base, the attributes that must be assigned to every single event. The primary key is the jobid, which is also one of -the required attributes. The other common attributes are currently the -timestamp of the event origin, generating component name and the event -sequence code (see Sect.~\ref{evprocess}). -\TODO{je jich mnohem vic (type, arrived, priority, ....) cili bud bych -je vyjmenoval vsechny, nebo napsal neco jako 'priklady nekterych dalsich jsou...'} +the required attributes. Among other common attributes belong currently the +timestamps of the event origin and of the event arrival to LB, +generating component name, the event type, its priority and sequence code +(see Sect.~\ref{evprocess}) and so on. For a complete list of attributes +see \cite{lbdg}. While the necessary and sufficient condition for a global jobid is to be Grid-wide unique, additional desired property relates to the @@ -911,5 +911,20 @@ functions, allowing the user to specify that similarly misbehaving CE's should be penalized or completely avoided when RB decides where jobs get submitted. -\TODO{zminit, ze umime zpracovavat condorove a PBS joby; jsou i v appendixu v seznamu eventu, tak by se asi nekde zminit mely} -% \subsubsection{Non-glite event sources} + +\subsubsection{Non-gLite event sources} +\TODO{ruda: zminit, ze umime zpracovavat condorove a PBS joby; jsou i v appendixu v +seznamu eventu, tak by se asi nekde zminit mely} + +\LB has been enhanced to support also non-gLite events, namely events from PBS +or Condor batch systems \cite{hpdc07}. These events are handeled differently from gLite events, +for a complete list of the PBS and Condor events see Appendix \ref{a:events}. +Since job states in the batch system slightly differ from the states of a job +defined in \LB (see also Appendix \ref{a:jobstat}), events are processed separately +from gLite events. Both PBS and Condor events has its own state machine that processes the +events and determines the now state of the job. + +Recently, there were also attempts to use \LB system to track syslog messages. +For a detailed description see \cite{TODO:LB4syslog}. + + diff --git a/org.glite.lb.doc/src/LBUG-Tools.tex b/org.glite.lb.doc/src/LBUG-Tools.tex index 66bee00..8df4d8d 100644 --- a/org.glite.lb.doc/src/LBUG-Tools.tex +++ b/org.glite.lb.doc/src/LBUG-Tools.tex @@ -1,37 +1,70 @@ \section{User tools} \label{s:lb-tools} -\TODO{honik: vytahnout WMS-UI do zvlastni kapitolky} +In this section we give a description of the CLI tools that a regular grid user +might want to use. If not stated otherwise, the tools are distributed in the +\verb'glite-lb-client' RPM. Behaviour of the commands can be often changed by +setting some of the following enviroment variables: -In this section we give a description of the tools that are installed -together with the \verb'glite-lb-client' RPM. These tools are \LB-specific. -Besides these tools, more general WMS commands \verb'glite-wms-job-status' -and \verb'glite-wms-job-logging-info' available -in \verb'glite-wms-ui-cli-python' RPM, provide also information about user jobs -and corresponding events. - -Behaviour of \verb'glite-lb-*' commands can be changed by setting the following enviroment variables: \begin{tabularx}{\textwidth}{lX} -GLITE\_WMS\_LOG\_DESTINATION & address of \verb'glite-lb-logd' daemon, in form \verb'hostname:port'\\ +GLITE\_WMS\_LOG\_DESTINATION & address of \verb'glite-lb-logd' daemon, in form \verb'hostname:port', for logging events\\ GLITE\_WMS\_LOG\_TIMEOUT & timeout (in seconds) for asynchronous logging\\ GLITE\_WMS\_LOG\_SYNC\_TIMEOUT & timeout (in seconds) for synchronous logging\\ GLITE\_WMS\_NOTIF\_SERVER& address of \verb'glite-lb-bkserver' daemon, in form \verb'hostname:port', for receiving notifications\\ GLITE\_WMS\_NOTIF\_TIMEOUT& timeout (in seconds) for notification registration\\ +GLITE\_WMS\_QUERY\_SERVER& address of \verb'glite-lb-bkserver' daemon, in form \verb'hostname:port', for queries\\ X509\_USER\_CERT and X509\_USER\_KEY & location of user credentials\\ \end{tabularx} -\TODO{zminil bych i EDG\_WL\_QUERY\_SERVER} For backward compatibility, all \verb'GLITE_WMS_*' variables can be prefixed by -\verb'EDG_WL_' instead. +\verb'EDG_WL_' instead. Setting the environment variable is for some commands mandatory, +so reading the documentaion below and appropriate manpages is highly recommended. + + +\subsection{glite-wms-job-status and glite-wms-job-logging-info} + +We start with tools that are distributed in \verb'glite-wms-ui-cli-python' RPM +and that can be found probably on all UI machines. Description of all user +commands that are used during the job submission process (generating proxy, +creating a JDL describing the job, submitting a job, retrieving output, +cancelling a job, etc.) is beoynd this document and it can be found for example +in \cite{wmsug}. We mention here only the commands that are related to +job monitoring. + +Once job has been submitted to WMS (by \verb'glite-wms-job-submit'), +a user can retrieve the job status by +\begin{verbatim} + glite-wms-job-status +\end{verbatim} +or if job ID is saved in the file +\begin{verbatim} + glite-wms-job-status -i +\end{verbatim} +or if user wants to see status of all his/her jobs +\begin{verbatim} + glite-wms-job-status --all +\end{verbatim} +List of all possible job states is summarised in Appendix \ref{a:jobstat}. + +Logging details on submitted job can be accessed via +\begin{verbatim} + glite-wms-job-logging-info -v +\end{verbatim} +or if job ID is saved in the file +\begin{verbatim} + glite-wms-job-logging-info -v -i +\end{verbatim} +where verbosity level can be from 0 to 3. + \input{logevent} \input{notify} + \subsection{Other useful tools} -\label{glite-lb-other} For debugging purposes, low-level commands for getting \LB job status and job related events are provided in -\verb'examples' directory (\verb'glite-lb-job-status' and \verb'glite-lb-job-log'). The same directory +\verb'examples' directory (\verb'glite-lb-job_status' and \verb'glite-lb-job_log'). The same directory contains also debugging commands for getting of all user jobs (\verb'glite-lb-user_jobs') and CE-reputability rank (see Section \ref{s:ce-rank}, \verb'glite-lb-stats'). -\TODO{job-log vs. job\_log - ujednotit} + diff --git a/org.glite.lb.doc/src/frontmatter.tex b/org.glite.lb.doc/src/frontmatter.tex index 4a38342..ffeda29 100644 --- a/org.glite.lb.doc/src/frontmatter.tex +++ b/org.glite.lb.doc/src/frontmatter.tex @@ -21,7 +21,9 @@ \begin{tabularx}{\textwidth}{|l|l|X|X|} \hline {\bf Issue } & {\bf Date } & {\bf Comment } & {\bf Author } \\ \hline -1.0.0 & August 1, 2008 & Initial version & CESNET team \\ +Initial version & August 1, 2008 & & CESNET team \\ \hline +Reviewer's comments &&& \\ \hline +Users' comments &&& \\ \hline \end{tabularx} diff --git a/org.glite.lb.doc/src/lbjp.bib b/org.glite.lb.doc/src/lbjp.bib index 7ff17df..0321dcc 100644 --- a/org.glite.lb.doc/src/lbjp.bib +++ b/org.glite.lb.doc/src/lbjp.bib @@ -693,7 +693,7 @@ @Misc{ lbug, author = "A. K\v{r}enek and others", title = "{L\&B User's Guide}", - howpublished = "\url{https://edms.cern.ch/file/571273/1/LB-guide.pdf}" + howpublished = "\url{https://edms.cern.ch/file/571273/2/LB-guide.pdf}" } @Misc{ lbdg, @@ -711,6 +711,12 @@ title = "{L\&B Test Plan}", } +@Misc{ wmsug, + author = "F. Pacini and others", + title = "{WMS User's Guide}", + howpublished = "\url{https://edms.cern.ch/file/572489/1/WMS-guide.pdf}" +} + @Misc{ lcg, author = "LCG", title = "{LHC Computing Project (LCG)}", @@ -722,3 +728,35 @@ howpublished = "\url{http://twiki.pasoa.ecs.soton.ac.uk/bin/view/PASOA/AboutPasoa}" } + +@InProceedings{ hpdc07, + author = {Miroslav Ruda and Ale\v{s} K\v{r}enek and Milo\v{s} + Mula\v{c} and Jan Posp\'{\i}\v{s}il and Zden\v{e}k \v{S}ustr}, + title = {A uniform job monitoring service in multiple job + universes}, + booktitle = {GMW '07: Proceedings of the 2007 workshop on Grid + monitoring}, + year = {2007}, + pages = {17--22}, + address = {New York, NY, USA}, + publisher = {ACM}, + isbn = {978-1-59593-716-2}, + location = {Monterey, California, USA}, + doi = {http://doi.acm.org/10.1145/1272680.1272684}, + abstract = {We describe an ongoing work of extending the gLite Logging + and Bookkeeping (\LB) service to be able to track + additional types of jobs, with the vision of being able to + uniformly follow jobs on the Grid, even when they pass + between different middleware domains. Details are given on + the simpler case of PBS jobs, which prove the cabability of + \LB to deal with additional job types, as well as started + more complex and challenging work on Condor jobs, where the + impact of eventual success is larger.} +} + + + + + + +