glite-lb-common & common files \\
glite-lb-client & client library and CLI tools\\
glite-lb-client-interface & client library interface (header files) \\
-glite-lb-harvester & enhanced \LB notification client (since \LBver{1.10}) \\
+%glite-lb-harvester & enhanced \LB notification client (since \LBver{1.10}) \\
glite-lb-logger & local-logger and inter-logger \\
glite-lb-proxy & proxy (restricted server used by WMS)\\
glite-lb-server & server \\
glite-security-gsoap-plugin & GSS wrapper and GSS plugin for gSoap
\end{tabularx}
-In \LBver{2.0}, the code has been restructured quite a lot, especially the dependencies were lightened,
+Starting with \LBver{2.0}, the code has been restructured quite a lot, especially the dependencies were lightened,
and the new list of packages is now the following:
\begin{tabularx}{\textwidth}{>{\tt}lX}
glite-lb-doc & documentation \\
glite-lb-common & common files \\
glite-lb-client & client library and CLI tools\\
-glite-lb-harvester & enhanced \LB notification client \\
+glite-lb-client-java & Java implementation of the client (since \LBver{2.1})\\
+glite-lb-harvester & enhanced \LB notification client (since \LBver{2.1})\\
glite-lb-logger & local-logger and inter-logger \\
glite-lb-server & server, including merged proxy functionality \\
glite-lb-state-machine & state machine and LB plugin for Job Provenance \\
glite-lb-utils & auxiliary utilities \\
glite-lb-ws-interface & web service interface \\
+glite-lb-yaim & YAIM initialization scripts for \LB (since \LBver{2.1}) \\
\end{tabularx}
More detailed description together with the dependencies can be read directly from each package,
\item \texttt{MYSQL\_PASSWORD} -- root password of MySQL server (mandatory)
\item \texttt{GLITE\_WMS\_LCGMON\_FILE} -- pathname of file where job state
export data are written for use by lgcmon/R-GMA
-(default: \texttt{/var/glite/logging/status.log}
+(default: \texttt{/var/glite/logging/status.log}). \emph{Note: This feature is now obsolete and only available in \LBver{1.x}.}
\item \texttt{GLITE\_LB\_EXPORT\_PURGE\_ARGS} -- purge timeouts (default: \texttt{--cleared 2d --aborted 15d --cancelled 15d --other 60d})
According to local retention policy you may want to use different purge timeouts (for example WLCG would need \texttt{--cleared 90d --aborted 90d --cancelled 90d --other 90d}).
database automatically. For existing LB server database the migration
process is not automatically handled.
-Note: If you want to add transaction when migrating to \LB 2.0 skip
-this section and use \LB 2.0 migration procedure. The migration of
-database to support transactions is included in \LB 2.0 migration procedure.
+Note: If you want to add transaction when migrating to \LB 2.x skip
+this section and use \LB 2.x migration procedure. The migration of
+database to support transactions is included in \LB 2.x migration procedure.
Steps:
\begin{itemize}
\end{itemize}
-\subsubsection{Migration to \LB 2.0}
+\subsubsection{Migration to \LB 2.x}
\label{inst:migrate20}
-The migration process of existing \LB 1.x database to the \LB 2.0 is
+The migration process of existing \LB 1.x database to the \LB 2.x is
not handled automatically. The database schema change is required due
to support of merged \LB server and proxy services using single
database, pointers to purged jobs (``zombies'') and other
Warning: There are two types of \LB database based on the fact that
you can have a \LB server or \LB proxy. For more information about \LB
-proxy please see \ref{inst:LBproxy}.
+proxy please see \ref{inst:LBproxy}
Steps:
\begin{itemize}
- \item \emph{Stop the server and upgrade to \LB 2.0.} Stop both a \LB
+ \item \emph{Stop the server and upgrade to \LB 2.x.} Stop both a \LB
server and a MySQL server. Making a fresh backup copy of database is
- a good idea. Do the upgrade to \LB 2.0, optionally you can move the
+ a good idea. Do the upgrade to \LB 2.x, optionally you can move the
database to new OS in this step (see \ref{inst:OSmigration}).
\item \emph{Before migration some database tuning is
required.} Especially parameter \texttt{innodb\_buffer\_pool\_size}
needs to be increased, to support bigger transactions. For details
- see Section~\ref{inst:db_tuning}.
+ see Section~\ref{inst:db_tuning}
\item \emph{Database type.} Check if you have a \LB server or a \LB
proxy. In the following step you must properly set the switch
\verb'-s' (server) or \verb'-p' (proxy).
\verb'mysql -u lbserver lbserver20 -e "alter table events drop index host"'
\end{quote}
\item \emph{(Optional) Check the \LB indexes.} You may need to add
- LastUpdateTime for monitoring services. See \ref{maintain:index}.
+ LastUpdateTime for monitoring services. See \ref{maintain:index}
\item \emph{Start the servers.} MySQL and \LB. Check logs.
\end{itemize}
\subsubsection{Export to R-GMA}
+\emph{Note: This feature is now obsolete and only available in \LBver{1.x}.}
+
{\sloppy
\LB server can export information on job state changes to R-GMA infrastructure through \verb'lcgmon'
in real time. This export is enabled by YAIM by default and uses \verb'GLITE_WMS_LCGMON_FILE'
data (jobs in Cleared state after two days, Aborted and Cancelled after 15 days, and other jobs
after 60 days of inactivity). It is recommended to run the cron jobs more often (in order to purge less jobs
during single run) if event queue backlogs form in client WMS machines when the purging cron jobs is running.
-For details on setting job purge timeouts, see Sect.~\ref{run:purge}.
+For details on setting job purge timeouts, see Sect.~\ref{run:purge}
\subsubsection{Exploiting parallelism}
\LB server only, not supported by proxy.
-(This functionality should not be confused with per-job dumps, Sect.~\ref{inst:purge} and \ref{run:purge}.)
+(This functionality should not be confused with per-job dumps, Sect.~\ref{inst:purge} and \ref{run:purge})
Besides setting up \LB server database on a~reliable storage or
backing it up directly (Sect.~\ref{inst:backup})
\begin{sloppypar}
In case of emergency (\eg corrupted file) the files can be examined
-with \verb'glite-lb-parse_eventsfile' %
-\footnote{Not fully supported tool, installed by \texttt{glite-lb-client} package among examples.}.
+with \verb'glite-lb-parse_eventsfile'.\footnote{Not fully supported tool, installed
+by \texttt{glite-lb-client} package among examples.}
It is possible to hand-edit the event files in emergency (remove corrupted lines).
However, glite-lb-interlogd must not be running, and the corresponding .ctl file
must be removed.
On the other hand, if it happens,
and an event with such a~jobid is logged,
\eg due to a~third-party job processing software bug,
-glite-lb-interlogd keeps trying to deliver it indefinitely%
-\footnote{Unless event expiration is set, though it is not done for normal events.}.
+glite-lb-interlogd keeps trying to deliver it indefinitely.\footnote{Unless
+event expiration is set, though it is not done for normal events.}
The unsuccessful attempts are reported via syslog.
The only solution is manual
removal of the corresponding files
one query but may arrive before a subsequent query is issued),
and (ii)~the dependence on a~remote service to provide vital state information
may block the local service if the remote one is not responding.
-These problems are addressed by providing \emph{local view} on the \LB data,
-see Sect.~\ref{local}.
+These problems are addressed by providing \emph{local view} of the \LB data,
+see Sect.~\ref{local}
Job representing collection or DAG can be used to monitor status of the set, including
information like how many subjobs is already finished etc.
Support for non-gLite jobs, namely for PBS or Condor systems, is described in section
-\ref{sec:nonglite}.
+\ref{sec:nonglite}
\subsubsection{Event ordering}%
the server where events belonging to a~job are gathered from multiple sources,
\LB infrastructure can store the logged events temporarily
on the event source, and perform the processing described
-in Sect.~\ref{evprocess}.
+in Sect.~\ref{evprocess}
In this setup, the logging vs.\ query semantics can be synchronous---it is
guaranteed that a~successfully logged event is reflected in the result of
an immediately following query,
Authorization information is also manipulated in this way.
Description of tools for event submission and ACL manipulation
-can be found in Section \ref{s:lb-tools}.
+can be found in Section \ref{s:lb-tools}
\subsubsection{Querying information}
information, but confortably wait for server to inform the user.
User registers for notifications via notification client \verb'glite-lb-notify', described in Section
-\ref{s:lb-notify}.
+\ref{s:lb-notify}
Conditions under which the notification is sent can be specified. For example - job XY reaches status
DONE. In \LBver{1.x}, one or more JOBID's are required in the condition and only a single occurence
of a specific attribute is allowed among ANDed conditions. In \LBver{2.x}, more complex conditions are allowed.
Thus \LB\ becomes a~data source for various real-tim Grid monitoring tools.
\subsubsection{R-GMA feed}
+\emph{Note: This feature is now obsolete and only available in \LBver{1.x}.}
+
The \LB server also supports streaming the most important data---the job
state changes---to another monitoring system. It works as the
notification service, sending information about job state changes to
the R-GMA infrastructure that is part of the Grid monitoring
infrastructure used in the EGEE Grid.
-Currently, only basic information about job state changes is provided
+Only basic information about job state changes is provided
this way, taking into account the security limitation of the R-GMA.
\subsubsection{\LB Job Statistics}
\subsubsection{CREAM jobs}
-\LB 2.1 implements basic support for CREAM jobs. \LB gathers events from CREAM, performing both the mapping of CREAM attributes to WMS attributes (if possible) and storing CREAM-specific atributes. Thus, the jobs submitted directly to computing elements can be tracked by \LB as well as extended data from WMS. The CREAM job states are mapped to \LB ones according to Appendix \ref{a:creammapping}.
+\LB 2.1 implements basic support for CREAM jobs. \LB gathers events from CREAM, performing both the mapping of CREAM attributes to WMS attributes (if possible) and storing CREAM-specific atributes. Thus, the jobs submitted directly to computing elements can be tracked by \LB as well as extended data from WMS. The CREAM job states are mapped to \LB ones according to Appendix \ref{a:creammapping}
CREAM events are generated by thee CREAM executor and LRMS, the generic \emph{CREAMStatus} event is generated when CREAM notifies that the job status has been changed.
Note that \LBver{1.x} supported only using VOMS group names, not full FQANs,
whose support has been introduced in \LBver{2.0}. \LBver{1.x} also did not
-allowed the users to use symbolic names for the values specifying ACL
+allow the users to use symbolic names for the values specifying ACL
setting and integers must be used instead. For example, to grant access
right on a \LBver{1.x} server one has to use following syntax:
and the job state updated
accordingly.
-On the other hand, the server exposes querying interface (Fig.~\ref{f:comp-query}%
-\ifx\insideUG\undefined\relax\else
-, Sect.~\ref{retrieve}
-\fi
-).
+On the other hand, the server exposes querying interface
+(Fig.~\ref{f:comp-query}\ifx\insideUG\undefined\relax\else, Sect.~\ref{retrieve}\fi).
The incoming user queries are transformed into SQL queries on the underlying
database engine.
The query result is filtered, authorization rules applied, and the result
% useful definitions
\def\LB{L\&B\xspace}
-\newcommand\LBver[1]{\textbf{\LB version {#1}\xspace}}
+\newcommand\LBver[1]{\textbf{\LB version {#1}}}
\def\JP{JP\xspace}
%\def\eg{e.\,g.}
\def\eg{for example\xspace}
\textbf{The following versions of \LB service are covered by these documents:}
\begin{itemize}
-\item \LBver{2.2}: current development version, available through CVS only,
-\item \LBver{2.1}: currently in certification, to be released in April 2010,
+\item \LBver{2.2}: in development, available through CVS only,
+\item \LBver{2.1}: replacement for \LBver{2.0} in gLite 3.2,
\item \LBver{2.0}: current stable version, in production as part of gLite 3.2,
\item \LBver{1.x}: old stable version, in production as part of gLite 3.1.
\end{itemize}
git://scientific.zcu.cz / jra1mw.git / commitdiff