clocks.
\fi
%
-We assume that the workload manager (WM) sends the job to a~computing element
-(CE)~A, where it starts running but the job dies in the middle.
-The failure is detected and the job is resubmitted back to the WM which sends it to CE~B then.
-However, if A's clock is ahead in time and B's clock is correct (which
-means behind the A's clock), the events in the right column are treated
-as delayed. The state machine will interpret events incorrectly, assuming
-the job has been run on B before sending it to A.
-The job would always (assuming the A's events arrive before B's events to
-the \LB) be reported as ``\emph{Running} at A'' despite
-the real state should follow the \emph{Waiting} \dots \emph{Running} sequence.
-Even the \emph{Done} event can be sent by B with a timestamp that says
-this happened before the job has been submitted to A and the job state
-will end with a discrepancy---it has been reported to finish on B while
-still reported to run on A.
-
-\begin{table}[hb]
+\begin{table}[h]
+\begin{center}
\begin{tabular}{rlrl}
1.&WM: Accept&
6.&WM: Accept\\
10.&CE~$B$: Run \\
\dots & $A$ dies\\
\end{tabular}
-
-\TODO{tahle tabulka je taky v 1.3, navic ve footnote}
-
+\end{center}
\caption{Simplified \LB events in the CE failure scenario}
\label{t:cefail}
\end{table}
+%
+We assume that the workload manager (WM) sends the job to a~computing element
+(CE)~A, where it starts running but the job dies in the middle.
+The failure is detected and the job is resubmitted back to the WM which sends it to CE~B then.
+However, if A's clock is ahead in time and B's clock is correct (which
+means behind the A's clock), the events in the right column are treated
+as delayed. The state machine will interpret events incorrectly, assuming
+the job has been run on B before sending it to A.
+The job would always (assuming the A's events arrive before B's events to
+the \LB) be reported as ``\emph{Running} at A'' despite
+the real state should follow the \emph{Waiting} \dots \emph{Running} sequence.
+Even the \emph{Done} event can be sent by B with a timestamp that says
+this happened before the job has been submitted to A and the job state
+will end with a discrepancy---it has been reported to finish on B while
+still reported to run on A.
Therefore we are looking for a~more robust and general solution. We can
discover severe clock bias if the timestamp on an event is in a future
\label{seqcode}
As discussed in Sect.~\ref{evorder}, sequence codes are used as logical
-timestamps to ensure proper event ordering on the \LB server. The
-sequence code counter is incremented whenever an event is logged and the
-sequence code must be passed between individual Grid components together
-with the job control.
+timestamps to ensure proper event ordering on the \LB server. The sequence code
+counter is incremented whenever an event is logged and the sequence code must
+be passed between individual Grid components together with the job control.
However, a single valued counter is not sufficient to support detection of
-branch forks within the execution tree.
-When considering again the Computing Element failure scenario described
-in Sect.~\ref{evorder}, there is no way to know that the counter value of
-the last event logged by the failed CE A is 5 (Tab.~\ref{t:cefail}).
+branch forks within the execution tree. When considering again the Computing
+Element failure scenario described in Sect.~\ref{evorder}, there is no way to
+know that the counter value of the last event logged by the failed CE A is 5
+(see Table~\ref{t:cefail} on page~\pageref{t:cefail}).
+
+Therefore we define a~hierarchical \emph{sequence code}---an array of
+counters, each corresponding to a~single Grid component class handling the job%
+\footnote{Currently the following gLite components: Network Server, Workload
+Manager, Job Controller, Log Monitor, Job Wrapper, and the application itself.}.
+Table~\ref{t:cefail2} below shows the same scenario with a~simplified two-counter
+sequence code. The counters correspond to the WM and CE component classes
+and they are incremented when each of the components logs an event.
+When WM receives the job back for resubmission,
+the CE counter becomes irrelevant (as the job control is on WM now),
+and the WM counter is incremented again.
-\begin{table}[b]
+\begin{table}[h]
+\begin{center}
\begin{tabular}{rlrl}
1:x&WM: Accept&
4:x&WM: Accept\\
6:2&CE~$B$: Run \\
\dots & $A$ dies\\
\end{tabular}
+\end{center}
\caption{The same CE failure scenario: hierarchical sequence codes.
``x'' denotes an undefined and unused value.}
\label{t:cefail2}
\end{table}
-
-Therefore we define a~hierarchical \emph{sequence code}---an array of
-counters, each corresponding to a~single Grid component class handling the job%
-\footnote{Currently the following gLite components: Network Server, Workload
-Manager, Job Controller, Log Monitor, Job Wrapper, and the application itself.}.
-Table~\ref{t:cefail2} shows the same scenario with a~simplified two-counter
-sequence code. The counters correspond to the WM and CE component classes
-and they are incremented when each of the components logs an event.
-When WM receives the job back for resubmission,
-the CE counter becomes irrelevant (as the job control is on WM now),
-and the WM counter is incremented again.
-
%Events on a~branch are ordered following the lexicographical order
%of the sequence codes.
%Branches are identified according to the WM counter as WM is
indicates how fast we are able to ``generate'' events in the feeding
program.
-\begin{table}[hbt]
+\begin{table}[h]
\begin{tabular}{l|r}
{\bf Component} & {\bf Job throughput (jobs/day)} \\
\hline