\section{Performance and Stress Tests}
+\TODO{review, reformat, update also \url{https://meta.cesnet.cz/mediawiki/index.php/LB_and_JP_Performance_Testing}, ...}
In this section we describe tests from layers 4 (stress tests) and 5 (performance tests).
-\TODO{review, reformat plus add megaJob results}
+General idea of performance and stress tests of LB components is the following:
+\begin{itemize}
+
+\item All source modifications for tests are in CVS, conditionaly compiled only
+with appropriate symbol.
+
+\item To compile LB with performance testing enabled, set environment variable
+\verb'LB_PERF' to 1 prior to LB build:
\begin{verbatim}
-- all source modifications for tests are in CVS, conditionaly compiled
- only with appropriate symbol
-
-- binaries for all tests are built using special property
- for ant target (or environment variable for Makefile), which
- compiles sources using the right #define combinations
-
-- component tests are run by shell scripts located under component
- directories, these tests may require binaries from other components,
- though
-
-- all tests use sequence of events for typical jobs (small job, big
- job, small DAG, big DAG) prepared beforehand. These events are
- stored in files in ULM format in CVS.
-
-- events are generated by stresslog program, which reads ULM text of
- events for particular test job and logs the event sequence directly
- by calling *_DoLogEvent<variant>. The number of test jobs is
- configurable. Stresslog inserts into every event timestamp when the
- event was generated and sent.*
-
-- event are consumed by breaking normal event processing either in the
- component being tested or the next component in chain, that is
- instrumented to read and discard events immediately. The consumption
- itself is done by calling special function which takes current time,
- extracts timestamp from event and prints the difference (ie. the
- event processing time).* These "break points" are chosen to measure
- throughput of the various component parts and to identify possible
- bottlenecks within the components.
-
- * the only exception is test of the logging library itself
-
-- test jobs are preregistered within the LB if the test includes
- bookkeeping server and/or proxy by the test script program and
- their id's are stored in separate file to enable re-use by other
- load-generating tools (status queries, for example)
-
-- test results:
- - some numbers must be reported by component themselves, not by
+ export LB_PERF=1
+ etics-build org.glite.lb
+\end{verbatim}
+
+\item Component tests are run by shell scripts located under component
+directories, these tests may require binaries from other components, though.
+
+\item All tests use sequence of events for typical jobs (small job, big job,
+small DAG, big DAG) prepared beforehand. These events are stored in files in
+ULM format in CVS (see \texttt{org.glite.lb.common/examples}).
+\TODO{new strategy? use real jobs...}
+
+\item Events are generated by \verb'glite-lb-stresslog' program, which reads
+ULM text of events for particular test job and logs the event sequence directly
+by calling \verb'*_DoLogEvent<variant>'. The number of test jobs is
+configurable. Stresslog inserts into every event timestamp when the event was
+generated and sent.
+
+\item Events are consumed by breaking normal event processing either in the
+component being tested or the next component in chain, that is instrumented to
+read and discard events immediately. The consumption itself is done by calling
+special function which takes current time, extracts timestamp from event and
+prints the difference (ie. the event processing time)\footnote{the only
+exception is test of the logging library itself}. These "break points" are
+chosen to measure throughput of the various component parts and to identify
+possible bottlenecks within the components.
+
+\item Test jobs are preregistered within the LB if the test includes
+bookkeeping server and/or proxy by the test script program and their id's are
+stored in separate file to enable re-use by other load-generating tools (status
+queries, for example).
+
+\item Test results:
+ \begin{itemize}
+ \item Some numbers must be reported by component themselves, not by
the event generator (due to the asynchronous LB nature). The
test script collects those numbers and presents them as the test
result at the end of testing.
- - after completion test scripts print the table described for the
+ \item After completion test scripts print the table described for the
respective tests filled in with measured values (ie. the table
- is not filled in manually by human tester)
+ is not filled in manually by human tester).
- - event throughput = 1/(time_delivered - time_arrived)
- * only if next event is sent after previous was delivered
-
-? measure job throughput for event patterns of typical jobs or deduce
-job throughput from throughput of selected types of events?
-
+ \item Measure event throughput bu
+ \[ \mbox{event\_throughput} = \frac{1}{\mbox{time\_delivered} - \mbox{time\_arrived}} \]
+\TODO{measure job throughput for event patterns of typical jobs or deduce
+job throughput from throughput of selected types of events?}
+ \item Publish the results on the web.
+\TODO{ETICS test? \url{https://meta.cesnet.cz/mediawiki/index.php/LB_and_JP_Performance_Testing}? etc.}
+ \end{itemize}
+% vztahuje se k cemu?
+% * only if next event is sent after previous was delivered
+%
+\end{itemize}
-I) Component tests
- ***************
+In the following subsections we describe performance and stress tests for
+individual LB compoments. They include both tests of the isolated components
+on one node (may require binaries from other components to produce/consume
+events) as well as tests of LB components among more nodes.
-- tests of the isolated components on one node
-- may require binaries from other components to produce/consume events
-
-\end{verbatim}
%--------------------
\subsection{Logging library test}
\end{verbatim}
+% ---------------------
+\subsection{"Real world" tests}
+% ---------------------
+
+Aim of these tests is to simulate real environment of the LB operation as
+closely as possible. We simulate the environment on WMS node as this is the
+most performance critical point. Events are sent through proxy and interlogger
+on one machine (WMS node) to the server on another machine (LB node), we also
+create some background noise by simultaneously polling the server and/or proxy
+for job status by several clients in parallel.
+
git://scientific.zcu.cz / jra1mw.git / commitdiff