--- /dev/null
+PURPOSE, EXPECTED USAGE, AND LIMITATIONS
+
+The purpose of the Job Provenance service (JP) is keeping track of the
+definition of submitted jobs, execution conditions and environment, and
+important points of the job life cycle for a long period (months to years).
+Those data can be used for debugging, post-mortem analysis, comparison of job
+execution within an evolving environment, as well as assisted re-execution of
+jobs. Only data of completed (either successful or failed) jobs are handled;
+tracking jobs during their active life is the task of L&B and Job Monitoring
+services.
+
+In general, gathered data are stored (i.e. copied) within the JP storage in
+order to really conserve a partial snapshot of the Grid environment when the
+job was executed, independently of changes of other Grid services. Obviously
+there are practical limitations of the extent to which it is feasible to record
+the entire job execution environment. (In the ideal case this would encompass a
+snapshot of the entire Grid!) We restrict the recorded data to those that are
+processed or somehow affect processing of the Workload Management and Computing
+Element services. On the other hand, snapshots of the state of other Grid
+services are not done, namely queries to the Data Catalog and their results are
+not stored, as well as contents of data files downloaded from and uploaded to
+Storage Elementsonly references to those are recorded if required.
+
+The JP Primary Storage, as the name suggests, is the primary interface for
+raw data storage. It is complemented with JP Index service, designed to
+provide indexing and execute various queries on these data.
+
+
+ENCOMPASSED DATA
+
+Currently, JP is designed to gather the following data:
+
+- JOB LIFE LOG is taken over from the L&B database (see Sect. 8.3.5) after job
+ completion. Among other information useful mainly for debugging and detailed
+ analysis of job execution it contains the complete definition of the job (in
+ terms of the submitted JDL), various timestamps (e.g. when the job was
+ submitted, matched for a particular CE, started and finished execution),
+ information on the chosen CE (or more of them, if the job was resubmitted),
+ various IDs assigned to the job (Condor, Globus, LRMS, ...), and the
+ result of execution including the return code. Another important information
+ available in L&B are the user-defined annotations (tags) which can be
+ specified either statically upon job submission, during its execution, or even
+ overridden after the job terminates.
+
+- INPUT/OUTPUT SANDBOX. The input sandbox of a job contains miscellaneous files
+ required for execution, and the output sandbox may contain various output
+ files, e.g. debug logs, or even a core file in the case of crash.
+
+- JP TAGS. JP tags are arbitrary "name=value" pairs attached to a singl job.
+ They are either standalone or override values of their LB counterparts.
+ However, JP tag values are still distinguishable those inherited from LB.
+ JP tags may be either strings or blobs.
+
+
+JP PRIMARY STORAGE INTERFACES AND OPERATIONS
+
+JP primary storage service exposes two interfaces:
+- WS control and query interface,
+- bulk file transfer interface (currently gsiftp).
+
+WS interface operations (see JobProvenancePS.wsdl for details):
+
+- RegisterJob - create the basic job record with JP primary.
+
+ Normally called by L&B upon job registration wigh L&B.
+
+- StartUpload - initiates upload of input/output sandbox or job log.
+ JP responds with upload destination URL, which points to its
+ bulk file transfer interface, and a time limit for the upload.
+
+ Normally called by L&B (job log) and WMS NS (sandboxes).
+
+- CommitUpload - confirm upload operation. This must be called after the
+ upload is done, and within the given time limit. If not done so,
+ the destination URL becomes invalid and any (partially) uploaded
+ file is discarded.
+
+ Normally called by L&B (job log) and WMS NS (sandboxes).
+
+- RecordTag - Record a value of JP user tag.
+
+ Called by any component which needs it.
+
+- FeedIndex - start feeding data into JP index server.
+
+ Called internally by JP index.
+
+- GetJob - Based on known JobId retrieve URL's of the job record (i.e. job log,
+ input/output sandboxes, and JP tags).
+
+ Mainly called by the user, either standalone, or after querying
+ index server.
+
+
+
+EXAMPLE PROGRAM
+
+The installed example program glite-jp-primary-test is a simple client
+calling all the service WS operations. Run "glite-jp-primary-test -?"
+for its current usage help.