hadoop-jobstat.git
9 years agoLates tunin of cluster usage summary. master
František Dvořák [Thu, 7 Jan 2016 15:37:12 +0000 (16:37 +0100)]
Lates tunin of cluster usage summary.

9 years agoMore details in the tasts sumary: map/reduce.
František Dvořák [Thu, 7 Jan 2016 00:15:15 +0000 (01:15 +0100)]
More details in the tasts sumary: map/reduce.

9 years agoRename node-specific chroust scripts.
František Dvořák [Wed, 6 Jan 2016 22:33:39 +0000 (23:33 +0100)]
Rename node-specific chroust scripts.

9 years agoOptimized version of analyzing nodes.
František Dvořák [Wed, 6 Jan 2016 22:32:25 +0000 (23:32 +0100)]
Optimized version of analyzing nodes.

9 years agoTuning: better SQL, better output.
František Dvořák [Wed, 6 Jan 2016 22:14:05 +0000 (23:14 +0100)]
Tuning: better SQL, better output.

10 years agoCustom min/max time in the node load analyzer script.
František Dvořák [Mon, 16 Mar 2015 14:05:48 +0000 (15:05 +0100)]
Custom min/max time in the node load analyzer script.

10 years agoStore also job counters in bookkeeping.
František Dvořák [Mon, 9 Mar 2015 14:23:32 +0000 (15:23 +0100)]
Store also job counters in bookkeeping.

10 years agoDon't add memory usage to PBS logs, cleanups.
František Dvořák [Mon, 9 Mar 2015 11:45:54 +0000 (12:45 +0100)]
Don't add memory usage to PBS logs, cleanups.

10 years agoPBS generator: reshuffle, add sched_nodespec and memory.
František Dvořák [Sun, 8 Mar 2015 22:16:53 +0000 (23:16 +0100)]
PBS generator: reshuffle, add sched_nodespec and memory.

10 years agoSync with MetaCenterCloudPuppet.
František Dvořák [Sun, 8 Mar 2015 13:24:55 +0000 (14:24 +0100)]
Sync with MetaCenterCloudPuppet.

10 years agoSupport for high availability of YARN Resource Manager.
František Dvořák [Sun, 8 Mar 2015 12:55:34 +0000 (13:55 +0100)]
Support for high availability of YARN Resource Manager.

10 years agoExplore YARN RM cluster status.
František Dvořák [Sun, 8 Mar 2015 12:30:11 +0000 (13:30 +0100)]
Explore YARN RM cluster status.

10 years agoHOWTO script.
František Dvořák [Sat, 7 Mar 2015 23:43:22 +0000 (00:43 +0100)]
HOWTO script.

10 years agoPlotting of the nodes usage.
František Dvořák [Sat, 7 Mar 2015 23:04:16 +0000 (00:04 +0100)]
Plotting of the nodes usage.

10 years agoProper startig time in each interval.
František Dvořák [Sat, 7 Mar 2015 23:00:37 +0000 (00:00 +0100)]
Proper startig time in each interval.

10 years agoChange default database name for bookkeeping from 'hadoop' to 'bookkeeping'.
František Dvořák [Fri, 6 Mar 2015 22:09:57 +0000 (23:09 +0100)]
Change default database name for bookkeeping from 'hadoop' to 'bookkeeping'.

10 years agoIndex also time cilumns in subjobs.
František Dvořák [Fri, 6 Mar 2015 22:03:50 +0000 (23:03 +0100)]
Index also time cilumns in subjobs.

10 years agoAlways take precedence to mapred before old data or yarn.
František Dvořák [Fri, 6 Mar 2015 21:50:03 +0000 (22:50 +0100)]
Always take precedence to mapred before old data or yarn.

10 years agoSync with MetaCenterCloudPuppet/cesnet-site_hadoop.
František Dvořák [Fri, 6 Mar 2015 20:54:19 +0000 (21:54 +0100)]
Sync with MetaCenterCloudPuppet/cesnet-site_hadoop.

10 years agoPerms.
František Dvořák [Fri, 6 Mar 2015 20:52:26 +0000 (21:52 +0100)]
Perms.

10 years agoFix init script.
František Dvořák [Fri, 6 Mar 2015 14:29:23 +0000 (15:29 +0100)]
Fix init script.

10 years agoInitial deployment scripts.
František Dvořák [Fri, 6 Mar 2015 14:27:03 +0000 (15:27 +0100)]
Initial deployment scripts.

10 years agoTune verbosity (print only errors by default).
František Dvořák [Fri, 6 Mar 2015 14:26:22 +0000 (15:26 +0100)]
Tune verbosity (print only errors by default).

10 years agoUsage statistics of each node mined from the subjobs (map/reduce tasks).
František Dvořák [Thu, 5 Mar 2015 23:34:39 +0000 (00:34 +0100)]
Usage statistics of each node mined from the subjobs (map/reduce tasks).

10 years agoSupport query parameters (it can be probably used for incremental checkouts).
František Dvořák [Thu, 5 Mar 2015 20:47:21 +0000 (21:47 +0100)]
Support query parameters (it can be probably used for incremental checkouts).

10 years agoFix node IDs in subjobs.
František Dvořák [Thu, 5 Mar 2015 20:02:45 +0000 (21:02 +0100)]
Fix node IDs in subjobs.

10 years agoDisable walltime for running jobs.
František Dvořák [Thu, 5 Mar 2015 17:12:12 +0000 (18:12 +0100)]
Disable walltime for running jobs.

10 years agoFix status.
František Dvořák [Thu, 5 Mar 2015 17:03:09 +0000 (18:03 +0100)]
Fix status.

10 years agoWrite also number of jobs processed.
František Dvořák [Thu, 5 Mar 2015 16:47:58 +0000 (17:47 +0100)]
Write also number of jobs processed.

10 years agoThe first real PBS logs.
František Dvořák [Thu, 5 Mar 2015 16:33:37 +0000 (17:33 +0100)]
The first real PBS logs.

10 years agoAdd job queue column.
František Dvořák [Thu, 5 Mar 2015 12:55:20 +0000 (13:55 +0100)]
Add job queue column.

10 years agoAdd more detailed subjobs table.
František Dvořák [Thu, 5 Mar 2015 12:14:37 +0000 (13:14 +0100)]
Add more detailed subjobs table.

10 years agoFetch details, of not found in the database.
František Dvořák [Thu, 5 Mar 2015 11:46:47 +0000 (12:46 +0100)]
Fetch details, of not found in the database.

10 years agoMove node hostnames to separated table.
František Dvořák [Thu, 5 Mar 2015 11:42:14 +0000 (12:42 +0100)]
Move node hostnames to separated table.

10 years agoInitial database consumer.
František Dvořák [Wed, 4 Mar 2015 23:04:47 +0000 (00:04 +0100)]
Initial database consumer.

10 years agoExample database initialization.
František Dvořák [Wed, 4 Mar 2015 23:04:23 +0000 (00:04 +0100)]
Example database initialization.

10 years agoSet timestamps more precicely on the end of the long transactions.
František Dvořák [Wed, 4 Mar 2015 23:00:29 +0000 (00:00 +0100)]
Set timestamps more precicely on the end of the long transactions.

10 years agoUse all database call properly. Add name also to new jobs.
František Dvořák [Wed, 4 Mar 2015 22:26:15 +0000 (23:26 +0100)]
Use all database call properly. Add name also to new jobs.

10 years agoAdded name column. Use rather symbolic constans in the code, proper SQL escaping.
František Dvořák [Wed, 4 Mar 2015 22:03:18 +0000 (23:03 +0100)]
Added name column. Use rather symbolic constans in the code, proper SQL escaping.

10 years agoGet node details (mapred tasks and attempts) only when job is updated in DB or when...
František Dvořák [Wed, 4 Mar 2015 20:57:27 +0000 (21:57 +0100)]
Get node details (mapred tasks and attempts) only when job is updated in DB or when printing.

10 years agoFine tunging of verbosity.
František Dvořák [Wed, 4 Mar 2015 20:37:52 +0000 (21:37 +0100)]
Fine tunging of verbosity.

10 years agoFix inserting new jobs.
František Dvořák [Wed, 4 Mar 2015 20:00:53 +0000 (21:00 +0100)]
Fix inserting new jobs.

10 years agoSupport database.
František Dvořák [Wed, 4 Mar 2015 16:40:16 +0000 (17:40 +0100)]
Support database.

10 years agoDatabase schema.
František Dvořák [Wed, 4 Mar 2015 14:41:48 +0000 (15:41 +0100)]
Database schema.

10 years agoFeatures: config file, single job query.
František Dvořák [Wed, 4 Mar 2015 14:31:08 +0000 (15:31 +0100)]
Features: config file, single job query.

10 years agoInitial version of jobs.py.
František Dvořák [Tue, 3 Mar 2015 16:05:24 +0000 (17:05 +0100)]
Initial version of jobs.py.

10 years agoBasic info.
František Dvořák [Tue, 3 Mar 2015 16:05:16 +0000 (17:05 +0100)]
Basic info.