Trace number 3321842

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-random? (MO) 8747.32 1246.41

General information on the benchmark

Namerandom/large/
unif-k7-r85-v400-c34000-S943787219-031-UNKNOWN.cnf
MD5SUM6105ec111e1e16da586afadcc6f09041
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables400
Number of clauses34000
Sum of the clauses size238000
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 534000

Solver Data 

0.35/0.68	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.35/0.68	c INFO: solving HOME/instance-3321842-1304368182.cnf
0.35/0.68	c INFO: computing task features
0.63/0.91	c INFO: computed variable-clause graph statistics
0.63/0.92	c INFO: computed clause balance statistics
1.84/2.11	c INFO: computed variable balance statistics
2.23/2.59	c INFO: computed small-clause counts
2.33/2.60	c INFO: computed Horn-clause counts
2.33/2.64	c INFO: computed variable graph statistics
2.33/2.64	c INFO: collected features for HOME/instance-3321842-1304368182.cnf in 1.96 s
2.33/2.68	c INFO: running gnovelty+2@0 for 97 with 39998 remaining (b = 0.38)
2.43/2.71	c INFO: running gnovelty+2@0 for 97 with 39997 remaining (b = 0.13)
2.43/2.75	c INFO: running adaptg2wsat2009++@0 for 97 with 39997 remaining (b = 0.08)
2.43/2.79	c INFO: running adaptg2wsat2009++@0 for 97 with 39997 remaining (b = 0.07)
2.52/2.83	c INFO: running adaptg2wsat2009++@0 for 97 with 39997 remaining (b = 0.05)
2.52/2.87	c INFO: running gnovelty+2@0 for 97 with 39997 remaining (b = 0.04)
2.62/2.91	c INFO: running FH@0 for 97 with 39997 remaining (b = 0.03)
2.62/2.95	c INFO: running adaptg2wsat2009++@0 for 97 with 39997 remaining (b = 0.03)
712.15/102.03	c INFO: running adaptg2wsat2009++@0 for 97 with 39898 remaining (b = 0.02)
718.41/102.98	c INFO: running FH@0 for 97 with 39897 remaining (b = 0.02)
719.79/103.12	c INFO: running hybridGM3@0 for 194 with 39897 remaining (b = 0.04)
720.35/103.24	c INFO: running gnovelty+2@0 for 97 with 39897 remaining (b = 0.02)
720.35/103.30	c INFO: running hybridGM3@0 for 194 with 39897 remaining (b = 0.03)
838.57/120.14	c INFO: running adaptg2wsat2009++@0 for 97 with 39880 remaining (b = 0.01)
886.73/127.07	c INFO: running hybridGM3@0 for 194 with 39873 remaining (b = 0.03)
1098.41/157.35	c INFO: running hybridGM3@0 for 194 with 39843 remaining (b = 0.02)
1418.57/203.27	c INFO: running TNM@0 for 1552 with 39797 remaining (b = 0.18)
1546.89/221.68	c INFO: running FH@0 for 97 with 39779 remaining (b = 0.01)
1579.60/226.38	c INFO: running adaptg2wsat2009++@0 for 97 with 39774 remaining (b = 0.01)
1580.30/226.45	c INFO: running adaptg2wsat2009++@0 for 97 with 39774 remaining (b = 0.01)
2246.19/321.98	c INFO: running adaptg2wsat2009++@0 for 97 with 39678 remaining (b = 0.01)
2258.02/323.67	c INFO: running adaptg2wsat2009++@99 for 97 with 39677 remaining (b = 0.01)
2260.84/324.02	c INFO: running hybridGM3@97 for 97 with 39676 remaining (b = 0.01)
2391.26/341.79	c INFO: running hybridGM3@101 for 97 with 39658 remaining (b = 0.01)
2436.73/347.84	c INFO: running hybridGM3@100 for 97 with 39652 remaining (b = 0.01)
2616.15/371.86	c INFO: running hybridGM3@0 for 194 with 39628 remaining (b = 0.02)
2707.21/384.08	c INFO: running hybridGM3@100 for 97 with 39616 remaining (b = 0.01)
3003.76/425.31	c INFO: running hybridGM3@294 for 97 with 39575 remaining (b = 0.01)
3044.98/431.30	c INFO: running hybridGM3@0 for 194 with 39569 remaining (b = 0.02)
3085.43/437.08	c INFO: running hybridGM3@299 for 97 with 39563 remaining (b = 0.01)
3127.96/443.12	c INFO: running TNM@0 for 1552 with 39557 remaining (b = 0.14)
3266.77/463.06	c INFO: running adaptg2wsat2009++@100 for 97 with 39537 remaining (b = 0.01)
3506.49/497.42	c INFO: running FH@0 for 291 with 39503 remaining (b = 0.03)
3751.84/532.64	c INFO: running gnovelty+2@0 for 97 with 39468 remaining (b = 0.01)
3917.67/556.44	c INFO: running hybridGM3@387 for 97 with 39444 remaining (b = 0.01)
4071.91/578.43	c INFO: running hybridGM3@394 for 97 with 39422 remaining (b = 0.01)
4072.67/578.58	c INFO: running adaptg2wsat2009++@294 for 291 with 39422 remaining (b = 0.02)
4536.92/645.14	c INFO: running iPAWS@0 for 97 with 39355 remaining (b = 0.01)
4638.57/659.78	c INFO: running FH@99 for 291 with 39340 remaining (b = 0.02)
4749.97/675.75	c INFO: running iPAWS@0 for 97 with 39325 remaining (b = 0.01)
4835.62/688.09	c INFO: running hybridGM3@94 for 97 with 39312 remaining (b = 0.01)
5235.27/745.47	c INFO: running hybridGM3@0 for 194 with 39255 remaining (b = 0.02)
5532.54/788.11	c INFO: running hybridGM3@0 for 194 with 39212 remaining (b = 0.01)
5651.73/805.23	c INFO: running TNM@0 for 1552 with 39195 remaining (b = 0.11)
5750.63/819.50	c INFO: running hybridGM3@192 for 97 with 39181 remaining (b = 0.01)
6483.20/920.83	c INFO: running adaptg2wsat2009++@102 for 97 with 39079 remaining (b = 0.01)
6724.80/955.57	c INFO: running FH@0 for 97 with 39045 remaining (b = 0.01)
6787.41/964.58	c INFO: running adaptg2wsat2009++@392 for 194 with 39036 remaining (b = 0.01)
6840.99/972.29	c INFO: running hybridGM3@100 for 97 with 39028 remaining (b = 0.01)
7285.81/1036.21	c INFO: running adaptg2wsat2009++@390 for 194 with 38964 remaining (b = 0.01)
7305.25/1038.93	c INFO: running iPAWS@0 for 97 with 38961 remaining (b = 0.01)
7507.08/1068.00	c INFO: running FH@294 for 97 with 38932 remaining (b = 0.01)
7671.98/1091.67	c INFO: running hybridGM3@192 for 97 with 38909 remaining (b = 0.01)
8162.62/1162.14	c INFO: running gnovelty+2@0 for 97 with 38838 remaining (b = 0.01)
8254.45/1175.37	c INFO: running hybridGM3@0 for 194 with 38825 remaining (b = 0.01)
8255.14/1175.48	c INFO: running NCVWr@0 for 485 with 38825 remaining (b = 0.03)
8460.84/1205.06	c INFO: running hybridGM3@386 for 194 with 38795 remaining (b = 0.01)
8744.42/1245.84	c INFO: running hybridGM3@0 for 194 with 38754 remaining (b = 0.01)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3321842-1304368182/watcher-3321842-1304368182 -o /tmp/evaluation-result-3321842-1304368182/solver-3321842-1304368182 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3321842-1304368182.cnf -seed 1828957852 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.79 8.17 5.55 3/292 13190
/proc/meminfo: memFree=18172460/32951124 swapFree=67111524/67111528
[pid=13190] ppid=13188 vsize=15764 CPUtime=0 cores=0-7
/proc/13190/stat : 13190 (python) R 13188 13190 12854 0 -1 4202496 849 0 0 0 0 0 0 0 22 0 1 0 529233863 16142336 731 33554432000 4194304 5683644 140734540582464 18446744073709551615 4540219 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/13190/statm: 3941 735 334 364 0 428 0

[startup+0.0546 s]
/proc/loadavg: 8.79 8.17 5.55 3/292 13190
/proc/meminfo: memFree=18172460/32951124 swapFree=67111524/67111528
[pid=13190] ppid=13188 vsize=25288 CPUtime=0.02 cores=0-7
/proc/13190/stat : 13190 (python) S 13188 13190 12854 0 -1 4202496 1347 0 0 0 2 0 0 0 22 0 1 0 529233863 25894912 1192 33554432000 4194304 5683644 140734540582464 18446744073709551615 264721481351 0 2147483391 16781312 2 18446744071699207358 0 0 17 2 0 0 0
/proc/13190/statm: 6836 1194 439 364 0 723 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 25288

[startup+0.10059 s]
/proc/loadavg: 8.79 8.17 5.55 3/292 13190
/proc/meminfo: memFree=18172460/32951124 swapFree=67111524/67111528
[pid=13190] ppid=13188 vsize=87700 CPUtime=0.05 cores=0-7
/proc/13190/stat : 13190 (python) S 13188 13190 12854 0 -1 4202496 2637 0 0 0 4 1 0 0 22 0 1 0 529233863 89804800 2353 33554432000 4194304 5683644 140734540582464 18446744073709551615 264726403072 0 2147483391 16781312 2 18446744071699207358 0 0 17 2 0 0 0
/proc/13190/statm: 21925 2353 717 364 0 1621 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 87700

[startup+0.300584 s]
/proc/loadavg: 8.79 8.17 5.55 3/292 13190
/proc/meminfo: memFree=18172460/32951124 swapFree=67111524/67111528
[pid=13190] ppid=13188 vsize=135748 CPUtime=0.17 cores=0-7
/proc/13190/stat : 13190 (python) R 13188 13190 12854 0 -1 4202496 5964 932 1 0 13 4 0 0 17 0 1 0 529233863 139005952 4151 33554432000 4194304 5683644 140734540582464 18446744073709551615 264726401029 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/13190/statm: 33937 4151 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.17
Current children cumulated vsize (KiB) 135748

[startup+0.700478 s]
/proc/loadavg: 8.79 8.17 5.55 3/292 13190
/proc/meminfo: memFree=18172460/32951124 swapFree=67111524/67111528
[pid=13190] ppid=13188 vsize=280028 CPUtime=0.43 cores=0-7
/proc/13190/stat : 13190 (python) R 13188 13190 12854 0 -1 4202496 9650 932 1 0 31 12 0 0 16 0 1 0 529233863 286748672 7331 33554432000 4194304 5683644 140734540582464 18446744073709551615 4602375 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/13190/statm: 70007 7331 1761 364 0 5931 0
Current children cumulated CPU time (s) 0.43
Current children cumulated vsize (KiB) 280028

[startup+1.50033 s]
/proc/loadavg: 8.25 8.07 5.53 3/293 13195
/proc/meminfo: memFree=18139716/32951124 swapFree=67111524/67111528
[pid=13190] ppid=13188 vsize=289496 CPUtime=1.23 cores=0-7
/proc/13190/stat : 13190 (python) R 13188 13190 12854 0 -1 4202496 13883 932 1 0 111 12 0 0 17 0 1 0 529233863 296443904 9759 33554432000 4194304 5683644 140734540582464 18446744073709551615 46938079978514 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/13190/statm: 72374 9759 1826 364 0 8298 0
Current children cumulated CPU time (s) 1.23
Current children cumulated vsize (KiB) 289496

[startup+3.10099 s]
/proc/loadavg: 8.25 8.07 5.53 3/293 13195
/proc/meminfo: memFree=18139220/32951124 swapFree=67111524/67111528
[pid=13190] ppid=13188 vsize=364424 CPUtime=2.68 cores=0-7
/proc/13190/stat : 13190 (python) S 13188 13190 12854 0 -1 4202496 20371 932 1 0 254 14 0 0 24 0 9 0 529233863 373170176 8079 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738231755 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/13190/statm: 91106 8079 1891 364 0 27003 0
Current children cumulated CPU time (s) 2.68
Current children cumulated vsize (KiB) 364424

[startup+6.30138 s]
/proc/loadavg: 8.31 8.08 5.55 10/317 13219
/proc/meminfo: memFree=18075664/32951124 swapFree=67111524/67111528
[pid=13190] ppid=13188 vsize=364424 CPUtime=2.68 cores=0-7
/proc/13190/stat : 13190 (python) S 13188 13190 12854 0 -1 4202496 20371 932 1 0 254 14 0 0 24 0 9 0 529233863 373170176 8079 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738231755 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/13190/statm: 91106 8079 1891 364 0 27003 0
[pid=13196] ppid=13190 vsize=282240 CPUtime=0 cores=0-7
/proc/13196/stat : 13196 (python) S 13190 13190 12854 0 -1 4202560 730 0 0 0 0 0 0 0 18 0 1 0 529234131 289013760 6331 33554432000 4194304 5683644 140734540582464 18446744073709551615 264726425103 0 0 16781312 514 1078519756073435396 0 0 17 0 0 0 0
/proc/13196/statm: 70560 6331 355 364 0 6478 0
[pid=13198] ppid=13196 vsize=1020188 CPUtime=3.61 cores=0-7
/proc/13198/stat : 13198 (gnovelty+2) R 13196 13198 13198 0 -1 4202496 1138 0 0 0 361 0 0 0 25 0 1 0 529234131 1044672512 877 33554432000 134512640 135663704 4287029968 18446744073709551615 134525255 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/13198/statm: 255047 877 113 282 0 254763 0
[pid=13199] ppid=13190 vsize=293128 CPUtime=0 cores=0-7
/proc/13199/stat : 13199 (python) S 13190 13190 12854 0 -1 4202560 739 0 0 0 0 0 0 0 18 0 1 0 529234135 300163072 6369 33554432000 4194304 5683644 140734540582464 18446744073709551615 264726425158 0 0 16781312 514 17114021086174871810 0 0 17 7 0 0 0
/proc/13199/statm: 73282 6369 349 364 0 9197 0
[pid=13201] ppid=13199 vsize=1020188 CPUtime=3.57 cores=0-7
/proc/13201/stat : 13201 (gnovelty+2) R 13199 13201 13201 0 -1 4202496 1138 0 0 0 357 0 0 0 25 0 1 0 529234135 1044672512 876 33554432000 134512640 135663704 4289437008 18446744073709551615 134525544 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/13201/statm: 255047 876 113 282 0 254763 0
[pid=13202] ppid=13190 vsize=302900 CPUtime=0 cores=0-7
/proc/13202/stat : 13202 (python) S 13190 13190 12854 0 -1 4202560 734 0 0 0 0 0 0 0 18 0 1 0 529234138 310169600 6505 33554432000 4194304 5683644 140734540582464 18446744073709551615 264726425158 0 0 16781312 514 4443971191945789698 0 0 17 2 0 0 0
/proc/13202/statm: 75725 6505 347 364 0 11637 0
[pid=13204] ppid=13202 vsize=421140 CPUtime=3.49 cores=0-7
/proc/13204/stat : 13204 (adaptg2wsat2009) R 13202 13204 13204 0 -1 4202496 1804 0 0 0 349 0 0 0 25 0 1 0 529234139 431247360 1539 33554432000 134512640 135107411 4289511184 18446744073709551615 134523424 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/13204/statm: 105285 1539 49 146 0 104161 0
[pid=13205] ppid=13190 vsize=313752 CPUtime=0 cores=0-7
/proc/13205/stat : 13205 (python) S 13190 13190 12854 0 -1 4202560 748 0 0 0 0 0 0 0 18 0 1 0 529234142 321282048 6512 33554432000 4194304 5683644 140734540582464 18446744073709551615 264726425158 0 0 16781312 514 17114017237884174595 0 0 17 0 0 0 0
/proc/13205/statm: 78438 6512 349 364 0 14347 0
[pid=13207] ppid=13205 vsize=421140 CPUtime=3.49 cores=0-7
/proc/13207/stat : 13207 (adaptg2wsat2009) R 13205 13207 13207 0 -1 4202496 1802 0 0 0 348 1 0 0 25 0 1 0 529234143 431247360 1538 33554432000 134512640 135107411 4292523872 18446744073709551615 134523317 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/13207/statm: 105285 1538 49 146 0 104161 0
[pid=13208] ppid=13190 vsize=323920 CPUtime=0 cores=0-7
/proc/13208/stat : 13208 (python) S 13190 13190 12854 0 -1 4202560 754 0 0 0 0 0 0 0 18 0 1 0 529234146 331694080 6516 33554432000 4194304 5683644 140734540582464 18446744073709551615 264726425158 0 0 16781312 514 18143966063505670404 0 0 17 0 0 0 0
/proc/13208/statm: 80980 6516 349 364 0 16886 0
[pid=13210] ppid=13208 vsize=421140 CPUtime=3.44 cores=0-7
/proc/13210/stat : 13210 (adaptg2wsat2009) R 13208 13210 13210 0 -1 4202496 1802 0 0 0 344 0 0 0 25 0 1 0 529234147 431247360 1538 33554432000 134512640 135107411 4291454496 18446744073709551615 134523452 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/13210/statm: 105285 1538 49 146 0 104161 0
[pid=13211] ppid=13190 vsize=333668 CPUtime=0 cores=0-7
/proc/13211/stat : 13211 (python) S 13190 13190 12854 0 -1 4202560 759 0 0 0 0 0 0 0 18 0 1 0 529234150 341676032 6524 33554432000 4194304 5683644 140734540582464 18446744073709551615 264726425158 0 0 16781312 514 11242487981484769538 0 0 17 6 0 0 0
/proc/13211/statm: 83417 6524 347 364 0 19320 0
[pid=13213] ppid=13211 vsize=1020188 CPUtime=3.4 cores=0-7
/proc/13213/stat : 13213 (gnovelty+2) R 13211 13213 13213 0 -1 4202496 1141 0 0 0 340 0 0 0 25 0 1 0 529234151 1044672512 876 33554432000 134512640 135663704 4289391488 18446744073709551615 134525286 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/13213/statm: 255047 876 113 282 0 254763 0
[pid=13214] ppid=13190 vsize=343924 CPUtime=0 cores=0-7
/proc/13214/stat : 13214 (python) S 13190 13190 12854 0 -1 4202560 765 0 0 0 0 0 0 0 18 0 1 0 529234154 352178176 6527 33554432000 4194304 5683644 140734540582464 18446744073709551615 264726425158 0 0 16781312 514 4343910960893755650 0 0 17 0 0 0 0
/proc/13214/statm: 85981 6527 347 364 0 21881 0
[pid=13216] ppid=13214 vsize=249144 CPUtime=3.33 cores=0-7
/proc/13216/stat : 13216 (FH) R 13214 13216 13216 0 -1 4202496 1558 0 0 0 333 0 0 0 25 0 1 0 529234157 255123456 1296 33554432000 134512640 134936988 4287268576 18446744073709551615 134523505 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/13216/statm: 62286 1296 37 104 0 61790 0
[pid=13217] ppid=13190 vsize=354788 CPUtime=0 cores=0-7
/proc/13217/stat : 13217 (python) S 13190 13190 12854 0 -1 4202560 797 0 0 0 0 0 0 0 18 0 1 0 529234159 363302912 6540 33554432000 4194304 5683644 140734540582464 18446744073709551615 264726425158 0 0 16781312 514 12942520424759001347 0 0 17 0 0 0 0
/proc/13217/statm: 88697 6540 355 364 0 24594 0
[pid=13219] ppid=13217 vsize=421140 CPUtime=1.66 cores=0-7
/proc/13219/stat : 13219 (adaptg2wsat2009) R 13217 13219 13219 0 -1 4202496 1808 0 0 0 166 0 0 0 25 0 1 0 529234159 431247360 1538 33554432000 134512640 135107411 4293036512 18446744073709551615 134523232 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/13219/statm: 105285 1538 49 146 0 104161 0
[pid=13190/tid=13197] ppid=13188 vsize=364424 CPUtime=0 cores=0-7
/proc/13190/task/13197/stat : 13197 (python) S 13188 13190 12854 0 -1 4202560 26 932 0 0 0 0 0 0 20 0 9 0 529234131 373170176 8079 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 6 0 0 0
[pid=13190/tid=13200] ppid=13188 vsize=364424 CPUtime=0 cores=0-7
/proc/13190/task/13200/stat : 13200 (python) S 13188 13190 12854 0 -1 4202560 24 932 0 0 0 0 0 0 19 0 9 0 529234135 373170176 8079 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 0 0 0 0
[pid=13190/tid=13203] ppid=13188 vsize=364424 CPUtime=0 cores=0-7
/proc/13190/task/13203/stat : 13203 (python) S 13188 13190 12854 0 -1 4202560 23 932 0 0 0 0 0 0 21 0 9 0 529234139 373170176 8079 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 6 0 0 0
[pid=13190/tid=13206] ppid=13188 vsize=364424 CPUtime=0 cores=0-7
/proc/13190/task/13206/stat : 13206 (python) S 13188 13190 12854 0 -1 4202560 24 932 0 0 0 0 0 0 23 0 9 0 529234143 373170176 8079 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 0 0 0 0
[pid=13190/tid=13209] ppid=13188 vsize=364424 CPUtime=0 cores=0-7
/proc/13190/task/13209/stat : 13209 (python) S 13188 13190 12854 0 -1 4202560 26 932 0 0 0 0 0 0 20 0 9 0 529234146 373170176 8079 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 0 0 0 0
[pid=13190/tid=13212] ppid=13188 vsize=364424 CPUtime=0 cores=0-7
/proc/13190/task/13212/stat : 13212 (python) S 13188 13190 12854 0 -1 4202560 28 932 0 0 0 0 0 0 20 0 9 0 529234150 373170176 8079 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 6 0 0 0
[pid=13190/tid=13215] ppid=13188 vsize=364424 CPUtime=0 cores=0-7
/proc/13190/task/13215/stat : 13215 (python) S 13188 13190 12854 0 -1 4202560 26 932 0 0 0 0 0 0 22 0 9 0 529234155 373170176 8079 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 0 0 0 0
[pid=13190/tid=13218] ppid=13188 vsize=364424 CPUtime=0 cores=0-7
/proc/13190/task/13218/stat : 13218 (python) S 13188 13190 12854 0 -1 4202560 81 932 0 0 0 0 0 0 25 0 9 0 529234159 373170176 8079 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 28.67
Current children cumulated vsize (KiB) 7907012

[startup+12.7012 s]
/proc/loadavg: 8.36 8.10 5.57 10/317 13219
/proc/meminfo: memFree=18069124/32951124 swapFree=67111524/67111528
[pid=13190] ppid=13188 vsize=364424 CPUtime=2.68 cores=0-7
/proc/13190/stat : 13190 (python) S 13188 13190 12854 0 -1 4202496 20371 932 1 0 254 14 0 0 24 0 9 0 529233863 373170176 8079 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738231755 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/13190/statm: 91106 8079 1891 364 0 27003 0
[pid=13196] ppid=13190 vsize=282240 CPUtime=0 cores=0-7
/proc/13196/stat : 13196 (python) S 13190 13190 12854 0 -1 4202560 892 0 0 0 0 0 0 0 18 0 1 0 529234131 289013760 6354 33554432000 4194304 5683644 140734540582464 18446744073709551615 264726425103 0 0 16781312 514 1078519756073435396 0 0 17 0 0 0 0
/proc/13196/statm: 70560 6354 378 364 0 6478 0
[pid=13198] ppid=13196 vsize=1020188 CPUtime=10.01 cores=0-7
/proc/13198/stat : 13198 (gnovelty+2) R 13196 13198 13198 0 -1 4202496 1138 0 0 0 1001 0 0 0 25 0 1 0 529234131 1044672512 877 33554432000 134512640 135663704 4287029968 18446744073709551615 134524918 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/13198/statm: 255047 877 113 282 0 254763 0
[pid=13199] ppid=13190 vsize=293128 CPUtime=0 cores=0-7
/proc/13199/stat : 13199 (python) S 13190 13190 12854 0 -1 4202560 911 0 0 0 0 0 0 0 18 0 1 0 529234135 300163072 6392 33554432000 4194304 5683644 140734540582464 18446744073709551615 264726425158 0 0 16781312 514 17114021086174871810 0 0 17 7 0 0 0
/proc/13199/statm: 73282 6392 372 364 0 9197 0
[pid=13201] ppid=13199 vsize=1020188 CPUtime=9.97 cores=0-7
/proc/13201/stat : 13201 (gnovelty+2) R 13199 13201 13201 0 -1 4202496 1138 0 0 0 997 0 0 0 25 0 1 0 529234135 1044672512 876 33554432000 134512640 135663704 4289437008 18446744073709551615 134544252 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/13201/statm: 255047 876 113 282 0 254763 0
[pid=13202] ppid=13190 vsize=302900 CPUtime=0 cores=0-7
/proc/13202/stat : 13202 (python) S 13190 13190 12854 0 -1 4202560 890 0 0 0 0 0 0 0 18 0 1 0 529234138 310169600 6528 33554432000 4194304 5683644 140734540582464 18446744073709551615 264726425158 0 0 16781312 514 4443971191945789698 0 0 17 2 0 0 0
/proc/13202/statm: 75725 6528 370 364 0 11637 0
[pid=13204] ppid=13202 vsize=421804 CPUtime=9.81 cores=0-7
/proc/13204/stat : 13204 (adaptg2wsat2009) R 13202 13204 13204 0 -1 4202496 2137 0 0 0 981 0 0 0 25 0 1 0 529234139 431927296 1872 33554432000 134512640 135107411 4289511184 18446744073709551615 134532313 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/13204/statm: 105451 1872 59 146 0 104327 0
[pid=13205] ppid=13190 vsize=313752 CPUtime=0 cores=0-7
/proc/13205/stat : 13205 (python) S 13190 13190 12854 0 -1 4202560 929 0 0 0 0 0 0 0 18 0 1 0 529234142 321282048 6535 33554432000 4194304 5683644 140734540582464 18446744073709551615 264726425158 0 0 16781312 514 17114017237884174595 0 0 17 0 0 0 0
/proc/13205/statm: 78438 6535 372 364 0 14347 0
[pid=13207] ppid=13205 vsize=421804 CPUtime=9.87 cores=0-7
/proc/13207/stat : 13207 (adaptg2wsat2009) R 13205 13207 13207 0 -1 4202496 2135 0 0 0 986 1 0 0 25 0 1 0 529234143 431927296 1871 33554432000 134512640 135107411 4292523872 18446744073709551615 134531435 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/13207/statm: 105451 1871 59 146 0 104327 0
[pid=13208] ppid=13190 vsize=323920 CPUtime=0 cores=0-7
/proc/13208/stat : 13208 (python) S 13190 13190 12854 0 -1 4202560 947 0 0 0 0 0 0 0 18 0 1 0 529234146 331694080 6539 33554432000 4194304 5683644 140734540582464 18446744073709551615 264726425158 0 0 16781312 514 18143966063505670404 0 0 17 0 0 0 0
/proc/13208/statm: 80980 6539 372 364 0 16886 0
[pid=13210] ppid=13208 vsize=421804 CPUtime=9.84 cores=0-7
/proc/13210/stat : 13210 (adaptg2wsat2009) R 13208 13210 13210 0 -1 4202496 2136 0 0 0 984 0 0 0 25 0 1 0 529234147 431927296 1872 33554432000 134512640 135107411 4291454496 18446744073709551615 134531709 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/13210/statm: 105451 1872 59 146 0 104327 0
[pid=13211] ppid=13190 vsize=333668 CPUtime=0 cores=0-7
/proc/13211/stat : 13211 (python) S 13190 13190 12854 0 -1 4202560 965 0 0 0 0 0 0 0 18 0 1 0 529234150 341676032 6547 33554432000 4194304 5683644 140734540582464 18446744073709551615 264726425158 0 0 16781312 514 11242487981484769538 0 0 17 6 0 0 0
/proc/13211/statm: 83417 6547 370 364 0 19320 0
[pid=13213] ppid=13211 vsize=1020188 CPUtime=9.78 cores=0-7
/proc/13213/stat : 13213 (gnovelty+2) R 13211 13213 13213 0 -1 4202496 1141 0 0 0 978 0 0 0 25 0 1 0 529234151 1044672512 876 33554432000 134512640 135663704 4289391488 18446744073709551615 134525309 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/13213/statm: 255047 876 113 282 0 254763 0
[pid=13214] ppid=13190 vsize=343924 CPUtime=0 cores=0-7
/proc/13214/stat : 13214 (python) S 13190 13190 12854 0 -1 4202560 969 0 0 0 0 0 0 0 18 0 1 0 529234154 352178176 6550 33554432000 4194304 5683644 140734540582464 18446744073709551615 264726425158 0 0 16781312 514 4343910960893755650 0 0 17 0 0 0 0
/proc/13214/statm: 85981 6550 370 364 0 21881 0

################
# More data... #
################

[pid=13292] ppid=13290 vsize=1020188 CPUtime=98.09 cores=0-7
/proc/13292/stat : 13292 (gnovelty+2) T 13290 13292 13292 0 -1 4202496 1153 0 0 0 9809 0 0 0 25 0 1 0 529287128 1044672512 876 33554432000 134512640 135663704 4288144480 18446744073709551615 134525258 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/13292/statm: 255047 876 113 282 0 254763 0
[pid=13290/tid=13297] ppid=13190 vsize=563864 CPUtime=0 cores=0-7
/proc/13290/task/13297/stat : 13297 (python) S 13190 13190 12854 0 -1 4202560 105 0 0 0 0 0 0 0 19 0 2 0 529298369 577396736 7111 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 514 18446604453939091456 0 0 -1 2 0 0 0
[pid=13298] ppid=13190 vsize=574120 CPUtime=0.25 cores=0-7
/proc/13298/stat : 13298 (python) S 13190 13190 12854 0 -1 4202560 1182 0 0 0 15 10 0 0 19 0 2 0 529298378 587898880 7084 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738231755 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/13298/statm: 143530 7084 407 364 0 79382 0
[pid=13300] ppid=13298 vsize=335996 CPUtime=97.3 cores=0-7
/proc/13300/stat : 13300 (iPAWS) T 13298 13300 13300 0 -1 4202496 2031 0 0 0 9730 0 0 0 25 0 1 0 529298378 344059904 1751 33554432000 134512640 135114423 4291593552 18446744073709551615 134546160 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/13300/statm: 83999 1751 59 147 0 83851 0
[pid=13298/tid=13308] ppid=13190 vsize=574120 CPUtime=0 cores=0-7
/proc/13298/task/13308/stat : 13308 (python) S 13190 13190 12854 0 -1 4202560 99 0 0 0 0 0 0 0 19 0 2 0 529308402 587898880 7084 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 514 18446604440278902912 0 0 -1 2 0 0 0
[pid=13302] ppid=13190 vsize=584376 CPUtime=0.28 cores=0-7
/proc/13302/stat : 13302 (python) S 13190 13190 12854 0 -1 4202560 1204 0 0 0 14 14 0 0 19 0 2 0 529301438 598401024 7213 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738231755 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/13302/statm: 146094 7213 406 364 0 81943 0
[pid=13304] ppid=13302 vsize=335996 CPUtime=100.21 cores=0-7
/proc/13304/stat : 13304 (iPAWS) T 13302 13304 13304 0 -1 4202496 2031 0 0 0 10021 0 0 0 25 0 1 0 529301439 344059904 1752 33554432000 134512640 135114423 4291655040 18446744073709551615 134514774 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/13304/statm: 83999 1752 59 147 0 83851 0
[pid=13302/tid=13313] ppid=13190 vsize=584376 CPUtime=0 cores=0-7
/proc/13302/task/13313/stat : 13313 (python) S 13190 13190 12854 0 -1 4202560 108 0 0 0 0 0 0 0 19 0 2 0 529312665 598401024 7213 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 514 18446604440278902016 0 0 -1 0 0 0 0
[pid=13309] ppid=13190 vsize=595260 CPUtime=1.27 cores=0-7
/proc/13309/stat : 13309 (python) S 13190 13190 12854 0 -1 4202560 1304 0 0 0 63 64 0 0 18 0 2 0 529308411 609546240 7107 33554432000 4194304 5683644 140734540582464 18446744073709551615 264726425158 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/13309/statm: 148815 7107 401 364 0 84661 0
[pid=13311] ppid=13309 vsize=268424 CPUtime=419.85 cores=0-7
/proc/13311/stat : 13311 (hybridGM3) R 13309 13311 13311 0 -1 4202496 5668 0 0 0 41984 1 0 0 25 0 1 0 529308411 274866176 5378 33554432000 134512640 135204385 4290422848 18446744073709551615 134537792 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/13311/statm: 67106 5378 63 169 0 66936 0
[pid=13309/tid=13328] ppid=13190 vsize=595260 CPUtime=0 cores=0-7
/proc/13309/task/13328/stat : 13328 (python) S 13190 13190 12854 0 -1 4202560 48 0 0 0 0 0 0 0 18 0 2 0 529331082 609546240 7107 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 514 18446604447397303296 0 0 -1 1 0 0 0
[pid=13314] ppid=13190 vsize=605020 CPUtime=0.63 cores=0-7
/proc/13314/stat : 13314 (python) S 13190 13190 12854 0 -1 4202560 1291 0 0 0 32 31 0 0 19 0 2 0 529312675 619540480 7110 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738231755 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/13314/statm: 151255 7110 401 364 0 87098 0
[pid=13316] ppid=13314 vsize=268424 CPUtime=195.67 cores=0-7
/proc/13316/stat : 13316 (hybridGM3) T 13314 13316 13316 0 -1 4202496 5659 0 0 0 19566 1 0 0 25 0 1 0 529312675 274866176 5378 33554432000 134512640 135204385 4286919840 18446744073709551615 134537792 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/13316/statm: 67106 5378 63 169 0 66936 0
[pid=13314/tid=13331] ppid=13190 vsize=605020 CPUtime=0 cores=0-7
/proc/13314/task/13331/stat : 13331 (python) S 13190 13190 12854 0 -1 4202560 211 0 0 0 0 0 0 0 19 0 2 0 529337747 619540480 7110 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 514 18446604451483599872 0 0 -1 0 0 0 0
[pid=13317] ppid=13190 vsize=615276 CPUtime=1.1 cores=0-7
/proc/13317/stat : 13317 (python) S 13190 13190 12854 0 -1 4202560 973 0 0 0 59 51 0 0 18 0 1 0 529314387 630042624 7157 33554432000 4194304 5683644 140734540582464 18446744073709551615 264726425158 0 0 16781312 514 11279177305819152641 0 0 17 1 0 0 0
/proc/13317/statm: 153819 7157 372 364 0 89659 0
[pid=13319] ppid=13317 vsize=459764 CPUtime=365.62 cores=0-7
/proc/13319/stat : 13319 (TNM) R 13317 13319 13319 0 -1 4202496 1940 0 0 0 36562 0 0 0 25 0 1 0 529314390 470798336 1657 33554432000 134512640 135008748 4294881360 18446744073709551615 134529176 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/13319/statm: 114941 1657 46 122 0 113841 0
[pid=13324] ppid=13190 vsize=625532 CPUtime=0.56 cores=0-7
/proc/13324/stat : 13324 (python) S 13190 13190 12854 0 -1 4202560 1217 0 0 0 30 26 0 0 18 0 2 0 529329420 640544768 7227 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738231755 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/13324/statm: 156383 7227 401 364 0 92220 0
[pid=13326] ppid=13324 vsize=251296 CPUtime=195.01 cores=0-7
/proc/13326/stat : 13326 (FH) T 13324 13326 13326 0 -1 4202496 1980 0 0 0 19501 0 0 0 25 0 1 0 529329426 257327104 1694 33554432000 134512640 134936988 4289676768 18446744073709551615 134545320 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/13326/statm: 62824 1694 41 104 0 62328 0
[pid=13324/tid=13335] ppid=13190 vsize=625532 CPUtime=0 cores=0-7
/proc/13324/task/13335/stat : 13335 (python) S 13190 13190 12854 0 -1 4202560 117 0 0 0 0 0 0 0 18 0 2 0 529340653 640544768 7227 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 514 18446604453915157696 0 0 -1 0 0 0 0
[pid=13332] ppid=13190 vsize=635788 CPUtime=0.34 cores=0-7
/proc/13332/stat : 13332 (python) S 13190 13190 12854 0 -1 4202560 1251 0 0 0 17 17 0 0 19 0 2 0 529337757 651046912 7184 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738231755 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/13332/statm: 158947 7184 406 364 0 94781 0
[pid=13334] ppid=13332 vsize=335996 CPUtime=99.89 cores=0-7
/proc/13334/stat : 13334 (iPAWS) T 13332 13334 13334 0 -1 4202496 2035 0 0 0 9989 0 0 0 25 0 1 0 529337757 344059904 1748 33554432000 134512640 135114423 4286783488 18446744073709551615 134516737 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/13334/statm: 83999 1748 59 147 0 83851 0
[pid=13332/tid=13342] ppid=13190 vsize=635788 CPUtime=0 cores=0-7
/proc/13332/task/13342/stat : 13342 (python) S 13190 13190 12854 0 -1 4202560 116 0 0 0 0 0 0 0 19 0 2 0 529351391 651046912 7184 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 514 18446604453921675264 0 0 -1 0 0 0 0
[pid=13339] ppid=13190 vsize=646044 CPUtime=0.22 cores=0-7
/proc/13339/stat : 13339 (python) S 13190 13190 12854 0 -1 4202560 1024 0 0 0 11 11 0 0 18 0 1 0 529350078 661549056 7276 33554432000 4194304 5683644 140734540582464 18446744073709551615 264726425158 0 0 16781312 514 11279177305819152643 0 0 17 1 0 0 0
/proc/13339/statm: 161511 7276 379 364 0 97342 0
[pid=13341] ppid=13339 vsize=1020188 CPUtime=76.02 cores=0-7
/proc/13341/stat : 13341 (gnovelty+2) R 13339 13341 13341 0 -1 4202496 1161 0 0 0 7602 0 0 0 25 0 1 0 529350078 1044672512 876 33554432000 134512640 135663704 4294032016 18446744073709551615 134524897 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/13341/statm: 255047 876 113 282 0 254763 0
[pid=13343] ppid=13190 vsize=656300 CPUtime=0.19 cores=0-7
/proc/13343/stat : 13343 (python) S 13190 13190 12854 0 -1 4202560 1037 0 0 0 9 10 0 0 18 0 1 0 529351401 672051200 7227 33554432000 4194304 5683644 140734540582464 18446744073709551615 264726425158 0 0 16781312 514 11279177305819152642 0 0 17 1 0 0 0
/proc/13343/statm: 164075 7227 379 364 0 99903 0
[pid=13345] ppid=13343 vsize=268424 CPUtime=49.8 cores=0-7
/proc/13345/stat : 13345 (hybridGM3) R 13343 13345 13345 0 -1 4202496 5670 0 0 0 4980 0 0 0 25 0 1 0 529351403 274866176 5378 33554432000 134512640 135204385 4288050208 18446744073709551615 134537792 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/13345/statm: 67106 5378 63 169 0 66936 0
[pid=13346] ppid=13190 vsize=666556 CPUtime=0.19 cores=0-7
/proc/13346/stat : 13346 (python) S 13190 13190 12854 0 -1 4202560 1048 0 0 0 12 7 0 0 18 0 1 0 529351412 682553344 7322 33554432000 4194304 5683644 140734540582464 18446744073709551615 264726425158 0 0 16781312 514 11279177305819152641 0 0 17 2 0 0 0
/proc/13346/statm: 166639 7322 378 364 0 102464 0
[pid=13348] ppid=13346 vsize=421344 CPUtime=60.08 cores=0-7
/proc/13348/stat : 13348 (NCVWr) R 13346 13348 13348 0 -1 4202496 1997 0 0 0 6008 0 0 0 25 0 1 0 529351414 431456256 1713 33554432000 134512640 135113558 4293038976 18446744073709551615 134534112 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/13348/statm: 105336 1713 58 147 0 104211 0
[pid=13351] ppid=13190 vsize=676812 CPUtime=0 cores=0-7
/proc/13351/stat : 13351 (python) S 13190 13190 12854 0 -1 4202560 936 0 0 0 0 0 0 0 18 0 1 0 529358447 693055488 7277 33554432000 4194304 5683644 140734540582464 18446744073709551615 264726425158 0 0 16781312 514 6451302291776471039 0 0 17 1 0 0 0
/proc/13351/statm: 169203 7277 349 364 0 105025 0
[pid=13353] ppid=13351 vsize=268424 CPUtime=0.2 cores=0-7
/proc/13353/stat : 13353 (hybridGM3) R 13351 13353 13353 0 -1 4202496 5670 0 0 0 20 0 0 0 21 0 1 0 529358448 274866176 5377 33554432000 134512640 135204385 4291419904 18446744073709551615 134538305 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/13353/statm: 67106 5377 63 169 0 66936 0
[pid=13190/tid=13197] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13197/stat : 13197 (python) S 13188 13190 12854 0 -1 4202560 26 932 0 0 0 0 0 0 20 0 34 0 529234131 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 6 0 0 0
[pid=13190/tid=13200] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13200/stat : 13200 (python) S 13188 13190 12854 0 -1 4202560 24 932 0 0 0 0 0 0 19 0 34 0 529234135 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 0 0 0 0
[pid=13190/tid=13203] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13203/stat : 13203 (python) S 13188 13190 12854 0 -1 4202560 27 932 0 0 0 0 0 0 18 0 34 0 529234139 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 6 0 0 0
[pid=13190/tid=13206] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13206/stat : 13206 (python) S 13188 13190 12854 0 -1 4202560 28 932 0 0 0 0 0 0 18 0 34 0 529234143 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 0 0 0 0
[pid=13190/tid=13209] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13209/stat : 13209 (python) S 13188 13190 12854 0 -1 4202560 29 932 0 0 0 0 0 0 18 0 34 0 529234146 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 0 0 0 0
[pid=13190/tid=13212] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13212/stat : 13212 (python) S 13188 13190 12854 0 -1 4202560 28 932 0 0 0 0 0 0 20 0 34 0 529234150 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 6 0 0 0
[pid=13190/tid=13215] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13215/stat : 13215 (python) S 13188 13190 12854 0 -1 4202560 32 932 0 0 0 0 0 0 18 0 34 0 529234155 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 0 0 0 0
[pid=13190/tid=13218] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13218/stat : 13218 (python) S 13188 13190 12854 0 -1 4202560 95 932 0 0 0 0 0 0 18 0 34 0 529234159 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 0 0 0 0
[pid=13190/tid=13225] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13225/stat : 13225 (python) S 13188 13190 12854 0 -1 4202560 39 932 0 0 0 0 0 0 18 0 34 0 529244067 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 3 0 0 0
[pid=13190/tid=13229] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13229/stat : 13229 (python) S 13188 13190 12854 0 -1 4202560 112 932 0 0 0 0 0 0 18 0 34 0 529244162 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 3 0 0 0
[pid=13190/tid=13233] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13233/stat : 13233 (python) S 13188 13190 12854 0 -1 4202560 36 932 0 0 0 0 0 0 18 0 34 0 529244176 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 2 0 0 0
[pid=13190/tid=13238] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13238/stat : 13238 (python) S 13188 13190 12854 0 -1 4202560 32 932 0 0 0 0 0 0 20 0 34 0 529244188 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 0 0 0 0
[pid=13190/tid=13241] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13241/stat : 13241 (python) S 13188 13190 12854 0 -1 4202560 44 932 0 0 0 0 0 0 18 0 34 0 529244193 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 2 0 0 0
[pid=13190/tid=13245] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13245/stat : 13245 (python) S 13188 13190 12854 0 -1 4202560 43 932 0 0 0 0 0 0 18 0 34 0 529245878 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 0 0 0 0
[pid=13190/tid=13249] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13249/stat : 13249 (python) S 13188 13190 12854 0 -1 4202560 41 932 0 0 0 0 0 0 18 0 34 0 529246570 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 0 0 0 0
[pid=13190/tid=13254] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13254/stat : 13254 (python) S 13188 13190 12854 0 -1 4202560 33 932 0 0 0 0 0 0 18 0 34 0 529249599 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 0 0 0 0
[pid=13190/tid=13259] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13259/stat : 13259 (python) S 13188 13190 12854 0 -1 4202560 26 932 0 0 0 0 0 0 19 0 34 0 529254191 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 0 0 0 0
[pid=13190/tid=13264] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13264/stat : 13264 (python) S 13188 13190 12854 0 -1 4202560 31 932 0 0 0 0 0 0 19 0 34 0 529256032 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 0 0 0 0
[pid=13190/tid=13277] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13277/stat : 13277 (python) S 13188 13190 12854 0 -1 4202560 30 932 0 0 0 0 0 0 20 0 34 0 529271050 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 2 0 0 0
[pid=13190/tid=13282] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13282/stat : 13282 (python) S 13188 13190 12854 0 -1 4202560 40 932 0 0 0 0 0 0 18 0 34 0 529276993 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 2 0 0 0
[pid=13190/tid=13286] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13286/stat : 13286 (python) S 13188 13190 12854 0 -1 4202560 32 932 0 0 0 0 0 0 20 0 34 0 529278176 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 3 0 0 0
[pid=13190/tid=13291] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13291/stat : 13291 (python) S 13188 13190 12854 0 -1 4202560 51 932 0 0 0 0 0 0 19 0 34 0 529287128 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 2 0 0 0
[pid=13190/tid=13299] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13299/stat : 13299 (python) S 13188 13190 12854 0 -1 4202560 36 932 0 0 0 0 0 0 20 0 34 0 529298378 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 1 0 0 0
[pid=13190/tid=13303] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13303/stat : 13303 (python) S 13188 13190 12854 0 -1 4202560 38 932 0 0 0 0 0 0 19 0 34 0 529301438 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 2 0 0 0
[pid=13190/tid=13310] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13310/stat : 13310 (python) S 13188 13190 12854 0 -1 4202560 45 932 0 0 0 0 0 0 18 0 34 0 529308411 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 3 0 0 0
[pid=13190/tid=13315] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13315/stat : 13315 (python) S 13188 13190 12854 0 -1 4202560 163 932 0 0 0 0 0 0 19 0 34 0 529312675 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 2 0 0 0
[pid=13190/tid=13318] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13318/stat : 13318 (python) S 13188 13190 12854 0 -1 4202560 35 932 0 0 0 0 0 0 19 0 34 0 529314387 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 2 0 0 0
[pid=13190/tid=13325] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13325/stat : 13325 (python) S 13188 13190 12854 0 -1 4202560 44 932 0 0 0 0 0 0 18 0 34 0 529329420 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 2 0 0 0
[pid=13190/tid=13333] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13333/stat : 13333 (python) S 13188 13190 12854 0 -1 4202560 46 932 0 0 0 0 0 0 19 0 34 0 529337757 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 2 0 0 0
[pid=13190/tid=13340] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13340/stat : 13340 (python) S 13188 13190 12854 0 -1 4202560 41 932 0 0 0 0 0 0 19 0 34 0 529350078 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 2 0 0 0
[pid=13190/tid=13344] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13344/stat : 13344 (python) S 13188 13190 12854 0 -1 4202560 35 932 0 0 0 0 0 0 19 0 34 0 529351401 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 2 0 0 0
[pid=13190/tid=13347] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13347/stat : 13347 (python) S 13188 13190 12854 0 -1 4202560 33 932 0 0 0 0 0 0 21 0 34 0 529351412 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 2 0 0 0
[pid=13190/tid=13352] ppid=13188 vsize=687056 CPUtime=0 cores=0-7
/proc/13190/task/13352/stat : 13352 (python) S 13188 13190 12854 0 -1 4202560 40 932 0 0 0 0 0 0 19 0 34 0 529358448 703545344 8874 33554432000 4194304 5683644 140734540582464 18446744073709551615 264738228657 0 0 16781312 2 18446604453926212544 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 8747.32
Current children cumulated vsize (KiB) 32368316

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 13190 and gives
#  childrusage.ru_utime.tv_sec=6
#  childrusage.ru_utime.tv_usec=581999
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=252961
# CPU time returned by wait4() is 6.83496
# while last known CPU time is 8747.32
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1246.41
CPU time (s): 8747.32
CPU user time (s): 8735.2
CPU system time (s): 12.12
CPU usage (%): 701.801
Max. virtual memory (cumulated for all children) (KiB): 32368316

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 6.582
system time used= 0.252961
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 65246
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3091
involuntary context switches= 90

runsolver used 12.2391 second user time and 29.3515 second system time

The end

Launcher Data 

Begin job on node141 at 2011-05-02 22:29:42
IDJOB=3321842
IDBENCH=82620
IDSOLVER=1712
FILE ID=node141/3321842-1304368182
RUNJOBID= node141-1304366465-12868
PBS_JOBID= 13181732
Free space on /tmp= 72872 MiB

SOLVER NAME= borg-sat 11.03.02-random
BENCH NAME= SAT11/random/large/unif-k7-r85-v400-c34000-S943787219-031-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3321842-1304368182/watcher-3321842-1304368182 -o /tmp/evaluation-result-3321842-1304368182/solver-3321842-1304368182 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3321842-1304368182.cnf -seed 1828957852 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 6105ec111e1e16da586afadcc6f09041
RANDOM SEED=1828957852

node141.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5362.96
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      18172624 kB
Buffers:       1432824 kB
Cached:        9007636 kB
SwapCached:          4 kB
Active:        7192088 kB
Inactive:      6973176 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      18172624 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:           32732 kB
Writeback:           0 kB
AnonPages:     3725036 kB
Mapped:          22692 kB
Slab:           519868 kB
PageTables:      26760 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 54636420 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72840 MiB
End job on node141 at 2011-05-02 22:50:31