Trace number 3311284

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
STRUCTure 0.1 (hbr=2)? (TO) 4800.54 1205.8

General information on the benchmark

Namerandom/large/
unif-k7-r85-v400-c34000-S586055573-067-UNKNOWN.cnf
MD5SUMd673a60901b4a946b283fdbb02f99fdc
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables400
Number of clauses34000
Sum of the clauses size238000
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 534000

Solver Data 

0.10/0.12	03:17 INFO : Reading from input HOME/instance-3311284-1304118197.cnf
0.71/0.54	Executor set job limit to 1000000
0.71/0.54	Executor set job limit to 1000000
0.71/0.54	Executor set job limit to 1000000
0.71/0.54	Executor set job limit to 1000000
0.71/0.54	Starting Executor!
0.71/0.54	Starting Executor!
0.71/0.54	Starting Executor!
0.71/0.54	Starting Executor!
2.10/1.10	03:18 INFO : 400 variables remaining out of 400
2.10/1.10	03:18 INFO : 272000 literals remaining out of 272000

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3311284-1304118197/watcher-3311284-1304118197 -o /tmp/evaluation-result-3311284-1304118197/solver-3311284-1304118197 -C 4800 -W 1300 -M 15500 run 15000 4 HOME HOME HOME/instance-3311284-1304118197.cnf -hbr 2 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 5.25 7.11 7.53 2/174 22152
/proc/meminfo: memFree=30595192/32951124 swapFree=67111416/67111528
[pid=22152] ppid=22150 vsize=23028 CPUtime=0 cores=1,3,5,7
/proc/22152/stat : 22152 (java) R 22150 22152 21203 0 -1 4202496 1092 0 0 0 0 0 0 0 21 0 1 0 523422083 23580672 578 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212431054613 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/22152/statm: 5757 590 443 9 0 124 0

[startup+0.0327039 s]
/proc/loadavg: 5.25 7.11 7.53 2/174 22152
/proc/meminfo: memFree=30595192/32951124 swapFree=67111416/67111528
[pid=22152] ppid=22150 vsize=15604116 CPUtime=0.02 cores=1,3,5,7
/proc/22152/stat : 22152 (java) S 22150 22152 21203 0 -1 4202496 3545 0 1 0 2 0 0 0 21 0 13 0 523422083 15978614784 2984 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447819749 0 0 0 16784584 18446744073709551615 0 0 17 5 0 0 0
/proc/22152/statm: 3901029 2984 1658 9 0 3891777 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 15604116

[startup+0.10471 s]
/proc/loadavg: 5.25 7.11 7.53 2/174 22152
/proc/meminfo: memFree=30595192/32951124 swapFree=67111416/67111528
[pid=22152] ppid=22150 vsize=15676760 CPUtime=0.1 cores=1,3,5,7
/proc/22152/stat : 22152 (java) S 22150 22152 21203 0 -1 4202496 5497 0 1 0 9 1 0 0 21 0 18 0 523422083 16053002240 4936 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447819749 0 0 0 16800975 18446744073709551615 0 0 17 5 0 0 0
/proc/22152/statm: 3919190 4936 2117 9 0 3909909 0
Current children cumulated CPU time (s) 0.1
Current children cumulated vsize (KiB) 15676760

[startup+0.300681 s]
/proc/loadavg: 5.25 7.11 7.53 2/174 22152
/proc/meminfo: memFree=30595192/32951124 swapFree=67111416/67111528
[pid=22152] ppid=22150 vsize=15680368 CPUtime=0.51 cores=1,3,5,7
/proc/22152/stat : 22152 (java) S 22150 22152 21203 0 -1 4202496 14283 0 1 0 49 2 0 0 21 0 18 0 523422083 16056696832 12776 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447819749 0 0 0 16800975 18446744073709551615 0 0 17 5 0 0 0
/proc/22152/statm: 3920092 12782 2207 9 0 3910811 0
Current children cumulated CPU time (s) 0.51
Current children cumulated vsize (KiB) 15680368

[startup+0.700579 s]
/proc/loadavg: 5.25 7.11 7.53 2/174 22152
/proc/meminfo: memFree=30595192/32951124 swapFree=67111416/67111528
[pid=22152] ppid=22150 vsize=15695568 CPUtime=1.05 cores=1,3,5,7
/proc/22152/stat : 22152 (java) S 22150 22152 21203 0 -1 4202496 45520 0 1 0 99 6 0 0 21 0 22 0 523422083 16072261632 42197 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447819749 0 0 0 16800975 18446744073709551615 0 0 17 5 0 0 0
/proc/22152/statm: 3923892 42199 2248 9 0 3914611 0
Current children cumulated CPU time (s) 1.05
Current children cumulated vsize (KiB) 15695568

[startup+1.50264 s]
/proc/loadavg: 5.07 7.04 7.51 5/216 22197
/proc/meminfo: memFree=30264028/32951124 swapFree=67111416/67111528
[pid=22152] ppid=22150 vsize=15709232 CPUtime=3.19 cores=1,3,5,7
/proc/22152/stat : 22152 (java) S 22150 22152 21203 0 -1 4202496 61661 0 1 0 310 9 0 0 21 0 22 0 523422083 16086253568 51524 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447819749 0 0 0 16800975 18446744073709551615 0 0 17 5 0 0 0
/proc/22152/statm: 3927308 51524 2273 9 0 3918027 0
[pid=22152/tid=22154] ppid=22150 vsize=15709232 CPUtime=1.08 cores=1,3,5,7
/proc/22152/task/22154/stat : 22154 (java) S 22150 22152 21203 0 -1 4202560 35693 0 1 0 104 4 0 0 20 0 22 0 523422083 16086253568 51524 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 1 0 0 0
[pid=22152/tid=22155] ppid=22150 vsize=15709232 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22155/stat : 22155 (java) R 22150 22152 21203 0 -1 4202560 851 0 0 0 0 0 0 0 15 0 22 0 523422084 16086253568 51524 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212435970359 0 4 0 16800975 0 0 0 -1 7 0 0 0
[pid=22152/tid=22156] ppid=22150 vsize=15709232 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22156/stat : 22156 (java) S 22150 22152 21203 0 -1 4202560 860 0 0 0 0 0 0 0 15 0 22 0 523422084 16086253568 51524 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 3 0 0 0
[pid=22152/tid=22157] ppid=22150 vsize=15709232 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22157/stat : 22157 (java) S 22150 22152 21203 0 -1 4202560 1 0 0 0 0 0 0 0 15 0 22 0 523422084 16086253568 51524 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22158] ppid=22150 vsize=15709232 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22158/stat : 22158 (java) S 22150 22152 21203 0 -1 4202560 1 0 0 0 0 0 0 0 15 0 22 0 523422084 16086253568 51524 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22159] ppid=22150 vsize=15709232 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22159/stat : 22159 (java) S 22150 22152 21203 0 -1 4202560 478 0 0 0 0 0 0 0 16 0 22 0 523422084 16086253568 51524 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 1 0 0 0
[pid=22152/tid=22160] ppid=22150 vsize=15709232 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22160/stat : 22160 (java) S 22150 22152 21203 0 -1 4202560 1 0 0 0 0 0 0 0 15 0 22 0 523422084 16086253568 51524 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22161] ppid=22150 vsize=15709232 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22161/stat : 22161 (java) S 22150 22152 21203 0 -1 4202560 1109 0 0 0 0 0 0 0 16 0 22 0 523422084 16086253568 51524 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22162] ppid=22150 vsize=15709232 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22162/stat : 22162 (java) S 22150 22152 21203 0 -1 4202560 349 0 0 0 0 0 0 0 15 0 22 0 523422084 16086253568 51524 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 1 0 0 0
[pid=22152/tid=22163] ppid=22150 vsize=15709232 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22163/stat : 22163 (java) S 22150 22152 21203 0 -1 4202560 11 0 0 0 0 0 0 0 16 0 22 0 523422085 16086253568 51524 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 0 0 16800975 18446604460393634560 0 0 -1 3 0 0 0
[pid=22152/tid=22164] ppid=22150 vsize=15709232 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22164/stat : 22164 (java) S 22150 22152 21203 0 -1 4202560 17 0 0 0 0 0 0 0 17 0 22 0 523422085 16086253568 51524 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22165] ppid=22150 vsize=15709232 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22165/stat : 22165 (java) S 22150 22152 21203 0 -1 4202560 9 0 0 0 0 0 0 0 18 0 22 0 523422085 16086253568 51524 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 7 0 0 0
[pid=22152/tid=22166] ppid=22150 vsize=15709232 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22166/stat : 22166 (java) S 22150 22152 21203 0 -1 4202560 1 0 0 0 0 0 0 0 19 0 22 0 523422088 16086253568 51524 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447840689 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22167] ppid=22150 vsize=15709232 CPUtime=0.63 cores=1,3,5,7
/proc/22152/task/22167/stat : 22167 (java) S 22150 22152 21203 0 -1 4202560 6780 0 0 0 62 1 0 0 15 0 22 0 523422088 16086253568 51524 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22168] ppid=22150 vsize=15709232 CPUtime=0.85 cores=1,3,5,7
/proc/22152/task/22168/stat : 22168 (java) S 22150 22152 21203 0 -1 4202560 14082 0 0 0 84 1 0 0 17 0 22 0 523422088 16086253568 51524 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 3 0 0 0
[pid=22152/tid=22169] ppid=22150 vsize=15709232 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22169/stat : 22169 (java) S 22150 22152 21203 0 -1 4202560 1 0 0 0 0 0 0 0 22 0 22 0 523422088 16086253568 51524 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22170] ppid=22150 vsize=15709232 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22170/stat : 22170 (java) S 22150 22152 21203 0 -1 4202560 1 0 0 0 0 0 0 0 15 0 22 0 523422088 16086253568 51524 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447833600 0 4 0 16800975 18446604460393634560 0 0 -1 3 0 0 0
[pid=22152/tid=22194] ppid=22150 vsize=15709232 CPUtime=0.27 cores=1,3,5,7
/proc/22152/task/22194/stat : 22194 (java) S 22150 22152 21203 0 -1 4202560 27 0 0 0 27 0 0 0 15 0 22 0 523422137 16086253568 51524 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 7 0 0 0
[pid=22152/tid=22195] ppid=22150 vsize=15709232 CPUtime=0.01 cores=1,3,5,7
/proc/22152/task/22195/stat : 22195 (java) S 22150 22152 21203 0 -1 4202560 13 0 0 0 1 0 0 0 15 0 22 0 523422137 16086253568 51524 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447833600 0 4 0 16800975 18446604460393634560 0 0 -1 3 0 0 0
[pid=22152/tid=22196] ppid=22150 vsize=15709232 CPUtime=0.16 cores=1,3,5,7
/proc/22152/task/22196/stat : 22196 (java) S 22150 22152 21203 0 -1 4202560 11 0 0 0 16 0 0 0 16 0 22 0 523422137 16086253568 51524 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 1 0 0 0
[pid=22152/tid=22197] ppid=22150 vsize=15709232 CPUtime=0.12 cores=1,3,5,7
/proc/22152/task/22197/stat : 22197 (java) S 22150 22152 21203 0 -1 4202560 7 0 0 0 12 0 0 0 15 0 22 0 523422137 16086253568 51524 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447833600 0 4 0 16800975 18446604460393634560 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3.19
Current children cumulated vsize (KiB) 15709232

[startup+3.10109 s]
/proc/loadavg: 5.07 7.04 7.51 8/220 22201
/proc/meminfo: memFree=30177384/32951124 swapFree=67111416/67111528
[pid=22152] ppid=22150 vsize=15709232 CPUtime=9.48 cores=1,3,5,7
/proc/22152/stat : 22152 (java) S 22150 22152 21203 0 -1 4202496 86926 0 1 0 935 13 0 0 21 0 22 0 523422083 16086253568 76709 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447819749 0 0 0 16800975 18446744073709551615 0 0 17 5 0 0 0
/proc/22152/statm: 3927308 76709 2273 9 0 3918027 0
[pid=22152/tid=22154] ppid=22150 vsize=15709232 CPUtime=1.08 cores=1,3,5,7
/proc/22152/task/22154/stat : 22154 (java) S 22150 22152 21203 0 -1 4202560 35693 0 1 0 104 4 0 0 20 0 22 0 523422083 16086253568 76709 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 1 0 0 0
[pid=22152/tid=22155] ppid=22150 vsize=15709232 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22155/stat : 22155 (java) S 22150 22152 21203 0 -1 4202560 1786 0 0 0 0 0 0 0 15 0 22 0 523422084 16086253568 76709 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 1 0 0 0
[pid=22152/tid=22156] ppid=22150 vsize=15709232 CPUtime=0.01 cores=1,3,5,7
/proc/22152/task/22156/stat : 22156 (java) S 22150 22152 21203 0 -1 4202560 4134 0 0 0 1 0 0 0 16 0 22 0 523422084 16086253568 76709 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 3 0 0 0
[pid=22152/tid=22157] ppid=22150 vsize=15709232 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22157/stat : 22157 (java) S 22150 22152 21203 0 -1 4202560 1123 0 0 0 0 0 0 0 15 0 22 0 523422084 16086253568 76709 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22158] ppid=22150 vsize=15709232 CPUtime=0.01 cores=1,3,5,7
/proc/22152/task/22158/stat : 22158 (java) S 22150 22152 21203 0 -1 4202560 4954 0 0 0 1 0 0 0 16 0 22 0 523422084 16086253568 76709 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22159] ppid=22150 vsize=15709232 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22159/stat : 22159 (java) S 22150 22152 21203 0 -1 4202560 3663 0 0 0 0 0 0 0 15 0 22 0 523422084 16086253568 76709 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 1 0 0 0
[pid=22152/tid=22160] ppid=22150 vsize=15709232 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22160/stat : 22160 (java) S 22150 22152 21203 0 -1 4202560 1394 0 0 0 0 0 0 0 15 0 22 0 523422084 16086253568 76709 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 3 0 0 0
[pid=22152/tid=22161] ppid=22150 vsize=15709232 CPUtime=0.02 cores=1,3,5,7
/proc/22152/task/22161/stat : 22161 (java) S 22150 22152 21203 0 -1 4202560 6788 0 0 0 1 1 0 0 16 0 22 0 523422084 16086253568 76709 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 7 0 0 0
[pid=22152/tid=22162] ppid=22150 vsize=15709232 CPUtime=0.01 cores=1,3,5,7
/proc/22152/task/22162/stat : 22162 (java) S 22150 22152 21203 0 -1 4202560 4949 0 0 0 1 0 0 0 16 0 22 0 523422084 16086253568 76709 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 1 0 0 0
[pid=22152/tid=22163] ppid=22150 vsize=15709232 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22163/stat : 22163 (java) S 22150 22152 21203 0 -1 4202560 109 0 0 0 0 0 0 0 15 0 22 0 523422085 16086253568 76709 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447833600 0 0 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22164] ppid=22150 vsize=15709232 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22164/stat : 22164 (java) S 22150 22152 21203 0 -1 4202560 17 0 0 0 0 0 0 0 15 0 22 0 523422085 16086253568 76709 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22165] ppid=22150 vsize=15709232 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22165/stat : 22165 (java) S 22150 22152 21203 0 -1 4202560 9 0 0 0 0 0 0 0 18 0 22 0 523422085 16086253568 76709 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 3 0 0 0
[pid=22152/tid=22166] ppid=22150 vsize=15709232 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22166/stat : 22166 (java) S 22150 22152 21203 0 -1 4202560 1 0 0 0 0 0 0 0 19 0 22 0 523422088 16086253568 76709 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447840689 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22167] ppid=22150 vsize=15709232 CPUtime=0.75 cores=1,3,5,7
/proc/22152/task/22167/stat : 22167 (java) S 22150 22152 21203 0 -1 4202560 6796 0 0 0 74 1 0 0 15 0 22 0 523422088 16086253568 76709 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 3 0 0 0
[pid=22152/tid=22168] ppid=22150 vsize=15709232 CPUtime=0.9 cores=1,3,5,7
/proc/22152/task/22168/stat : 22168 (java) S 22150 22152 21203 0 -1 4202560 14085 0 0 0 88 2 0 0 15 0 22 0 523422088 16086253568 76709 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22169] ppid=22150 vsize=15709232 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22169/stat : 22169 (java) S 22150 22152 21203 0 -1 4202560 1 0 0 0 0 0 0 0 22 0 22 0 523422088 16086253568 76709 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22170] ppid=22150 vsize=15709232 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22170/stat : 22170 (java) S 22150 22152 21203 0 -1 4202560 1 0 0 0 0 0 0 0 15 0 22 0 523422088 16086253568 76709 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447833600 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22194] ppid=22150 vsize=15709232 CPUtime=1.83 cores=1,3,5,7
/proc/22152/task/22194/stat : 22194 (java) R 22150 22152 21203 0 -1 4202560 28 0 0 0 183 0 0 0 17 0 22 0 523422137 16086253568 76709 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 46912505978515 0 4 0 16800975 0 0 0 -1 7 0 0 0
[pid=22152/tid=22195] ppid=22150 vsize=15709232 CPUtime=1.49 cores=1,3,5,7
/proc/22152/task/22195/stat : 22195 (java) R 22150 22152 21203 0 -1 4202560 15 0 0 0 149 0 0 0 17 0 22 0 523422137 16086253568 76709 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 46912505916325 0 4 0 16800975 0 0 0 -1 3 0 0 0
[pid=22152/tid=22196] ppid=22150 vsize=15709232 CPUtime=1.68 cores=1,3,5,7
/proc/22152/task/22196/stat : 22196 (java) R 22150 22152 21203 0 -1 4202560 13 0 0 0 168 0 0 0 17 0 22 0 523422137 16086253568 76709 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 46912505873651 0 4 0 16800975 0 0 0 -1 5 0 0 0
[pid=22152/tid=22197] ppid=22150 vsize=15709232 CPUtime=1.58 cores=1,3,5,7
/proc/22152/task/22197/stat : 22197 (java) R 22150 22152 21203 0 -1 4202560 9 0 0 0 158 0 0 0 17 0 22 0 523422137 16086253568 76709 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 46912505917612 0 4 0 16800975 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 9.48
Current children cumulated vsize (KiB) 15709232

[startup+6.30148 s]
/proc/loadavg: 5.07 7.04 7.51 10/220 22201
/proc/meminfo: memFree=28929728/32951124 swapFree=67111416/67111528
[pid=22152] ppid=22150 vsize=15709232 CPUtime=22.24 cores=1,3,5,7
/proc/22152/stat : 22152 (java) S 22150 22152 21203 0 -1 4202496 203391 0 1 0 2194 30 0 0 21 0 22 0 523422083 16086253568 193161 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447819749 0 0 0 16800975 18446744073709551615 0 0 17 5 0 0 0
/proc/22152/statm: 3927308 193161 2273 9 0 3918027 0
[pid=22152/tid=22154] ppid=22150 vsize=15709232 CPUtime=1.08 cores=1,3,5,7
/proc/22152/task/22154/stat : 22154 (java) S 22150 22152 21203 0 -1 4202560 35693 0 1 0 104 4 0 0 20 0 22 0 523422083 16086253568 193161 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 1 0 0 0
[pid=22152/tid=22155] ppid=22150 vsize=15709232 CPUtime=0.03 cores=1,3,5,7
/proc/22152/task/22155/stat : 22155 (java) S 22150 22152 21203 0 -1 4202560 9600 0 0 0 2 1 0 0 15 0 22 0 523422084 16086253568 193161 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 1 0 0 0
[pid=22152/tid=22156] ppid=22150 vsize=15709232 CPUtime=0.03 cores=1,3,5,7
/proc/22152/task/22156/stat : 22156 (java) S 22150 22152 21203 0 -1 4202560 10244 0 0 0 2 1 0 0 16 0 22 0 523422084 16086253568 193161 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 1 0 0 0
[pid=22152/tid=22157] ppid=22150 vsize=15709232 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22157/stat : 22157 (java) S 22150 22152 21203 0 -1 4202560 1123 0 0 0 0 0 0 0 15 0 22 0 523422084 16086253568 193161 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22158] ppid=22150 vsize=15709232 CPUtime=0.04 cores=1,3,5,7
/proc/22152/task/22158/stat : 22158 (java) S 22150 22152 21203 0 -1 4202560 15006 0 0 0 2 2 0 0 16 0 22 0 523422084 16086253568 193161 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22159] ppid=22150 vsize=15709232 CPUtime=0.05 cores=1,3,5,7
/proc/22152/task/22159/stat : 22159 (java) S 22150 22152 21203 0 -1 4202560 16141 0 0 0 3 2 0 0 16 0 22 0 523422084 16086253568 193161 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 3 0 0 0
[pid=22152/tid=22160] ppid=22150 vsize=15709232 CPUtime=0.01 cores=1,3,5,7
/proc/22152/task/22160/stat : 22160 (java) S 22150 22152 21203 0 -1 4202560 5104 0 0 0 1 0 0 0 15 0 22 0 523422084 16086253568 193161 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 3 0 0 0
[pid=22152/tid=22161] ppid=22150 vsize=15709232 CPUtime=0.05 cores=1,3,5,7
/proc/22152/task/22161/stat : 22161 (java) S 22150 22152 21203 0 -1 4202560 17910 0 0 0 3 2 0 0 16 0 22 0 523422084 16086253568 193161 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 1 0 0 0
[pid=22152/tid=22162] ppid=22150 vsize=15709232 CPUtime=0.04 cores=1,3,5,7
/proc/22152/task/22162/stat : 22162 (java) S 22150 22152 21203 0 -1 4202560 13868 0 0 0 2 2 0 0 15 0 22 0 523422084 16086253568 193161 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22163] ppid=22150 vsize=15709232 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22163/stat : 22163 (java) S 22150 22152 21203 0 -1 4202560 410 0 0 0 0 0 0 0 15 0 22 0 523422085 16086253568 193161 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447833600 0 0 0 16800975 18446604460393634560 0 0 -1 7 0 0 0
[pid=22152/tid=22164] ppid=22150 vsize=15709232 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22164/stat : 22164 (java) S 22150 22152 21203 0 -1 4202560 17 0 0 0 0 0 0 0 15 0 22 0 523422085 16086253568 193161 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22165] ppid=22150 vsize=15709232 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22165/stat : 22165 (java) S 22150 22152 21203 0 -1 4202560 9 0 0 0 0 0 0 0 18 0 22 0 523422085 16086253568 193161 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 3 0 0 0
[pid=22152/tid=22166] ppid=22150 vsize=15709232 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22166/stat : 22166 (java) S 22150 22152 21203 0 -1 4202560 1 0 0 0 0 0 0 0 19 0 22 0 523422088 16086253568 193161 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447840689 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22167] ppid=22150 vsize=15709232 CPUtime=0.75 cores=1,3,5,7
/proc/22152/task/22167/stat : 22167 (java) S 22150 22152 21203 0 -1 4202560 6796 0 0 0 74 1 0 0 15 0 22 0 523422088 16086253568 193161 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 3 0 0 0

################
# More data... #
################

[pid=22152/tid=22195] ppid=22150 vsize=15699072 CPUtime=1108.01 cores=1,3,5,7
/proc/22152/task/22195/stat : 22195 (java) R 22150 22152 21203 0 -1 4202560 226568 0 0 0 110767 34 0 0 25 0 22 0 523422137 16075849728 177251 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 18446744073699065863 0 4 0 16800975 0 0 0 -1 3 0 0 0
[pid=22152/tid=22196] ppid=22150 vsize=15699072 CPUtime=1107.65 cores=1,3,5,7
/proc/22152/task/22196/stat : 22196 (java) R 22150 22152 21203 0 -1 4202560 221786 0 0 0 110732 33 0 0 25 0 22 0 523422137 16075849728 177251 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 18446744073699065904 0 4 0 16800975 0 0 0 -1 7 0 0 0
[pid=22152/tid=22197] ppid=22150 vsize=15699072 CPUtime=1108.55 cores=1,3,5,7
/proc/22152/task/22197/stat : 22197 (java) R 22150 22152 21203 0 -1 4202560 223128 0 0 0 110822 33 0 0 25 0 22 0 523422137 16075849728 177251 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 46912505743361 0 4 0 16800975 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4468.07
Current children cumulated vsize (KiB) 15699072

[startup+1182.3 s]
/proc/loadavg: 5.04 5.21 6.06 6/220 22236
/proc/meminfo: memFree=14508628/32951124 swapFree=67111416/67111528
[pid=22152] ppid=22150 vsize=15699072 CPUtime=4707.02 cores=1,3,5,7
/proc/22152/stat : 22152 (java) S 22150 22152 21203 0 -1 4202496 2011837 0 1 0 470291 411 0 0 21 0 22 0 523422083 16075849728 162967 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447819749 0 0 0 16800975 18446744073709551615 0 0 17 5 0 0 0
/proc/22152/statm: 3924768 162967 2297 9 0 3915487 0
[pid=22152/tid=22154] ppid=22150 vsize=15699072 CPUtime=1.08 cores=1,3,5,7
/proc/22152/task/22154/stat : 22154 (java) S 22150 22152 21203 0 -1 4202560 35693 0 1 0 104 4 0 0 20 0 22 0 523422083 16075849728 162967 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 1 0 0 0
[pid=22152/tid=22155] ppid=22150 vsize=15699072 CPUtime=3.92 cores=1,3,5,7
/proc/22152/task/22155/stat : 22155 (java) S 22150 22152 21203 0 -1 4202560 120248 0 0 0 368 24 0 0 16 0 22 0 523422084 16075849728 162967 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 3 0 0 0
[pid=22152/tid=22156] ppid=22150 vsize=15699072 CPUtime=3.99 cores=1,3,5,7
/proc/22152/task/22156/stat : 22156 (java) S 22150 22152 21203 0 -1 4202560 134005 0 0 0 372 27 0 0 16 0 22 0 523422084 16075849728 162967 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 1 0 0 0
[pid=22152/tid=22157] ppid=22150 vsize=15699072 CPUtime=3.88 cores=1,3,5,7
/proc/22152/task/22157/stat : 22157 (java) S 22150 22152 21203 0 -1 4202560 113890 0 0 0 364 24 0 0 15 0 22 0 523422084 16075849728 162967 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 1 0 0 0
[pid=22152/tid=22158] ppid=22150 vsize=15699072 CPUtime=3.97 cores=1,3,5,7
/proc/22152/task/22158/stat : 22158 (java) S 22150 22152 21203 0 -1 4202560 124304 0 0 0 372 25 0 0 16 0 22 0 523422084 16075849728 162967 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 7 0 0 0
[pid=22152/tid=22159] ppid=22150 vsize=15699072 CPUtime=4.04 cores=1,3,5,7
/proc/22152/task/22159/stat : 22159 (java) S 22150 22152 21203 0 -1 4202560 133471 0 0 0 378 26 0 0 16 0 22 0 523422084 16075849728 162967 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22160] ppid=22150 vsize=15699072 CPUtime=3.94 cores=1,3,5,7
/proc/22152/task/22160/stat : 22160 (java) S 22150 22152 21203 0 -1 4202560 123738 0 0 0 367 27 0 0 15 0 22 0 523422084 16075849728 162967 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 3 0 0 0
[pid=22152/tid=22161] ppid=22150 vsize=15699072 CPUtime=3.99 cores=1,3,5,7
/proc/22152/task/22161/stat : 22161 (java) S 22150 22152 21203 0 -1 4202560 133561 0 0 0 374 25 0 0 15 0 22 0 523422084 16075849728 162967 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22162] ppid=22150 vsize=15699072 CPUtime=4.02 cores=1,3,5,7
/proc/22152/task/22162/stat : 22162 (java) S 22150 22152 21203 0 -1 4202560 128963 0 0 0 376 26 0 0 15 0 22 0 523422084 16075849728 162967 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 1 0 0 0
[pid=22152/tid=22163] ppid=22150 vsize=15699072 CPUtime=1.42 cores=1,3,5,7
/proc/22152/task/22163/stat : 22163 (java) S 22150 22152 21203 0 -1 4202560 6221 0 0 0 91 51 0 0 15 0 22 0 523422085 16075849728 162967 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447833600 0 0 0 16800975 18446604460393634560 0 0 -1 7 0 0 0
[pid=22152/tid=22164] ppid=22150 vsize=15699072 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22164/stat : 22164 (java) S 22150 22152 21203 0 -1 4202560 17 0 0 0 0 0 0 0 15 0 22 0 523422085 16075849728 162967 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22165] ppid=22150 vsize=15699072 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22165/stat : 22165 (java) S 22150 22152 21203 0 -1 4202560 9 0 0 0 0 0 0 0 18 0 22 0 523422085 16075849728 162967 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 3 0 0 0
[pid=22152/tid=22166] ppid=22150 vsize=15699072 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22166/stat : 22166 (java) S 22150 22152 21203 0 -1 4202560 1 0 0 0 0 0 0 0 19 0 22 0 523422088 16075849728 162967 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447840689 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22167] ppid=22150 vsize=15699072 CPUtime=1.09 cores=1,3,5,7
/proc/22152/task/22167/stat : 22167 (java) S 22150 22152 21203 0 -1 4202560 6837 0 0 0 108 1 0 0 15 0 22 0 523422088 16075849728 162967 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 3 0 0 0
[pid=22152/tid=22168] ppid=22150 vsize=15699072 CPUtime=1.23 cores=1,3,5,7
/proc/22152/task/22168/stat : 22168 (java) S 22150 22152 21203 0 -1 4202560 14146 0 0 0 121 2 0 0 15 0 22 0 523422088 16075849728 162967 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22169] ppid=22150 vsize=15699072 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22169/stat : 22169 (java) S 22150 22152 21203 0 -1 4202560 1 0 0 0 0 0 0 0 22 0 22 0 523422088 16075849728 162967 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22170] ppid=22150 vsize=15699072 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22170/stat : 22170 (java) S 22150 22152 21203 0 -1 4202560 62 0 0 0 0 0 0 0 15 0 22 0 523422088 16075849728 162967 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447833600 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22194] ppid=22150 vsize=15699072 CPUtime=1168.26 cores=1,3,5,7
/proc/22152/task/22194/stat : 22194 (java) R 22150 22152 21203 0 -1 4202560 231469 0 0 0 116791 35 0 0 25 0 22 0 523422137 16075849728 162967 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 46912505872771 0 4 0 16800975 0 0 0 -1 7 0 0 0
[pid=22152/tid=22195] ppid=22150 vsize=15699072 CPUtime=1167.29 cores=1,3,5,7
/proc/22152/task/22195/stat : 22195 (java) R 22150 22152 21203 0 -1 4202560 236975 0 0 0 116693 36 0 0 25 0 22 0 523422137 16075849728 162967 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 46912505971857 0 4 0 16800975 0 0 0 -1 5 0 0 0
[pid=22152/tid=22196] ppid=22150 vsize=15699072 CPUtime=1166.82 cores=1,3,5,7
/proc/22152/task/22196/stat : 22196 (java) R 22150 22152 21203 0 -1 4202560 233044 0 0 0 116647 35 0 0 25 0 22 0 523422137 16075849728 162967 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 46912505972065 0 4 0 16800975 0 0 0 -1 1 0 0 0
[pid=22152/tid=22197] ppid=22150 vsize=15699072 CPUtime=1167.92 cores=1,3,5,7
/proc/22152/task/22197/stat : 22197 (java) R 22150 22152 21203 0 -1 4202560 233824 0 0 0 116757 35 0 0 25 0 22 0 523422137 16075849728 162967 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 46912505957930 0 4 0 16800975 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4707.02
Current children cumulated vsize (KiB) 15699072



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1205.7 s]
/proc/loadavg: 5.03 5.19 6.03 6/220 22236
/proc/meminfo: memFree=14494332/32951124 swapFree=67111416/67111528
[pid=22152] ppid=22150 vsize=15699072 CPUtime=4800.24 cores=1,3,5,7
/proc/22152/stat : 22152 (java) S 22150 22152 21203 0 -1 4202496 2051606 0 1 0 479604 420 0 0 21 0 22 0 523422083 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447819749 0 0 0 16800975 18446744073709551615 0 0 17 5 0 0 0
/proc/22152/statm: 3924768 167373 2297 9 0 3915487 0
[pid=22152/tid=22154] ppid=22150 vsize=15699072 CPUtime=1.08 cores=1,3,5,7
/proc/22152/task/22154/stat : 22154 (java) S 22150 22152 21203 0 -1 4202560 35693 0 1 0 104 4 0 0 20 0 22 0 523422083 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 1 0 0 0
[pid=22152/tid=22155] ppid=22150 vsize=15699072 CPUtime=4 cores=1,3,5,7
/proc/22152/task/22155/stat : 22155 (java) S 22150 22152 21203 0 -1 4202560 122569 0 0 0 376 24 0 0 16 0 22 0 523422084 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22156] ppid=22150 vsize=15699072 CPUtime=4.07 cores=1,3,5,7
/proc/22152/task/22156/stat : 22156 (java) S 22150 22152 21203 0 -1 4202560 136258 0 0 0 380 27 0 0 16 0 22 0 523422084 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 7 0 0 0
[pid=22152/tid=22157] ppid=22150 vsize=15699072 CPUtime=3.98 cores=1,3,5,7
/proc/22152/task/22157/stat : 22157 (java) S 22150 22152 21203 0 -1 4202560 116308 0 0 0 373 25 0 0 15 0 22 0 523422084 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 7 0 0 0
[pid=22152/tid=22158] ppid=22150 vsize=15699072 CPUtime=4.07 cores=1,3,5,7
/proc/22152/task/22158/stat : 22158 (java) S 22150 22152 21203 0 -1 4202560 126019 0 0 0 381 26 0 0 15 0 22 0 523422084 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 1 0 0 0
[pid=22152/tid=22159] ppid=22150 vsize=15699072 CPUtime=4.12 cores=1,3,5,7
/proc/22152/task/22159/stat : 22159 (java) S 22150 22152 21203 0 -1 4202560 134981 0 0 0 386 26 0 0 16 0 22 0 523422084 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 1 0 0 0
[pid=22152/tid=22160] ppid=22150 vsize=15699072 CPUtime=4.03 cores=1,3,5,7
/proc/22152/task/22160/stat : 22160 (java) S 22150 22152 21203 0 -1 4202560 125835 0 0 0 376 27 0 0 16 0 22 0 523422084 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 3 0 0 0
[pid=22152/tid=22161] ppid=22150 vsize=15699072 CPUtime=4.08 cores=1,3,5,7
/proc/22152/task/22161/stat : 22161 (java) S 22150 22152 21203 0 -1 4202560 135738 0 0 0 383 25 0 0 15 0 22 0 523422084 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 7 0 0 0
[pid=22152/tid=22162] ppid=22150 vsize=15699072 CPUtime=4.09 cores=1,3,5,7
/proc/22152/task/22162/stat : 22162 (java) S 22150 22152 21203 0 -1 4202560 131555 0 0 0 382 27 0 0 15 0 22 0 523422084 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 3 0 0 0
[pid=22152/tid=22163] ppid=22150 vsize=15699072 CPUtime=1.44 cores=1,3,5,7
/proc/22152/task/22163/stat : 22163 (java) S 22150 22152 21203 0 -1 4202560 6364 0 0 0 92 52 0 0 15 0 22 0 523422085 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447833600 0 0 0 16800975 18446604460393634560 0 0 -1 7 0 0 0
[pid=22152/tid=22164] ppid=22150 vsize=15699072 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22164/stat : 22164 (java) S 22150 22152 21203 0 -1 4202560 17 0 0 0 0 0 0 0 15 0 22 0 523422085 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22165] ppid=22150 vsize=15699072 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22165/stat : 22165 (java) S 22150 22152 21203 0 -1 4202560 9 0 0 0 0 0 0 0 18 0 22 0 523422085 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 3 0 0 0
[pid=22152/tid=22166] ppid=22150 vsize=15699072 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22166/stat : 22166 (java) S 22150 22152 21203 0 -1 4202560 1 0 0 0 0 0 0 0 19 0 22 0 523422088 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447840689 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22167] ppid=22150 vsize=15699072 CPUtime=1.09 cores=1,3,5,7
/proc/22152/task/22167/stat : 22167 (java) S 22150 22152 21203 0 -1 4202560 6837 0 0 0 108 1 0 0 15 0 22 0 523422088 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 3 0 0 0
[pid=22152/tid=22168] ppid=22150 vsize=15699072 CPUtime=1.23 cores=1,3,5,7
/proc/22152/task/22168/stat : 22168 (java) S 22150 22152 21203 0 -1 4202560 14146 0 0 0 121 2 0 0 15 0 22 0 523422088 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22169] ppid=22150 vsize=15699072 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22169/stat : 22169 (java) S 22150 22152 21203 0 -1 4202560 1 0 0 0 0 0 0 0 22 0 22 0 523422088 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22170] ppid=22150 vsize=15699072 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22170/stat : 22170 (java) S 22150 22152 21203 0 -1 4202560 64 0 0 0 0 0 0 0 15 0 22 0 523422088 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447833600 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22194] ppid=22150 vsize=15699072 CPUtime=1191.27 cores=1,3,5,7
/proc/22152/task/22194/stat : 22194 (java) R 22150 22152 21203 0 -1 4202560 237300 0 0 0 119091 36 0 0 25 0 22 0 523422137 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 46912505985675 0 4 0 16800975 0 0 0 -1 7 0 0 0
[pid=22152/tid=22195] ppid=22150 vsize=15699072 CPUtime=1190.45 cores=1,3,5,7
/proc/22152/task/22195/stat : 22195 (java) R 22150 22152 21203 0 -1 4202560 243238 0 0 0 119008 37 0 0 25 0 22 0 523422137 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 46912505800123 0 4 0 16800975 0 0 0 -1 5 0 0 0
[pid=22152/tid=22196] ppid=22150 vsize=15699072 CPUtime=1189.98 cores=1,3,5,7
/proc/22152/task/22196/stat : 22196 (java) R 22150 22152 21203 0 -1 4202560 238362 0 0 0 118962 36 0 0 25 0 22 0 523422137 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 46912505800314 0 4 0 16800975 0 0 0 -1 1 0 0 0
[pid=22152/tid=22197] ppid=22150 vsize=15699072 CPUtime=1191.09 cores=1,3,5,7
/proc/22152/task/22197/stat : 22197 (java) R 22150 22152 21203 0 -1 4202560 238953 0 0 0 119073 36 0 0 25 0 22 0 523422137 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 46912505873710 0 4 0 16800975 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4800.24
Current children cumulated vsize (KiB) 15699072

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1205.7 s]
/proc/loadavg: 5.03 5.19 6.03 6/220 22236
/proc/meminfo: memFree=14494332/32951124 swapFree=67111416/67111528
[pid=22152] ppid=22150 vsize=15699072 CPUtime=4800.24 cores=1,3,5,7
/proc/22152/stat : 22152 (java) S 22150 22152 21203 0 -1 4202496 2051606 0 1 0 479604 420 0 0 21 0 22 0 523422083 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447819749 0 0 0 16800975 18446744073709551615 0 0 17 5 0 0 0
/proc/22152/statm: 3924768 167373 2297 9 0 3915487 0
[pid=22152/tid=22154] ppid=22150 vsize=15699072 CPUtime=1.08 cores=1,3,5,7
/proc/22152/task/22154/stat : 22154 (java) S 22150 22152 21203 0 -1 4202560 35693 0 1 0 104 4 0 0 20 0 22 0 523422083 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 1 0 0 0
[pid=22152/tid=22155] ppid=22150 vsize=15699072 CPUtime=4 cores=1,3,5,7
/proc/22152/task/22155/stat : 22155 (java) S 22150 22152 21203 0 -1 4202560 122569 0 0 0 376 24 0 0 16 0 22 0 523422084 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22156] ppid=22150 vsize=15699072 CPUtime=4.07 cores=1,3,5,7
/proc/22152/task/22156/stat : 22156 (java) S 22150 22152 21203 0 -1 4202560 136258 0 0 0 380 27 0 0 16 0 22 0 523422084 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 7 0 0 0
[pid=22152/tid=22157] ppid=22150 vsize=15699072 CPUtime=3.98 cores=1,3,5,7
/proc/22152/task/22157/stat : 22157 (java) S 22150 22152 21203 0 -1 4202560 116308 0 0 0 373 25 0 0 15 0 22 0 523422084 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 7 0 0 0
[pid=22152/tid=22158] ppid=22150 vsize=15699072 CPUtime=4.07 cores=1,3,5,7
/proc/22152/task/22158/stat : 22158 (java) S 22150 22152 21203 0 -1 4202560 126019 0 0 0 381 26 0 0 15 0 22 0 523422084 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 1 0 0 0
[pid=22152/tid=22159] ppid=22150 vsize=15699072 CPUtime=4.12 cores=1,3,5,7
/proc/22152/task/22159/stat : 22159 (java) S 22150 22152 21203 0 -1 4202560 134981 0 0 0 386 26 0 0 16 0 22 0 523422084 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 1 0 0 0
[pid=22152/tid=22160] ppid=22150 vsize=15699072 CPUtime=4.03 cores=1,3,5,7
/proc/22152/task/22160/stat : 22160 (java) S 22150 22152 21203 0 -1 4202560 125835 0 0 0 376 27 0 0 16 0 22 0 523422084 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 3 0 0 0
[pid=22152/tid=22161] ppid=22150 vsize=15699072 CPUtime=4.08 cores=1,3,5,7
/proc/22152/task/22161/stat : 22161 (java) S 22150 22152 21203 0 -1 4202560 135738 0 0 0 383 25 0 0 15 0 22 0 523422084 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 7 0 0 0
[pid=22152/tid=22162] ppid=22150 vsize=15699072 CPUtime=4.09 cores=1,3,5,7
/proc/22152/task/22162/stat : 22162 (java) S 22150 22152 21203 0 -1 4202560 131555 0 0 0 382 27 0 0 15 0 22 0 523422084 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 3 0 0 0
[pid=22152/tid=22163] ppid=22150 vsize=15699072 CPUtime=1.44 cores=1,3,5,7
/proc/22152/task/22163/stat : 22163 (java) S 22150 22152 21203 0 -1 4202560 6364 0 0 0 92 52 0 0 15 0 22 0 523422085 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447833600 0 0 0 16800975 18446604460393634560 0 0 -1 7 0 0 0
[pid=22152/tid=22164] ppid=22150 vsize=15699072 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22164/stat : 22164 (java) S 22150 22152 21203 0 -1 4202560 17 0 0 0 0 0 0 0 15 0 22 0 523422085 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22165] ppid=22150 vsize=15699072 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22165/stat : 22165 (java) S 22150 22152 21203 0 -1 4202560 9 0 0 0 0 0 0 0 18 0 22 0 523422085 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 3 0 0 0
[pid=22152/tid=22166] ppid=22150 vsize=15699072 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22166/stat : 22166 (java) S 22150 22152 21203 0 -1 4202560 1 0 0 0 0 0 0 0 19 0 22 0 523422088 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447840689 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22167] ppid=22150 vsize=15699072 CPUtime=1.09 cores=1,3,5,7
/proc/22152/task/22167/stat : 22167 (java) S 22150 22152 21203 0 -1 4202560 6837 0 0 0 108 1 0 0 15 0 22 0 523422088 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 3 0 0 0
[pid=22152/tid=22168] ppid=22150 vsize=15699072 CPUtime=1.23 cores=1,3,5,7
/proc/22152/task/22168/stat : 22168 (java) S 22150 22152 21203 0 -1 4202560 14146 0 0 0 121 2 0 0 15 0 22 0 523422088 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22169] ppid=22150 vsize=15699072 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22169/stat : 22169 (java) S 22150 22152 21203 0 -1 4202560 1 0 0 0 0 0 0 0 22 0 22 0 523422088 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447832985 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22170] ppid=22150 vsize=15699072 CPUtime=0 cores=1,3,5,7
/proc/22152/task/22170/stat : 22170 (java) S 22150 22152 21203 0 -1 4202560 64 0 0 0 0 0 0 0 15 0 22 0 523422088 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 212447833600 0 4 0 16800975 18446604460393634560 0 0 -1 5 0 0 0
[pid=22152/tid=22194] ppid=22150 vsize=15699072 CPUtime=1191.27 cores=1,3,5,7
/proc/22152/task/22194/stat : 22194 (java) R 22150 22152 21203 0 -1 4202560 237300 0 0 0 119091 36 0 0 25 0 22 0 523422137 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 46912505985675 0 4 0 16800975 0 0 0 -1 7 0 0 0
[pid=22152/tid=22195] ppid=22150 vsize=15699072 CPUtime=1190.45 cores=1,3,5,7
/proc/22152/task/22195/stat : 22195 (java) R 22150 22152 21203 0 -1 4202560 243238 0 0 0 119008 37 0 0 25 0 22 0 523422137 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 46912505800123 0 4 0 16800975 0 0 0 -1 5 0 0 0
[pid=22152/tid=22196] ppid=22150 vsize=15699072 CPUtime=1189.98 cores=1,3,5,7
/proc/22152/task/22196/stat : 22196 (java) R 22150 22152 21203 0 -1 4202560 238362 0 0 0 118962 36 0 0 25 0 22 0 523422137 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 46912505800314 0 4 0 16800975 0 0 0 -1 1 0 0 0
[pid=22152/tid=22197] ppid=22150 vsize=15699072 CPUtime=1191.09 cores=1,3,5,7
/proc/22152/task/22197/stat : 22197 (java) R 22150 22152 21203 0 -1 4202560 238953 0 0 0 119073 36 0 0 25 0 22 0 523422137 16075849728 167373 33554432000 1073741824 1073778376 140734500816240 18446744073709551615 46912505873710 0 4 0 16800975 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4800.24
Current children cumulated vsize (KiB) 15699072

Child status: 143
Real time (s): 1205.8
CPU time (s): 4800.54
CPU user time (s): 4796.31
CPU system time (s): 4.23136
CPU usage (%): 398.12
Max. virtual memory (cumulated for all children) (KiB): 15710856

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4796.31
system time used= 4.23136
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2051632
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 143367
involuntary context switches= 96028

runsolver used 1.9467 second user time and 4.69529 second system time

The end

Launcher Data 

Begin job on node102 at 2011-04-30 01:03:17
IDJOB=3311284
IDBENCH=82669
IDSOLVER=1698
FILE ID=node102/3311284-1304118197
RUNJOBID= node102-1304118197-22139
PBS_JOBID= 13170678
Free space on /tmp= 71560 MiB

SOLVER NAME= STRUCTure 0.1 (hbr=2)
BENCH NAME= SAT11/random/large/unif-k7-r85-v400-c34000-S586055573-067-UNKNOWN.cnf
COMMAND LINE= run 15000 NBCORE TMPDIR DIR BENCHNAME -hbr 2
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3311284-1304118197/watcher-3311284-1304118197 -o /tmp/evaluation-result-3311284-1304118197/solver-3311284-1304118197 -C 4800 -W 1300 -M 15500  run 15000 4 HOME HOME HOME/instance-3311284-1304118197.cnf -hbr 2

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= d673a60901b4a946b283fdbb02f99fdc
RANDOM SEED=1651718147

node102.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      30595472 kB
Buffers:        603824 kB
Cached:        1268128 kB
SwapCached:          0 kB
Active:         141536 kB
Inactive:      1798588 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      30595472 kB
SwapTotal:    67111528 kB
SwapFree:     67111416 kB
Dirty:            4276 kB
Writeback:           0 kB
AnonPages:       67696 kB
Mapped:          14488 kB
Slab:           351668 kB
PageTables:       4728 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   195488 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71552 MiB
End job on node102 at 2011-04-30 01:23:25