Trace number 3371184

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.11 1203.02

General information on the benchmark

Namecrafted/spence/sgen/
sgen3-n120-s55656844-unsat.cnf
MD5SUM4877ca9f3fe317c84d3087c8296a2a0b
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark219.038
Satisfiable
(Un)Satisfiability was proved
Number of variables121
Number of clauses252
Sum of the clauses size756
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3252
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3371184-1305424416/watcher-3371184-1305424416 -o /tmp/evaluation-result-3371184-1305424416/solver-3371184-1305424416 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3371184-1305424416.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.74 7.99 7.73 5/295 29459
/proc/meminfo: memFree=24278328/32951124 swapFree=63485424/67111528
[pid=29459] ppid=29457 vsize=115368 CPUtime=0.01 cores=0,2,4,6
/proc/29459/stat : 29459 (plingeling) S 29457 29459 28842 0 -1 4202496 350 1252 0 0 1 0 0 0 25 0 5 0 634857548 118136832 277 33554432000 4194304 4417452 140736834740000 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29459/statm: 28842 277 176 55 0 26728 0
[pid=29459/tid=29465] ppid=29457 vsize=115368 CPUtime=0 cores=0,2,4,6
/proc/29459/task/29465/stat : 29465 (plingeling) R 29457 29459 28842 0 -1 4202560 29 1252 0 0 0 0 0 0 25 0 5 0 634857548 118136832 281 33554432000 4194304 4417452 140736834740000 18446744073709551615 4359092 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29459/tid=29466] ppid=29457 vsize=115368 CPUtime=0 cores=0,2,4,6
/proc/29459/task/29466/stat : 29466 (plingeling) R 29457 29459 28842 0 -1 4202560 5 1252 0 0 0 0 0 0 25 0 5 0 634857548 118136832 281 33554432000 4194304 4417452 140736834740000 18446744073709551615 4272148 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29459/tid=29467] ppid=29457 vsize=115368 CPUtime=0 cores=0,2,4,6
/proc/29459/task/29467/stat : 29467 (plingeling) R 29457 29459 28842 0 -1 4202560 19 1252 0 0 0 0 0 0 25 0 5 0 634857548 118136832 281 33554432000 4194304 4417452 140736834740000 18446744073709551615 220815930339 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29459/tid=29468] ppid=29457 vsize=115368 CPUtime=0 cores=0,2,4,6
/proc/29459/task/29468/stat : 29468 (plingeling) R 29457 29459 28842 0 -1 4202560 3 1252 0 0 0 0 0 0 25 0 5 0 634857548 118136832 281 33554432000 4194304 4417452 140736834740000 18446744073709551615 220816341486 0 0 4096 17442 0 0 0 -1 4 0 0 0

[startup+0.11712 s]
/proc/loadavg: 7.74 7.99 7.73 5/295 29459
/proc/meminfo: memFree=24278328/32951124 swapFree=63485424/67111528
[pid=29459] ppid=29457 vsize=118496 CPUtime=0.44 cores=0,2,4,6
/proc/29459/stat : 29459 (plingeling) S 29457 29459 28842 0 -1 4202496 1451 1252 0 0 44 0 0 0 25 0 5 0 634857548 121339904 1072 33554432000 4194304 4417452 140736834740000 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29459/statm: 29624 1072 179 55 0 27510 0
[pid=29459/tid=29465] ppid=29457 vsize=118496 CPUtime=0.11 cores=0,2,4,6
/proc/29459/task/29465/stat : 29465 (plingeling) R 29457 29459 28842 0 -1 4202560 188 1252 0 0 11 0 0 0 25 0 5 0 634857548 121339904 1072 33554432000 4194304 4417452 140736834740000 18446744073709551615 4252365 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29459/tid=29466] ppid=29457 vsize=118496 CPUtime=0.11 cores=0,2,4,6
/proc/29459/task/29466/stat : 29466 (plingeling) R 29457 29459 28842 0 -1 4202560 244 1252 0 0 11 0 0 0 25 0 5 0 634857548 121339904 1072 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364548 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29459/tid=29467] ppid=29457 vsize=118496 CPUtime=0.11 cores=0,2,4,6
/proc/29459/task/29467/stat : 29467 (plingeling) R 29457 29459 28842 0 -1 4202560 410 1252 0 0 11 0 0 0 25 0 5 0 634857548 121339904 1072 33554432000 4194304 4417452 140736834740000 18446744073709551615 4232626 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29459/tid=29468] ppid=29457 vsize=118496 CPUtime=0.1 cores=0,2,4,6
/proc/29459/task/29468/stat : 29468 (plingeling) R 29457 29459 28842 0 -1 4202560 311 1252 0 0 10 0 0 0 25 0 5 0 634857548 121339904 1072 33554432000 4194304 4417452 140736834740000 18446744073709551615 4250502 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 0.44
Current children cumulated vsize (KiB) 118496

[startup+0.201106 s]
/proc/loadavg: 7.74 7.99 7.73 5/295 29459
/proc/meminfo: memFree=24278328/32951124 swapFree=63485424/67111528
[pid=29459] ppid=29457 vsize=120224 CPUtime=0.77 cores=0,2,4,6
/proc/29459/stat : 29459 (plingeling) S 29457 29459 28842 0 -1 4202496 2394 1252 0 0 77 0 0 0 25 0 5 0 634857548 123109376 1506 33554432000 4194304 4417452 140736834740000 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29459/statm: 30056 1506 179 55 0 27942 0
[pid=29459/tid=29465] ppid=29457 vsize=120224 CPUtime=0.19 cores=0,2,4,6
/proc/29459/task/29465/stat : 29465 (plingeling) R 29457 29459 28842 0 -1 4202560 518 1252 0 0 19 0 0 0 25 0 5 0 634857548 123109376 1506 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29459/tid=29466] ppid=29457 vsize=120224 CPUtime=0.19 cores=0,2,4,6
/proc/29459/task/29466/stat : 29466 (plingeling) R 29457 29459 28842 0 -1 4202560 498 1252 0 0 19 0 0 0 25 0 5 0 634857548 123109376 1506 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364657 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29459/tid=29467] ppid=29457 vsize=120224 CPUtime=0.19 cores=0,2,4,6
/proc/29459/task/29467/stat : 29467 (plingeling) R 29457 29459 28842 0 -1 4202560 601 1252 0 0 19 0 0 0 25 0 5 0 634857548 123109376 1506 33554432000 4194304 4417452 140736834740000 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29459/tid=29468] ppid=29457 vsize=120224 CPUtime=0.18 cores=0,2,4,6
/proc/29459/task/29468/stat : 29468 (plingeling) R 29457 29459 28842 0 -1 4202560 479 1252 0 0 18 0 0 0 25 0 5 0 634857548 123109376 1506 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 0.77
Current children cumulated vsize (KiB) 120224

[startup+0.301087 s]
/proc/loadavg: 7.74 7.99 7.73 5/295 29459
/proc/meminfo: memFree=24278328/32951124 swapFree=63485424/67111528
[pid=29459] ppid=29457 vsize=121120 CPUtime=1.17 cores=0,2,4,6
/proc/29459/stat : 29459 (plingeling) S 29457 29459 28842 0 -1 4202496 3577 1252 0 0 117 0 0 0 25 0 5 0 634857548 124026880 1730 33554432000 4194304 4417452 140736834740000 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29459/statm: 30280 1730 179 55 0 28166 0
[pid=29459/tid=29465] ppid=29457 vsize=121120 CPUtime=0.29 cores=0,2,4,6
/proc/29459/task/29465/stat : 29465 (plingeling) R 29457 29459 28842 0 -1 4202560 790 1252 0 0 29 0 0 0 25 0 5 0 634857548 124026880 1730 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364737 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29459/tid=29466] ppid=29457 vsize=121120 CPUtime=0.29 cores=0,2,4,6
/proc/29459/task/29466/stat : 29466 (plingeling) R 29457 29459 28842 0 -1 4202560 804 1252 0 0 29 0 0 0 25 0 5 0 634857548 124026880 1730 33554432000 4194304 4417452 140736834740000 18446744073709551615 4321303 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29459/tid=29467] ppid=29457 vsize=121120 CPUtime=0.29 cores=0,2,4,6
/proc/29459/task/29467/stat : 29467 (plingeling) R 29457 29459 28842 0 -1 4202560 948 1252 0 0 29 0 0 0 25 0 5 0 634857548 124026880 1730 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29459/tid=29468] ppid=29457 vsize=121120 CPUtime=0.28 cores=0,2,4,6
/proc/29459/task/29468/stat : 29468 (plingeling) R 29457 29459 28842 0 -1 4202560 737 1252 0 0 28 0 0 0 25 0 5 0 634857548 124026880 1730 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 1.17
Current children cumulated vsize (KiB) 121120

[startup+0.701014 s]
/proc/loadavg: 7.74 7.99 7.73 5/295 29459
/proc/meminfo: memFree=24278328/32951124 swapFree=63485424/67111528
[pid=29459] ppid=29457 vsize=122088 CPUtime=2.77 cores=0,2,4,6
/proc/29459/stat : 29459 (plingeling) S 29457 29459 28842 0 -1 4202496 6388 1252 0 0 277 0 0 0 25 0 5 0 634857548 125018112 1972 33554432000 4194304 4417452 140736834740000 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29459/statm: 30522 1972 179 55 0 28408 0
[pid=29459/tid=29465] ppid=29457 vsize=122088 CPUtime=0.69 cores=0,2,4,6
/proc/29459/task/29465/stat : 29465 (plingeling) R 29457 29459 28842 0 -1 4202560 1623 1252 0 0 69 0 0 0 25 0 5 0 634857548 125018112 1972 33554432000 4194304 4417452 140736834740000 18446744073709551615 4250845 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29459/tid=29466] ppid=29457 vsize=122088 CPUtime=0.69 cores=0,2,4,6
/proc/29459/task/29466/stat : 29466 (plingeling) R 29457 29459 28842 0 -1 4202560 1431 1252 0 0 69 0 0 0 25 0 5 0 634857548 125018112 1972 33554432000 4194304 4417452 140736834740000 18446744073709551615 4365118 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29459/tid=29467] ppid=29457 vsize=122088 CPUtime=0.69 cores=0,2,4,6
/proc/29459/task/29467/stat : 29467 (plingeling) R 29457 29459 28842 0 -1 4202560 1709 1252 0 0 69 0 0 0 25 0 5 0 634857548 125018112 1972 33554432000 4194304 4417452 140736834740000 18446744073709551615 4365027 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29459/tid=29468] ppid=29457 vsize=122088 CPUtime=0.68 cores=0,2,4,6
/proc/29459/task/29468/stat : 29468 (plingeling) R 29457 29459 28842 0 -1 4202560 1327 1252 0 0 68 0 0 0 25 0 5 0 634857548 125018112 1972 33554432000 4194304 4417452 140736834740000 18446744073709551615 4358921 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 2.77
Current children cumulated vsize (KiB) 122088

[startup+1.50087 s]
/proc/loadavg: 7.74 7.99 7.73 9/300 29469
/proc/meminfo: memFree=24263652/32951124 swapFree=63485424/67111528
[pid=29459] ppid=29457 vsize=127388 CPUtime=5.96 cores=0,2,4,6
/proc/29459/stat : 29459 (plingeling) S 29457 29459 28842 0 -1 4202496 12271 1252 0 0 595 1 0 0 25 0 5 0 634857548 130445312 3291 33554432000 4194304 4417452 140736834740000 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29459/statm: 31847 3291 179 55 0 29733 0
[pid=29459/tid=29465] ppid=29457 vsize=127388 CPUtime=1.49 cores=0,2,4,6
/proc/29459/task/29465/stat : 29465 (plingeling) R 29457 29459 28842 0 -1 4202560 3079 1252 0 0 149 0 0 0 25 0 5 0 634857548 130445312 3291 33554432000 4194304 4417452 140736834740000 18446744073709551615 4251045 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29459/tid=29466] ppid=29457 vsize=127388 CPUtime=1.49 cores=0,2,4,6
/proc/29459/task/29466/stat : 29466 (plingeling) R 29457 29459 28842 0 -1 4202560 2756 1252 0 0 149 0 0 0 25 0 5 0 634857548 130445312 3291 33554432000 4194304 4417452 140736834740000 18446744073709551615 4365042 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29459/tid=29467] ppid=29457 vsize=127388 CPUtime=1.49 cores=0,2,4,6
/proc/29459/task/29467/stat : 29467 (plingeling) R 29457 29459 28842 0 -1 4202560 3200 1252 0 0 149 0 0 0 25 0 5 0 634857548 130445312 3291 33554432000 4194304 4417452 140736834740000 18446744073709551615 4356852 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29459/tid=29468] ppid=29457 vsize=127388 CPUtime=1.47 cores=0,2,4,6
/proc/29459/task/29468/stat : 29468 (plingeling) R 29457 29459 28842 0 -1 4202560 2938 1252 0 0 147 0 0 0 25 0 5 0 634857548 130445312 3291 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364587 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 5.96
Current children cumulated vsize (KiB) 127388

[startup+3.10058 s]
/proc/loadavg: 7.74 7.99 7.73 9/300 29469
/proc/meminfo: memFree=24261792/32951124 swapFree=63485424/67111528
[pid=29459] ppid=29457 vsize=132140 CPUtime=12.35 cores=0,2,4,6
/proc/29459/stat : 29459 (plingeling) S 29457 29459 28842 0 -1 4202496 20534 1252 0 0 1232 3 0 0 25 0 5 0 634857548 135311360 4478 33554432000 4194304 4417452 140736834740000 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29459/statm: 33035 4478 179 55 0 30921 0
[pid=29459/tid=29465] ppid=29457 vsize=132140 CPUtime=3.08 cores=0,2,4,6
/proc/29459/task/29465/stat : 29465 (plingeling) R 29457 29459 28842 0 -1 4202560 5494 1252 0 0 308 0 0 0 25 0 5 0 634857548 135311360 4478 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29459/tid=29466] ppid=29457 vsize=132140 CPUtime=3.08 cores=0,2,4,6
/proc/29459/task/29466/stat : 29466 (plingeling) R 29457 29459 28842 0 -1 4202560 5118 1252 0 0 308 0 0 0 25 0 5 0 634857548 135311360 4478 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29459/tid=29467] ppid=29457 vsize=132140 CPUtime=3.09 cores=0,2,4,6
/proc/29459/task/29467/stat : 29467 (plingeling) R 29457 29459 28842 0 -1 4202560 5163 1252 0 0 309 0 0 0 25 0 5 0 634857548 135311360 4478 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29459/tid=29468] ppid=29457 vsize=132140 CPUtime=3.06 cores=0,2,4,6
/proc/29459/task/29468/stat : 29468 (plingeling) R 29457 29459 28842 0 -1 4202560 4461 1252 0 0 306 0 0 0 25 0 5 0 634857548 135311360 4478 33554432000 4194304 4417452 140736834740000 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 12.35
Current children cumulated vsize (KiB) 132140

[startup+6.30098 s]
/proc/loadavg: 7.76 7.99 7.73 9/300 29469
/proc/meminfo: memFree=24241696/32951124 swapFree=63485424/67111528
[pid=29459] ppid=29457 vsize=138104 CPUtime=25.12 cores=0,2,4,6
/proc/29459/stat : 29459 (plingeling) S 29457 29459 28842 0 -1 4202496 34861 1252 0 0 2508 4 0 0 25 0 5 0 634857548 141418496 5972 33554432000 4194304 4417452 140736834740000 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29459/statm: 34526 5972 179 55 0 32412 0
[pid=29459/tid=29465] ppid=29457 vsize=138104 CPUtime=6.29 cores=0,2,4,6
/proc/29459/task/29465/stat : 29465 (plingeling) R 29457 29459 28842 0 -1 4202560 8680 1252 0 0 628 1 0 0 25 0 5 0 634857548 141418496 5972 33554432000 4194304 4417452 140736834740000 18446744073709551615 4366079 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29459/tid=29466] ppid=29457 vsize=138104 CPUtime=6.29 cores=0,2,4,6
/proc/29459/task/29466/stat : 29466 (plingeling) R 29457 29459 28842 0 -1 4202560 9094 1252 0 0 628 1 0 0 25 0 5 0 634857548 141418496 5972 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364885 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29459/tid=29467] ppid=29457 vsize=138104 CPUtime=6.28 cores=0,2,4,6
/proc/29459/task/29467/stat : 29467 (plingeling) R 29457 29459 28842 0 -1 4202560 9263 1252 0 0 628 0 0 0 25 0 5 0 634857548 141418496 5972 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364650 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29459/tid=29468] ppid=29457 vsize=138104 CPUtime=6.23 cores=0,2,4,6
/proc/29459/task/29468/stat : 29468 (plingeling) R 29457 29459 28842 0 -1 4202560 7526 1252 0 0 622 1 0 0 25 0 5 0 634857548 141418496 5972 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 25.12
Current children cumulated vsize (KiB) 138104

[startup+12.7008 s]
/proc/loadavg: 7.86 8.00 7.74 9/300 29469
/proc/meminfo: memFree=24230660/32951124 swapFree=63485424/67111528
[pid=29459] ppid=29457 vsize=139236 CPUtime=50.65 cores=0,2,4,6
/proc/29459/stat : 29459 (plingeling) S 29457 29459 28842 0 -1 4202496 59353 1252 0 0 5057 8 0 0 25 0 5 0 634857548 142577664 6240 33554432000 4194304 4417452 140736834740000 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29459/statm: 34809 6240 179 55 0 32695 0
[pid=29459/tid=29465] ppid=29457 vsize=139236 CPUtime=12.68 cores=0,2,4,6
/proc/29459/task/29465/stat : 29465 (plingeling) R 29457 29459 28842 0 -1 4202560 15259 1252 0 0 1267 1 0 0 25 0 5 0 634857548 142577664 6240 33554432000 4194304 4417452 140736834740000 18446744073709551615 4321386 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29459/tid=29466] ppid=29457 vsize=139236 CPUtime=12.69 cores=0,2,4,6
/proc/29459/task/29466/stat : 29466 (plingeling) R 29457 29459 28842 0 -1 4202560 15719 1252 0 0 1267 2 0 0 25 0 5 0 634857548 142577664 6240 33554432000 4194304 4417452 140736834740000 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29459/tid=29467] ppid=29457 vsize=139236 CPUtime=12.69 cores=0,2,4,6
/proc/29459/task/29467/stat : 29467 (plingeling) R 29457 29459 28842 0 -1 4202560 15285 1252 0 0 1268 1 0 0 25 0 5 0 634857548 142577664 6240 33554432000 4194304 4417452 140736834740000 18446744073709551615 4355572 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29459/tid=29468] ppid=29457 vsize=139236 CPUtime=12.57 cores=0,2,4,6
/proc/29459/task/29468/stat : 29468 (plingeling) R 29457 29459 28842 0 -1 4202560 12792 1252 0 0 1255 2 0 0 25 0 5 0 634857548 142577664 6240 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 50.65
Current children cumulated vsize (KiB) 139236

[startup+25.5004 s]
/proc/loadavg: 7.96 8.02 7.74 9/300 29469
/proc/meminfo: memFree=24248020/32951124 swapFree=63485424/67111528
[pid=29459] ppid=29457 vsize=152356 CPUtime=101.73 cores=0,2,4,6
/proc/29459/stat : 29459 (plingeling) S 29457 29459 28842 0 -1 4202496 101756 1252 0 0 10158 15 0 0 25 0 5 0 634857548 156012544 9507 33554432000 4194304 4417452 140736834740000 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29459/statm: 38089 9507 179 55 0 35975 0
[pid=29459/tid=29465] ppid=29457 vsize=152356 CPUtime=25.48 cores=0,2,4,6
/proc/29459/task/29465/stat : 29465 (plingeling) R 29457 29459 28842 0 -1 4202560 24959 1252 0 0 2545 3 0 0 25 0 5 0 634857548 156012544 9507 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364972 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29459/tid=29466] ppid=29457 vsize=152356 CPUtime=25.49 cores=0,2,4,6
/proc/29459/task/29466/stat : 29466 (plingeling) R 29457 29459 28842 0 -1 4202560 26344 1252 0 0 2546 3 0 0 25 0 5 0 634857548 156012544 9507 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29459/tid=29467] ppid=29457 vsize=152356 CPUtime=25.49 cores=0,2,4,6
/proc/29459/task/29467/stat : 29467 (plingeling) R 29457 29459 28842 0 -1 4202560 26103 1252 0 0 2546 3 0 0 25 0 5 0 634857548 156012544 9507 33554432000 4194304 4417452 140736834740000 18446744073709551615 4365023 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29459/tid=29468] ppid=29457 vsize=152356 CPUtime=25.25 cores=0,2,4,6
/proc/29459/task/29468/stat : 29468 (plingeling) R 29457 29459 28842 0 -1 4202560 24052 1252 0 0 2521 4 0 0 25 0 5 0 634857548 156012544 9507 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 101.73
Current children cumulated vsize (KiB) 152356

[startup+51.1007 s]
/proc/loadavg: 7.97 8.02 7.75 9/300 29470
/proc/meminfo: memFree=24240404/32951124 swapFree=63485424/67111528
[pid=29459] ppid=29457 vsize=165092 CPUtime=203.91 cores=0,2,4,6
/proc/29459/stat : 29459 (plingeling) S 29457 29459 28842 0 -1 4202496 161569 1252 0 0 20369 22 0 0 25 0 5 0 634857548 169054208 12721 33554432000 4194304 4417452 140736834740000 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29459/statm: 41273 12721 179 55 0 39159 0
[pid=29459/tid=29465] ppid=29457 vsize=165092 CPUtime=51.08 cores=0,2,4,6
/proc/29459/task/29465/stat : 29465 (plingeling) R 29457 29459 28842 0 -1 4202560 40029 1252 0 0 5103 5 0 0 25 0 5 0 634857548 169054208 12721 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29459/tid=29466] ppid=29457 vsize=165092 CPUtime=51.08 cores=0,2,4,6
/proc/29459/task/29466/stat : 29466 (plingeling) R 29457 29459 28842 0 -1 4202560 40604 1252 0 0 5104 4 0 0 25 0 5 0 634857548 169054208 12721 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364780 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29459/tid=29467] ppid=29457 vsize=165092 CPUtime=51.09 cores=0,2,4,6

################
# More data... #
################

[pid=29459/tid=29466] ppid=29457 vsize=255980 CPUtime=762.23 cores=0,2,4,6
/proc/29459/task/29466/stat : 29466 (plingeling) R 29457 29459 28842 0 -1 4202560 274794 1252 0 0 76185 38 0 0 25 0 5 0 634857548 262123520 35437 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29459/tid=29467] ppid=29457 vsize=255980 CPUtime=762.18 cores=0,2,4,6
/proc/29459/task/29467/stat : 29467 (plingeling) R 29457 29459 28842 0 -1 4202560 263759 1252 0 0 76169 49 0 0 25 0 5 0 634857548 262123520 35437 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29459/tid=29468] ppid=29457 vsize=255980 CPUtime=755.23 cores=0,2,4,6
/proc/29459/task/29468/stat : 29468 (plingeling) R 29457 29459 28842 0 -1 4202560 283796 1252 0 0 75481 42 0 0 25 0 5 0 634857548 262123520 35437 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3041.24
Current children cumulated vsize (KiB) 255980

[startup+822.301 s]
/proc/loadavg: 5.77 7.78 7.90 5/290 31773
/proc/meminfo: memFree=24160192/32951124 swapFree=63485424/67111528
[pid=29459] ppid=29457 vsize=251952 CPUtime=3280.66 cores=0,2,4,6
/proc/29459/stat : 29459 (plingeling) S 29457 29459 28842 0 -1 4202496 1154820 1252 0 0 327885 181 0 0 25 0 5 0 634857548 257998848 34406 33554432000 4194304 4417452 140736834740000 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29459/statm: 62988 34406 179 55 0 60874 0
[pid=29459/tid=29465] ppid=29457 vsize=251952 CPUtime=821.56 cores=0,2,4,6
/proc/29459/task/29465/stat : 29465 (plingeling) R 29457 29459 28842 0 -1 4202560 286437 1252 0 0 82114 42 0 0 25 0 5 0 634857548 257998848 34406 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29459/tid=29466] ppid=29457 vsize=251952 CPUtime=822.23 cores=0,2,4,6
/proc/29459/task/29466/stat : 29466 (plingeling) R 29457 29459 28842 0 -1 4202560 289271 1252 0 0 82183 40 0 0 25 0 5 0 634857548 257998848 34406 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29459/tid=29467] ppid=29457 vsize=251952 CPUtime=822.17 cores=0,2,4,6
/proc/29459/task/29467/stat : 29467 (plingeling) R 29457 29459 28842 0 -1 4202560 281066 1252 0 0 82164 53 0 0 25 0 5 0 634857548 257998848 34406 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29459/tid=29468] ppid=29457 vsize=251952 CPUtime=814.67 cores=0,2,4,6
/proc/29459/task/29468/stat : 29468 (plingeling) R 29457 29459 28842 0 -1 4202560 297748 1252 0 0 81423 44 0 0 25 0 5 0 634857548 257998848 34406 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3280.66
Current children cumulated vsize (KiB) 251952

[startup+882.311 s]
/proc/loadavg: 4.70 7.10 7.65 5/290 31774
/proc/meminfo: memFree=24166216/32951124 swapFree=63485424/67111528
[pid=29459] ppid=29457 vsize=251424 CPUtime=3520.15 cores=0,2,4,6
/proc/29459/stat : 29459 (plingeling) S 29457 29459 28842 0 -1 4202496 1214543 1252 0 0 351826 189 0 0 25 0 5 0 634857548 257458176 34274 33554432000 4194304 4417452 140736834740000 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29459/statm: 62856 34274 179 55 0 60742 0
[pid=29459/tid=29465] ppid=29457 vsize=251424 CPUtime=881.57 cores=0,2,4,6
/proc/29459/task/29465/stat : 29465 (plingeling) R 29457 29459 28842 0 -1 4202560 303399 1252 0 0 88112 45 0 0 25 0 5 0 634857548 257458176 34274 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29459/tid=29466] ppid=29457 vsize=251424 CPUtime=882.24 cores=0,2,4,6
/proc/29459/task/29466/stat : 29466 (plingeling) R 29457 29459 28842 0 -1 4202560 306210 1252 0 0 88182 42 0 0 25 0 5 0 634857548 257458176 34274 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29459/tid=29467] ppid=29457 vsize=251424 CPUtime=882.19 cores=0,2,4,6
/proc/29459/task/29467/stat : 29467 (plingeling) R 29457 29459 28842 0 -1 4202560 291701 1252 0 0 88164 55 0 0 25 0 5 0 634857548 257458176 34274 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29459/tid=29468] ppid=29457 vsize=251424 CPUtime=874.13 cores=0,2,4,6
/proc/29459/task/29468/stat : 29468 (plingeling) R 29457 29459 28842 0 -1 4202560 312935 1252 0 0 87367 46 0 0 25 0 5 0 634857548 257458176 34274 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3520.15
Current children cumulated vsize (KiB) 251424

[startup+942.301 s]
/proc/loadavg: 4.30 6.55 7.43 5/290 31776
/proc/meminfo: memFree=24172248/32951124 swapFree=63485424/67111528
[pid=29459] ppid=29457 vsize=238348 CPUtime=3759.57 cores=0,2,4,6
/proc/29459/stat : 29459 (plingeling) S 29457 29459 28842 0 -1 4202496 1269324 1252 0 0 375761 196 0 0 25 0 5 0 634857548 244068352 31011 33554432000 4194304 4417452 140736834740000 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29459/statm: 59587 31011 179 55 0 57473 0
[pid=29459/tid=29465] ppid=29457 vsize=238348 CPUtime=941.56 cores=0,2,4,6
/proc/29459/task/29465/stat : 29465 (plingeling) R 29457 29459 28842 0 -1 4202560 315928 1252 0 0 94110 46 0 0 25 0 5 0 634857548 244068352 31011 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29459/tid=29466] ppid=29457 vsize=238348 CPUtime=942.23 cores=0,2,4,6
/proc/29459/task/29466/stat : 29466 (plingeling) R 29457 29459 28842 0 -1 4202560 316563 1252 0 0 94180 43 0 0 25 0 5 0 634857548 244068352 31011 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29459/tid=29467] ppid=29457 vsize=238348 CPUtime=942.17 cores=0,2,4,6
/proc/29459/task/29467/stat : 29467 (plingeling) R 29457 29459 28842 0 -1 4202560 308572 1252 0 0 94160 57 0 0 25 0 5 0 634857548 244068352 31011 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29459/tid=29468] ppid=29457 vsize=238348 CPUtime=933.58 cores=0,2,4,6
/proc/29459/task/29468/stat : 29468 (plingeling) R 29457 29459 28842 0 -1 4202560 327963 1252 0 0 93310 48 0 0 25 0 5 0 634857548 244068352 31011 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364807 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3759.57
Current children cumulated vsize (KiB) 238348

[startup+1002.3 s]
/proc/loadavg: 4.15 6.10 7.22 5/290 31778
/proc/meminfo: memFree=24144792/32951124 swapFree=63485424/67111528
[pid=29459] ppid=29457 vsize=264936 CPUtime=3999.03 cores=0,2,4,6
/proc/29459/stat : 29459 (plingeling) S 29457 29459 28842 0 -1 4202496 1327284 1252 0 0 399699 204 0 0 25 0 5 0 634857548 271294464 37652 33554432000 4194304 4417452 140736834740000 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29459/statm: 66234 37652 179 55 0 64120 0
[pid=29459/tid=29465] ppid=29457 vsize=264936 CPUtime=1001.57 cores=0,2,4,6
/proc/29459/task/29465/stat : 29465 (plingeling) R 29457 29459 28842 0 -1 4202560 324999 1252 0 0 100109 48 0 0 25 0 5 0 634857548 271294464 37652 33554432000 4194304 4417452 140736834740000 18446744073709551615 4354217 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29459/tid=29466] ppid=29457 vsize=264936 CPUtime=1002.23 cores=0,2,4,6
/proc/29459/task/29466/stat : 29466 (plingeling) R 29457 29459 28842 0 -1 4202560 335323 1252 0 0 100178 45 0 0 25 0 5 0 634857548 271294464 37652 33554432000 4194304 4417452 140736834740000 18446744073709551615 4366400 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29459/tid=29467] ppid=29457 vsize=264936 CPUtime=1002.18 cores=0,2,4,6
/proc/29459/task/29467/stat : 29467 (plingeling) R 29457 29459 28842 0 -1 4202560 325985 1252 0 0 100159 59 0 0 25 0 5 0 634857548 271294464 37652 33554432000 4194304 4417452 140736834740000 18446744073709551615 4366804 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29459/tid=29468] ppid=29457 vsize=264936 CPUtime=993.03 cores=0,2,4,6
/proc/29459/task/29468/stat : 29468 (plingeling) R 29457 29459 28842 0 -1 4202560 340679 1252 0 0 99253 50 0 0 25 0 5 0 634857548 271294464 37652 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3999.03
Current children cumulated vsize (KiB) 264936

[startup+1062.3 s]
/proc/loadavg: 4.10 5.73 7.02 5/290 31779
/proc/meminfo: memFree=24151468/32951124 swapFree=63485424/67111528
[pid=29459] ppid=29457 vsize=258380 CPUtime=4238.5 cores=0,2,4,6
/proc/29459/stat : 29459 (plingeling) S 29457 29459 28842 0 -1 4202496 1377354 1252 0 0 423639 211 0 0 25 0 5 0 634857548 264581120 36013 33554432000 4194304 4417452 140736834740000 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29459/statm: 64595 36013 179 55 0 62481 0
[pid=29459/tid=29465] ppid=29457 vsize=258380 CPUtime=1061.57 cores=0,2,4,6
/proc/29459/task/29465/stat : 29465 (plingeling) R 29457 29459 28842 0 -1 4202560 332129 1252 0 0 106108 49 0 0 25 0 5 0 634857548 264581120 36013 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29459/tid=29466] ppid=29457 vsize=258380 CPUtime=1062.23 cores=0,2,4,6
/proc/29459/task/29466/stat : 29466 (plingeling) R 29457 29459 28842 0 -1 4202560 347239 1252 0 0 106176 47 0 0 25 0 5 0 634857548 264581120 36013 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29459/tid=29467] ppid=29457 vsize=258380 CPUtime=1062.18 cores=0,2,4,6
/proc/29459/task/29467/stat : 29467 (plingeling) R 29457 29459 28842 0 -1 4202560 343019 1252 0 0 106157 61 0 0 25 0 5 0 634857548 264581120 36013 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364558 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29459/tid=29468] ppid=29457 vsize=258380 CPUtime=1052.49 cores=0,2,4,6
/proc/29459/task/29468/stat : 29468 (plingeling) R 29457 29459 28842 0 -1 4202560 354669 1252 0 0 105197 52 0 0 25 0 5 0 634857548 264581120 36013 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4238.5
Current children cumulated vsize (KiB) 258380

[startup+1122.3 s]
/proc/loadavg: 4.07 5.43 6.83 5/290 31781
/proc/meminfo: memFree=24171132/32951124 swapFree=63485424/67111528
[pid=29459] ppid=29457 vsize=234648 CPUtime=4477.95 cores=0,2,4,6
/proc/29459/stat : 29459 (plingeling) S 29457 29459 28842 0 -1 4202496 1432584 1252 0 0 447577 218 0 0 25 0 5 0 634857548 240279552 30080 33554432000 4194304 4417452 140736834740000 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29459/statm: 58662 30080 179 55 0 56548 0
[pid=29459/tid=29465] ppid=29457 vsize=234648 CPUtime=1121.56 cores=0,2,4,6
/proc/29459/task/29465/stat : 29465 (plingeling) R 29457 29459 28842 0 -1 4202560 345338 1252 0 0 112106 50 0 0 25 0 5 0 634857548 240279552 30080 33554432000 4194304 4417452 140736834740000 18446744073709551615 4365088 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29459/tid=29466] ppid=29457 vsize=234648 CPUtime=1122.24 cores=0,2,4,6
/proc/29459/task/29466/stat : 29466 (plingeling) R 29457 29459 28842 0 -1 4202560 363087 1252 0 0 112174 50 0 0 25 0 5 0 634857548 240279552 30080 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29459/tid=29467] ppid=29457 vsize=234648 CPUtime=1122.18 cores=0,2,4,6
/proc/29459/task/29467/stat : 29467 (plingeling) R 29457 29459 28842 0 -1 4202560 356823 1252 0 0 112155 63 0 0 25 0 5 0 634857548 240279552 30080 33554432000 4194304 4417452 140736834740000 18446744073709551615 4365027 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29459/tid=29468] ppid=29457 vsize=234648 CPUtime=1111.94 cores=0,2,4,6
/proc/29459/task/29468/stat : 29468 (plingeling) R 29457 29459 28842 0 -1 4202560 367038 1252 0 0 111140 54 0 0 25 0 5 0 634857548 240279552 30080 33554432000 4194304 4417452 140736834740000 18446744073709551615 4365023 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4477.95
Current children cumulated vsize (KiB) 234648

[startup+1182.3 s]
/proc/loadavg: 4.06 5.18 6.66 5/290 31782
/proc/meminfo: memFree=24149892/32951124 swapFree=63485424/67111528
[pid=29459] ppid=29457 vsize=261916 CPUtime=4717.42 cores=0,2,4,6
/proc/29459/stat : 29459 (plingeling) S 29457 29459 28842 0 -1 4202496 1493922 1252 0 0 471517 225 0 0 25 0 5 0 634857548 268201984 36897 33554432000 4194304 4417452 140736834740000 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29459/statm: 65479 36897 179 55 0 63365 0
[pid=29459/tid=29465] ppid=29457 vsize=261916 CPUtime=1181.56 cores=0,2,4,6
/proc/29459/task/29465/stat : 29465 (plingeling) R 29457 29459 28842 0 -1 4202560 355330 1252 0 0 118105 51 0 0 25 0 5 0 634857548 268201984 36897 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29459/tid=29466] ppid=29457 vsize=261916 CPUtime=1182.24 cores=0,2,4,6
/proc/29459/task/29466/stat : 29466 (plingeling) R 29457 29459 28842 0 -1 4202560 380548 1252 0 0 118172 52 0 0 25 0 5 0 634857548 268201984 36897 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29459/tid=29467] ppid=29457 vsize=261916 CPUtime=1182.18 cores=0,2,4,6
/proc/29459/task/29467/stat : 29467 (plingeling) R 29457 29459 28842 0 -1 4202560 374873 1252 0 0 118153 65 0 0 25 0 5 0 634857548 268201984 36897 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29459/tid=29468] ppid=29457 vsize=261916 CPUtime=1171.41 cores=0,2,4,6
/proc/29459/task/29468/stat : 29468 (plingeling) R 29457 29459 28842 0 -1 4202560 382873 1252 0 0 117085 56 0 0 25 0 5 0 634857548 268201984 36897 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4717.42
Current children cumulated vsize (KiB) 261916



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203 s]
/proc/loadavg: 4.10 5.12 6.60 5/290 31783
/proc/meminfo: memFree=24136848/32951124 swapFree=63485424/67111528
[pid=29459] ppid=29457 vsize=272952 CPUtime=4800.06 cores=0,2,4,6
/proc/29459/stat : 29459 (plingeling) S 29457 29459 28842 0 -1 4202496 1514252 1252 0 0 479778 228 0 0 25 0 5 0 634857548 279502848 39656 33554432000 4194304 4417452 140736834740000 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29459/statm: 68238 39656 179 55 0 66124 0
[pid=29459/tid=29465] ppid=29457 vsize=272952 CPUtime=1202.28 cores=0,2,4,6
/proc/29459/task/29465/stat : 29465 (plingeling) R 29457 29459 28842 0 -1 4202560 361683 1252 0 0 120175 53 0 0 25 0 5 0 634857548 279502848 39656 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29459/tid=29466] ppid=29457 vsize=272952 CPUtime=1202.95 cores=0,2,4,6
/proc/29459/task/29466/stat : 29466 (plingeling) R 29457 29459 28842 0 -1 4202560 384906 1252 0 0 120243 52 0 0 25 0 5 0 634857548 279502848 39656 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364873 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29459/tid=29467] ppid=29457 vsize=272952 CPUtime=1202.89 cores=0,2,4,6
/proc/29459/task/29467/stat : 29467 (plingeling) R 29457 29459 28842 0 -1 4202560 379441 1252 0 0 120224 65 0 0 25 0 5 0 634857548 279502848 39656 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29459/tid=29468] ppid=29457 vsize=272952 CPUtime=1191.91 cores=0,2,4,6
/proc/29459/task/29468/stat : 29468 (plingeling) R 29457 29459 28842 0 -1 4202560 387924 1252 0 0 119135 56 0 0 25 0 5 0 634857548 279502848 39656 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4800.06
Current children cumulated vsize (KiB) 272952

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1203 s]
/proc/loadavg: 4.10 5.12 6.60 5/290 31783
/proc/meminfo: memFree=24136848/32951124 swapFree=63485424/67111528
[pid=29459] ppid=29457 vsize=272952 CPUtime=4800.06 cores=0,2,4,6
/proc/29459/stat : 29459 (plingeling) S 29457 29459 28842 0 -1 4202496 1514252 1252 0 0 479778 228 0 0 25 0 5 0 634857548 279502848 39656 33554432000 4194304 4417452 140736834740000 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29459/statm: 68238 39656 179 55 0 66124 0
[pid=29459/tid=29465] ppid=29457 vsize=272952 CPUtime=1202.28 cores=0,2,4,6
/proc/29459/task/29465/stat : 29465 (plingeling) R 29457 29459 28842 0 -1 4202560 361683 1252 0 0 120175 53 0 0 25 0 5 0 634857548 279502848 39656 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29459/tid=29466] ppid=29457 vsize=272952 CPUtime=1202.95 cores=0,2,4,6
/proc/29459/task/29466/stat : 29466 (plingeling) R 29457 29459 28842 0 -1 4202560 384906 1252 0 0 120243 52 0 0 25 0 5 0 634857548 279502848 39656 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364873 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29459/tid=29467] ppid=29457 vsize=272952 CPUtime=1202.89 cores=0,2,4,6
/proc/29459/task/29467/stat : 29467 (plingeling) R 29457 29459 28842 0 -1 4202560 379441 1252 0 0 120224 65 0 0 25 0 5 0 634857548 279502848 39656 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29459/tid=29468] ppid=29457 vsize=272952 CPUtime=1191.91 cores=0,2,4,6
/proc/29459/task/29468/stat : 29468 (plingeling) R 29457 29459 28842 0 -1 4202560 387924 1252 0 0 119135 56 0 0 25 0 5 0 634857548 279502848 39656 33554432000 4194304 4417452 140736834740000 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4800.06
Current children cumulated vsize (KiB) 272952

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1203.02
CPU time (s): 4800.11
CPU user time (s): 4797.81
CPU system time (s): 2.29665
CPU usage (%): 399.004
Max. virtual memory (cumulated for all children) (KiB): 295892

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4797.81
system time used= 2.29665
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1515504
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 141
involuntary context switches= 24444

runsolver used 3.37049 second user time and 7.58685 second system time

The end

Launcher Data 

Begin job on node147 at 2011-05-15 03:53:36
IDJOB=3371184
IDBENCH=83147
IDSOLVER=1848
FILE ID=node147/3371184-1305424416
RUNJOBID= node147-1305423211-29247
PBS_JOBID= 13324574
Free space on /tmp= 73108 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/crafted/spence/sgen/sgen3-n120-s55656844-unsat.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3371184-1305424416/watcher-3371184-1305424416 -o /tmp/evaluation-result-3371184-1305424416/solver-3371184-1305424416 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3371184-1305424416.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 4877ca9f3fe317c84d3087c8296a2a0b
RANDOM SEED=420448241

node147.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      24278492 kB
Buffers:        294636 kB
Cached:        5015720 kB
SwapCached:      13132 kB
Active:        1740456 kB
Inactive:      6699820 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      24278492 kB
SwapTotal:    67111528 kB
SwapFree:     63485424 kB
Dirty:              20 kB
Writeback:           0 kB
AnonPages:     3124944 kB
Mapped:          15256 kB
Slab:           140664 kB
PageTables:      29156 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 44991676 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73160 MiB
End job on node147 at 2011-05-15 04:13:41