Trace number 3249580

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 4800.4 1202.06

General information on the benchmark

Namecrafted/mosoi/289/
289-unsat-9x32.cnf
MD5SUMbcc12436eba65ad776b55f23c6f42298
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables1152
Number of clauses73440
Sum of the clauses size290304
Maximum clause length4
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 21728
Number of clauses of size 30
Number of clauses of size 471712
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3249580-1304086912/watcher-3249580-1304086912 -o /tmp/evaluation-result-3249580-1304086912/solver-3249580-1304086912 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3249580-1304086912.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.73 3.95 4.04 5/181 22105
/proc/meminfo: memFree=30635836/32951124 swapFree=67111356/67111528
[pid=22105] ppid=22103 vsize=9616 CPUtime=0 cores=1,3,5,7
/proc/22105/stat : 22105 (plingeling) R 22103 22105 20894 0 -1 4202496 468 1250 0 0 0 0 0 0 25 0 1 0 501107485 9846784 397 33554432000 4194304 4417332 140735446891360 18446744073709551615 212039655168 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/22105/statm: 2404 408 146 55 0 290 0

[startup+0.057929 s]
/proc/loadavg: 3.73 3.95 4.04 5/181 22105
/proc/meminfo: memFree=30635836/32951124 swapFree=67111356/67111528
[pid=22105] ppid=22103 vsize=21824 CPUtime=0.05 cores=1,3,5,7
/proc/22105/stat : 22105 (plingeling) R 22103 22105 20894 0 -1 4202496 3709 1250 0 0 5 0 0 0 25 0 1 0 501107485 22347776 3478 33554432000 4194304 4417332 140735446891360 18446744073709551615 212039655168 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/22105/statm: 5456 3478 147 55 0 3342 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 21824

[startup+0.190859 s]
/proc/loadavg: 3.73 3.95 4.04 5/181 22105
/proc/meminfo: memFree=30635836/32951124 swapFree=67111356/67111528
[pid=22105] ppid=22103 vsize=69376 CPUtime=0.48 cores=1,3,5,7
/proc/22105/stat : 22105 (plingeling) S 22103 22105 20894 0 -1 4202496 6259 1250 0 0 48 0 0 0 25 0 5 0 501107485 71041024 5135 33554432000 4194304 4417332 140735446891360 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22105/statm: 17344 5135 176 55 0 15230 0
Current children cumulated CPU time (s) 0.48
Current children cumulated vsize (KiB) 69376

[startup+0.300843 s]
/proc/loadavg: 3.73 3.95 4.04 5/181 22105
/proc/meminfo: memFree=30635836/32951124 swapFree=67111356/67111528
[pid=22105] ppid=22103 vsize=73388 CPUtime=0.93 cores=1,3,5,7
/proc/22105/stat : 22105 (plingeling) S 22103 22105 20894 0 -1 4202496 7492 1250 0 0 92 1 0 0 25 0 5 0 501107485 75149312 6160 33554432000 4194304 4417332 140735446891360 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22105/statm: 18382 6182 179 55 0 16268 0
Current children cumulated CPU time (s) 0.93
Current children cumulated vsize (KiB) 73388

[startup+0.700768 s]
/proc/loadavg: 3.73 3.95 4.04 5/181 22105
/proc/meminfo: memFree=30635836/32951124 swapFree=67111356/67111528
[pid=22105] ppid=22103 vsize=83340 CPUtime=2.53 cores=1,3,5,7
/proc/22105/stat : 22105 (plingeling) S 22103 22105 20894 0 -1 4202496 10592 1250 0 0 251 2 0 0 25 0 5 0 501107485 85340160 8653 33554432000 4194304 4417332 140735446891360 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22105/statm: 20835 8653 179 55 0 18721 0
Current children cumulated CPU time (s) 2.53
Current children cumulated vsize (KiB) 83340

[startup+1.50063 s]
/proc/loadavg: 3.73 3.95 4.04 9/186 22115
/proc/meminfo: memFree=30571812/32951124 swapFree=67111356/67111528
[pid=22105] ppid=22103 vsize=90256 CPUtime=5.72 cores=1,3,5,7
/proc/22105/stat : 22105 (plingeling) S 22103 22105 20894 0 -1 4202496 14492 1250 0 0 570 2 0 0 25 0 5 0 501107485 92422144 10350 33554432000 4194304 4417332 140735446891360 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22105/statm: 22564 10350 179 55 0 20450 0
[pid=22105/tid=22112] ppid=22103 vsize=90256 CPUtime=1.41 cores=1,3,5,7
/proc/22105/task/22112/stat : 22112 (plingeling) R 22103 22105 20894 0 -1 4202560 1649 1250 0 0 141 0 0 0 25 0 5 0 501107494 92422144 10350 33554432000 4194304 4417332 140735446891360 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22105/tid=22113] ppid=22103 vsize=90256 CPUtime=1.4 cores=1,3,5,7
/proc/22105/task/22113/stat : 22113 (plingeling) R 22103 22105 20894 0 -1 4202560 1667 1250 0 0 140 0 0 0 25 0 5 0 501107494 92422144 10350 33554432000 4194304 4417332 140735446891360 18446744073709551615 4249110 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22105/tid=22114] ppid=22103 vsize=90256 CPUtime=1.4 cores=1,3,5,7
/proc/22105/task/22114/stat : 22114 (plingeling) R 22103 22105 20894 0 -1 4202560 1569 1250 0 0 140 0 0 0 25 0 5 0 501107494 92422144 10350 33554432000 4194304 4417332 140735446891360 18446744073709551615 4365991 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=22105/tid=22115] ppid=22103 vsize=90256 CPUtime=1.41 cores=1,3,5,7
/proc/22105/task/22115/stat : 22115 (plingeling) R 22103 22105 20894 0 -1 4202560 4862 1250 0 0 140 1 0 0 25 0 5 0 501107494 92422144 10350 33554432000 4194304 4417332 140735446891360 18446744073709551615 4250660 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 5.72
Current children cumulated vsize (KiB) 90256

[startup+3.10034 s]
/proc/loadavg: 3.73 3.95 4.04 9/186 22115
/proc/meminfo: memFree=30556140/32951124 swapFree=67111356/67111528
[pid=22105] ppid=22103 vsize=167464 CPUtime=12.11 cores=1,3,5,7
/proc/22105/stat : 22105 (plingeling) S 22103 22105 20894 0 -1 4202496 30217 1250 0 0 1206 5 0 0 25 0 5 0 501107485 171483136 13517 33554432000 4194304 4417332 140735446891360 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22105/statm: 41866 13517 180 55 0 39752 0
[pid=22105/tid=22112] ppid=22103 vsize=167464 CPUtime=3 cores=1,3,5,7
/proc/22105/task/22112/stat : 22112 (plingeling) R 22103 22105 20894 0 -1 4202560 4884 1250 0 0 300 0 0 0 25 0 5 0 501107494 171483136 13517 33554432000 4194304 4417332 140735446891360 18446744073709551615 4366359 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22105/tid=22113] ppid=22103 vsize=167464 CPUtime=3 cores=1,3,5,7
/proc/22105/task/22113/stat : 22113 (plingeling) R 22103 22105 20894 0 -1 4202560 5938 1250 0 0 299 1 0 0 25 0 5 0 501107494 171483136 13517 33554432000 4194304 4417332 140735446891360 18446744073709551615 4320455 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22105/tid=22114] ppid=22103 vsize=167464 CPUtime=2.99 cores=1,3,5,7
/proc/22105/task/22114/stat : 22114 (plingeling) R 22103 22105 20894 0 -1 4202560 7281 1250 0 0 298 1 0 0 25 0 5 0 501107494 171483136 13517 33554432000 4194304 4417332 140735446891360 18446744073709551615 4249209 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=22105/tid=22115] ppid=22103 vsize=167464 CPUtime=3 cores=1,3,5,7
/proc/22105/task/22115/stat : 22115 (plingeling) R 22103 22105 20894 0 -1 4202560 7369 1250 0 0 299 1 0 0 25 0 5 0 501107494 171483136 13517 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 12.11
Current children cumulated vsize (KiB) 167464

[startup+6.30077 s]
/proc/loadavg: 4.07 4.01 4.06 9/186 22115
/proc/meminfo: memFree=30512360/32951124 swapFree=67111356/67111528
[pid=22105] ppid=22103 vsize=185384 CPUtime=24.9 cores=1,3,5,7
/proc/22105/stat : 22105 (plingeling) S 22103 22105 20894 0 -1 4202496 55310 1250 0 0 2481 9 0 0 25 0 5 0 501107485 189833216 17782 33554432000 4194304 4417332 140735446891360 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22105/statm: 46346 17782 180 55 0 44232 0
[pid=22105/tid=22112] ppid=22103 vsize=185384 CPUtime=6.21 cores=1,3,5,7
/proc/22105/task/22112/stat : 22112 (plingeling) R 22103 22105 20894 0 -1 4202560 12513 1250 0 0 619 2 0 0 25 0 5 0 501107494 189833216 17782 33554432000 4194304 4417332 140735446891360 18446744073709551615 4244345 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22105/tid=22113] ppid=22103 vsize=185384 CPUtime=6.2 cores=1,3,5,7
/proc/22105/task/22113/stat : 22113 (plingeling) R 22103 22105 20894 0 -1 4202560 7995 1250 0 0 619 1 0 0 25 0 5 0 501107494 189833216 17782 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364683 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22105/tid=22114] ppid=22103 vsize=185384 CPUtime=6.17 cores=1,3,5,7
/proc/22105/task/22114/stat : 22114 (plingeling) R 22103 22105 20894 0 -1 4202560 17757 1250 0 0 615 2 0 0 25 0 5 0 501107494 189833216 17782 33554432000 4194304 4417332 140735446891360 18446744073709551615 4355716 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=22105/tid=22115] ppid=22103 vsize=185384 CPUtime=6.2 cores=1,3,5,7
/proc/22105/task/22115/stat : 22115 (plingeling) R 22103 22105 20894 0 -1 4202560 12300 1250 0 0 618 2 0 0 25 0 5 0 501107494 189833216 17782 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364470 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 24.9
Current children cumulated vsize (KiB) 185384

[startup+12.7006 s]
/proc/loadavg: 4.38 4.08 4.08 9/186 22115
/proc/meminfo: memFree=30426208/32951124 swapFree=67111356/67111528
[pid=22105] ppid=22103 vsize=262968 CPUtime=50.45 cores=1,3,5,7
/proc/22105/stat : 22105 (plingeling) S 22103 22105 20894 0 -1 4202496 108817 1250 0 0 5028 17 0 0 25 0 5 0 501107485 269279232 37178 33554432000 4194304 4417332 140735446891360 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22105/statm: 65742 37178 180 55 0 63628 0
[pid=22105/tid=22112] ppid=22103 vsize=262968 CPUtime=12.61 cores=1,3,5,7
/proc/22105/task/22112/stat : 22112 (plingeling) R 22103 22105 20894 0 -1 4202560 26000 1250 0 0 1257 4 0 0 25 0 5 0 501107494 269279232 37178 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364715 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22105/tid=22113] ppid=22103 vsize=262968 CPUtime=12.6 cores=1,3,5,7
/proc/22105/task/22113/stat : 22113 (plingeling) R 22103 22105 20894 0 -1 4202560 22604 1250 0 0 1257 3 0 0 25 0 5 0 501107494 269279232 37178 33554432000 4194304 4417332 140735446891360 18446744073709551615 4250669 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22105/tid=22114] ppid=22103 vsize=262968 CPUtime=12.54 cores=1,3,5,7
/proc/22105/task/22114/stat : 22114 (plingeling) R 22103 22105 20894 0 -1 4202560 34015 1250 0 0 1249 5 0 0 25 0 5 0 501107494 269279232 37178 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364962 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=22105/tid=22115] ppid=22103 vsize=262968 CPUtime=12.61 cores=1,3,5,7
/proc/22105/task/22115/stat : 22115 (plingeling) R 22103 22105 20894 0 -1 4202560 21453 1250 0 0 1257 4 0 0 25 0 5 0 501107494 269279232 37178 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364923 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 50.45
Current children cumulated vsize (KiB) 262968

[startup+25.5003 s]
/proc/loadavg: 5.18 4.27 4.14 9/186 22116
/proc/meminfo: memFree=30305176/32951124 swapFree=67111356/67111528
[pid=22105] ppid=22103 vsize=306772 CPUtime=101.59 cores=1,3,5,7
/proc/22105/stat : 22105 (plingeling) S 22103 22105 20894 0 -1 4202496 178448 1250 0 0 10130 29 0 0 25 0 5 0 501107485 314134528 48129 33554432000 4194304 4417332 140735446891360 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22105/statm: 76693 48129 180 55 0 74579 0
[pid=22105/tid=22112] ppid=22103 vsize=306772 CPUtime=25.4 cores=1,3,5,7
/proc/22105/task/22112/stat : 22112 (plingeling) R 22103 22105 20894 0 -1 4202560 44533 1250 0 0 2534 6 0 0 25 0 5 0 501107494 314134528 48129 33554432000 4194304 4417332 140735446891360 18446744073709551615 18446744073699065863 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22105/tid=22113] ppid=22103 vsize=306772 CPUtime=25.4 cores=1,3,5,7
/proc/22105/task/22113/stat : 22113 (plingeling) R 22103 22105 20894 0 -1 4202560 35202 1250 0 0 2535 5 0 0 25 0 5 0 501107494 314134528 48129 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364547 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22105/tid=22114] ppid=22103 vsize=306772 CPUtime=25.26 cores=1,3,5,7
/proc/22105/task/22114/stat : 22114 (plingeling) R 22103 22105 20894 0 -1 4202560 63333 1250 0 0 2517 9 0 0 25 0 5 0 501107494 314134528 48129 33554432000 4194304 4417332 140735446891360 18446744073709551615 4365963 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=22105/tid=22115] ppid=22103 vsize=306772 CPUtime=25.41 cores=1,3,5,7
/proc/22105/task/22115/stat : 22115 (plingeling) R 22103 22105 20894 0 -1 4202560 30635 1250 0 0 2535 6 0 0 25 0 5 0 501107494 314134528 48129 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 101.59
Current children cumulated vsize (KiB) 306772

[startup+51.1008 s]
/proc/loadavg: 6.22 4.59 4.25 9/186 22116
/proc/meminfo: memFree=30044420/32951124 swapFree=67111356/67111528
[pid=22105] ppid=22103 vsize=444280 CPUtime=203.84 cores=1,3,5,7
/proc/22105/stat : 22105 (plingeling) S 22103 22105 20894 0 -1 4202496 291295 1250 0 0 20337 47 0 0 25 0 5 0 501107485 454942720 82506 33554432000 4194304 4417332 140735446891360 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22105/statm: 111070 82506 180 55 0 108956 0
[pid=22105/tid=22112] ppid=22103 vsize=444280 CPUtime=51.01 cores=1,3,5,7
/proc/22105/task/22112/stat : 22112 (plingeling) R 22103 22105 20894 0 -1 4202560 67749 1250 0 0 5091 10 0 0 25 0 5 0 501107494 454942720 82506 33554432000 4194304 4417332 140735446891360 18446744073709551615 4357169 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22105/tid=22113] ppid=22103 vsize=444280 CPUtime=51 cores=1,3,5,7
/proc/22105/task/22113/stat : 22113 (plingeling) R 22103 22105 20894 0 -1 4202560 58408 1250 0 0 5092 8 0 0 25 0 5 0 501107494 454942720 82506 33554432000 4194304 4417332 140735446891360 18446744073709551615 4254955 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22105/tid=22114] ppid=22103 vsize=444280 CPUtime=50.73 cores=1,3,5,7
/proc/22105/task/22114/stat : 22114 (plingeling) R 22103 22105 20894 0 -1 4202560 107739 1250 0 0 5056 17 0 0 25 0 5 0 501107494 454942720 82506 33554432000 4194304 4417332 140735446891360 18446744073709551615 4366011 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=22105/tid=22115] ppid=22103 vsize=444280 CPUtime=51 cores=1,3,5,7
/proc/22105/task/22115/stat : 22115 (plingeling) R 22103 22105 20894 0 -1 4202560 52654 1250 0 0 5089 11 0 0 25 0 5 0 501107494 454942720 82506 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 203.84
Current children cumulated vsize (KiB) 444280

[startup+102.307 s]
/proc/loadavg: 7.28 5.13 4.45 9/186 22118
/proc/meminfo: memFree=29696948/32951124 swapFree=67111356/67111528
[pid=22105] ppid=22103 vsize=577380 CPUtime=408.38 cores=1,3,5,7
/proc/22105/stat : 22105 (plingeling) S 22103 22105 20894 0 -1 4202496 480817 1250 0 0 40763 75 0 0 25 0 5 0 501107485 591237120 115781 33554432000 4194304 4417332 140735446891360 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22105/statm: 144345 115781 180 55 0 142231 0
[pid=22105/tid=22112] ppid=22103 vsize=577380 CPUtime=102.21 cores=1,3,5,7
/proc/22105/task/22112/stat : 22112 (plingeling) R 22103 22105 20894 0 -1 4202560 141643 1250 0 0 10199 22 0 0 25 0 5 0 501107494 591237120 115781 33554432000 4194304 4417332 140735446891360 18446744073709551615 4357984 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22105/tid=22113] ppid=22103 vsize=577380 CPUtime=102.21 cores=1,3,5,7
/proc/22105/task/22113/stat : 22113 (plingeling) R 22103 22105 20894 0 -1 4202560 86045 1250 0 0 10210 11 0 0 25 0 5 0 501107494 591237120 115781 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22105/tid=22114] ppid=22103 vsize=577380 CPUtime=101.65 cores=1,3,5,7
/proc/22105/task/22114/stat : 22114 (plingeling) R 22103 22105 20894 0 -1 4202560 173473 1250 0 0 10139 26 0 0 25 0 5 0 501107494 591237120 115781 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=22105/tid=22115] ppid=22103 vsize=577380 CPUtime=102.2 cores=1,3,5,7
/proc/22105/task/22115/stat : 22115 (plingeling) R 22103 22105 20894 0 -1 4202560 74911 1250 0 0 10206 14 0 0 25 0 5 0 501107494 591237120 115781 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 408.38
Current children cumulated vsize (KiB) 577380

[startup+162.301 s]
/proc/loadavg: 7.78 5.66 4.67 9/186 22119
/proc/meminfo: memFree=29447800/32951124 swapFree=67111356/67111528
[pid=22105] ppid=22103 vsize=587920 CPUtime=648.03 cores=1,3,5,7
/proc/22105/stat : 22105 (plingeling) S 22103 22105 20894 0 -1 4202496 590351 1250 0 0 64713 90 0 0 25 0 5 0 501107485 602030080 118416 33554432000 4194304 4417332 140735446891360 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22105/statm: 146980 118416 180 55 0 144866 0
[pid=22105/tid=22112] ppid=22103 vsize=587920 CPUtime=162.21 cores=1,3,5,7
/proc/22105/task/22112/stat : 22112 (plingeling) R 22103 22105 20894 0 -1 4202560 184615 1250 0 0 16193 28 0 0 25 0 5 0 501107494 602030080 118416 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364572 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22105/tid=22113] ppid=22103 vsize=587920 CPUtime=162.2 cores=1,3,5,7
/proc/22105/task/22113/stat : 22113 (plingeling) R 22103 22105 20894 0 -1 4202560 89295 1250 0 0 16209 11 0 0 25 0 5 0 501107494 602030080 118416 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364734 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22105/tid=22114] ppid=22103 vsize=587920 CPUtime=161.32 cores=1,3,5,7
/proc/22105/task/22114/stat : 22114 (plingeling) R 22103 22105 20894 0 -1 4202560 225345 1250 0 0 16099 33 0 0 25 0 5 0 501107494 602030080 118416 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364683 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=22105/tid=22115] ppid=22103 vsize=587920 CPUtime=162.2 cores=1,3,5,7
/proc/22105/task/22115/stat : 22115 (plingeling) R 22103 22105 20894 0 -1 4202560 86351 1250 0 0 16204 16 0 0 25 0 5 0 501107494 602030080 118416 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 648.03
Current children cumulated vsize (KiB) 587920


################
# More data... #
################

[pid=22105/tid=22113] ppid=22103 vsize=1413696 CPUtime=699.11 cores=1,3,5,7
/proc/22105/task/22113/stat : 22113 (plingeling) R 22103 22105 20894 0 -1 4202560 457026 1250 0 0 69850 61 0 0 25 0 5 0 501107494 1447624704 324860 33554432000 4194304 4417332 140735446891360 18446744073709551615 4278271 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=22105/tid=22114] ppid=22103 vsize=1413696 CPUtime=700.64 cores=1,3,5,7
/proc/22105/task/22114/stat : 22114 (plingeling) R 22103 22105 20894 0 -1 4202560 821012 1250 0 0 69940 124 0 0 25 0 5 0 501107494 1447624704 324860 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364942 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22105/tid=22115] ppid=22103 vsize=1413696 CPUtime=702.15 cores=1,3,5,7
/proc/22105/task/22115/stat : 22115 (plingeling) R 22103 22105 20894 0 -1 4202560 228210 1250 0 0 70179 36 0 0 25 0 5 0 501107494 1447624704 324860 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 2804.24
Current children cumulated vsize (KiB) 1413696

[startup+762.307 s]
/proc/loadavg: 8.11 7.82 6.32 9/186 22138
/proc/meminfo: memFree=27857348/32951124 swapFree=67111356/67111528
[pid=22105] ppid=22103 vsize=1445540 CPUtime=3043.93 cores=1,3,5,7
/proc/22105/stat : 22105 (plingeling) S 22103 22105 20894 0 -1 4202496 2226563 1250 0 0 304061 332 0 0 25 0 5 0 501107485 1480232960 332821 33554432000 4194304 4417332 140735446891360 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22105/statm: 361385 332821 180 55 0 359271 0
[pid=22105/tid=22112] ppid=22103 vsize=1445540 CPUtime=762.22 cores=1,3,5,7
/proc/22105/task/22112/stat : 22112 (plingeling) R 22103 22105 20894 0 -1 4202560 608595 1250 0 0 76133 89 0 0 25 0 5 0 501107494 1480232960 332821 33554432000 4194304 4417332 140735446891360 18446744073709551615 4232670 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22105/tid=22113] ppid=22103 vsize=1445540 CPUtime=758.79 cores=1,3,5,7
/proc/22105/task/22113/stat : 22113 (plingeling) R 22103 22105 20894 0 -1 4202560 509584 1250 0 0 75811 68 0 0 25 0 5 0 501107494 1480232960 332821 33554432000 4194304 4417332 140735446891360 18446744073709551615 4321690 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=22105/tid=22114] ppid=22103 vsize=1445540 CPUtime=760.65 cores=1,3,5,7
/proc/22105/task/22114/stat : 22114 (plingeling) R 22103 22105 20894 0 -1 4202560 875429 1250 0 0 75928 137 0 0 25 0 5 0 501107494 1480232960 332821 33554432000 4194304 4417332 140735446891360 18446744073709551615 4270300 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22105/tid=22115] ppid=22103 vsize=1445540 CPUtime=762.15 cores=1,3,5,7
/proc/22105/task/22115/stat : 22115 (plingeling) R 22103 22105 20894 0 -1 4202560 228210 1250 0 0 76179 36 0 0 25 0 5 0 501107494 1480232960 332821 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3043.93
Current children cumulated vsize (KiB) 1445540

[startup+822.301 s]
/proc/loadavg: 8.10 7.86 6.43 9/186 22140
/proc/meminfo: memFree=27220208/32951124 swapFree=67111356/67111528
[pid=22105] ppid=22103 vsize=1758548 CPUtime=3283.58 cores=1,3,5,7
/proc/22105/stat : 22105 (plingeling) S 22103 22105 20894 0 -1 4202496 2329275 1250 0 0 328006 352 0 0 25 0 5 0 501107485 1800753152 411073 33554432000 4194304 4417332 140735446891360 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22105/statm: 439637 411073 180 55 0 437523 0
[pid=22105/tid=22112] ppid=22103 vsize=1758548 CPUtime=822.22 cores=1,3,5,7
/proc/22105/task/22112/stat : 22112 (plingeling) R 22103 22105 20894 0 -1 4202560 612895 1250 0 0 82132 90 0 0 25 0 5 0 501107494 1800753152 411073 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22105/tid=22113] ppid=22103 vsize=1758548 CPUtime=818.46 cores=1,3,5,7
/proc/22105/task/22113/stat : 22113 (plingeling) R 22103 22105 20894 0 -1 4202560 530272 1250 0 0 81774 72 0 0 25 0 5 0 501107494 1800753152 411073 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364736 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=22105/tid=22114] ppid=22103 vsize=1758548 CPUtime=820.65 cores=1,3,5,7
/proc/22105/task/22114/stat : 22114 (plingeling) R 22103 22105 20894 0 -1 4202560 953153 1250 0 0 81913 152 0 0 25 0 5 0 501107494 1800753152 411073 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22105/tid=22115] ppid=22103 vsize=1758548 CPUtime=822.14 cores=1,3,5,7
/proc/22105/task/22115/stat : 22115 (plingeling) R 22103 22105 20894 0 -1 4202560 228210 1250 0 0 82178 36 0 0 25 0 5 0 501107494 1800753152 411073 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3283.58
Current children cumulated vsize (KiB) 1758548

[startup+882.3 s]
/proc/loadavg: 8.09 7.90 6.53 9/186 22141
/proc/meminfo: memFree=27419564/32951124 swapFree=67111356/67111528
[pid=22105] ppid=22103 vsize=1659668 CPUtime=3523.25 cores=1,3,5,7
/proc/22105/stat : 22105 (plingeling) S 22103 22105 20894 0 -1 4202496 2394712 1250 0 0 351962 363 0 0 25 0 5 0 501107485 1699500032 386353 33554432000 4194304 4417332 140735446891360 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22105/statm: 414917 386353 180 55 0 412803 0
[pid=22105/tid=22112] ppid=22103 vsize=1659668 CPUtime=882.21 cores=1,3,5,7
/proc/22105/task/22112/stat : 22112 (plingeling) R 22103 22105 20894 0 -1 4202560 626762 1250 0 0 88130 91 0 0 25 0 5 0 501107494 1699500032 386353 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22105/tid=22113] ppid=22103 vsize=1659668 CPUtime=878.14 cores=1,3,5,7
/proc/22105/task/22113/stat : 22113 (plingeling) R 22103 22105 20894 0 -1 4202560 573331 1250 0 0 87734 80 0 0 25 0 5 0 501107494 1699500032 386353 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364536 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=22105/tid=22114] ppid=22103 vsize=1659668 CPUtime=880.65 cores=1,3,5,7
/proc/22105/task/22114/stat : 22114 (plingeling) R 22103 22105 20894 0 -1 4202560 953153 1250 0 0 87913 152 0 0 25 0 5 0 501107494 1699500032 386353 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22105/tid=22115] ppid=22103 vsize=1659668 CPUtime=882.14 cores=1,3,5,7
/proc/22105/task/22115/stat : 22115 (plingeling) R 22103 22105 20894 0 -1 4202560 236721 1250 0 0 88176 38 0 0 25 0 5 0 501107494 1699500032 386353 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364957 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3523.25
Current children cumulated vsize (KiB) 1659668

[startup+942.301 s]
/proc/loadavg: 8.03 7.92 6.62 9/186 22143
/proc/meminfo: memFree=27433912/32951124 swapFree=67111356/67111528
[pid=22105] ppid=22103 vsize=1491988 CPUtime=3762.92 cores=1,3,5,7
/proc/22105/stat : 22105 (plingeling) S 22103 22105 20894 0 -1 4202496 2609139 1250 0 0 375895 397 0 0 25 0 5 0 501107485 1527795712 344433 33554432000 4194304 4417332 140735446891360 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22105/statm: 372997 344433 180 55 0 370883 0
[pid=22105/tid=22112] ppid=22103 vsize=1491988 CPUtime=942.21 cores=1,3,5,7
/proc/22105/task/22112/stat : 22112 (plingeling) R 22103 22105 20894 0 -1 4202560 669780 1250 0 0 94125 96 0 0 25 0 5 0 501107494 1527795712 344433 33554432000 4194304 4417332 140735446891360 18446744073709551615 4250873 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22105/tid=22113] ppid=22103 vsize=1491988 CPUtime=937.8 cores=1,3,5,7
/proc/22105/task/22113/stat : 22113 (plingeling) R 22103 22105 20894 0 -1 4202560 612255 1250 0 0 93694 86 0 0 25 0 5 0 501107494 1527795712 344433 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=22105/tid=22114] ppid=22103 vsize=1491988 CPUtime=940.65 cores=1,3,5,7
/proc/22105/task/22114/stat : 22114 (plingeling) R 22103 22105 20894 0 -1 4202560 1066673 1250 0 0 93895 170 0 0 25 0 5 0 501107494 1527795712 344433 33554432000 4194304 4417332 140735446891360 18446744073709551615 4365003 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22105/tid=22115] ppid=22103 vsize=1491988 CPUtime=942.13 cores=1,3,5,7
/proc/22105/task/22115/stat : 22115 (plingeling) R 22103 22105 20894 0 -1 4202560 255686 1250 0 0 94171 42 0 0 25 0 5 0 501107494 1527795712 344433 33554432000 4194304 4417332 140735446891360 18446744073709551615 4246388 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3762.92
Current children cumulated vsize (KiB) 1491988

[startup+1002.3 s]
/proc/loadavg: 8.01 7.93 6.70 9/186 22145
/proc/meminfo: memFree=27772644/32951124 swapFree=67111356/67111528
[pid=22105] ppid=22103 vsize=1427672 CPUtime=4002.59 cores=1,3,5,7
/proc/22105/stat : 22105 (plingeling) S 22103 22105 20894 0 -1 4202496 2678668 1250 0 0 399852 407 0 0 25 0 5 0 501107485 1461936128 328354 33554432000 4194304 4417332 140735446891360 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22105/statm: 356918 328354 180 55 0 354804 0
[pid=22105/tid=22112] ppid=22103 vsize=1427672 CPUtime=1002.21 cores=1,3,5,7
/proc/22105/task/22112/stat : 22112 (plingeling) R 22103 22105 20894 0 -1 4202560 715736 1250 0 0 100119 102 0 0 25 0 5 0 501107494 1461936128 328354 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22105/tid=22113] ppid=22103 vsize=1427672 CPUtime=997.47 cores=1,3,5,7
/proc/22105/task/22113/stat : 22113 (plingeling) R 22103 22105 20894 0 -1 4202560 612255 1250 0 0 99661 86 0 0 25 0 5 0 501107494 1461936128 328354 33554432000 4194304 4417332 140735446891360 18446744073709551615 4365963 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=22105/tid=22114] ppid=22103 vsize=1427672 CPUtime=1000.65 cores=1,3,5,7
/proc/22105/task/22114/stat : 22114 (plingeling) R 22103 22105 20894 0 -1 4202560 1067472 1250 0 0 99895 170 0 0 25 0 5 0 501107494 1461936128 328354 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22105/tid=22115] ppid=22103 vsize=1427672 CPUtime=1002.13 cores=1,3,5,7
/proc/22105/task/22115/stat : 22115 (plingeling) R 22103 22105 20894 0 -1 4202560 278460 1250 0 0 100167 46 0 0 25 0 5 0 501107494 1461936128 328354 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4002.59
Current children cumulated vsize (KiB) 1427672

[startup+1062.3 s]
/proc/loadavg: 8.08 7.96 6.79 9/186 22146
/proc/meminfo: memFree=27539492/32951124 swapFree=67111356/67111528
[pid=22105] ppid=22103 vsize=1393600 CPUtime=4242.27 cores=1,3,5,7
/proc/22105/stat : 22105 (plingeling) S 22103 22105 20894 0 -1 4202496 2806385 1250 0 0 423803 424 0 0 25 0 5 0 501107485 1427046400 319836 33554432000 4194304 4417332 140735446891360 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22105/statm: 348400 319836 180 55 0 346286 0
[pid=22105/tid=22112] ppid=22103 vsize=1393600 CPUtime=1062.22 cores=1,3,5,7
/proc/22105/task/22112/stat : 22112 (plingeling) R 22103 22105 20894 0 -1 4202560 733055 1250 0 0 106117 105 0 0 25 0 5 0 501107494 1427046400 319836 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22105/tid=22113] ppid=22103 vsize=1393600 CPUtime=1057.15 cores=1,3,5,7
/proc/22105/task/22113/stat : 22113 (plingeling) R 22103 22105 20894 0 -1 4202560 701607 1250 0 0 105616 99 0 0 25 0 5 0 501107494 1427046400 319836 33554432000 4194304 4417332 140735446891360 18446744073709551615 4320914 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=22105/tid=22114] ppid=22103 vsize=1393600 CPUtime=1060.65 cores=1,3,5,7
/proc/22105/task/22114/stat : 22114 (plingeling) R 22103 22105 20894 0 -1 4202560 1088518 1250 0 0 105893 172 0 0 25 0 5 0 501107494 1427046400 319836 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22105/tid=22115] ppid=22103 vsize=1393600 CPUtime=1062.13 cores=1,3,5,7
/proc/22105/task/22115/stat : 22115 (plingeling) R 22103 22105 20894 0 -1 4202560 278460 1250 0 0 106167 46 0 0 25 0 5 0 501107494 1427046400 319836 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4242.27
Current children cumulated vsize (KiB) 1393600

[startup+1122.3 s]
/proc/loadavg: 8.09 7.98 6.87 9/186 22148
/proc/meminfo: memFree=27340452/32951124 swapFree=67111356/67111528
[pid=22105] ppid=22103 vsize=1627328 CPUtime=4481.93 cores=1,3,5,7
/proc/22105/stat : 22105 (plingeling) S 22103 22105 20894 0 -1 4202496 2908404 1250 0 0 447753 440 0 0 25 0 5 0 501107485 1666383872 378268 33554432000 4194304 4417332 140735446891360 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22105/statm: 406832 378268 180 55 0 404718 0
[pid=22105/tid=22112] ppid=22103 vsize=1627328 CPUtime=1122.22 cores=1,3,5,7
/proc/22105/task/22112/stat : 22112 (plingeling) R 22103 22105 20894 0 -1 4202560 767852 1250 0 0 112111 111 0 0 25 0 5 0 501107494 1666383872 378268 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22105/tid=22113] ppid=22103 vsize=1627328 CPUtime=1116.83 cores=1,3,5,7
/proc/22105/task/22113/stat : 22113 (plingeling) R 22103 22105 20894 0 -1 4202560 737793 1250 0 0 111579 104 0 0 25 0 5 0 501107494 1666383872 378268 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=22105/tid=22114] ppid=22103 vsize=1627328 CPUtime=1120.66 cores=1,3,5,7
/proc/22105/task/22114/stat : 22114 (plingeling) R 22103 22105 20894 0 -1 4202560 1091714 1250 0 0 111893 173 0 0 25 0 5 0 501107494 1666383872 378268 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364590 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22105/tid=22115] ppid=22103 vsize=1627328 CPUtime=1122.12 cores=1,3,5,7
/proc/22105/task/22115/stat : 22115 (plingeling) R 22103 22105 20894 0 -1 4202560 306300 1250 0 0 112161 51 0 0 25 0 5 0 501107494 1666383872 378268 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4481.93
Current children cumulated vsize (KiB) 1627328

[startup+1182.3 s]
/proc/loadavg: 8.09 7.99 6.94 9/186 22149
/proc/meminfo: memFree=27151052/32951124 swapFree=67111356/67111528
[pid=22105] ppid=22103 vsize=1874940 CPUtime=4721.61 cores=1,3,5,7
/proc/22105/stat : 22105 (plingeling) S 22103 22105 20894 0 -1 4202496 3010395 1250 0 0 471707 454 0 0 25 0 5 0 501107485 1919938560 440171 33554432000 4194304 4417332 140735446891360 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22105/statm: 468735 440171 180 55 0 466621 0
[pid=22105/tid=22112] ppid=22103 vsize=1874940 CPUtime=1182.22 cores=1,3,5,7
/proc/22105/task/22112/stat : 22112 (plingeling) R 22103 22105 20894 0 -1 4202560 776883 1250 0 0 118110 112 0 0 25 0 5 0 501107494 1919938560 440171 33554432000 4194304 4417332 140735446891360 18446744073709551615 4365975 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22105/tid=22113] ppid=22103 vsize=1874940 CPUtime=1176.5 cores=1,3,5,7
/proc/22105/task/22113/stat : 22113 (plingeling) R 22103 22105 20894 0 -1 4202560 765397 1250 0 0 117542 108 0 0 25 0 5 0 501107494 1919938560 440171 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=22105/tid=22114] ppid=22103 vsize=1874940 CPUtime=1180.65 cores=1,3,5,7
/proc/22105/task/22114/stat : 22114 (plingeling) R 22103 22105 20894 0 -1 4202560 1156573 1250 0 0 117884 181 0 0 25 0 5 0 501107494 1919938560 440171 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22105/tid=22115] ppid=22103 vsize=1874940 CPUtime=1182.12 cores=1,3,5,7
/proc/22105/task/22115/stat : 22115 (plingeling) R 22103 22105 20894 0 -1 4202560 306797 1250 0 0 118161 51 0 0 25 0 5 0 501107494 1919938560 440171 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364544 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4721.61
Current children cumulated vsize (KiB) 1874940



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1202 s]
/proc/loadavg: 8.07 7.99 6.96 5/180 22151
/proc/meminfo: memFree=29268140/32951124 swapFree=67111356/67111528
[pid=22105] ppid=22103 vsize=1549184 CPUtime=4800.32 cores=1,3,5,7
/proc/22105/stat : 22105 (plingeling) S 22103 22105 20894 0 -1 4202496 3011674 1250 0 0 479576 456 0 0 25 0 5 0 501107485 1586364416 358732 33554432000 4194304 4417332 140735446891360 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22105/statm: 387296 358732 180 55 0 385182 0
[pid=22105/tid=22112] ppid=22103 vsize=1549184 CPUtime=1201.92 cores=1,3,5,7
/proc/22105/task/22112/stat : 22112 (plingeling) R 22103 22105 20894 0 -1 4202560 778099 1250 0 0 120080 112 0 0 25 0 5 0 501107494 1586364416 358732 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364938 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22105/tid=22113] ppid=22103 vsize=1549184 CPUtime=1196.09 cores=1,3,5,7
/proc/22105/task/22113/stat : 22113 (plingeling) R 22103 22105 20894 0 -1 4202560 765460 1250 0 0 119500 109 0 0 25 0 5 0 501107494 1586364416 358732 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=22105/tid=22114] ppid=22103 vsize=1549184 CPUtime=1200.36 cores=1,3,5,7
/proc/22105/task/22114/stat : 22114 (plingeling) R 22103 22105 20894 0 -1 4202560 1156573 1250 0 0 119855 181 0 0 25 0 5 0 501107494 1586364416 358732 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364732 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22105/tid=22115] ppid=22103 vsize=1549184 CPUtime=1201.82 cores=1,3,5,7
/proc/22105/task/22115/stat : 22115 (plingeling) R 22103 22105 20894 0 -1 4202560 306797 1250 0 0 120131 51 0 0 25 0 5 0 501107494 1586364416 358732 33554432000 4194304 4417332 140735446891360 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4800.32
Current children cumulated vsize (KiB) 1549184

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1202.06
CPU time (s): 4800.4
CPU user time (s): 4795.79
CPU system time (s): 4.6153
CPU usage (%): 399.346
Max. virtual memory (cumulated for all children) (KiB): 1983940

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4795.79
system time used= 4.6153
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3012924
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 103
involuntary context switches= 37082

runsolver used 1.90071 second user time and 4.66329 second system time

The end

Launcher Data 

Begin job on node146 at 2011-04-29 16:21:52
IDJOB=3249580
IDBENCH=83177
IDSOLVER=1605
FILE ID=node146/3249580-1304086912
RUNJOBID= node146-1304086911-21969
PBS_JOBID= 13170499
Free space on /tmp= 72992 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT11/crafted/mosoi/289/289-unsat-9x32.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3249580-1304086912/watcher-3249580-1304086912 -o /tmp/evaluation-result-3249580-1304086912/solver-3249580-1304086912 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3249580-1304086912.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= bcc12436eba65ad776b55f23c6f42298
RANDOM SEED=1796370456

node146.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      30636116 kB
Buffers:        604460 kB
Cached:        1185184 kB
SwapCached:         32 kB
Active:         204656 kB
Inactive:      1721608 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      30636116 kB
SwapTotal:    67111528 kB
SwapFree:     67111356 kB
Dirty:            6624 kB
Writeback:           0 kB
AnonPages:      136112 kB
Mapped:          15272 kB
Slab:           326352 kB
PageTables:       4568 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   318060 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72996 MiB
End job on node146 at 2011-04-29 16:41:56