Trace number 3263222

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 4800.33 1201.68

General information on the benchmark

NameSAT03/handmade/anton/UNSAT/
SGI_30_70_18_80_10-dir.shuffled-as.sat03-140.cnf
MD5SUM1037a2754a0a53f62ba8ec901741c9e9
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables540
Number of clauses40510
Sum of the clauses size81524
Maximum clause length30
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 240492
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 518

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3263222-1304099236/watcher-3263222-1304099236 -o /tmp/evaluation-result-3263222-1304099236/solver-3263222-1304099236 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3263222-1304099236.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.18 8.07 7.75 5/181 20431
/proc/meminfo: memFree=29869672/32951124 swapFree=67111364/67111528
[pid=20431] ppid=20429 vsize=9296 CPUtime=0 cores=0,2,4,6
/proc/20431/stat : 20431 (plingeling) R 20429 20431 19972 0 -1 4202496 416 1256 0 0 0 0 0 0 25 0 1 0 521524036 9519104 345 33554432000 4194304 4417332 140733637483024 18446744073709551615 253640859392 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/20431/statm: 2324 345 146 55 0 210 0

[startup+0.139352 s]
/proc/loadavg: 8.18 8.07 7.75 5/181 20431
/proc/meminfo: memFree=29869672/32951124 swapFree=67111364/67111528
[pid=20431] ppid=20429 vsize=123140 CPUtime=0.45 cores=0,2,4,6
/proc/20431/stat : 20431 (plingeling) S 20429 20431 19972 0 -1 4202496 3229 1256 0 0 45 0 0 0 25 0 5 0 521524036 126095360 2195 33554432000 4194304 4417332 140733637483024 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/20431/statm: 30785 2195 177 55 0 28671 0
Current children cumulated CPU time (s) 0.45
Current children cumulated vsize (KiB) 123140

[startup+0.200342 s]
/proc/loadavg: 8.18 8.07 7.75 5/181 20431
/proc/meminfo: memFree=29869672/32951124 swapFree=67111364/67111528
[pid=20431] ppid=20429 vsize=125104 CPUtime=0.69 cores=0,2,4,6
/proc/20431/stat : 20431 (plingeling) S 20429 20431 19972 0 -1 4202496 3958 1256 0 0 69 0 0 0 25 0 5 0 521524036 128106496 2690 33554432000 4194304 4417332 140733637483024 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/20431/statm: 31276 2690 181 55 0 29162 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 125104

[startup+0.300323 s]
/proc/loadavg: 8.18 8.07 7.75 5/181 20431
/proc/meminfo: memFree=29869672/32951124 swapFree=67111364/67111528
[pid=20431] ppid=20429 vsize=127772 CPUtime=1.09 cores=0,2,4,6
/proc/20431/stat : 20431 (plingeling) S 20429 20431 19972 0 -1 4202496 4625 1256 0 0 109 0 0 0 25 0 5 0 521524036 130838528 3357 33554432000 4194304 4417332 140733637483024 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/20431/statm: 31943 3357 181 55 0 29829 0
Current children cumulated CPU time (s) 1.09
Current children cumulated vsize (KiB) 127772

[startup+0.70025 s]
/proc/loadavg: 8.18 8.07 7.75 5/181 20431
/proc/meminfo: memFree=29869672/32951124 swapFree=67111364/67111528
[pid=20431] ppid=20429 vsize=130900 CPUtime=2.69 cores=0,2,4,6
/proc/20431/stat : 20431 (plingeling) S 20429 20431 19972 0 -1 4202496 6640 1256 0 0 269 0 0 0 25 0 5 0 521524036 134041600 4139 33554432000 4194304 4417332 140733637483024 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/20431/statm: 32725 4139 181 55 0 30611 0
Current children cumulated CPU time (s) 2.69
Current children cumulated vsize (KiB) 130900

[startup+1.50111 s]
/proc/loadavg: 8.18 8.07 7.75 9/186 20441
/proc/meminfo: memFree=29847816/32951124 swapFree=67111364/67111528
[pid=20431] ppid=20429 vsize=142220 CPUtime=5.89 cores=0,2,4,6
/proc/20431/stat : 20431 (plingeling) S 20429 20431 19972 0 -1 4202496 11149 1256 0 0 588 1 0 0 25 0 5 0 521524036 145633280 6909 33554432000 4194304 4417332 140733637483024 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/20431/statm: 35555 6909 181 55 0 33441 0
[pid=20431/tid=20438] ppid=20429 vsize=142220 CPUtime=1.46 cores=0,2,4,6
/proc/20431/task/20438/stat : 20438 (plingeling) R 20429 20431 19972 0 -1 4202560 690 1256 0 0 146 0 0 0 25 0 5 0 521524039 145633280 6909 33554432000 4194304 4417332 140733637483024 18446744073709551615 4250072 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=20431/tid=20439] ppid=20429 vsize=142220 CPUtime=1.46 cores=0,2,4,6
/proc/20431/task/20439/stat : 20439 (plingeling) R 20429 20431 19972 0 -1 4202560 668 1256 0 0 146 0 0 0 25 0 5 0 521524039 145633280 6909 33554432000 4194304 4417332 140733637483024 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=20431/tid=20440] ppid=20429 vsize=142220 CPUtime=1.46 cores=0,2,4,6
/proc/20431/task/20440/stat : 20440 (plingeling) R 20429 20431 19972 0 -1 4202560 1271 1256 0 0 146 0 0 0 25 0 5 0 521524039 145633280 6909 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364899 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20431/tid=20441] ppid=20429 vsize=142220 CPUtime=1.45 cores=0,2,4,6
/proc/20431/task/20441/stat : 20441 (plingeling) R 20429 20431 19972 0 -1 4202560 7063 1256 0 0 145 0 0 0 25 0 5 0 521524039 145633280 6909 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 5.89
Current children cumulated vsize (KiB) 142220

[startup+3.10081 s]
/proc/loadavg: 8.18 8.07 7.75 9/186 20441
/proc/meminfo: memFree=29843972/32951124 swapFree=67111364/67111528
[pid=20431] ppid=20429 vsize=151412 CPUtime=12.29 cores=0,2,4,6
/proc/20431/stat : 20431 (plingeling) S 20429 20431 19972 0 -1 4202496 17614 1256 0 0 1227 2 0 0 25 0 5 0 521524036 155045888 9295 33554432000 4194304 4417332 140733637483024 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/20431/statm: 37853 9295 181 55 0 35739 0
[pid=20431/tid=20438] ppid=20429 vsize=151412 CPUtime=3.06 cores=0,2,4,6
/proc/20431/task/20438/stat : 20438 (plingeling) R 20429 20431 19972 0 -1 4202560 690 1256 0 0 306 0 0 0 25 0 5 0 521524039 155045888 9295 33554432000 4194304 4417332 140733637483024 18446744073709551615 4250017 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=20431/tid=20439] ppid=20429 vsize=151412 CPUtime=3.06 cores=0,2,4,6
/proc/20431/task/20439/stat : 20439 (plingeling) R 20429 20431 19972 0 -1 4202560 668 1256 0 0 306 0 0 0 25 0 5 0 521524039 155045888 9295 33554432000 4194304 4417332 140733637483024 18446744073709551615 4236508 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=20431/tid=20440] ppid=20429 vsize=151412 CPUtime=3.06 cores=0,2,4,6
/proc/20431/task/20440/stat : 20440 (plingeling) R 20429 20431 19972 0 -1 4202560 2472 1256 0 0 306 0 0 0 25 0 5 0 521524039 155045888 9295 33554432000 4194304 4417332 140733637483024 18446744073709551615 4353458 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20431/tid=20441] ppid=20429 vsize=151412 CPUtime=3.05 cores=0,2,4,6
/proc/20431/task/20441/stat : 20441 (plingeling) R 20429 20431 19972 0 -1 4202560 12327 1256 0 0 304 1 0 0 25 0 5 0 521524039 155045888 9295 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 12.29
Current children cumulated vsize (KiB) 151412

[startup+6.30023 s]
/proc/loadavg: 8.16 8.07 7.75 9/186 20441
/proc/meminfo: memFree=29809352/32951124 swapFree=67111364/67111528
[pid=20431] ppid=20429 vsize=157296 CPUtime=25.07 cores=0,2,4,6
/proc/20431/stat : 20431 (plingeling) S 20429 20431 19972 0 -1 4202496 35907 1256 0 0 2501 6 0 0 25 0 5 0 521524036 161071104 10786 33554432000 4194304 4417332 140733637483024 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/20431/statm: 39324 10786 181 55 0 37210 0
[pid=20431/tid=20438] ppid=20429 vsize=157296 CPUtime=6.26 cores=0,2,4,6
/proc/20431/task/20438/stat : 20438 (plingeling) R 20429 20431 19972 0 -1 4202560 2467 1256 0 0 626 0 0 0 25 0 5 0 521524039 161071104 10786 33554432000 4194304 4417332 140733637483024 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=20431/tid=20439] ppid=20429 vsize=157296 CPUtime=6.26 cores=0,2,4,6
/proc/20431/task/20439/stat : 20439 (plingeling) R 20429 20431 19972 0 -1 4202560 1939 1256 0 0 626 0 0 0 25 0 5 0 521524039 161071104 10786 33554432000 4194304 4417332 140733637483024 18446744073709551615 4249675 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=20431/tid=20440] ppid=20429 vsize=157296 CPUtime=6.26 cores=0,2,4,6
/proc/20431/task/20440/stat : 20440 (plingeling) R 20429 20431 19972 0 -1 4202560 6198 1256 0 0 625 1 0 0 25 0 5 0 521524039 161071104 10786 33554432000 4194304 4417332 140733637483024 18446744073709551615 4249601 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20431/tid=20441] ppid=20429 vsize=157296 CPUtime=6.23 cores=0,2,4,6
/proc/20431/task/20441/stat : 20441 (plingeling) R 20429 20431 19972 0 -1 4202560 23846 1256 0 0 620 3 0 0 25 0 5 0 521524039 161071104 10786 33554432000 4194304 4417332 140733637483024 18446744073709551615 4357641 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 25.07
Current children cumulated vsize (KiB) 157296

[startup+12.7011 s]
/proc/loadavg: 8.23 8.09 7.76 9/186 20441
/proc/meminfo: memFree=29766820/32951124 swapFree=67111364/67111528
[pid=20431] ppid=20429 vsize=192080 CPUtime=50.64 cores=0,2,4,6
/proc/20431/stat : 20431 (plingeling) S 20429 20431 19972 0 -1 4202496 66271 1256 0 0 5054 10 0 0 25 0 5 0 521524036 196689920 19506 33554432000 4194304 4417332 140733637483024 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/20431/statm: 48020 19506 181 55 0 45906 0
[pid=20431/tid=20438] ppid=20429 vsize=192080 CPUtime=12.66 cores=0,2,4,6
/proc/20431/task/20438/stat : 20438 (plingeling) R 20429 20431 19972 0 -1 4202560 8366 1256 0 0 1265 1 0 0 25 0 5 0 521524039 196689920 19506 33554432000 4194304 4417332 140733637483024 18446744073709551615 4354401 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=20431/tid=20439] ppid=20429 vsize=192080 CPUtime=12.66 cores=0,2,4,6
/proc/20431/task/20439/stat : 20439 (plingeling) R 20429 20431 19972 0 -1 4202560 6251 1256 0 0 1266 0 0 0 25 0 5 0 521524039 196689920 19506 33554432000 4194304 4417332 140733637483024 18446744073709551615 4250571 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=20431/tid=20440] ppid=20429 vsize=192080 CPUtime=12.66 cores=0,2,4,6
/proc/20431/task/20440/stat : 20440 (plingeling) R 20429 20431 19972 0 -1 4202560 12933 1256 0 0 1264 2 0 0 25 0 5 0 521524039 196689920 19506 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364973 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20431/tid=20441] ppid=20429 vsize=192080 CPUtime=12.6 cores=0,2,4,6
/proc/20431/task/20441/stat : 20441 (plingeling) R 20429 20431 19972 0 -1 4202560 37264 1256 0 0 1255 5 0 0 25 0 5 0 521524039 196689920 19506 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 50.64
Current children cumulated vsize (KiB) 192080

[startup+25.5007 s]
/proc/loadavg: 8.18 8.08 7.76 9/186 20441
/proc/meminfo: memFree=29695880/32951124 swapFree=67111364/67111528
[pid=20431] ppid=20429 vsize=288504 CPUtime=101.78 cores=0,2,4,6
/proc/20431/stat : 20431 (plingeling) S 20429 20431 19972 0 -1 4202496 123637 1256 0 0 10160 18 0 0 25 0 5 0 521524036 295428096 43659 33554432000 4194304 4417332 140733637483024 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/20431/statm: 72126 43659 181 55 0 70012 0
[pid=20431/tid=20438] ppid=20429 vsize=288504 CPUtime=25.46 cores=0,2,4,6
/proc/20431/task/20438/stat : 20438 (plingeling) R 20429 20431 19972 0 -1 4202560 21652 1256 0 0 2543 3 0 0 25 0 5 0 521524039 295428096 43659 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364590 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=20431/tid=20439] ppid=20429 vsize=288504 CPUtime=25.46 cores=0,2,4,6
/proc/20431/task/20439/stat : 20439 (plingeling) R 20429 20431 19972 0 -1 4202560 21169 1256 0 0 2544 2 0 0 25 0 5 0 521524039 295428096 43659 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=20431/tid=20440] ppid=20429 vsize=288504 CPUtime=25.46 cores=0,2,4,6
/proc/20431/task/20440/stat : 20440 (plingeling) R 20429 20431 19972 0 -1 4202560 26071 1256 0 0 2543 3 0 0 25 0 5 0 521524039 295428096 43659 33554432000 4194304 4417332 140733637483024 18446744073709551615 4307546 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20431/tid=20441] ppid=20429 vsize=288504 CPUtime=25.34 cores=0,2,4,6
/proc/20431/task/20441/stat : 20441 (plingeling) R 20429 20431 19972 0 -1 4202560 53288 1256 0 0 2526 8 0 0 25 0 5 0 521524039 295428096 43659 33554432000 4194304 4417332 140733637483024 18446744073709551615 4237290 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 101.78
Current children cumulated vsize (KiB) 288504

[startup+51.1011 s]
/proc/loadavg: 8.12 8.07 7.77 9/186 20442
/proc/meminfo: memFree=29600888/32951124 swapFree=67111364/67111528
[pid=20431] ppid=20429 vsize=332956 CPUtime=204.06 cores=0,2,4,6
/proc/20431/stat : 20431 (plingeling) S 20429 20431 19972 0 -1 4202496 225504 1256 0 0 20372 34 0 0 25 0 5 0 521524036 340946944 54911 33554432000 4194304 4417332 140733637483024 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/20431/statm: 83239 54911 181 55 0 81125 0
[pid=20431/tid=20438] ppid=20429 vsize=332956 CPUtime=51.06 cores=0,2,4,6
/proc/20431/task/20438/stat : 20438 (plingeling) R 20429 20431 19972 0 -1 4202560 51295 1256 0 0 5098 8 0 0 25 0 5 0 521524039 340946944 54911 33554432000 4194304 4417332 140733637483024 18446744073709551615 4344093 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=20431/tid=20439] ppid=20429 vsize=332956 CPUtime=51.06 cores=0,2,4,6
/proc/20431/task/20439/stat : 20439 (plingeling) R 20429 20431 19972 0 -1 4202560 46287 1256 0 0 5101 5 0 0 25 0 5 0 521524039 340946944 54911 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364736 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=20431/tid=20440] ppid=20429 vsize=332956 CPUtime=51.06 cores=0,2,4,6
/proc/20431/task/20440/stat : 20440 (plingeling) R 20429 20431 19972 0 -1 4202560 48393 1256 0 0 5099 7 0 0 25 0 5 0 521524039 340946944 54911 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20431/tid=20441] ppid=20429 vsize=332956 CPUtime=50.82 cores=0,2,4,6
/proc/20431/task/20441/stat : 20441 (plingeling) R 20429 20431 19972 0 -1 4202560 78072 1256 0 0 5070 12 0 0 25 0 5 0 521524039 340946944 54911 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 204.06
Current children cumulated vsize (KiB) 332956

[startup+102.301 s]
/proc/loadavg: 8.09 8.08 7.78 9/186 20443
/proc/meminfo: memFree=29501308/32951124 swapFree=67111364/67111528
[pid=20431] ppid=20429 vsize=453564 CPUtime=408.63 cores=0,2,4,6
/proc/20431/stat : 20431 (plingeling) S 20429 20431 19972 0 -1 4202496 360478 1256 0 0 40811 52 0 0 25 0 5 0 521524036 464449536 85054 33554432000 4194304 4417332 140733637483024 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/20431/statm: 113391 85054 181 55 0 111277 0
[pid=20431/tid=20438] ppid=20429 vsize=453564 CPUtime=102.27 cores=0,2,4,6
/proc/20431/task/20438/stat : 20438 (plingeling) R 20429 20431 19972 0 -1 4202560 80263 1256 0 0 10215 12 0 0 25 0 5 0 521524039 464449536 85054 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364502 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=20431/tid=20439] ppid=20429 vsize=453564 CPUtime=102.26 cores=0,2,4,6
/proc/20431/task/20439/stat : 20439 (plingeling) R 20429 20431 19972 0 -1 4202560 82482 1256 0 0 10216 10 0 0 25 0 5 0 521524039 464449536 85054 33554432000 4194304 4417332 140733637483024 18446744073709551615 4365963 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=20431/tid=20440] ppid=20429 vsize=453564 CPUtime=102.27 cores=0,2,4,6
/proc/20431/task/20440/stat : 20440 (plingeling) R 20429 20431 19972 0 -1 4202560 93021 1256 0 0 10214 13 0 0 25 0 5 0 521524039 464449536 85054 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364742 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20431/tid=20441] ppid=20429 vsize=453564 CPUtime=101.77 cores=0,2,4,6
/proc/20431/task/20441/stat : 20441 (plingeling) R 20429 20431 19972 0 -1 4202560 103255 1256 0 0 10162 15 0 0 25 0 5 0 521524039 464449536 85054 33554432000 4194304 4417332 140733637483024 18446744073709551615 4365977 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 408.63
Current children cumulated vsize (KiB) 453564

[startup+162.301 s]
/proc/loadavg: 8.03 8.06 7.79 9/186 20445
/proc/meminfo: memFree=29408924/32951124 swapFree=67111364/67111528
[pid=20431] ppid=20429 vsize=506344 CPUtime=648.31 cores=0,2,4,6
/proc/20431/stat : 20431 (plingeling) S 20429 20431 19972 0 -1 4202496 518503 1256 0 0 64760 71 0 0 25 0 5 0 521524036 518496256 98270 33554432000 4194304 4417332 140733637483024 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/20431/statm: 126586 98270 181 55 0 124472 0
[pid=20431/tid=20438] ppid=20429 vsize=506344 CPUtime=162.27 cores=0,2,4,6
/proc/20431/task/20438/stat : 20438 (plingeling) R 20429 20431 19972 0 -1 4202560 130962 1256 0 0 16208 19 0 0 25 0 5 0 521524039 518496256 98270 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364796 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=20431/tid=20439] ppid=20429 vsize=506344 CPUtime=162.26 cores=0,2,4,6
/proc/20431/task/20439/stat : 20439 (plingeling) R 20429 20431 19972 0 -1 4202560 126098 1256 0 0 16210 16 0 0 25 0 5 0 521524039 518496256 98270 33554432000 4194304 4417332 140733637483024 18446744073709551615 4311289 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=20431/tid=20440] ppid=20429 vsize=506344 CPUtime=162.26 cores=0,2,4,6
/proc/20431/task/20440/stat : 20440 (plingeling) R 20429 20431 19972 0 -1 4202560 130668 1256 0 0 16209 17 0 0 25 0 5 0 521524039 518496256 98270 33554432000 4194304 4417332 140733637483024 18446744073709551615 4255615 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20431/tid=20441] ppid=20429 vsize=506344 CPUtime=161.47 cores=0,2,4,6
/proc/20431/task/20441/stat : 20441 (plingeling) R 20429 20431 19972 0 -1 4202560 129318 1256 0 0 16129 18 0 0 25 0 5 0 521524039 518496256 98270 33554432000 4194304 4417332 140733637483024 18446744073709551615 4365007 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 648.31
Current children cumulated vsize (KiB) 506344


################
# More data... #
################

[pid=20431/tid=20439] ppid=20429 vsize=1361912 CPUtime=762.28 cores=0,2,4,6
/proc/20431/task/20439/stat : 20439 (plingeling) R 20429 20431 19972 0 -1 4202560 392893 1256 0 0 76176 52 0 0 25 0 5 0 521524039 1394597888 312884 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=20431/tid=20440] ppid=20429 vsize=1361912 CPUtime=762.29 cores=0,2,4,6
/proc/20431/task/20440/stat : 20440 (plingeling) R 20429 20431 19972 0 -1 4202560 457451 1256 0 0 76165 64 0 0 25 0 5 0 521524039 1394597888 312884 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364899 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20431/tid=20441] ppid=20429 vsize=1361912 CPUtime=758.41 cores=0,2,4,6
/proc/20431/task/20441/stat : 20441 (plingeling) R 20429 20431 19972 0 -1 4202560 295084 1256 0 0 75799 42 0 0 25 0 5 0 521524039 1394597888 312884 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3045.31
Current children cumulated vsize (KiB) 1361912

[startup+822.301 s]
/proc/loadavg: 8.14 8.09 7.93 9/186 20500
/proc/meminfo: memFree=28461340/32951124 swapFree=67111364/67111528
[pid=20431] ppid=20429 vsize=1515964 CPUtime=3284.98 cores=0,2,4,6
/proc/20431/stat : 20431 (plingeling) S 20429 20431 19972 0 -1 4202496 1683116 1256 0 0 328257 241 0 0 25 0 5 0 521524036 1552347136 351397 33554432000 4194304 4417332 140733637483024 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/20431/statm: 378991 351397 181 55 0 376877 0
[pid=20431/tid=20438] ppid=20429 vsize=1515964 CPUtime=822.28 cores=0,2,4,6
/proc/20431/task/20438/stat : 20438 (plingeling) R 20429 20431 19972 0 -1 4202560 474812 1256 0 0 82157 71 0 0 25 0 5 0 521524039 1552347136 351397 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=20431/tid=20439] ppid=20429 vsize=1515964 CPUtime=822.28 cores=0,2,4,6
/proc/20431/task/20439/stat : 20439 (plingeling) R 20429 20431 19972 0 -1 4202560 425184 1256 0 0 82171 57 0 0 25 0 5 0 521524039 1552347136 351397 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=20431/tid=20440] ppid=20429 vsize=1515964 CPUtime=822.28 cores=0,2,4,6
/proc/20431/task/20440/stat : 20440 (plingeling) R 20429 20431 19972 0 -1 4202560 457451 1256 0 0 82164 64 0 0 25 0 5 0 521524039 1552347136 351397 33554432000 4194304 4417332 140733637483024 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20431/tid=20441] ppid=20429 vsize=1515964 CPUtime=818.08 cores=0,2,4,6
/proc/20431/task/20441/stat : 20441 (plingeling) R 20429 20431 19972 0 -1 4202560 324212 1256 0 0 81761 47 0 0 25 0 5 0 521524039 1552347136 351397 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3284.98
Current children cumulated vsize (KiB) 1515964

[startup+882.306 s]
/proc/loadavg: 8.10 8.09 7.94 9/186 20510
/proc/meminfo: memFree=28342004/32951124 swapFree=67111364/67111528
[pid=20431] ppid=20429 vsize=1658708 CPUtime=3524.67 cores=0,2,4,6
/proc/20431/stat : 20431 (plingeling) S 20429 20431 19972 0 -1 4202496 1751850 1256 0 0 352215 252 0 0 25 0 5 0 521524036 1698516992 387083 33554432000 4194304 4417332 140733637483024 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/20431/statm: 414677 387083 181 55 0 412563 0
[pid=20431/tid=20438] ppid=20429 vsize=1658708 CPUtime=882.28 cores=0,2,4,6
/proc/20431/task/20438/stat : 20438 (plingeling) R 20429 20431 19972 0 -1 4202560 474812 1256 0 0 88156 72 0 0 25 0 5 0 521524039 1698516992 387083 33554432000 4194304 4417332 140733637483024 18446744073709551615 4365960 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=20431/tid=20439] ppid=20429 vsize=1658708 CPUtime=882.28 cores=0,2,4,6
/proc/20431/task/20439/stat : 20439 (plingeling) R 20429 20431 19972 0 -1 4202560 425184 1256 0 0 88171 57 0 0 25 0 5 0 521524039 1698516992 387083 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364656 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=20431/tid=20440] ppid=20429 vsize=1658708 CPUtime=882.29 cores=0,2,4,6
/proc/20431/task/20440/stat : 20440 (plingeling) R 20429 20431 19972 0 -1 4202560 514782 1256 0 0 88157 72 0 0 25 0 5 0 521524039 1698516992 387083 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20431/tid=20441] ppid=20429 vsize=1658708 CPUtime=877.76 cores=0,2,4,6
/proc/20431/task/20441/stat : 20441 (plingeling) R 20429 20431 19972 0 -1 4202560 335615 1256 0 0 87727 49 0 0 25 0 5 0 521524039 1698516992 387083 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3524.67
Current children cumulated vsize (KiB) 1658708

[startup+942.3 s]
/proc/loadavg: 8.04 8.07 7.94 9/186 20512
/proc/meminfo: memFree=28350800/32951124 swapFree=67111364/67111528
[pid=20431] ppid=20429 vsize=1635052 CPUtime=3764.34 cores=0,2,4,6
/proc/20431/stat : 20431 (plingeling) S 20429 20431 19972 0 -1 4202496 1830921 1256 0 0 376172 262 0 0 25 0 5 0 521524036 1674293248 381275 33554432000 4194304 4417332 140733637483024 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/20431/statm: 408763 381275 181 55 0 406649 0
[pid=20431/tid=20438] ppid=20429 vsize=1635052 CPUtime=942.27 cores=0,2,4,6
/proc/20431/task/20438/stat : 20438 (plingeling) R 20429 20431 19972 0 -1 4202560 509208 1256 0 0 94151 76 0 0 25 0 5 0 521524039 1674293248 381275 33554432000 4194304 4417332 140733637483024 18446744073709551615 4263180 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=20431/tid=20439] ppid=20429 vsize=1635052 CPUtime=942.28 cores=0,2,4,6
/proc/20431/task/20439/stat : 20439 (plingeling) R 20429 20431 19972 0 -1 4202560 461667 1256 0 0 94166 62 0 0 25 0 5 0 521524039 1674293248 381275 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=20431/tid=20440] ppid=20429 vsize=1635052 CPUtime=942.28 cores=0,2,4,6
/proc/20431/task/20440/stat : 20440 (plingeling) R 20429 20431 19972 0 -1 4202560 514782 1256 0 0 94156 72 0 0 25 0 5 0 521524039 1674293248 381275 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20431/tid=20441] ppid=20429 vsize=1635052 CPUtime=937.44 cores=0,2,4,6
/proc/20431/task/20441/stat : 20441 (plingeling) R 20429 20431 19972 0 -1 4202560 343807 1256 0 0 93694 50 0 0 25 0 5 0 521524039 1674293248 381275 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364652 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3764.34
Current children cumulated vsize (KiB) 1635052

[startup+1002.3 s]
/proc/loadavg: 8.24 8.10 7.95 10/187 20516
/proc/meminfo: memFree=28020812/32951124 swapFree=67111364/67111528
[pid=20431] ppid=20429 vsize=1779128 CPUtime=4004.02 cores=0,2,4,6
/proc/20431/stat : 20431 (plingeling) S 20429 20431 19972 0 -1 4202496 1878651 1256 0 0 400132 270 0 0 25 0 5 0 521524036 1821827072 417294 33554432000 4194304 4417332 140733637483024 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/20431/statm: 444782 417294 181 55 0 442668 0
[pid=20431/tid=20438] ppid=20429 vsize=1779128 CPUtime=1002.27 cores=0,2,4,6
/proc/20431/task/20438/stat : 20438 (plingeling) R 20429 20431 19972 0 -1 4202560 517400 1256 0 0 100150 77 0 0 25 0 5 0 521524039 1821827072 417294 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=20431/tid=20439] ppid=20429 vsize=1779128 CPUtime=1002.28 cores=0,2,4,6
/proc/20431/task/20439/stat : 20439 (plingeling) R 20429 20431 19972 0 -1 4202560 487117 1256 0 0 100162 66 0 0 25 0 5 0 521524039 1821827072 417294 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=20431/tid=20440] ppid=20429 vsize=1779128 CPUtime=1002.29 cores=0,2,4,6
/proc/20431/task/20440/stat : 20440 (plingeling) R 20429 20431 19972 0 -1 4202560 522974 1256 0 0 100155 74 0 0 25 0 5 0 521524039 1821827072 417294 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20431/tid=20441] ppid=20429 vsize=1779128 CPUtime=997.12 cores=0,2,4,6
/proc/20431/task/20441/stat : 20441 (plingeling) R 20429 20431 19972 0 -1 4202560 349703 1256 0 0 99661 51 0 0 25 0 5 0 521524039 1821827072 417294 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364519 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4004.02
Current children cumulated vsize (KiB) 1779128

[startup+1062.3 s]
/proc/loadavg: 8.26 8.15 7.98 9/186 20518
/proc/meminfo: memFree=27826412/32951124 swapFree=67111364/67111528
[pid=20431] ppid=20429 vsize=1921348 CPUtime=4243.71 cores=0,2,4,6
/proc/20431/stat : 20431 (plingeling) S 20429 20431 19972 0 -1 4202496 1940974 1256 0 0 424091 280 0 0 25 0 5 0 521524036 1967460352 452849 33554432000 4194304 4417332 140733637483024 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/20431/statm: 480337 452849 181 55 0 478223 0
[pid=20431/tid=20438] ppid=20429 vsize=1921348 CPUtime=1062.27 cores=0,2,4,6
/proc/20431/task/20438/stat : 20438 (plingeling) R 20429 20431 19972 0 -1 4202560 517400 1256 0 0 106150 77 0 0 25 0 5 0 521524039 1967460352 452849 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=20431/tid=20439] ppid=20429 vsize=1921348 CPUtime=1062.28 cores=0,2,4,6
/proc/20431/task/20439/stat : 20439 (plingeling) R 20429 20431 19972 0 -1 4202560 487117 1256 0 0 106162 66 0 0 25 0 5 0 521524039 1967460352 452849 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=20431/tid=20440] ppid=20429 vsize=1921348 CPUtime=1062.28 cores=0,2,4,6
/proc/20431/task/20440/stat : 20440 (plingeling) R 20429 20431 19972 0 -1 4202560 577105 1256 0 0 106145 83 0 0 25 0 5 0 521524039 1967460352 452849 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364973 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20431/tid=20441] ppid=20429 vsize=1921348 CPUtime=1056.8 cores=0,2,4,6
/proc/20431/task/20441/stat : 20441 (plingeling) R 20429 20431 19972 0 -1 4202560 357895 1256 0 0 105629 51 0 0 25 0 5 0 521524039 1967460352 452849 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4243.71
Current children cumulated vsize (KiB) 1921348

[startup+1122.3 s]
/proc/loadavg: 8.13 8.14 7.98 9/186 20520
/proc/meminfo: memFree=27714552/32951124 swapFree=67111364/67111528
[pid=20431] ppid=20429 vsize=1906672 CPUtime=4483.4 cores=0,2,4,6
/proc/20431/stat : 20431 (plingeling) S 20429 20431 19972 0 -1 4202496 2024575 1256 0 0 448048 292 0 0 25 0 5 0 521524036 1952432128 449264 33554432000 4194304 4417332 140733637483024 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/20431/statm: 476668 449264 181 55 0 474554 0
[pid=20431/tid=20438] ppid=20429 vsize=1906672 CPUtime=1122.28 cores=0,2,4,6
/proc/20431/task/20438/stat : 20438 (plingeling) R 20429 20431 19972 0 -1 4202560 575754 1256 0 0 112143 85 0 0 25 0 5 0 521524039 1952432128 449264 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=20431/tid=20439] ppid=20429 vsize=1906672 CPUtime=1122.29 cores=0,2,4,6
/proc/20431/task/20439/stat : 20439 (plingeling) R 20429 20431 19972 0 -1 4202560 495309 1256 0 0 112161 68 0 0 25 0 5 0 521524039 1952432128 449264 33554432000 4194304 4417332 140733637483024 18446744073709551615 4365977 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=20431/tid=20440] ppid=20429 vsize=1906672 CPUtime=1122.29 cores=0,2,4,6
/proc/20431/task/20440/stat : 20440 (plingeling) R 20429 20431 19972 0 -1 4202560 577105 1256 0 0 112146 83 0 0 25 0 5 0 521524039 1952432128 449264 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20431/tid=20441] ppid=20429 vsize=1906672 CPUtime=1116.49 cores=0,2,4,6
/proc/20431/task/20441/stat : 20441 (plingeling) R 20429 20431 19972 0 -1 4202560 374950 1256 0 0 111595 54 0 0 25 0 5 0 521524039 1952432128 449264 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364722 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4483.4
Current children cumulated vsize (KiB) 1906672

[startup+1182.3 s]
/proc/loadavg: 8.08 8.12 7.99 9/186 20521
/proc/meminfo: memFree=27788452/32951124 swapFree=67111364/67111528
[pid=20431] ppid=20429 vsize=1961544 CPUtime=4723.11 cores=0,2,4,6
/proc/20431/stat : 20431 (plingeling) S 20429 20431 19972 0 -1 4202496 2042389 1256 0 0 472015 296 0 0 25 0 5 0 521524036 2008621056 462982 33554432000 4194304 4417332 140733637483024 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/20431/statm: 490386 462982 181 55 0 488272 0
[pid=20431/tid=20438] ppid=20429 vsize=1961544 CPUtime=1182.28 cores=0,2,4,6
/proc/20431/task/20438/stat : 20438 (plingeling) R 20429 20431 19972 0 -1 4202560 575754 1256 0 0 118143 85 0 0 25 0 5 0 521524039 2008621056 462982 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364683 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=20431/tid=20439] ppid=20429 vsize=1961544 CPUtime=1182.29 cores=0,2,4,6
/proc/20431/task/20439/stat : 20439 (plingeling) R 20429 20431 19972 0 -1 4202560 509027 1256 0 0 118158 71 0 0 25 0 5 0 521524039 2008621056 462982 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=20431/tid=20440] ppid=20429 vsize=1961544 CPUtime=1182.29 cores=0,2,4,6
/proc/20431/task/20440/stat : 20440 (plingeling) R 20429 20431 19972 0 -1 4202560 577105 1256 0 0 118146 83 0 0 25 0 5 0 521524039 2008621056 462982 33554432000 4194304 4417332 140733637483024 18446744073709551615 4346399 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20431/tid=20441] ppid=20429 vsize=1961544 CPUtime=1176.19 cores=0,2,4,6
/proc/20431/task/20441/stat : 20441 (plingeling) R 20429 20431 19972 0 -1 4202560 379046 1256 0 0 117564 55 0 0 25 0 5 0 521524039 2008621056 462982 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4723.11
Current children cumulated vsize (KiB) 1961544



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.6 s]
/proc/loadavg: 8.06 8.11 7.99 9/186 20522
/proc/meminfo: memFree=27701520/32951124 swapFree=67111364/67111528
[pid=20431] ppid=20429 vsize=2011716 CPUtime=4800.22 cores=0,2,4,6
/proc/20431/stat : 20431 (plingeling) S 20429 20431 19972 0 -1 4202496 2095287 1256 0 0 479717 305 0 0 25 0 5 0 521524036 2059997184 475525 33554432000 4194304 4417332 140733637483024 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/20431/statm: 502929 475525 181 55 0 500815 0
[pid=20431/tid=20438] ppid=20429 vsize=2011716 CPUtime=1201.59 cores=0,2,4,6
/proc/20431/task/20438/stat : 20438 (plingeling) R 20429 20431 19972 0 -1 4202560 575754 1256 0 0 120074 85 0 0 25 0 5 0 521524039 2059997184 475525 33554432000 4194304 4417332 140733637483024 18446744073709551615 4263080 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=20431/tid=20439] ppid=20429 vsize=2011716 CPUtime=1201.6 cores=0,2,4,6
/proc/20431/task/20439/stat : 20439 (plingeling) R 20429 20431 19972 0 -1 4202560 509027 1256 0 0 120089 71 0 0 25 0 5 0 521524039 2059997184 475525 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=20431/tid=20440] ppid=20429 vsize=2011716 CPUtime=1201.59 cores=0,2,4,6
/proc/20431/task/20440/stat : 20440 (plingeling) R 20429 20431 19972 0 -1 4202560 630003 1256 0 0 120067 92 0 0 25 0 5 0 521524039 2059997184 475525 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20431/tid=20441] ppid=20429 vsize=2011716 CPUtime=1195.39 cores=0,2,4,6
/proc/20431/task/20441/stat : 20441 (plingeling) R 20429 20431 19972 0 -1 4202560 379046 1256 0 0 119484 55 0 0 25 0 5 0 521524039 2059997184 475525 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364639 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4800.22
Current children cumulated vsize (KiB) 2011716

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.6 s]
/proc/loadavg: 8.06 8.11 7.99 9/186 20522
/proc/meminfo: memFree=27701520/32951124 swapFree=67111364/67111528
[pid=20431] ppid=20429 vsize=2011716 CPUtime=4800.22 cores=0,2,4,6
/proc/20431/stat : 20431 (plingeling) S 20429 20431 19972 0 -1 4202496 2095287 1256 0 0 479717 305 0 0 25 0 5 0 521524036 2059997184 475525 33554432000 4194304 4417332 140733637483024 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/20431/statm: 502929 475525 181 55 0 500815 0
[pid=20431/tid=20438] ppid=20429 vsize=2011716 CPUtime=1201.59 cores=0,2,4,6
/proc/20431/task/20438/stat : 20438 (plingeling) R 20429 20431 19972 0 -1 4202560 575754 1256 0 0 120074 85 0 0 25 0 5 0 521524039 2059997184 475525 33554432000 4194304 4417332 140733637483024 18446744073709551615 4263080 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=20431/tid=20439] ppid=20429 vsize=2011716 CPUtime=1201.6 cores=0,2,4,6
/proc/20431/task/20439/stat : 20439 (plingeling) R 20429 20431 19972 0 -1 4202560 509027 1256 0 0 120089 71 0 0 25 0 5 0 521524039 2059997184 475525 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=20431/tid=20440] ppid=20429 vsize=2011716 CPUtime=1201.59 cores=0,2,4,6
/proc/20431/task/20440/stat : 20440 (plingeling) R 20429 20431 19972 0 -1 4202560 630003 1256 0 0 120067 92 0 0 25 0 5 0 521524039 2059997184 475525 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20431/tid=20441] ppid=20429 vsize=2011716 CPUtime=1195.39 cores=0,2,4,6
/proc/20431/task/20441/stat : 20441 (plingeling) R 20429 20431 19972 0 -1 4202560 379046 1256 0 0 119484 55 0 0 25 0 5 0 521524039 2059997184 475525 33554432000 4194304 4417332 140733637483024 18446744073709551615 4364639 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4800.22
Current children cumulated vsize (KiB) 2011716

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1201.68
CPU time (s): 4800.33
CPU user time (s): 4797.2
CPU system time (s): 3.13052
CPU usage (%): 399.468
Max. virtual memory (cumulated for all children) (KiB): 2011716

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4797.2
system time used= 3.13052
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2096543
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 117
involuntary context switches= 21813

runsolver used 2.02669 second user time and 4.18336 second system time

The end

Launcher Data 

Begin job on node106 at 2011-04-29 19:47:16
IDJOB=3263222
IDBENCH=83945
IDSOLVER=1605
FILE ID=node106/3263222-1304099236
RUNJOBID= node106-1304097373-19990
PBS_JOBID= 13170612
Free space on /tmp= 72160 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT03/handmade/anton/UNSAT/SGI_30_70_18_80_10-dir.shuffled-as.sat03-140.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3263222-1304099236/watcher-3263222-1304099236 -o /tmp/evaluation-result-3263222-1304099236/solver-3263222-1304099236 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3263222-1304099236.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 1037a2754a0a53f62ba8ec901741c9e9
RANDOM SEED=238386681

node106.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.20
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      29869836 kB
Buffers:        253392 kB
Cached:        2075296 kB
SwapCached:         88 kB
Active:        1263884 kB
Inactive:      1605796 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      29869836 kB
SwapTotal:    67111528 kB
SwapFree:     67111364 kB
Dirty:             844 kB
Writeback:           0 kB
AnonPages:      540640 kB
Mapped:          15280 kB
Slab:           146456 kB
PageTables:       5716 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   727596 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72160 MiB
End job on node106 at 2011-04-29 20:07:20