Trace number 3265114

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 4800.4 1201.8

General information on the benchmark

NameSAT05/crafted/sabharwal05/counting/php/unsat/
easier/php-045-035.shuffled-as.sat05-1181.cnf
MD5SUM057b78ff24cc0773c9c67e770a7b9de2
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark44.4072
Satisfiable
(Un)Satisfiability was proved
Number of variables1575
Number of clauses34695
Sum of the clauses size70875
Maximum clause length35
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 234650
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 545

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3265114-1304105614/watcher-3265114-1304105614 -o /tmp/evaluation-result-3265114-1304105614/solver-3265114-1304105614 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3265114-1304105614.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.12 7.97 7.95 5/181 20660
/proc/meminfo: memFree=30448508/32951124 swapFree=67111416/67111528
[pid=20660] ppid=20658 vsize=9504 CPUtime=0 cores=1,3,5,7
/proc/20660/stat : 20660 (plingeling) R 20658 20660 19850 0 -1 4202496 460 1255 0 0 0 0 0 0 25 0 1 0 522163760 9732096 389 33554432000 4194304 4417332 140737404153680 18446744073709551615 212436016896 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/20660/statm: 2376 389 146 55 0 262 0

[startup+0.00714408 s]
/proc/loadavg: 8.12 7.97 7.95 5/181 20660
/proc/meminfo: memFree=30448508/32951124 swapFree=67111416/67111528
[pid=20660] ppid=20658 vsize=9664 CPUtime=0 cores=1,3,5,7
/proc/20660/stat : 20660 (plingeling) R 20658 20660 19850 0 -1 4202496 508 1255 0 0 0 0 0 0 25 0 1 0 522163760 9895936 437 33554432000 4194304 4417332 140737404153680 18446744073709551615 212436016896 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/20660/statm: 2416 437 146 55 0 302 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9664

[startup+0.137075 s]
/proc/loadavg: 8.12 7.97 7.95 5/181 20660
/proc/meminfo: memFree=30448508/32951124 swapFree=67111416/67111528
[pid=20660] ppid=20658 vsize=124468 CPUtime=0.43 cores=1,3,5,7
/proc/20660/stat : 20660 (plingeling) S 20658 20660 19850 0 -1 4202496 3347 1255 0 0 43 0 0 0 25 0 5 0 522163760 127455232 2542 33554432000 4194304 4417332 140737404153680 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/20660/statm: 31117 2542 179 55 0 29003 0
Current children cumulated CPU time (s) 0.43
Current children cumulated vsize (KiB) 124468

[startup+0.301043 s]
/proc/loadavg: 8.12 7.97 7.95 5/181 20660
/proc/meminfo: memFree=30448508/32951124 swapFree=67111416/67111528
[pid=20660] ppid=20658 vsize=126376 CPUtime=1.08 cores=1,3,5,7
/proc/20660/stat : 20660 (plingeling) S 20658 20660 19850 0 -1 4202496 4464 1255 0 0 108 0 0 0 25 0 5 0 522163760 129409024 3020 33554432000 4194304 4417332 140737404153680 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/20660/statm: 31594 3020 180 55 0 29480 0
Current children cumulated CPU time (s) 1.08
Current children cumulated vsize (KiB) 126376

[startup+0.700968 s]
/proc/loadavg: 8.12 7.97 7.95 5/181 20660
/proc/meminfo: memFree=30448508/32951124 swapFree=67111416/67111528
[pid=20660] ppid=20658 vsize=134720 CPUtime=2.69 cores=1,3,5,7
/proc/20660/stat : 20660 (plingeling) S 20658 20660 19850 0 -1 4202496 9096 1255 0 0 268 1 0 0 25 0 5 0 522163760 137953280 5220 33554432000 4194304 4417332 140737404153680 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/20660/statm: 33680 5220 180 55 0 31566 0
Current children cumulated CPU time (s) 2.69
Current children cumulated vsize (KiB) 134720

[startup+1.50082 s]
/proc/loadavg: 8.12 7.97 7.95 9/186 20671
/proc/meminfo: memFree=30424792/32951124 swapFree=67111416/67111528
[pid=20660] ppid=20658 vsize=147212 CPUtime=5.88 cores=1,3,5,7
/proc/20660/stat : 20660 (plingeling) S 20658 20660 19850 0 -1 4202496 24553 1255 0 0 584 4 0 0 25 0 5 0 522163760 150745088 8487 33554432000 4194304 4417332 140737404153680 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/20660/statm: 36803 8487 180 55 0 34689 0
[pid=20660/tid=20667] ppid=20658 vsize=147212 CPUtime=1.46 cores=1,3,5,7
/proc/20660/task/20667/stat : 20667 (plingeling) R 20658 20660 19850 0 -1 4202560 4410 1255 0 0 145 1 0 0 25 0 5 0 522163763 150745088 8487 33554432000 4194304 4417332 140737404153680 18446744073709551615 4351325 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20660/tid=20668] ppid=20658 vsize=147212 CPUtime=1.45 cores=1,3,5,7
/proc/20660/task/20668/stat : 20668 (plingeling) R 20658 20660 19850 0 -1 4202560 5124 1255 0 0 145 0 0 0 25 0 5 0 522163763 150745088 8487 33554432000 4194304 4417332 140737404153680 18446744073709551615 4244642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20660/tid=20669] ppid=20658 vsize=147212 CPUtime=1.46 cores=1,3,5,7
/proc/20660/task/20669/stat : 20669 (plingeling) R 20658 20660 19850 0 -1 4202560 4014 1255 0 0 146 0 0 0 25 0 5 0 522163763 150745088 8487 33554432000 4194304 4417332 140737404153680 18446744073709551615 4365977 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20660/tid=20670] ppid=20658 vsize=147212 CPUtime=1.45 cores=1,3,5,7
/proc/20660/task/20670/stat : 20670 (plingeling) R 20658 20660 19850 0 -1 4202560 9456 1255 0 0 144 1 0 0 25 0 5 0 522163763 150745088 8487 33554432000 4194304 4417332 140737404153680 18446744073709551615 212436074922 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 5.88
Current children cumulated vsize (KiB) 147212

[startup+3.10051 s]
/proc/loadavg: 8.11 7.97 7.95 9/186 20671
/proc/meminfo: memFree=30402596/32951124 swapFree=67111416/67111528
[pid=20660] ppid=20658 vsize=163204 CPUtime=12.27 cores=1,3,5,7
/proc/20660/stat : 20660 (plingeling) S 20658 20660 19850 0 -1 4202496 52419 1255 0 0 1219 8 0 0 25 0 5 0 522163760 167120896 12376 33554432000 4194304 4417332 140737404153680 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/20660/statm: 40801 12376 180 55 0 38687 0
[pid=20660/tid=20667] ppid=20658 vsize=163204 CPUtime=3.06 cores=1,3,5,7
/proc/20660/task/20667/stat : 20667 (plingeling) R 20658 20660 19850 0 -1 4202560 9429 1255 0 0 305 1 0 0 25 0 5 0 522163763 167120896 12376 33554432000 4194304 4417332 140737404153680 18446744073709551615 4249143 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20660/tid=20668] ppid=20658 vsize=163204 CPUtime=3.05 cores=1,3,5,7
/proc/20660/task/20668/stat : 20668 (plingeling) R 20658 20660 19850 0 -1 4202560 11205 1255 0 0 304 1 0 0 25 0 5 0 522163763 167120896 12376 33554432000 4194304 4417332 140737404153680 18446744073709551615 4353461 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20660/tid=20669] ppid=20658 vsize=163204 CPUtime=3.06 cores=1,3,5,7
/proc/20660/task/20669/stat : 20669 (plingeling) R 20658 20660 19850 0 -1 4202560 9306 1255 0 0 305 1 0 0 25 0 5 0 522163763 167120896 12376 33554432000 4194304 4417332 140737404153680 18446744073709551615 4249597 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20660/tid=20670] ppid=20658 vsize=163204 CPUtime=3.05 cores=1,3,5,7
/proc/20660/task/20670/stat : 20670 (plingeling) R 20658 20660 19850 0 -1 4202560 20930 1255 0 0 302 3 0 0 25 0 5 0 522163763 167120896 12376 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 12.27
Current children cumulated vsize (KiB) 163204

[startup+6.30091 s]
/proc/loadavg: 8.11 7.97 7.95 9/186 20671
/proc/meminfo: memFree=30374756/32951124 swapFree=67111416/67111528
[pid=20660] ppid=20658 vsize=203232 CPUtime=25.06 cores=1,3,5,7
/proc/20660/stat : 20660 (plingeling) S 20658 20660 19850 0 -1 4202496 106248 1255 0 0 2489 17 0 0 25 0 5 0 522163760 208109568 22719 33554432000 4194304 4417332 140737404153680 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/20660/statm: 50808 22719 180 55 0 48694 0
[pid=20660/tid=20667] ppid=20658 vsize=203232 CPUtime=6.26 cores=1,3,5,7
/proc/20660/task/20667/stat : 20667 (plingeling) R 20658 20660 19850 0 -1 4202560 24714 1255 0 0 622 4 0 0 25 0 5 0 522163763 208109568 22719 33554432000 4194304 4417332 140737404153680 18446744073709551615 4270223 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20660/tid=20668] ppid=20658 vsize=203232 CPUtime=6.25 cores=1,3,5,7
/proc/20660/task/20668/stat : 20668 (plingeling) R 20658 20660 19850 0 -1 4202560 25505 1255 0 0 622 3 0 0 25 0 5 0 522163763 208109568 22719 33554432000 4194304 4417332 140737404153680 18446744073709551615 4256265 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20660/tid=20669] ppid=20658 vsize=203232 CPUtime=6.26 cores=1,3,5,7
/proc/20660/task/20669/stat : 20669 (plingeling) R 20658 20660 19850 0 -1 4202560 23172 1255 0 0 623 3 0 0 25 0 5 0 522163763 208109568 22719 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20660/tid=20670] ppid=20658 vsize=203232 CPUtime=6.23 cores=1,3,5,7
/proc/20660/task/20670/stat : 20670 (plingeling) R 20658 20660 19850 0 -1 4202560 31308 1255 0 0 618 5 0 0 25 0 5 0 522163763 208109568 22719 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 25.06
Current children cumulated vsize (KiB) 203232

[startup+12.7007 s]
/proc/loadavg: 8.16 7.98 7.96 9/186 20671
/proc/meminfo: memFree=30308492/32951124 swapFree=67111416/67111528
[pid=20660] ppid=20658 vsize=252376 CPUtime=50.62 cores=1,3,5,7
/proc/20660/stat : 20660 (plingeling) S 20658 20660 19850 0 -1 4202496 194741 1255 0 0 5032 30 0 0 25 0 5 0 522163760 258433024 34683 33554432000 4194304 4417332 140737404153680 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/20660/statm: 63094 34683 181 55 0 60980 0
[pid=20660/tid=20667] ppid=20658 vsize=252376 CPUtime=12.66 cores=1,3,5,7
/proc/20660/task/20667/stat : 20667 (plingeling) R 20658 20660 19850 0 -1 4202560 47874 1255 0 0 1259 7 0 0 25 0 5 0 522163763 258433024 34683 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20660/tid=20668] ppid=20658 vsize=252376 CPUtime=12.65 cores=1,3,5,7
/proc/20660/task/20668/stat : 20668 (plingeling) R 20658 20660 19850 0 -1 4202560 48108 1255 0 0 1258 7 0 0 25 0 5 0 522163763 258433024 34683 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20660/tid=20669] ppid=20658 vsize=252376 CPUtime=12.66 cores=1,3,5,7
/proc/20660/task/20669/stat : 20669 (plingeling) R 20658 20660 19850 0 -1 4202560 48041 1255 0 0 1259 7 0 0 25 0 5 0 522163763 258433024 34683 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364536 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20660/tid=20670] ppid=20658 vsize=252376 CPUtime=12.59 cores=1,3,5,7
/proc/20660/task/20670/stat : 20670 (plingeling) R 20658 20660 19850 0 -1 4202560 49169 1255 0 0 1252 7 0 0 25 0 5 0 522163763 258433024 34683 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364732 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 50.62
Current children cumulated vsize (KiB) 252376

[startup+25.5003 s]
/proc/loadavg: 8.14 7.98 7.96 9/186 20671
/proc/meminfo: memFree=30146308/32951124 swapFree=67111416/67111528
[pid=20660] ppid=20658 vsize=395884 CPUtime=101.74 cores=1,3,5,7
/proc/20660/stat : 20660 (plingeling) S 20658 20660 19850 0 -1 4202496 355611 1255 0 0 10119 55 0 0 25 0 5 0 522163760 405385216 71683 33554432000 4194304 4417332 140737404153680 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/20660/statm: 98971 71683 181 55 0 96857 0
[pid=20660/tid=20667] ppid=20658 vsize=395884 CPUtime=25.46 cores=1,3,5,7
/proc/20660/task/20667/stat : 20667 (plingeling) R 20658 20660 19850 0 -1 4202560 85832 1255 0 0 2534 12 0 0 25 0 5 0 522163763 405385216 71683 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20660/tid=20668] ppid=20658 vsize=395884 CPUtime=25.45 cores=1,3,5,7
/proc/20660/task/20668/stat : 20668 (plingeling) R 20658 20660 19850 0 -1 4202560 96828 1255 0 0 2530 15 0 0 25 0 5 0 522163763 405385216 71683 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364973 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20660/tid=20669] ppid=20658 vsize=395884 CPUtime=25.46 cores=1,3,5,7
/proc/20660/task/20669/stat : 20669 (plingeling) R 20658 20660 19850 0 -1 4202560 83669 1255 0 0 2533 13 0 0 25 0 5 0 522163763 405385216 71683 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20660/tid=20670] ppid=20658 vsize=395884 CPUtime=25.32 cores=1,3,5,7
/proc/20660/task/20670/stat : 20670 (plingeling) R 20658 20660 19850 0 -1 4202560 87733 1255 0 0 2518 14 0 0 25 0 5 0 522163763 405385216 71683 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364526 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 101.74
Current children cumulated vsize (KiB) 395884

[startup+51.1004 s]
/proc/loadavg: 8.09 7.98 7.96 9/186 20672
/proc/meminfo: memFree=30014224/32951124 swapFree=67111416/67111528
[pid=20660] ppid=20658 vsize=508352 CPUtime=204.01 cores=1,3,5,7
/proc/20660/stat : 20660 (plingeling) S 20658 20660 19850 0 -1 4202496 594318 1255 0 0 20312 89 0 0 25 0 5 0 522163760 520552448 100985 33554432000 4194304 4417332 140737404153680 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/20660/statm: 127088 100985 181 55 0 124974 0
[pid=20660/tid=20667] ppid=20658 vsize=508352 CPUtime=51.06 cores=1,3,5,7
/proc/20660/task/20667/stat : 20667 (plingeling) R 20658 20660 19850 0 -1 4202560 149317 1255 0 0 5087 19 0 0 25 0 5 0 522163763 520552448 100985 33554432000 4194304 4417332 140737404153680 18446744073709551615 4310394 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20660/tid=20668] ppid=20658 vsize=508352 CPUtime=51.05 cores=1,3,5,7
/proc/20660/task/20668/stat : 20668 (plingeling) R 20658 20660 19850 0 -1 4202560 163533 1255 0 0 5081 24 0 0 25 0 5 0 522163763 520552448 100985 33554432000 4194304 4417332 140737404153680 18446744073709551615 4311285 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20660/tid=20669] ppid=20658 vsize=508352 CPUtime=51.05 cores=1,3,5,7
/proc/20660/task/20669/stat : 20669 (plingeling) R 20658 20660 19850 0 -1 4202560 149726 1255 0 0 5083 22 0 0 25 0 5 0 522163763 520552448 100985 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20660/tid=20670] ppid=20658 vsize=508352 CPUtime=50.79 cores=1,3,5,7
/proc/20660/task/20670/stat : 20670 (plingeling) R 20658 20660 19850 0 -1 4202560 130193 1255 0 0 5057 22 0 0 25 0 5 0 522163763 520552448 100985 33554432000 4194304 4417332 140737404153680 18446744073709551615 4365977 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 204.01
Current children cumulated vsize (KiB) 508352

[startup+102.307 s]
/proc/loadavg: 8.08 8.00 7.97 9/186 20673
/proc/meminfo: memFree=29832408/32951124 swapFree=67111416/67111528
[pid=20660] ppid=20658 vsize=681256 CPUtime=408.55 cores=1,3,5,7
/proc/20660/stat : 20660 (plingeling) S 20658 20660 19850 0 -1 4202496 971376 1255 0 0 40711 144 0 0 25 0 5 0 522163760 697606144 140295 33554432000 4194304 4417332 140737404153680 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/20660/statm: 170314 140295 181 55 0 168200 0
[pid=20660/tid=20667] ppid=20658 vsize=681256 CPUtime=102.26 cores=1,3,5,7
/proc/20660/task/20667/stat : 20667 (plingeling) R 20658 20660 19850 0 -1 4202560 247402 1255 0 0 10192 34 0 0 25 0 5 0 522163763 697606144 140295 33554432000 4194304 4417332 140737404153680 18446744073709551615 4343677 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20660/tid=20668] ppid=20658 vsize=681256 CPUtime=102.26 cores=1,3,5,7
/proc/20660/task/20668/stat : 20668 (plingeling) R 20658 20660 19850 0 -1 4202560 270054 1255 0 0 10188 38 0 0 25 0 5 0 522163763 697606144 140295 33554432000 4194304 4417332 140737404153680 18446744073709551615 4310483 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20660/tid=20669] ppid=20658 vsize=681256 CPUtime=102.26 cores=1,3,5,7
/proc/20660/task/20669/stat : 20669 (plingeling) R 20658 20660 19850 0 -1 4202560 246405 1255 0 0 10190 36 0 0 25 0 5 0 522163763 697606144 140295 33554432000 4194304 4417332 140737404153680 18446744073709551615 4356996 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20660/tid=20670] ppid=20658 vsize=681256 CPUtime=101.71 cores=1,3,5,7
/proc/20660/task/20670/stat : 20670 (plingeling) R 20658 20660 19850 0 -1 4202560 205966 1255 0 0 10137 34 0 0 25 0 5 0 522163763 697606144 140295 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364713 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 408.55
Current children cumulated vsize (KiB) 681256

[startup+162.301 s]
/proc/loadavg: 8.06 8.01 7.97 9/186 20675
/proc/meminfo: memFree=29570096/32951124 swapFree=67111416/67111528
[pid=20660] ppid=20658 vsize=883552 CPUtime=648.21 cores=1,3,5,7
/proc/20660/stat : 20660 (plingeling) S 20658 20660 19850 0 -1 4202496 1462857 1255 0 0 64602 219 0 0 25 0 5 0 522163760 904757248 192325 33554432000 4194304 4417332 140737404153680 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/20660/statm: 220888 192325 181 55 0 218774 0
[pid=20660/tid=20667] ppid=20658 vsize=883552 CPUtime=162.25 cores=1,3,5,7
/proc/20660/task/20667/stat : 20667 (plingeling) R 20658 20660 19850 0 -1 4202560 362956 1255 0 0 16175 50 0 0 25 0 5 0 522163763 904757248 192325 33554432000 4194304 4417332 140737404153680 18446744073709551615 4365963 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20660/tid=20668] ppid=20658 vsize=883552 CPUtime=162.26 cores=1,3,5,7
/proc/20660/task/20668/stat : 20668 (plingeling) R 20658 20660 19850 0 -1 4202560 432963 1255 0 0 16163 63 0 0 25 0 5 0 522163763 904757248 192325 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20660/tid=20669] ppid=20658 vsize=883552 CPUtime=162.26 cores=1,3,5,7
/proc/20660/task/20669/stat : 20669 (plingeling) R 20658 20660 19850 0 -1 4202560 404207 1255 0 0 16165 61 0 0 25 0 5 0 522163763 904757248 192325 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20660/tid=20670] ppid=20658 vsize=883552 CPUtime=161.39 cores=1,3,5,7
/proc/20660/task/20670/stat : 20670 (plingeling) R 20658 20660 19850 0 -1 4202560 261182 1255 0 0 16096 43 0 0 25 0 5 0 522163763 904757248 192325 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364887 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 648.21
Current children cumulated vsize (KiB) 883552


################
# More data... #
################

[pid=20660/tid=20668] ppid=20658 vsize=2155980 CPUtime=762.28 cores=1,3,5,7
/proc/20660/task/20668/stat : 20668 (plingeling) R 20658 20660 19850 0 -1 4202560 1159557 1255 0 0 76065 163 0 0 25 0 5 0 522163763 2207723520 511117 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20660/tid=20669] ppid=20658 vsize=2155980 CPUtime=762.27 cores=1,3,5,7
/proc/20660/task/20669/stat : 20669 (plingeling) R 20658 20660 19850 0 -1 4202560 1304625 1255 0 0 76046 181 0 0 25 0 5 0 522163763 2207723520 511117 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364919 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20660/tid=20670] ppid=20658 vsize=2155980 CPUtime=758.14 cores=1,3,5,7
/proc/20660/task/20670/stat : 20670 (plingeling) R 20658 20660 19850 0 -1 4202560 615143 1255 0 0 75714 100 0 0 25 0 5 0 522163763 2207723520 511117 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364656 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3045.03
Current children cumulated vsize (KiB) 2155980

[startup+822.301 s]
/proc/loadavg: 8.10 8.10 8.02 9/186 20692
/proc/meminfo: memFree=28019368/32951124 swapFree=67111416/67111528
[pid=20660] ppid=20658 vsize=2259400 CPUtime=3284.68 cores=1,3,5,7
/proc/20660/stat : 20660 (plingeling) S 20658 20660 19850 0 -1 4202496 4311317 1255 0 0 327860 608 0 0 25 0 5 0 522163760 2313625600 536972 33554432000 4194304 4417332 140737404153680 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/20660/statm: 564850 536972 181 55 0 562736 0
[pid=20660/tid=20667] ppid=20658 vsize=2259400 CPUtime=822.26 cores=1,3,5,7
/proc/20660/task/20667/stat : 20667 (plingeling) R 20658 20660 19850 0 -1 4202560 1088425 1255 0 0 82083 143 0 0 25 0 5 0 522163763 2313625600 536972 33554432000 4194304 4417332 140737404153680 18446744073709551615 4305857 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20660/tid=20668] ppid=20658 vsize=2259400 CPUtime=822.28 cores=1,3,5,7
/proc/20660/task/20668/stat : 20668 (plingeling) R 20658 20660 19850 0 -1 4202560 1247730 1255 0 0 82052 176 0 0 25 0 5 0 522163763 2313625600 536972 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20660/tid=20669] ppid=20658 vsize=2259400 CPUtime=822.27 cores=1,3,5,7
/proc/20660/task/20669/stat : 20669 (plingeling) R 20658 20660 19850 0 -1 4202560 1342000 1255 0 0 82041 186 0 0 25 0 5 0 522163763 2313625600 536972 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364796 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20660/tid=20670] ppid=20658 vsize=2259400 CPUtime=817.82 cores=1,3,5,7
/proc/20660/task/20670/stat : 20670 (plingeling) R 20658 20660 19850 0 -1 4202560 631613 1255 0 0 81680 102 0 0 25 0 5 0 522163763 2313625600 536972 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3284.68
Current children cumulated vsize (KiB) 2259400

[startup+882.3 s]
/proc/loadavg: 8.04 8.08 8.02 9/186 20694
/proc/meminfo: memFree=27852312/32951124 swapFree=67111416/67111528
[pid=20660] ppid=20658 vsize=2417376 CPUtime=3524.36 cores=1,3,5,7
/proc/20660/stat : 20660 (plingeling) S 20658 20660 19850 0 -1 4202496 4413497 1255 0 0 351811 625 0 0 25 0 5 0 522163760 2475393024 576466 33554432000 4194304 4417332 140737404153680 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/20660/statm: 604344 576466 181 55 0 602230 0
[pid=20660/tid=20667] ppid=20658 vsize=2417376 CPUtime=882.25 cores=1,3,5,7
/proc/20660/task/20667/stat : 20667 (plingeling) R 20658 20660 19850 0 -1 4202560 1107617 1255 0 0 88079 146 0 0 25 0 5 0 522163763 2475393024 576466 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20660/tid=20668] ppid=20658 vsize=2417376 CPUtime=882.28 cores=1,3,5,7
/proc/20660/task/20668/stat : 20668 (plingeling) R 20658 20660 19850 0 -1 4202560 1289536 1255 0 0 88045 183 0 0 25 0 5 0 522163763 2475393024 576466 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20660/tid=20669] ppid=20658 vsize=2417376 CPUtime=882.27 cores=1,3,5,7
/proc/20660/task/20669/stat : 20669 (plingeling) R 20658 20660 19850 0 -1 4202560 1383182 1255 0 0 88036 191 0 0 25 0 5 0 522163763 2475393024 576466 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20660/tid=20670] ppid=20658 vsize=2417376 CPUtime=877.49 cores=1,3,5,7
/proc/20660/task/20670/stat : 20670 (plingeling) R 20658 20660 19850 0 -1 4202560 631613 1255 0 0 87647 102 0 0 25 0 5 0 522163763 2475393024 576466 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3524.36
Current children cumulated vsize (KiB) 2417376

[startup+942.301 s]
/proc/loadavg: 8.01 8.06 8.01 9/186 20696
/proc/meminfo: memFree=27964504/32951124 swapFree=67111416/67111528
[pid=20660] ppid=20658 vsize=2235952 CPUtime=3764.05 cores=1,3,5,7
/proc/20660/stat : 20660 (plingeling) S 20658 20660 19850 0 -1 4202496 4626776 1255 0 0 375751 654 0 0 25 0 5 0 522163760 2289614848 531110 33554432000 4194304 4417332 140737404153680 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/20660/statm: 558988 531110 181 55 0 556874 0
[pid=20660/tid=20667] ppid=20658 vsize=2235952 CPUtime=942.26 cores=1,3,5,7
/proc/20660/task/20667/stat : 20667 (plingeling) R 20658 20660 19850 0 -1 4202560 1138553 1255 0 0 94075 151 0 0 25 0 5 0 522163763 2289614848 531110 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20660/tid=20668] ppid=20658 vsize=2235952 CPUtime=942.29 cores=1,3,5,7
/proc/20660/task/20668/stat : 20668 (plingeling) R 20658 20660 19850 0 -1 4202560 1322690 1255 0 0 94041 188 0 0 25 0 5 0 522163763 2289614848 531110 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364713 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20660/tid=20669] ppid=20658 vsize=2235952 CPUtime=942.27 cores=1,3,5,7
/proc/20660/task/20669/stat : 20669 (plingeling) R 20658 20660 19850 0 -1 4202560 1498997 1255 0 0 94019 208 0 0 25 0 5 0 522163763 2289614848 531110 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364722 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20660/tid=20670] ppid=20658 vsize=2235952 CPUtime=937.17 cores=1,3,5,7
/proc/20660/task/20670/stat : 20670 (plingeling) R 20658 20660 19850 0 -1 4202560 664987 1255 0 0 93611 106 0 0 25 0 5 0 522163763 2289614848 531110 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3764.05
Current children cumulated vsize (KiB) 2235952

[startup+1002.3 s]
/proc/loadavg: 8.00 8.05 8.01 9/186 20697
/proc/meminfo: memFree=27325624/32951124 swapFree=67111416/67111528
[pid=20660] ppid=20658 vsize=2809628 CPUtime=4003.72 cores=1,3,5,7
/proc/20660/stat : 20660 (plingeling) S 20658 20660 19850 0 -1 4202496 4827907 1255 0 0 399685 687 0 0 25 0 5 0 522163760 2877059072 674529 33554432000 4194304 4417332 140737404153680 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/20660/statm: 702407 674529 181 55 0 700293 0
[pid=20660/tid=20667] ppid=20658 vsize=2809628 CPUtime=1002.25 cores=1,3,5,7
/proc/20660/task/20667/stat : 20667 (plingeling) R 20658 20660 19850 0 -1 4202560 1241826 1255 0 0 100056 169 0 0 25 0 5 0 522163763 2877059072 674529 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364732 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20660/tid=20668] ppid=20658 vsize=2809628 CPUtime=1002.29 cores=1,3,5,7
/proc/20660/task/20668/stat : 20668 (plingeling) R 20658 20660 19850 0 -1 4202560 1356377 1255 0 0 100036 193 0 0 25 0 5 0 522163763 2877059072 674529 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20660/tid=20669] ppid=20658 vsize=2809628 CPUtime=1002.28 cores=1,3,5,7
/proc/20660/task/20669/stat : 20669 (plingeling) R 20658 20660 19850 0 -1 4202560 1530228 1255 0 0 100017 211 0 0 25 0 5 0 522163763 2877059072 674529 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20660/tid=20670] ppid=20658 vsize=2809628 CPUtime=996.84 cores=1,3,5,7
/proc/20660/task/20670/stat : 20670 (plingeling) R 20658 20660 19850 0 -1 4202560 697927 1255 0 0 99572 112 0 0 25 0 5 0 522163763 2877059072 674529 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364531 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4003.72
Current children cumulated vsize (KiB) 2809628

[startup+1062.3 s]
/proc/loadavg: 7.71 7.94 7.97 9/186 20808
/proc/meminfo: memFree=27982924/32951124 swapFree=67111416/67111528
[pid=20660] ppid=20658 vsize=2551940 CPUtime=4243.41 cores=1,3,5,7
/proc/20660/stat : 20660 (plingeling) S 20658 20660 19850 0 -1 4202496 4981294 1255 0 0 423632 709 0 0 25 0 5 0 522163760 2613186560 610107 33554432000 4194304 4417332 140737404153680 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/20660/statm: 637985 610107 181 55 0 635871 0
[pid=20660/tid=20667] ppid=20658 vsize=2551940 CPUtime=1062.26 cores=1,3,5,7
/proc/20660/task/20667/stat : 20667 (plingeling) R 20658 20660 19850 0 -1 4202560 1313238 1255 0 0 106047 179 0 0 25 0 5 0 522163763 2613186560 610107 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20660/tid=20668] ppid=20658 vsize=2551940 CPUtime=1062.29 cores=1,3,5,7
/proc/20660/task/20668/stat : 20668 (plingeling) R 20658 20660 19850 0 -1 4202560 1406806 1255 0 0 106027 202 0 0 25 0 5 0 522163763 2613186560 610107 33554432000 4194304 4417332 140737404153680 18446744073709551615 4309503 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20660/tid=20669] ppid=20658 vsize=2551940 CPUtime=1062.28 cores=1,3,5,7
/proc/20660/task/20669/stat : 20669 (plingeling) R 20658 20660 19850 0 -1 4202560 1561774 1255 0 0 106013 215 0 0 25 0 5 0 522163763 2613186560 610107 33554432000 4194304 4417332 140737404153680 18446744073709551615 4365963 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20660/tid=20670] ppid=20658 vsize=2551940 CPUtime=1056.52 cores=1,3,5,7
/proc/20660/task/20670/stat : 20670 (plingeling) R 20658 20660 19850 0 -1 4202560 697927 1255 0 0 105540 112 0 0 25 0 5 0 522163763 2613186560 610107 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4243.41
Current children cumulated vsize (KiB) 2551940

[startup+1122.3 s]
/proc/loadavg: 7.89 7.95 7.97 9/186 20809
/proc/meminfo: memFree=27488596/32951124 swapFree=67111416/67111528
[pid=20660] ppid=20658 vsize=2915516 CPUtime=4483.08 cores=1,3,5,7
/proc/20660/stat : 20660 (plingeling) S 20658 20660 19850 0 -1 4202496 5235992 1255 0 0 447566 742 0 0 25 0 5 0 522163760 2985488384 701001 33554432000 4194304 4417332 140737404153680 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/20660/statm: 728879 701001 181 55 0 726765 0
[pid=20660/tid=20667] ppid=20658 vsize=2915516 CPUtime=1122.26 cores=1,3,5,7
/proc/20660/task/20667/stat : 20667 (plingeling) R 20658 20660 19850 0 -1 4202560 1323602 1255 0 0 112046 180 0 0 25 0 5 0 522163763 2985488384 701001 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20660/tid=20668] ppid=20658 vsize=2915516 CPUtime=1122.29 cores=1,3,5,7
/proc/20660/task/20668/stat : 20668 (plingeling) R 20658 20660 19850 0 -1 4202560 1530423 1255 0 0 112014 215 0 0 25 0 5 0 522163763 2985488384 701001 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364509 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20660/tid=20669] ppid=20658 vsize=2915516 CPUtime=1122.27 cores=1,3,5,7
/proc/20660/task/20669/stat : 20669 (plingeling) R 20658 20660 19850 0 -1 4202560 1591655 1255 0 0 112009 218 0 0 25 0 5 0 522163763 2985488384 701001 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364732 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20660/tid=20670] ppid=20658 vsize=2915516 CPUtime=1116.19 cores=1,3,5,7
/proc/20660/task/20670/stat : 20670 (plingeling) R 20658 20660 19850 0 -1 4202560 788763 1255 0 0 111492 127 0 0 25 0 5 0 522163763 2985488384 701001 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4483.08
Current children cumulated vsize (KiB) 2915516

[startup+1182.3 s]
/proc/loadavg: 8.01 7.97 7.98 9/186 20811
/proc/meminfo: memFree=27788512/32951124 swapFree=67111416/67111528
[pid=20660] ppid=20658 vsize=2598028 CPUtime=4722.76 cores=1,3,5,7
/proc/20660/stat : 20660 (plingeling) S 20658 20660 19850 0 -1 4202496 5532761 1255 0 0 471493 783 0 0 25 0 5 0 522163760 2660380672 621629 33554432000 4194304 4417332 140737404153680 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/20660/statm: 649507 621629 181 55 0 647393 0
[pid=20660/tid=20667] ppid=20658 vsize=2598028 CPUtime=1182.26 cores=1,3,5,7
/proc/20660/task/20667/stat : 20667 (plingeling) R 20658 20660 19850 0 -1 4202560 1467201 1255 0 0 118027 199 0 0 25 0 5 0 522163763 2660380672 621629 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20660/tid=20668] ppid=20658 vsize=2598028 CPUtime=1182.3 cores=1,3,5,7
/proc/20660/task/20668/stat : 20668 (plingeling) R 20658 20660 19850 0 -1 4202560 1557602 1255 0 0 118011 219 0 0 25 0 5 0 522163763 2660380672 621629 33554432000 4194304 4417332 140737404153680 18446744073709551615 4343680 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20660/tid=20669] ppid=20658 vsize=2598028 CPUtime=1182.28 cores=1,3,5,7
/proc/20660/task/20669/stat : 20669 (plingeling) R 20658 20660 19850 0 -1 4202560 1600120 1255 0 0 118009 219 0 0 25 0 5 0 522163763 2660380672 621629 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20660/tid=20670] ppid=20658 vsize=2598028 CPUtime=1175.87 cores=1,3,5,7
/proc/20660/task/20670/stat : 20670 (plingeling) R 20658 20660 19850 0 -1 4202560 906289 1255 0 0 117443 144 0 0 25 0 5 0 522163763 2660380672 621629 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4722.76
Current children cumulated vsize (KiB) 2598028



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.7 s]
/proc/loadavg: 8.01 7.97 7.98 9/186 20812
/proc/meminfo: memFree=27725996/32951124 swapFree=67111416/67111528
[pid=20660] ppid=20658 vsize=2640872 CPUtime=4800.28 cores=1,3,5,7
/proc/20660/stat : 20660 (plingeling) S 20658 20660 19850 0 -1 4202496 5552195 1255 0 0 479242 786 0 0 25 0 5 0 522163760 2704252928 632340 33554432000 4194304 4417332 140737404153680 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/20660/statm: 660218 632340 181 55 0 658104 0
[pid=20660/tid=20667] ppid=20658 vsize=2640872 CPUtime=1201.66 cores=1,3,5,7
/proc/20660/task/20667/stat : 20667 (plingeling) R 20658 20660 19850 0 -1 4202560 1467201 1255 0 0 119967 199 0 0 25 0 5 0 522163763 2704252928 632340 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20660/tid=20668] ppid=20658 vsize=2640872 CPUtime=1201.7 cores=1,3,5,7
/proc/20660/task/20668/stat : 20668 (plingeling) R 20658 20660 19850 0 -1 4202560 1570878 1255 0 0 119949 221 0 0 25 0 5 0 522163763 2704252928 632340 33554432000 4194304 4417332 140737404153680 18446744073709551615 4365963 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20660/tid=20669] ppid=20658 vsize=2640872 CPUtime=1201.69 cores=1,3,5,7
/proc/20660/task/20669/stat : 20669 (plingeling) R 20658 20660 19850 0 -1 4202560 1606278 1255 0 0 119949 220 0 0 25 0 5 0 522163763 2704252928 632340 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364732 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20660/tid=20670] ppid=20658 vsize=2640872 CPUtime=1195.17 cores=1,3,5,7
/proc/20660/task/20670/stat : 20670 (plingeling) R 20658 20660 19850 0 -1 4202560 906289 1255 0 0 119373 144 0 0 25 0 5 0 522163763 2704252928 632340 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4800.28
Current children cumulated vsize (KiB) 2640872

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.7 s]
/proc/loadavg: 8.01 7.97 7.98 9/186 20812
/proc/meminfo: memFree=27725996/32951124 swapFree=67111416/67111528
[pid=20660] ppid=20658 vsize=2640872 CPUtime=4800.28 cores=1,3,5,7
/proc/20660/stat : 20660 (plingeling) S 20658 20660 19850 0 -1 4202496 5552195 1255 0 0 479242 786 0 0 25 0 5 0 522163760 2704252928 632340 33554432000 4194304 4417332 140737404153680 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/20660/statm: 660218 632340 181 55 0 658104 0
[pid=20660/tid=20667] ppid=20658 vsize=2640872 CPUtime=1201.66 cores=1,3,5,7
/proc/20660/task/20667/stat : 20667 (plingeling) R 20658 20660 19850 0 -1 4202560 1467201 1255 0 0 119967 199 0 0 25 0 5 0 522163763 2704252928 632340 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20660/tid=20668] ppid=20658 vsize=2640872 CPUtime=1201.7 cores=1,3,5,7
/proc/20660/task/20668/stat : 20668 (plingeling) R 20658 20660 19850 0 -1 4202560 1570878 1255 0 0 119949 221 0 0 25 0 5 0 522163763 2704252928 632340 33554432000 4194304 4417332 140737404153680 18446744073709551615 4365963 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20660/tid=20669] ppid=20658 vsize=2640872 CPUtime=1201.69 cores=1,3,5,7
/proc/20660/task/20669/stat : 20669 (plingeling) R 20658 20660 19850 0 -1 4202560 1606278 1255 0 0 119949 220 0 0 25 0 5 0 522163763 2704252928 632340 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364732 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20660/tid=20670] ppid=20658 vsize=2640872 CPUtime=1195.17 cores=1,3,5,7
/proc/20660/task/20670/stat : 20670 (plingeling) R 20658 20660 19850 0 -1 4202560 906289 1255 0 0 119373 144 0 0 25 0 5 0 522163763 2704252928 632340 33554432000 4194304 4417332 140737404153680 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4800.28
Current children cumulated vsize (KiB) 2640872

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1201.8
CPU time (s): 4800.4
CPU user time (s): 4792.45
CPU system time (s): 7.95579
CPU usage (%): 399.433
Max. virtual memory (cumulated for all children) (KiB): 3182480

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4792.45
system time used= 7.95579
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 5553450
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 181
involuntary context switches= 34033

runsolver used 1.92171 second user time and 4.6223 second system time

The end

Launcher Data 

Begin job on node102 at 2011-04-29 21:33:34
IDJOB=3265114
IDBENCH=85438
IDSOLVER=1605
FILE ID=node102/3265114-1304105614
RUNJOBID= node102-1304105437-20556
PBS_JOBID= 13170630
Free space on /tmp= 71560 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT05/crafted/sabharwal05/counting/php/unsat/easier/php-045-035.shuffled-as.sat05-1181.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3265114-1304105614/watcher-3265114-1304105614 -o /tmp/evaluation-result-3265114-1304105614/solver-3265114-1304105614 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3265114-1304105614.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 057b78ff24cc0773c9c67e770a7b9de2
RANDOM SEED=992312303

node102.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      30448788 kB
Buffers:        596660 kB
Cached:        1251016 kB
SwapCached:          0 kB
Active:         294148 kB
Inactive:      1791904 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      30448788 kB
SwapTotal:    67111528 kB
SwapFree:     67111416 kB
Dirty:             440 kB
Writeback:           0 kB
AnonPages:      238268 kB
Mapped:          15072 kB
Slab:           351404 kB
PageTables:       4832 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   423788 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71548 MiB
End job on node102 at 2011-04-29 21:53:38