Trace number 3370949

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.42 1202.15

General information on the benchmark

Namecrafted/skvortsov/automata-synchronization/
crn_40_1521_s.cnf
MD5SUM4e35cb9894b1ca841d462da2d6957cc5
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables63882
Number of clauses123981
Sum of the clauses size369562
Maximum clause length3
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 22381
Number of clauses of size 3121600
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3370949-1307643561/watcher-3370949-1307643561 -o /tmp/evaluation-result-3370949-1307643561/solver-3370949-1307643561 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3370949-1307643561.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.48 1.92 2.32 2/181 4806
/proc/meminfo: memFree=32529680/32951124 swapFree=67056964/67111528
[pid=4806] ppid=4804 vsize=8728 CPUtime=0 cores=1,3,5,7
/proc/4806/stat : 4806 (plingeling) S 4804 4806 4755 0 -1 4202496 216 0 0 0 0 0 0 0 24 0 1 0 856771967 8937472 146 33554432000 4194304 4417452 140734173982688 18446744073709551615 233510297344 0 0 4096 0 18446744071562362840 0 0 17 5 0 0 0
/proc/4806/statm: 2182 146 122 55 0 68 0

[startup+0.035141 s]
/proc/loadavg: 1.48 1.92 2.32 2/181 4806
/proc/meminfo: memFree=32529680/32951124 swapFree=67056964/67111528
[pid=4806] ppid=4804 vsize=8728 CPUtime=0 cores=1,3,5,7
/proc/4806/stat : 4806 (plingeling) S 4804 4806 4755 0 -1 4202496 216 0 0 0 0 0 0 0 24 0 1 0 856771967 8937472 146 33554432000 4194304 4417452 140734173982688 18446744073709551615 233510297344 0 0 4096 0 18446744071562362840 0 0 17 5 0 0 0
/proc/4806/statm: 2182 146 122 55 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8728

[startup+0.101124 s]
/proc/loadavg: 1.48 1.92 2.32 2/181 4806
/proc/meminfo: memFree=32529680/32951124 swapFree=67056964/67111528
[pid=4806] ppid=4804 vsize=21596 CPUtime=0.04 cores=1,3,5,7
/proc/4806/stat : 4806 (plingeling) R 4804 4806 4755 0 -1 4202496 3495 1248 0 3 4 0 0 0 24 0 1 0 856771967 22114304 3423 33554432000 4194304 4417452 140734173982688 18446744073709551615 4264909 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/4806/statm: 5399 3423 149 55 0 3285 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 21596

[startup+0.301056 s]
/proc/loadavg: 1.48 1.92 2.32 2/181 4806
/proc/meminfo: memFree=32529680/32951124 swapFree=67056964/67111528
[pid=4806] ppid=4804 vsize=156212 CPUtime=0.38 cores=1,3,5,7
/proc/4806/stat : 4806 (plingeling) S 4804 4806 4755 0 -1 4202496 12620 1248 0 3 37 1 0 0 25 0 5 0 856771967 159961088 10512 33554432000 4194304 4417452 140734173982688 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/4806/statm: 39053 10512 171 55 0 36939 0
Current children cumulated CPU time (s) 0.38
Current children cumulated vsize (KiB) 156212

[startup+0.700986 s]
/proc/loadavg: 1.48 1.92 2.32 2/181 4806
/proc/meminfo: memFree=32529680/32951124 swapFree=67056964/67111528
[pid=4806] ppid=4804 vsize=158392 CPUtime=1.98 cores=1,3,5,7
/proc/4806/stat : 4806 (plingeling) S 4804 4806 4755 0 -1 4202496 27964 1248 0 3 195 3 0 0 25 0 5 0 856771967 162193408 12706 33554432000 4194304 4417452 140734173982688 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/4806/statm: 39598 12706 174 55 0 37484 0
Current children cumulated CPU time (s) 1.98
Current children cumulated vsize (KiB) 158392

[startup+1.50084 s]
/proc/loadavg: 1.48 1.92 2.32 9/186 4820
/proc/meminfo: memFree=32470464/32951124 swapFree=67056964/67111528
[pid=4806] ppid=4804 vsize=149700 CPUtime=5.17 cores=1,3,5,7
/proc/4806/stat : 4806 (plingeling) S 4804 4806 4755 0 -1 4202496 50145 1248 0 3 511 6 0 0 25 0 5 0 856771967 153292800 11933 33554432000 4194304 4417452 140734173982688 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/4806/statm: 37425 11933 174 55 0 35311 0
[pid=4806/tid=4817] ppid=4804 vsize=149700 CPUtime=1.24 cores=1,3,5,7
/proc/4806/task/4817/stat : 4817 (plingeling) R 4804 4806 4755 0 -1 4202560 12372 1248 0 3 123 1 0 0 25 0 5 0 856771992 153292800 11933 33554432000 4194304 4417452 140734173982688 18446744073709551615 4290403 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4806/tid=4818] ppid=4804 vsize=149700 CPUtime=1.24 cores=1,3,5,7
/proc/4806/task/4818/stat : 4818 (plingeling) R 4804 4806 4755 0 -1 4202560 17490 1248 0 3 122 2 0 0 25 0 5 0 856771992 153292800 11933 33554432000 4194304 4417452 140734173982688 18446744073709551615 4306643 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4806/tid=4819] ppid=4804 vsize=149700 CPUtime=1.23 cores=1,3,5,7
/proc/4806/task/4819/stat : 4819 (plingeling) R 4804 4806 4755 0 -1 4202560 10303 1248 0 3 122 1 0 0 25 0 5 0 856771992 153292800 11933 33554432000 4194304 4417452 140734173982688 18446744073709551615 4302061 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4806/tid=4820] ppid=4804 vsize=149700 CPUtime=1.24 cores=1,3,5,7
/proc/4806/task/4820/stat : 4820 (plingeling) R 4804 4806 4755 0 -1 4202560 577 1248 0 3 124 0 0 0 19 0 5 0 856771992 153292800 11933 33554432000 4194304 4417452 140734173982688 18446744073709551615 4232664 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 5.17
Current children cumulated vsize (KiB) 149700

[startup+3.10055 s]
/proc/loadavg: 1.48 1.92 2.32 9/186 4867
/proc/meminfo: memFree=32472016/32951124 swapFree=67056964/67111528
[pid=4806] ppid=4804 vsize=186636 CPUtime=11.56 cores=1,3,5,7
/proc/4806/stat : 4806 (plingeling) S 4804 4806 4755 0 -1 4202496 68021 1248 0 3 1148 8 0 0 25 0 5 0 856771967 191115264 19516 33554432000 4194304 4417452 140734173982688 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/4806/statm: 46659 19516 180 55 0 44545 0
[pid=4806/tid=4817] ppid=4804 vsize=186636 CPUtime=2.85 cores=1,3,5,7
/proc/4806/task/4817/stat : 4817 (plingeling) R 4804 4806 4755 0 -1 4202560 18808 1248 0 3 283 2 0 0 25 0 5 0 856771992 191115264 19516 33554432000 4194304 4417452 140734173982688 18446744073709551615 4252389 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4806/tid=4818] ppid=4804 vsize=186636 CPUtime=2.84 cores=1,3,5,7
/proc/4806/task/4818/stat : 4818 (plingeling) R 4804 4806 4755 0 -1 4202560 21428 1248 0 3 281 3 0 0 25 0 5 0 856771992 191115264 19516 33554432000 4194304 4417452 140734173982688 18446744073709551615 4273064 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4806/tid=4819] ppid=4804 vsize=186636 CPUtime=2.83 cores=1,3,5,7
/proc/4806/task/4819/stat : 4819 (plingeling) R 4804 4806 4755 0 -1 4202560 17805 1248 0 3 281 2 0 0 25 0 5 0 856771992 191115264 19516 33554432000 4194304 4417452 140734173982688 18446744073709551615 4277545 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4806/tid=4820] ppid=4804 vsize=186636 CPUtime=2.84 cores=1,3,5,7
/proc/4806/task/4820/stat : 4820 (plingeling) R 4804 4806 4755 0 -1 4202560 577 1248 0 3 284 0 0 0 23 0 5 0 856771992 191115264 19516 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 11.56
Current children cumulated vsize (KiB) 186636

[startup+6.30098 s]
/proc/loadavg: 2.01 2.02 2.35 9/186 4867
/proc/meminfo: memFree=32425716/32951124 swapFree=67056964/67111528
[pid=4806] ppid=4804 vsize=194828 CPUtime=24.35 cores=1,3,5,7
/proc/4806/stat : 4806 (plingeling) S 4804 4806 4755 0 -1 4202496 70361 1248 0 3 2427 8 0 0 25 0 5 0 856771967 199503872 21824 33554432000 4194304 4417452 140734173982688 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/4806/statm: 48707 21824 180 55 0 46593 0
[pid=4806/tid=4817] ppid=4804 vsize=194828 CPUtime=6.05 cores=1,3,5,7
/proc/4806/task/4817/stat : 4817 (plingeling) R 4804 4806 4755 0 -1 4202560 20091 1248 0 3 603 2 0 0 25 0 5 0 856771992 199503872 21824 33554432000 4194304 4417452 140734173982688 18446744073709551615 4257197 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4806/tid=4818] ppid=4804 vsize=194828 CPUtime=6.04 cores=1,3,5,7
/proc/4806/task/4818/stat : 4818 (plingeling) R 4804 4806 4755 0 -1 4202560 22454 1248 0 3 601 3 0 0 25 0 5 0 856771992 199503872 21824 33554432000 4194304 4417452 140734173982688 18446744073709551615 4253360 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4806/tid=4819] ppid=4804 vsize=194828 CPUtime=6.01 cores=1,3,5,7
/proc/4806/task/4819/stat : 4819 (plingeling) R 4804 4806 4755 0 -1 4202560 17836 1248 0 3 599 2 0 0 25 0 5 0 856771992 199503872 21824 33554432000 4194304 4417452 140734173982688 18446744073709551615 4265433 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4806/tid=4820] ppid=4804 vsize=194828 CPUtime=6.05 cores=1,3,5,7
/proc/4806/task/4820/stat : 4820 (plingeling) R 4804 4806 4755 0 -1 4202560 577 1248 0 3 605 0 0 0 25 0 5 0 856771992 199503872 21824 33554432000 4194304 4417452 140734173982688 18446744073709551615 4265360 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 24.35
Current children cumulated vsize (KiB) 194828

[startup+12.7008 s]
/proc/loadavg: 2.49 2.12 2.38 9/186 4867
/proc/meminfo: memFree=32413652/32951124 swapFree=67056964/67111528
[pid=4806] ppid=4804 vsize=197116 CPUtime=49.92 cores=1,3,5,7
/proc/4806/stat : 4806 (plingeling) S 4804 4806 4755 0 -1 4202496 74014 1248 0 3 4983 9 0 0 25 0 5 0 856771967 201846784 22402 33554432000 4194304 4417452 140734173982688 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/4806/statm: 49279 22402 180 55 0 47165 0
[pid=4806/tid=4817] ppid=4804 vsize=197116 CPUtime=12.45 cores=1,3,5,7
/proc/4806/task/4817/stat : 4817 (plingeling) R 4804 4806 4755 0 -1 4202560 20091 1248 0 3 1243 2 0 0 25 0 5 0 856771992 201846784 22402 33554432000 4194304 4417452 140734173982688 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4806/tid=4818] ppid=4804 vsize=197116 CPUtime=12.44 cores=1,3,5,7
/proc/4806/task/4818/stat : 4818 (plingeling) R 4804 4806 4755 0 -1 4202560 24503 1248 0 3 1241 3 0 0 25 0 5 0 856771992 201846784 22402 33554432000 4194304 4417452 140734173982688 18446744073709551615 4265465 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4806/tid=4819] ppid=4804 vsize=197116 CPUtime=12.38 cores=1,3,5,7
/proc/4806/task/4819/stat : 4819 (plingeling) R 4804 4806 4755 0 -1 4202560 18416 1248 0 3 1236 2 0 0 25 0 5 0 856771992 201846784 22402 33554432000 4194304 4417452 140734173982688 18446744073709551615 4257128 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4806/tid=4820] ppid=4804 vsize=197116 CPUtime=12.45 cores=1,3,5,7
/proc/4806/task/4820/stat : 4820 (plingeling) R 4804 4806 4755 0 -1 4202560 1601 1248 0 3 1245 0 0 0 25 0 5 0 856771992 201846784 22402 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 49.92
Current children cumulated vsize (KiB) 197116

[startup+25.5005 s]
/proc/loadavg: 3.71 2.41 2.47 9/186 4867
/proc/meminfo: memFree=32390572/32951124 swapFree=67056964/67111528
[pid=4806] ppid=4804 vsize=229236 CPUtime=101.04 cores=1,3,5,7
/proc/4806/stat : 4806 (plingeling) S 4804 4806 4755 0 -1 4202496 88866 1248 0 3 10093 11 0 0 25 0 5 0 856771967 234737664 30432 33554432000 4194304 4417452 140734173982688 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/4806/statm: 57309 30432 180 55 0 55195 0
[pid=4806/tid=4817] ppid=4804 vsize=229236 CPUtime=25.25 cores=1,3,5,7
/proc/4806/task/4817/stat : 4817 (plingeling) R 4804 4806 4755 0 -1 4202560 25371 1248 0 3 2522 3 0 0 25 0 5 0 856771992 234737664 30432 33554432000 4194304 4417452 140734173982688 18446744073709551615 4253071 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4806/tid=4818] ppid=4804 vsize=229236 CPUtime=25.23 cores=1,3,5,7
/proc/4806/task/4818/stat : 4818 (plingeling) R 4804 4806 4755 0 -1 4202560 27168 1248 0 3 2520 3 0 0 25 0 5 0 856771992 234737664 30432 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364604 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4806/tid=4819] ppid=4804 vsize=229236 CPUtime=25.1 cores=1,3,5,7
/proc/4806/task/4819/stat : 4819 (plingeling) R 4804 4806 4755 0 -1 4202560 23057 1248 0 3 2508 2 0 0 25 0 5 0 856771992 234737664 30432 33554432000 4194304 4417452 140734173982688 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4806/tid=4820] ppid=4804 vsize=229236 CPUtime=25.24 cores=1,3,5,7
/proc/4806/task/4820/stat : 4820 (plingeling) R 4804 4806 4755 0 -1 4202560 3867 1248 0 3 2524 0 0 0 25 0 5 0 856771992 234737664 30432 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364607 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 101.04
Current children cumulated vsize (KiB) 229236

[startup+51.1009 s]
/proc/loadavg: 5.24 2.87 2.62 9/186 4868
/proc/meminfo: memFree=32365808/32951124 swapFree=67056964/67111528
[pid=4806] ppid=4804 vsize=245496 CPUtime=203.31 cores=1,3,5,7
/proc/4806/stat : 4806 (plingeling) S 4804 4806 4755 0 -1 4202496 118364 1248 0 3 20316 15 0 0 25 0 5 0 856771967 251387904 34497 33554432000 4194304 4417452 140734173982688 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/4806/statm: 61374 34497 180 55 0 59260 0
[pid=4806/tid=4817] ppid=4804 vsize=245496 CPUtime=50.85 cores=1,3,5,7
/proc/4806/task/4817/stat : 4817 (plingeling) R 4804 4806 4755 0 -1 4202560 32844 1248 0 3 5081 4 0 0 25 0 5 0 856771992 251387904 34497 33554432000 4194304 4417452 140734173982688 18446744073709551615 4264940 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4806/tid=4818] ppid=4804 vsize=245496 CPUtime=50.84 cores=1,3,5,7
/proc/4806/task/4818/stat : 4818 (plingeling) R 4804 4806 4755 0 -1 4202560 36950 1248 0 3 5079 5 0 0 25 0 5 0 856771992 251387904 34497 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364711 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4806/tid=4819] ppid=4804 vsize=245496 CPUtime=50.56 cores=1,3,5,7
/proc/4806/task/4819/stat : 4819 (plingeling) R 4804 4806 4755 0 -1 4202560 29302 1248 0 3 5053 3 0 0 25 0 5 0 856771992 251387904 34497 33554432000 4194304 4417452 140734173982688 18446744073709551615 4265429 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4806/tid=4820] ppid=4804 vsize=245496 CPUtime=50.84 cores=1,3,5,7
/proc/4806/task/4820/stat : 4820 (plingeling) R 4804 4806 4755 0 -1 4202560 9865 1248 0 3 5084 0 0 0 25 0 5 0 856771992 251387904 34497 33554432000 4194304 4417452 140734173982688 18446744073709551615 4353680 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 203.31
Current children cumulated vsize (KiB) 245496

[startup+102.307 s]
/proc/loadavg: 6.96 3.70 2.92 9/186 4964
/proc/meminfo: memFree=32233424/32951124 swapFree=67056964/67111528
[pid=4806] ppid=4804 vsize=293036 CPUtime=407.85 cores=1,3,5,7
/proc/4806/stat : 4806 (plingeling) S 4804 4806 4755 0 -1 4202496 182079 1248 0 3 40763 22 0 0 25 0 5 0 856771967 300068864 46382 33554432000 4194304 4417452 140734173982688 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/4806/statm: 73259 46382 180 55 0 71145 0
[pid=4806/tid=4817] ppid=4804 vsize=293036 CPUtime=102.06 cores=1,3,5,7
/proc/4806/task/4817/stat : 4817 (plingeling) R 4804 4806 4755 0 -1 4202560 50180 1248 0 3 10200 6 0 0 25 0 5 0 856771992 300068864 46382 33554432000 4194304 4417452 140734173982688 18446744073709551615 4251909 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4806/tid=4818] ppid=4804 vsize=293036 CPUtime=102.05 cores=1,3,5,7
/proc/4806/task/4818/stat : 4818 (plingeling) R 4804 4806 4755 0 -1 4202560 59212 1248 0 3 10198 7 0 0 25 0 5 0 856771992 300068864 46382 33554432000 4194304 4417452 140734173982688 18446744073709551615 4366099 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4806/tid=4819] ppid=4804 vsize=293036 CPUtime=101.49 cores=1,3,5,7
/proc/4806/task/4819/stat : 4819 (plingeling) R 4804 4806 4755 0 -1 4202560 44119 1248 0 3 10144 5 0 0 25 0 5 0 856771992 300068864 46382 33554432000 4194304 4417452 140734173982688 18446744073709551615 4357207 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4806/tid=4820] ppid=4804 vsize=293036 CPUtime=102.05 cores=1,3,5,7
/proc/4806/task/4820/stat : 4820 (plingeling) R 4804 4806 4755 0 -1 4202560 19165 1248 0 3 10203 2 0 0 25 0 5 0 856771992 300068864 46382 33554432000 4194304 4417452 140734173982688 18446744073709551615 4356980 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 407.85
Current children cumulated vsize (KiB) 293036

[startup+162.301 s]
/proc/loadavg: 7.68 4.50 3.24 9/186 4965
/proc/meminfo: memFree=32166676/32951124 swapFree=67056964/67111528
[pid=4806] ppid=4804 vsize=342740 CPUtime=647.52 cores=1,3,5,7
/proc/4806/stat : 4806 (plingeling) S 4804 4806 4755 0 -1 4202496 270907 1248 0 3 64718 34 0 0 25 0 5 0 856771967 350965760 58250 33554432000 4194304 4417452 140734173982688 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/4806/statm: 85685 58250 180 55 0 83571 0
[pid=4806/tid=4817] ppid=4804 vsize=342740 CPUtime=162.05 cores=1,3,5,7
/proc/4806/task/4817/stat : 4817 (plingeling) R 4804 4806 4755 0 -1 4202560 60509 1248 0 3 16197 8 0 0 25 0 5 0 856771992 350965760 58250 33554432000 4194304 4417452 140734173982688 18446744073709551615 4353474 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4806/tid=4818] ppid=4804 vsize=342740 CPUtime=162.04 cores=1,3,5,7
/proc/4806/task/4818/stat : 4818 (plingeling) R 4804 4806 4755 0 -1 4202560 84917 1248 0 3 16194 10 0 0 25 0 5 0 856771992 350965760 58250 33554432000 4194304 4417452 140734173982688 18446744073709551615 4257626 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4806/tid=4819] ppid=4804 vsize=342740 CPUtime=161.16 cores=1,3,5,7
/proc/4806/task/4819/stat : 4819 (plingeling) R 4804 4806 4755 0 -1 4202560 72454 1248 0 3 16108 8 0 0 25 0 5 0 856771992 350965760 58250 33554432000 4194304 4417452 140734173982688 18446744073709551615 4252539 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4806/tid=4820] ppid=4804 vsize=342740 CPUtime=162.05 cores=1,3,5,7
/proc/4806/task/4820/stat : 4820 (plingeling) R 4804 4806 4755 0 -1 4202560 43624 1248 0 3 16200 5 0 0 25 0 5 0 856771992 350965760 58250 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 647.52
Current children cumulated vsize (KiB) 342740


################
# More data... #
################

[pid=4806/tid=4818] ppid=4804 vsize=1064780 CPUtime=759.7 cores=1,3,5,7
/proc/4806/task/4818/stat : 4818 (plingeling) R 4804 4806 4755 0 -1 4202560 568021 1248 0 3 75890 80 0 0 25 0 5 0 856771992 1090334720 239318 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4806/tid=4819] ppid=4804 vsize=1064780 CPUtime=760.26 cores=1,3,5,7
/proc/4806/task/4819/stat : 4819 (plingeling) R 4804 4806 4755 0 -1 4202560 487139 1248 0 3 75953 73 0 0 25 0 5 0 856771992 1090334720 239318 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4806/tid=4820] ppid=4804 vsize=1064780 CPUtime=762.06 cores=1,3,5,7
/proc/4806/task/4820/stat : 4820 (plingeling) R 4804 4806 4755 0 -1 4202560 157736 1248 0 3 76184 22 0 0 25 0 5 0 856771992 1090334720 239318 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3043.5
Current children cumulated vsize (KiB) 1064780

[startup+822.301 s]
/proc/loadavg: 8.14 7.72 5.71 9/186 4986
/proc/meminfo: memFree=31240196/32951124 swapFree=67056964/67111528
[pid=4806] ppid=4804 vsize=1160944 CPUtime=3283.16 cores=1,3,5,7
/proc/4806/stat : 4806 (plingeling) S 4804 4806 4755 0 -1 4202496 1879207 1248 0 3 328045 271 0 0 25 0 5 0 856771967 1188806656 261165 33554432000 4194304 4417452 140734173982688 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/4806/statm: 290236 261165 180 55 0 288122 0
[pid=4806/tid=4817] ppid=4804 vsize=1160944 CPUtime=821.25 cores=1,3,5,7
/proc/4806/task/4817/stat : 4817 (plingeling) R 4804 4806 4755 0 -1 4202560 551326 1248 0 3 82047 78 0 0 25 0 5 0 856771992 1188806656 261165 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364737 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4806/tid=4818] ppid=4804 vsize=1160944 CPUtime=819.38 cores=1,3,5,7
/proc/4806/task/4818/stat : 4818 (plingeling) R 4804 4806 4755 0 -1 4202560 620022 1248 0 3 81849 89 0 0 25 0 5 0 856771992 1188806656 261165 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4806/tid=4819] ppid=4804 vsize=1160944 CPUtime=820.26 cores=1,3,5,7
/proc/4806/task/4819/stat : 4819 (plingeling) R 4804 4806 4755 0 -1 4202560 540720 1248 0 3 81946 80 0 0 25 0 5 0 856771992 1188806656 261165 33554432000 4194304 4417452 140734173982688 18446744073709551615 4358110 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4806/tid=4820] ppid=4804 vsize=1160944 CPUtime=822.06 cores=1,3,5,7
/proc/4806/task/4820/stat : 4820 (plingeling) R 4804 4806 4755 0 -1 4202560 157736 1248 0 3 82184 22 0 0 25 0 5 0 856771992 1188806656 261165 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3283.16
Current children cumulated vsize (KiB) 1160944

[startup+882.301 s]
/proc/loadavg: 8.10 7.78 5.85 9/186 4987
/proc/meminfo: memFree=31052276/32951124 swapFree=67056964/67111528
[pid=4806] ppid=4804 vsize=1338908 CPUtime=3522.83 cores=1,3,5,7
/proc/4806/stat : 4806 (plingeling) S 4804 4806 4755 0 -1 4202496 1991179 1248 0 3 351994 289 0 0 25 0 5 0 856771967 1371041792 305656 33554432000 4194304 4417452 140734173982688 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/4806/statm: 334727 305656 180 55 0 332613 0
[pid=4806/tid=4817] ppid=4804 vsize=1338908 CPUtime=881.24 cores=1,3,5,7
/proc/4806/task/4817/stat : 4817 (plingeling) R 4804 4806 4755 0 -1 4202560 594069 1248 0 3 88040 84 0 0 25 0 5 0 856771992 1371041792 305656 33554432000 4194304 4417452 140734173982688 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4806/tid=4818] ppid=4804 vsize=1338908 CPUtime=879.05 cores=1,3,5,7
/proc/4806/task/4818/stat : 4818 (plingeling) R 4804 4806 4755 0 -1 4202560 649165 1248 0 3 87813 92 0 0 25 0 5 0 856771992 1371041792 305656 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4806/tid=4819] ppid=4804 vsize=1338908 CPUtime=880.26 cores=1,3,5,7
/proc/4806/task/4819/stat : 4819 (plingeling) R 4804 4806 4755 0 -1 4202560 573518 1248 0 3 87939 87 0 0 25 0 5 0 856771992 1371041792 305656 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4806/tid=4820] ppid=4804 vsize=1338908 CPUtime=882.07 cores=1,3,5,7
/proc/4806/task/4820/stat : 4820 (plingeling) R 4804 4806 4755 0 -1 4202560 165024 1248 0 3 88183 24 0 0 25 0 5 0 856771992 1371041792 305656 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364616 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3522.83
Current children cumulated vsize (KiB) 1338908

[startup+942.301 s]
/proc/loadavg: 8.15 7.85 5.99 9/186 4989
/proc/meminfo: memFree=31165804/32951124 swapFree=67056964/67111528
[pid=4806] ppid=4804 vsize=1202212 CPUtime=3762.52 cores=1,3,5,7
/proc/4806/stat : 4806 (plingeling) S 4804 4806 4755 0 -1 4202496 2163500 1248 0 3 375936 316 0 0 25 0 5 0 856771967 1231065088 273676 33554432000 4194304 4417452 140734173982688 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/4806/statm: 300553 273676 180 55 0 298439 0
[pid=4806/tid=4817] ppid=4804 vsize=1202212 CPUtime=941.25 cores=1,3,5,7
/proc/4806/task/4817/stat : 4817 (plingeling) R 4804 4806 4755 0 -1 4202560 641583 1248 0 3 94035 90 0 0 25 0 5 0 856771992 1231065088 273676 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4806/tid=4818] ppid=4804 vsize=1202212 CPUtime=938.73 cores=1,3,5,7
/proc/4806/task/4818/stat : 4818 (plingeling) R 4804 4806 4755 0 -1 4202560 693473 1248 0 3 93773 100 0 0 25 0 5 0 856771992 1231065088 273676 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364629 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4806/tid=4819] ppid=4804 vsize=1202212 CPUtime=940.26 cores=1,3,5,7
/proc/4806/task/4819/stat : 4819 (plingeling) R 4804 4806 4755 0 -1 4202560 638580 1248 0 3 93929 97 0 0 25 0 5 0 856771992 1231065088 273676 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4806/tid=4820] ppid=4804 vsize=1202212 CPUtime=942.06 cores=1,3,5,7
/proc/4806/task/4820/stat : 4820 (plingeling) R 4804 4806 4755 0 -1 4202560 180461 1248 0 3 94180 26 0 0 25 0 5 0 856771992 1231065088 273676 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3762.52
Current children cumulated vsize (KiB) 1202212

[startup+1002.3 s]
/proc/loadavg: 8.13 7.89 6.12 9/186 4990
/proc/meminfo: memFree=31147388/32951124 swapFree=67056964/67111528
[pid=4806] ppid=4804 vsize=1208272 CPUtime=4002.2 cores=1,3,5,7
/proc/4806/stat : 4806 (plingeling) S 4804 4806 4755 0 -1 4202496 2291708 1248 0 3 399886 334 0 0 25 0 5 0 856771967 1237270528 275191 33554432000 4194304 4417452 140734173982688 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/4806/statm: 302068 275191 180 55 0 299954 0
[pid=4806/tid=4817] ppid=4804 vsize=1208272 CPUtime=1001.25 cores=1,3,5,7
/proc/4806/task/4817/stat : 4817 (plingeling) R 4804 4806 4755 0 -1 4202560 681483 1248 0 3 100028 97 0 0 25 0 5 0 856771992 1237270528 275191 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364765 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4806/tid=4818] ppid=4804 vsize=1208272 CPUtime=998.4 cores=1,3,5,7
/proc/4806/task/4818/stat : 4818 (plingeling) R 4804 4806 4755 0 -1 4202560 739058 1248 0 3 99735 105 0 0 25 0 5 0 856771992 1237270528 275191 33554432000 4194304 4417452 140734173982688 18446744073709551615 4250961 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4806/tid=4819] ppid=4804 vsize=1208272 CPUtime=1000.26 cores=1,3,5,7
/proc/4806/task/4819/stat : 4819 (plingeling) R 4804 4806 4755 0 -1 4202560 671348 1248 0 3 99926 100 0 0 25 0 5 0 856771992 1237270528 275191 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4806/tid=4820] ppid=4804 vsize=1208272 CPUtime=1002.06 cores=1,3,5,7
/proc/4806/task/4820/stat : 4820 (plingeling) R 4804 4806 4755 0 -1 4202560 190416 1248 0 3 100178 28 0 0 25 0 5 0 856771992 1237270528 275191 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4002.2
Current children cumulated vsize (KiB) 1208272

[startup+1062.3 s]
/proc/loadavg: 8.16 7.94 6.25 9/186 4992
/proc/meminfo: memFree=31178580/32951124 swapFree=67056964/67111528
[pid=4806] ppid=4804 vsize=1256696 CPUtime=4241.88 cores=1,3,5,7
/proc/4806/stat : 4806 (plingeling) S 4804 4806 4755 0 -1 4202496 2402318 1248 0 3 423840 348 0 0 25 0 5 0 856771967 1286856704 284038 33554432000 4194304 4417452 140734173982688 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/4806/statm: 314174 284066 180 55 0 312060 0
[pid=4806/tid=4817] ppid=4804 vsize=1256696 CPUtime=1061.25 cores=1,3,5,7
/proc/4806/task/4817/stat : 4817 (plingeling) R 4804 4806 4755 0 -1 4202560 727252 1248 0 3 106022 103 0 0 25 0 5 0 856771992 1286856704 284078 33554432000 4194304 4417452 140734173982688 18446744073709551615 233509991517 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4806/tid=4818] ppid=4804 vsize=1256696 CPUtime=1058.08 cores=1,3,5,7
/proc/4806/task/4818/stat : 4818 (plingeling) R 4804 4806 4755 0 -1 4202560 803924 1248 0 3 105694 114 0 0 25 0 5 0 856771992 1286856704 284094 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4806/tid=4819] ppid=4804 vsize=1256696 CPUtime=1060.26 cores=1,3,5,7
/proc/4806/task/4819/stat : 4819 (plingeling) R 4804 4806 4755 0 -1 4202560 671364 1248 0 3 105926 100 0 0 25 0 5 0 856771992 1286856704 284110 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4806/tid=4820] ppid=4804 vsize=1256696 CPUtime=1062.06 cores=1,3,5,7
/proc/4806/task/4820/stat : 4820 (plingeling) R 4804 4806 4755 0 -1 4202560 190416 1248 0 3 106178 28 0 0 25 0 5 0 856771992 1286856704 284125 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4241.88
Current children cumulated vsize (KiB) 1256696

[startup+1122.3 s]
/proc/loadavg: 8.16 7.98 6.36 9/186 4994
/proc/meminfo: memFree=30903740/32951124 swapFree=67056964/67111528
[pid=4806] ppid=4804 vsize=1370084 CPUtime=4481.56 cores=1,3,5,7
/proc/4806/stat : 4806 (plingeling) S 4804 4806 4755 0 -1 4202496 2545710 1248 0 3 447789 367 0 0 25 0 5 0 856771967 1402966016 315644 33554432000 4194304 4417452 140734173982688 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/4806/statm: 342521 315644 180 55 0 340407 0
[pid=4806/tid=4817] ppid=4804 vsize=1370084 CPUtime=1121.26 cores=1,3,5,7
/proc/4806/task/4817/stat : 4817 (plingeling) R 4804 4806 4755 0 -1 4202560 769222 1248 0 3 112016 110 0 0 25 0 5 0 856771992 1402966016 315644 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364838 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4806/tid=4818] ppid=4804 vsize=1370084 CPUtime=1117.75 cores=1,3,5,7
/proc/4806/task/4818/stat : 4818 (plingeling) R 4804 4806 4755 0 -1 4202560 868525 1248 0 3 111653 122 0 0 25 0 5 0 856771992 1402966016 315644 33554432000 4194304 4417452 140734173982688 18446744073709551615 4265437 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4806/tid=4819] ppid=4804 vsize=1370084 CPUtime=1120.26 cores=1,3,5,7
/proc/4806/task/4819/stat : 4819 (plingeling) R 4804 4806 4755 0 -1 4202560 704132 1248 0 3 111923 103 0 0 25 0 5 0 856771992 1402966016 315644 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4806/tid=4820] ppid=4804 vsize=1370084 CPUtime=1122.07 cores=1,3,5,7
/proc/4806/task/4820/stat : 4820 (plingeling) R 4804 4806 4755 0 -1 4202560 194428 1248 0 3 112178 29 0 0 25 0 5 0 856771992 1402966016 315644 33554432000 4194304 4417452 140734173982688 18446744073709551615 4366385 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4481.56
Current children cumulated vsize (KiB) 1370084

[startup+1182.3 s]
/proc/loadavg: 8.15 8.01 6.47 9/186 4995
/proc/meminfo: memFree=31268616/32951124 swapFree=67056964/67111528
[pid=4806] ppid=4804 vsize=1086108 CPUtime=4721.24 cores=1,3,5,7
/proc/4806/stat : 4806 (plingeling) S 4804 4806 4755 0 -1 4202496 2693324 1248 0 3 471734 390 0 0 25 0 5 0 856771967 1112174592 244650 33554432000 4194304 4417452 140734173982688 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/4806/statm: 271527 244650 180 55 0 269413 0
[pid=4806/tid=4817] ppid=4804 vsize=1086108 CPUtime=1181.25 cores=1,3,5,7
/proc/4806/task/4817/stat : 4817 (plingeling) R 4804 4806 4755 0 -1 4202560 808515 1248 0 3 118010 115 0 0 25 0 5 0 856771992 1112174592 244650 33554432000 4194304 4417452 140734173982688 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4806/tid=4818] ppid=4804 vsize=1086108 CPUtime=1177.42 cores=1,3,5,7
/proc/4806/task/4818/stat : 4818 (plingeling) R 4804 4806 4755 0 -1 4202560 893810 1248 0 3 117617 125 0 0 25 0 5 0 856771992 1112174592 244650 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4806/tid=4819] ppid=4804 vsize=1086108 CPUtime=1180.26 cores=1,3,5,7
/proc/4806/task/4819/stat : 4819 (plingeling) R 4804 4806 4755 0 -1 4202560 787168 1248 0 3 117909 117 0 0 25 0 5 0 856771992 1112174592 244650 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4806/tid=4820] ppid=4804 vsize=1086108 CPUtime=1182.07 cores=1,3,5,7
/proc/4806/task/4820/stat : 4820 (plingeling) R 4804 4806 4755 0 -1 4202560 194428 1248 0 3 118178 29 0 0 25 0 5 0 856771992 1112174592 244650 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4721.24
Current children cumulated vsize (KiB) 1086108



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1202.1 s]
/proc/loadavg: 8.18 8.02 6.51 9/186 4996
/proc/meminfo: memFree=31253696/32951124 swapFree=67056964/67111528
[pid=4806] ppid=4804 vsize=1119412 CPUtime=4800.35 cores=1,3,5,7
/proc/4806/stat : 4806 (plingeling) S 4804 4806 4755 0 -1 4202496 2743673 1248 0 3 479637 398 0 0 25 0 5 0 856771967 1146277888 252976 33554432000 4194304 4417452 140734173982688 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/4806/statm: 279853 252976 180 55 0 277739 0
[pid=4806/tid=4817] ppid=4804 vsize=1119412 CPUtime=1201.06 cores=1,3,5,7
/proc/4806/task/4817/stat : 4817 (plingeling) R 4804 4806 4755 0 -1 4202560 811655 1248 0 3 119991 115 0 0 25 0 5 0 856771992 1146277888 252976 33554432000 4194304 4417452 140734173982688 18446744073709551615 4319442 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4806/tid=4818] ppid=4804 vsize=1119412 CPUtime=1197.12 cores=1,3,5,7
/proc/4806/task/4818/stat : 4818 (plingeling) R 4804 4806 4755 0 -1 4202560 911747 1248 0 3 119583 129 0 0 25 0 5 0 856771992 1146277888 252976 33554432000 4194304 4417452 140734173982688 18446744073709551615 4259603 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4806/tid=4819] ppid=4804 vsize=1119412 CPUtime=1200.07 cores=1,3,5,7
/proc/4806/task/4819/stat : 4819 (plingeling) R 4804 4806 4755 0 -1 4202560 816440 1248 0 3 119885 122 0 0 25 0 5 0 856771992 1146277888 252976 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364741 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4806/tid=4820] ppid=4804 vsize=1119412 CPUtime=1201.88 cores=1,3,5,7
/proc/4806/task/4820/stat : 4820 (plingeling) R 4804 4806 4755 0 -1 4202560 194428 1248 0 3 120159 29 0 0 25 0 5 0 856771992 1146277888 252976 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364780 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4800.35
Current children cumulated vsize (KiB) 1119412

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1202.1 s]
/proc/loadavg: 8.18 8.02 6.51 9/186 4996
/proc/meminfo: memFree=31253696/32951124 swapFree=67056964/67111528
[pid=4806] ppid=4804 vsize=1119412 CPUtime=4800.35 cores=1,3,5,7
/proc/4806/stat : 4806 (plingeling) S 4804 4806 4755 0 -1 4202496 2743673 1248 0 3 479637 398 0 0 25 0 5 0 856771967 1146277888 252976 33554432000 4194304 4417452 140734173982688 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/4806/statm: 279853 252976 180 55 0 277739 0
[pid=4806/tid=4817] ppid=4804 vsize=1119412 CPUtime=1201.06 cores=1,3,5,7
/proc/4806/task/4817/stat : 4817 (plingeling) R 4804 4806 4755 0 -1 4202560 811655 1248 0 3 119991 115 0 0 25 0 5 0 856771992 1146277888 252976 33554432000 4194304 4417452 140734173982688 18446744073709551615 4319442 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4806/tid=4818] ppid=4804 vsize=1119412 CPUtime=1197.12 cores=1,3,5,7
/proc/4806/task/4818/stat : 4818 (plingeling) R 4804 4806 4755 0 -1 4202560 911747 1248 0 3 119583 129 0 0 25 0 5 0 856771992 1146277888 252976 33554432000 4194304 4417452 140734173982688 18446744073709551615 4259603 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4806/tid=4819] ppid=4804 vsize=1119412 CPUtime=1200.07 cores=1,3,5,7
/proc/4806/task/4819/stat : 4819 (plingeling) R 4804 4806 4755 0 -1 4202560 816440 1248 0 3 119885 122 0 0 25 0 5 0 856771992 1146277888 252976 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364741 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4806/tid=4820] ppid=4804 vsize=1119412 CPUtime=1201.88 cores=1,3,5,7
/proc/4806/task/4820/stat : 4820 (plingeling) R 4804 4806 4755 0 -1 4202560 194428 1248 0 3 120159 29 0 0 25 0 5 0 856771992 1146277888 252976 33554432000 4194304 4417452 140734173982688 18446744073709551615 4364780 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4800.35
Current children cumulated vsize (KiB) 1119412

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1202.15
CPU time (s): 4800.42
CPU user time (s): 4796.39
CPU system time (s): 4.02939
CPU usage (%): 399.319
Max. virtual memory (cumulated for all children) (KiB): 1456660

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4796.39
system time used= 4.02939
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2744921
page faults= 3
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 115
involuntary context switches= 32820

runsolver used 1.91471 second user time and 4.65529 second system time

The end

Launcher Data 

Begin job on node145 at 2011-06-09 20:19:21
IDJOB=3370949
IDBENCH=83191
IDSOLVER=1848
FILE ID=node145/3370949-1307643561
RUNJOBID= node145-1307643561-4774
PBS_JOBID= 13549405
Free space on /tmp= 74196 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/crafted/skvortsov/automata-synchronization/crn_40_1521_s.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3370949-1307643561/watcher-3370949-1307643561 -o /tmp/evaluation-result-3370949-1307643561/solver-3370949-1307643561 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3370949-1307643561.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 4e35cb9894b1ca841d462da2d6957cc5
RANDOM SEED=1500989152

node145.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.57
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      32531388 kB
Buffers:         47252 kB
Cached:         211788 kB
SwapCached:       7944 kB
Active:          92708 kB
Inactive:       185660 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      32531388 kB
SwapTotal:    67111528 kB
SwapFree:     67056964 kB
Dirty:            3192 kB
Writeback:           0 kB
AnonPages:       17984 kB
Mapped:          12444 kB
Slab:            74404 kB
PageTables:       5004 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   185504 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 74196 MiB
End job on node145 at 2011-06-09 20:39:25