Trace number 3249712

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 4800.34 1201.97

General information on the benchmark

Namecrafted/mosoi/289/
289-unsat-14x34.cnf
MD5SUM72677fc46303044d3d9e832fa24a37b2
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables1904
Number of clauses207536
Sum of the clauses size824432
Maximum clause length4
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 22856
Number of clauses of size 30
Number of clauses of size 4204680
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3249712-1304087470/watcher-3249712-1304087470 -o /tmp/evaluation-result-3249712-1304087470/solver-3249712-1304087470 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3249712-1304087470.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.46 7.98 8.01 1/175 16522
/proc/meminfo: memFree=30709800/32950928 swapFree=67111428/67111528
[pid=16522] ppid=16520 vsize=9628 CPUtime=0 cores=0,2,4,6
/proc/16522/stat : 16522 (plingeling) R 16520 16522 15826 0 -1 4202496 483 1251 0 0 0 0 0 0 22 0 1 0 520346862 9859072 412 33554432000 4194304 4417332 140734601858192 18446744073709551615 248089392089 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/16522/statm: 2407 412 146 55 0 293 0

[startup+0.0834331 s]
/proc/loadavg: 7.46 7.98 8.01 1/175 16522
/proc/meminfo: memFree=30709800/32950928 swapFree=67111428/67111528
[pid=16522] ppid=16520 vsize=25932 CPUtime=0.07 cores=0,2,4,6
/proc/16522/stat : 16522 (plingeling) R 16520 16522 15826 0 -1 4202496 4700 1251 0 0 7 0 0 0 22 0 1 0 520346862 26554368 4505 33554432000 4194304 4417332 140734601858192 18446744073709551615 4235973 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/16522/statm: 6483 4505 147 55 0 4369 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 25932

[startup+0.100427 s]
/proc/loadavg: 7.46 7.98 8.01 1/175 16522
/proc/meminfo: memFree=30709800/32950928 swapFree=67111428/67111528
[pid=16522] ppid=16520 vsize=25932 CPUtime=0.09 cores=0,2,4,6
/proc/16522/stat : 16522 (plingeling) R 16520 16522 15826 0 -1 4202496 4700 1251 0 0 9 0 0 0 22 0 1 0 520346862 26554368 4505 33554432000 4194304 4417332 140734601858192 18446744073709551615 248089316950 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/16522/statm: 6483 4505 147 55 0 4369 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 25932

[startup+0.300395 s]
/proc/loadavg: 7.46 7.98 8.01 1/175 16522
/proc/meminfo: memFree=30709800/32950928 swapFree=67111428/67111528
[pid=16522] ppid=16520 vsize=99620 CPUtime=0.4 cores=0,2,4,6
/proc/16522/stat : 16522 (plingeling) S 16520 16522 15826 0 -1 4202496 12981 1251 0 0 39 1 0 0 23 0 5 0 520346862 102010880 12682 33554432000 4194304 4417332 140734601858192 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16522/statm: 24905 12682 170 55 0 22791 0
Current children cumulated CPU time (s) 0.4
Current children cumulated vsize (KiB) 99620

[startup+0.700327 s]
/proc/loadavg: 7.46 7.98 8.01 1/175 16522
/proc/meminfo: memFree=30709800/32950928 swapFree=67111428/67111528
[pid=16522] ppid=16520 vsize=118936 CPUtime=1.99 cores=0,2,4,6
/proc/16522/stat : 16522 (plingeling) S 16520 16522 15826 0 -1 4202496 18890 1251 0 0 197 2 0 0 23 0 5 0 520346862 121790464 17551 33554432000 4194304 4417332 140734601858192 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16522/statm: 29734 17551 178 55 0 27620 0
Current children cumulated CPU time (s) 1.99
Current children cumulated vsize (KiB) 118936

[startup+1.50119 s]
/proc/loadavg: 7.46 7.98 8.01 6/177 16589
/proc/meminfo: memFree=30618632/32950928 swapFree=67111428/67111528
[pid=16522] ppid=16520 vsize=142152 CPUtime=5.19 cores=0,2,4,6
/proc/16522/stat : 16522 (plingeling) S 16520 16522 15826 0 -1 4202496 25928 1251 0 0 516 3 0 0 23 0 5 0 520346862 145563648 23279 33554432000 4194304 4417332 140734601858192 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16522/statm: 35538 23302 179 55 0 33424 0
[pid=16522/tid=16565] ppid=16520 vsize=142152 CPUtime=1.23 cores=0,2,4,6
/proc/16522/task/16565/stat : 16565 (plingeling) R 16520 16522 15826 0 -1 4202560 3263 1251 0 0 123 0 0 0 25 0 5 0 520346888 145563648 23314 33554432000 4194304 4417332 140734601858192 18446744073709551615 4239393 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=16522/tid=16566] ppid=16520 vsize=142152 CPUtime=1.22 cores=0,2,4,6
/proc/16522/task/16566/stat : 16566 (plingeling) R 16520 16522 15826 0 -1 4202560 3274 1251 0 0 122 0 0 0 25 0 5 0 520346888 145563648 23330 33554432000 4194304 4417332 140734601858192 18446744073709551615 4239605 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16522/tid=16567] ppid=16520 vsize=142152 CPUtime=1.22 cores=0,2,4,6
/proc/16522/task/16567/stat : 16567 (plingeling) R 16520 16522 15826 0 -1 4202560 3026 1251 0 0 122 0 0 0 25 0 5 0 520346888 145563648 23344 33554432000 4194304 4417332 140734601858192 18446744073709551615 4239418 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16522/tid=16568] ppid=16520 vsize=142152 CPUtime=1.23 cores=0,2,4,6
/proc/16522/task/16568/stat : 16568 (plingeling) R 16520 16522 15826 0 -1 4202560 3538 1251 0 0 123 0 0 0 25 0 5 0 520346888 145563648 23356 33554432000 4194304 4417332 140734601858192 18446744073709551615 248089392187 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 5.19
Current children cumulated vsize (KiB) 142152

[startup+3.1009 s]
/proc/loadavg: 7.26 7.93 8.00 9/186 16604
/proc/meminfo: memFree=30463188/32950928 swapFree=67111428/67111528
[pid=16522] ppid=16520 vsize=161348 CPUtime=11.59 cores=0,2,4,6
/proc/16522/stat : 16522 (plingeling) S 16520 16522 15826 0 -1 4202496 33421 1251 0 0 1154 5 0 0 23 0 5 0 520346862 165220352 28123 33554432000 4194304 4417332 140734601858192 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16522/statm: 40337 28123 179 55 0 38223 0
[pid=16522/tid=16565] ppid=16520 vsize=161348 CPUtime=2.82 cores=0,2,4,6
/proc/16522/task/16565/stat : 16565 (plingeling) R 16520 16522 15826 0 -1 4202560 5262 1251 0 0 282 0 0 0 25 0 5 0 520346888 165220352 28123 33554432000 4194304 4417332 140734601858192 18446744073709551615 4249181 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=16522/tid=16566] ppid=16520 vsize=161348 CPUtime=2.83 cores=0,2,4,6
/proc/16522/task/16566/stat : 16566 (plingeling) R 16520 16522 15826 0 -1 4202560 5273 1251 0 0 282 1 0 0 25 0 5 0 520346888 165220352 28123 33554432000 4194304 4417332 140734601858192 18446744073709551615 4236224 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16522/tid=16567] ppid=16520 vsize=161348 CPUtime=2.81 cores=0,2,4,6
/proc/16522/task/16567/stat : 16567 (plingeling) R 16520 16522 15826 0 -1 4202560 3026 1251 0 0 281 0 0 0 25 0 5 0 520346888 165220352 28123 33554432000 4194304 4417332 140734601858192 18446744073709551615 4249171 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16522/tid=16568] ppid=16520 vsize=161348 CPUtime=2.82 cores=0,2,4,6
/proc/16522/task/16568/stat : 16568 (plingeling) R 16520 16522 15826 0 -1 4202560 6956 1251 0 0 282 0 0 0 25 0 5 0 520346888 165220352 28123 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 11.59
Current children cumulated vsize (KiB) 161348

[startup+6.30033 s]
/proc/loadavg: 7.26 7.93 8.00 10/186 16604
/proc/meminfo: memFree=30336128/32950928 swapFree=67111428/67111528
[pid=16522] ppid=16520 vsize=183232 CPUtime=24.36 cores=0,2,4,6
/proc/16522/stat : 16522 (plingeling) S 16520 16522 15826 0 -1 4202496 41846 1251 0 0 2430 6 0 0 23 0 5 0 520346862 187629568 33594 33554432000 4194304 4417332 140734601858192 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16522/statm: 45808 33594 179 55 0 43694 0
[pid=16522/tid=16565] ppid=16520 vsize=183232 CPUtime=6.02 cores=0,2,4,6
/proc/16522/task/16565/stat : 16565 (plingeling) R 16520 16522 15826 0 -1 4202560 5262 1251 0 0 602 0 0 0 25 0 5 0 520346888 187629568 33594 33554432000 4194304 4417332 140734601858192 18446744073709551615 4249222 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=16522/tid=16566] ppid=16520 vsize=183232 CPUtime=6.02 cores=0,2,4,6
/proc/16522/task/16566/stat : 16566 (plingeling) R 16520 16522 15826 0 -1 4202560 5273 1251 0 0 601 1 0 0 25 0 5 0 520346888 187629568 33594 33554432000 4194304 4417332 140734601858192 18446744073709551615 4249584 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16522/tid=16567] ppid=16520 vsize=183232 CPUtime=5.99 cores=0,2,4,6
/proc/16522/task/16567/stat : 16567 (plingeling) R 16520 16522 15826 0 -1 4202560 5364 1251 0 0 599 0 0 0 25 0 5 0 520346888 187629568 33594 33554432000 4194304 4417332 140734601858192 18446744073709551615 4251133 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16522/tid=16568] ppid=16520 vsize=183232 CPUtime=6.02 cores=0,2,4,6
/proc/16522/task/16568/stat : 16568 (plingeling) R 16520 16522 15826 0 -1 4202560 13043 1251 0 0 601 1 0 0 25 0 5 0 520346888 187629568 33594 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364509 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 24.36
Current children cumulated vsize (KiB) 183232

[startup+12.7002 s]
/proc/loadavg: 7.45 7.95 8.00 9/186 16604
/proc/meminfo: memFree=30238340/32950928 swapFree=67111428/67111528
[pid=16522] ppid=16520 vsize=199372 CPUtime=49.92 cores=0,2,4,6
/proc/16522/stat : 16522 (plingeling) S 16520 16522 15826 0 -1 4202496 62868 1251 0 0 4983 9 0 0 23 0 5 0 520346862 204156928 37629 33554432000 4194304 4417332 140734601858192 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16522/statm: 49843 37629 179 55 0 47729 0
[pid=16522/tid=16565] ppid=16520 vsize=199372 CPUtime=12.43 cores=0,2,4,6
/proc/16522/task/16565/stat : 16565 (plingeling) R 16520 16522 15826 0 -1 4202560 6075 1251 0 0 1242 1 0 0 25 0 5 0 520346888 204156928 37629 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364463 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=16522/tid=16566] ppid=16520 vsize=199372 CPUtime=12.42 cores=0,2,4,6
/proc/16522/task/16566/stat : 16566 (plingeling) R 16520 16522 15826 0 -1 4202560 6915 1251 0 0 1241 1 0 0 25 0 5 0 520346888 204156928 37629 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364732 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16522/tid=16567] ppid=16520 vsize=199372 CPUtime=12.36 cores=0,2,4,6
/proc/16522/task/16567/stat : 16567 (plingeling) R 16520 16522 15826 0 -1 4202560 15337 1251 0 0 1234 2 0 0 25 0 5 0 520346888 204156928 37629 33554432000 4194304 4417332 140734601858192 18446744073709551615 4249584 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16522/tid=16568] ppid=16520 vsize=199372 CPUtime=12.42 cores=0,2,4,6
/proc/16522/task/16568/stat : 16568 (plingeling) R 16520 16522 15826 0 -1 4202560 21637 1251 0 0 1239 3 0 0 25 0 5 0 520346888 204156928 37629 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 49.92
Current children cumulated vsize (KiB) 199372

[startup+25.5009 s]
/proc/loadavg: 7.53 7.95 8.00 10/186 16605
/proc/meminfo: memFree=30154672/32950928 swapFree=67111428/67111528
[pid=16522] ppid=16520 vsize=248768 CPUtime=101.05 cores=0,2,4,6
/proc/16522/stat : 16522 (plingeling) S 16520 16522 15826 0 -1 4202496 97939 1251 0 0 10091 14 0 0 23 0 5 0 520346862 254738432 49978 33554432000 4194304 4417332 140734601858192 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16522/statm: 62192 49978 179 55 0 60078 0
[pid=16522/tid=16565] ppid=16520 vsize=248768 CPUtime=25.23 cores=0,2,4,6
/proc/16522/task/16565/stat : 16565 (plingeling) R 16520 16522 15826 0 -1 4202560 7377 1251 0 0 2522 1 0 0 25 0 5 0 520346888 254738432 49978 33554432000 4194304 4417332 140734601858192 18446744073709551615 4249171 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=16522/tid=16566] ppid=16520 vsize=248768 CPUtime=25.22 cores=0,2,4,6
/proc/16522/task/16566/stat : 16566 (plingeling) R 16520 16522 15826 0 -1 4202560 20143 1251 0 0 2519 3 0 0 25 0 5 0 520346888 254738432 49978 33554432000 4194304 4417332 140734601858192 18446744073709551615 4249222 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16522/tid=16567] ppid=16520 vsize=248768 CPUtime=25.09 cores=0,2,4,6
/proc/16522/task/16567/stat : 16567 (plingeling) R 16520 16522 15826 0 -1 4202560 25841 1251 0 0 2506 3 0 0 25 0 5 0 520346888 254738432 49978 33554432000 4194304 4417332 140734601858192 18446744073709551615 4235685 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16522/tid=16568] ppid=16520 vsize=248768 CPUtime=25.22 cores=0,2,4,6
/proc/16522/task/16568/stat : 16568 (plingeling) R 16520 16522 15826 0 -1 4202560 31674 1251 0 0 2518 4 0 0 25 0 5 0 520346888 254738432 49978 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 101.05
Current children cumulated vsize (KiB) 248768

[startup+51.1004 s]
/proc/loadavg: 7.69 7.95 8.00 9/186 16605
/proc/meminfo: memFree=29949868/32950928 swapFree=67111428/67111528
[pid=16522] ppid=16520 vsize=352136 CPUtime=203.3 cores=0,2,4,6
/proc/16522/stat : 16522 (plingeling) S 16520 16522 15826 0 -1 4202496 158565 1251 0 0 20308 22 0 0 23 0 5 0 520346862 360587264 75821 33554432000 4194304 4417332 140734601858192 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16522/statm: 88034 75821 180 55 0 85920 0
[pid=16522/tid=16565] ppid=16520 vsize=352136 CPUtime=50.83 cores=0,2,4,6
/proc/16522/task/16565/stat : 16565 (plingeling) R 16520 16522 15826 0 -1 4202560 16251 1251 0 0 5081 2 0 0 25 0 5 0 520346888 360587264 75821 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=16522/tid=16566] ppid=16520 vsize=352136 CPUtime=50.83 cores=0,2,4,6
/proc/16522/task/16566/stat : 16566 (plingeling) R 16520 16522 15826 0 -1 4202560 25616 1251 0 0 5079 4 0 0 25 0 5 0 520346888 360587264 75821 33554432000 4194304 4417332 140734601858192 18446744073709551615 4211504 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16522/tid=16567] ppid=16520 vsize=352136 CPUtime=50.56 cores=0,2,4,6
/proc/16522/task/16567/stat : 16567 (plingeling) R 16520 16522 15826 0 -1 4202560 48330 1251 0 0 5049 7 0 0 25 0 5 0 520346888 360587264 75821 33554432000 4194304 4417332 140734601858192 18446744073709551615 4366303 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16522/tid=16568] ppid=16520 vsize=352136 CPUtime=50.81 cores=0,2,4,6
/proc/16522/task/16568/stat : 16568 (plingeling) R 16520 16522 15826 0 -1 4202560 55464 1251 0 0 5074 7 0 0 25 0 5 0 520346888 360587264 75821 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364565 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 203.3
Current children cumulated vsize (KiB) 352136

[startup+102.306 s]
/proc/loadavg: 7.92 7.97 8.00 10/186 16607
/proc/meminfo: memFree=29475816/32950928 swapFree=67111428/67111528
[pid=16522] ppid=16520 vsize=626792 CPUtime=407.85 cores=0,2,4,6
/proc/16522/stat : 16522 (plingeling) S 16520 16522 15826 0 -1 4202496 311314 1251 0 0 40741 44 0 0 23 0 5 0 520346862 641835008 128102 33554432000 4194304 4417332 140734601858192 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16522/statm: 156698 128102 180 55 0 154584 0
[pid=16522/tid=16565] ppid=16520 vsize=626792 CPUtime=102.03 cores=0,2,4,6
/proc/16522/task/16565/stat : 16565 (plingeling) R 16520 16522 15826 0 -1 4202560 73399 1251 0 0 10194 9 0 0 25 0 5 0 520346888 641835008 128102 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364547 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=16522/tid=16566] ppid=16520 vsize=626792 CPUtime=102.03 cores=0,2,4,6
/proc/16522/task/16566/stat : 16566 (plingeling) R 16520 16522 15826 0 -1 4202560 40978 1251 0 0 10198 5 0 0 25 0 5 0 520346888 641835008 128102 33554432000 4194304 4417332 140734601858192 18446744073709551615 4259903 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16522/tid=16567] ppid=16520 vsize=626792 CPUtime=101.47 cores=0,2,4,6
/proc/16522/task/16567/stat : 16567 (plingeling) R 16520 16522 15826 0 -1 4202560 109449 1251 0 0 10131 16 0 0 25 0 5 0 520346888 641835008 128102 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16522/tid=16568] ppid=16520 vsize=626792 CPUtime=102.02 cores=0,2,4,6
/proc/16522/task/16568/stat : 16568 (plingeling) R 16520 16522 15826 0 -1 4202560 74584 1251 0 0 10191 11 0 0 25 0 5 0 520346888 641835008 128102 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364473 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 407.85
Current children cumulated vsize (KiB) 626792

[startup+162.301 s]
/proc/loadavg: 8.00 7.99 8.00 10/186 16608
/proc/meminfo: memFree=29116296/32950928 swapFree=67111428/67111528
[pid=16522] ppid=16520 vsize=871328 CPUtime=647.5 cores=0,2,4,6
/proc/16522/stat : 16522 (plingeling) S 16520 16522 15826 0 -1 4202496 488720 1251 0 0 64681 69 0 0 23 0 5 0 520346862 892239872 189236 33554432000 4194304 4417332 140734601858192 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16522/statm: 217832 189236 180 55 0 215718 0
[pid=16522/tid=16565] ppid=16520 vsize=871328 CPUtime=162.03 cores=0,2,4,6
/proc/16522/task/16565/stat : 16565 (plingeling) R 16520 16522 15826 0 -1 4202560 122227 1251 0 0 16188 15 0 0 25 0 5 0 520346888 892239872 189236 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364919 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=16522/tid=16566] ppid=16520 vsize=871328 CPUtime=162.02 cores=0,2,4,6
/proc/16522/task/16566/stat : 16566 (plingeling) R 16520 16522 15826 0 -1 4202560 97531 1251 0 0 16189 13 0 0 25 0 5 0 520346888 892239872 189236 33554432000 4194304 4417332 140734601858192 18446744073709551615 4249584 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16522/tid=16567] ppid=16520 vsize=871328 CPUtime=161.15 cores=0,2,4,6
/proc/16522/task/16567/stat : 16567 (plingeling) R 16520 16522 15826 0 -1 4202560 147552 1251 0 0 16092 23 0 0 25 0 5 0 520346888 892239872 189236 33554432000 4194304 4417332 140734601858192 18446744073709551615 4357108 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16522/tid=16568] ppid=16520 vsize=871328 CPUtime=162.01 cores=0,2,4,6
/proc/16522/task/16568/stat : 16568 (plingeling) R 16520 16522 15826 0 -1 4202560 108506 1251 0 0 16185 16 0 0 25 0 5 0 520346888 892239872 189236 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 647.5
Current children cumulated vsize (KiB) 871328


################
# More data... #
################

[pid=16522/tid=16566] ppid=16520 vsize=1594268 CPUtime=762.04 cores=0,2,4,6
/proc/16522/task/16566/stat : 16566 (plingeling) R 16520 16522 15826 0 -1 4202560 442012 1251 0 0 76138 66 0 0 25 0 5 0 520346888 1632530432 369979 33554432000 4194304 4417332 140734601858192 18446744073709551615 4265437 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16522/tid=16567] ppid=16520 vsize=1594268 CPUtime=757.91 cores=0,2,4,6
/proc/16522/task/16567/stat : 16567 (plingeling) R 16520 16522 15826 0 -1 4202560 808842 1251 0 0 75665 126 0 0 25 0 5 0 520346888 1632530432 369979 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364509 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16522/tid=16568] ppid=16520 vsize=1594268 CPUtime=761.97 cores=0,2,4,6
/proc/16522/task/16568/stat : 16568 (plingeling) R 16520 16522 15826 0 -1 4202560 254680 1251 0 0 76157 40 0 0 25 0 5 0 520346888 1632530432 369979 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364479 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3044.26
Current children cumulated vsize (KiB) 1594268

[startup+822.3 s]
/proc/loadavg: 8.08 8.08 8.02 10/186 16626
/proc/meminfo: memFree=27593812/32950928 swapFree=67111428/67111528
[pid=16522] ppid=16520 vsize=1375844 CPUtime=3283.91 cores=0,2,4,6
/proc/16522/stat : 16522 (plingeling) S 16520 16522 15826 0 -1 4202496 2125618 1251 0 0 328071 320 0 0 23 0 5 0 520346862 1408864256 315373 33554432000 4194304 4417332 140734601858192 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16522/statm: 343961 315373 180 55 0 341847 0
[pid=16522/tid=16565] ppid=16520 vsize=1375844 CPUtime=822.05 cores=0,2,4,6
/proc/16522/task/16565/stat : 16565 (plingeling) R 16520 16522 15826 0 -1 4202560 539197 1251 0 0 82133 72 0 0 25 0 5 0 520346888 1408864256 315373 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364502 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=16522/tid=16566] ppid=16520 vsize=1375844 CPUtime=822.04 cores=0,2,4,6
/proc/16522/task/16566/stat : 16566 (plingeling) R 16520 16522 15826 0 -1 4202560 455899 1251 0 0 82136 68 0 0 25 0 5 0 520346888 1408864256 315373 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364753 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16522/tid=16567] ppid=16520 vsize=1375844 CPUtime=817.58 cores=0,2,4,6
/proc/16522/task/16567/stat : 16567 (plingeling) R 16520 16522 15826 0 -1 4202560 839514 1251 0 0 81625 133 0 0 25 0 5 0 520346888 1408864256 315373 33554432000 4194304 4417332 140734601858192 18446744073709551615 4366303 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16522/tid=16568] ppid=16520 vsize=1375844 CPUtime=821.96 cores=0,2,4,6
/proc/16522/task/16568/stat : 16568 (plingeling) R 16520 16522 15826 0 -1 4202560 278104 1251 0 0 82151 45 0 0 25 0 5 0 520346888 1408864256 315373 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3283.91
Current children cumulated vsize (KiB) 1375844

[startup+882.301 s]
/proc/loadavg: 8.12 8.09 8.02 9/186 16627
/proc/meminfo: memFree=27663696/32950928 swapFree=67111428/67111528
[pid=16522] ppid=16520 vsize=1512784 CPUtime=3523.59 cores=0,2,4,6
/proc/16522/stat : 16522 (plingeling) S 16520 16522 15826 0 -1 4202496 2210532 1251 0 0 352024 335 0 0 23 0 5 0 520346862 1549090816 349608 33554432000 4194304 4417332 140734601858192 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16522/statm: 378196 349608 180 55 0 376082 0
[pid=16522/tid=16565] ppid=16520 vsize=1512784 CPUtime=882.05 cores=0,2,4,6
/proc/16522/task/16565/stat : 16565 (plingeling) R 16520 16522 15826 0 -1 4202560 580674 1251 0 0 88127 78 0 0 25 0 5 0 520346888 1549090816 349608 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=16522/tid=16566] ppid=16520 vsize=1512784 CPUtime=882.04 cores=0,2,4,6
/proc/16522/task/16566/stat : 16566 (plingeling) R 16520 16522 15826 0 -1 4202560 477283 1251 0 0 88133 71 0 0 25 0 5 0 520346888 1549090816 349608 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364724 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16522/tid=16567] ppid=16520 vsize=1512784 CPUtime=877.25 cores=0,2,4,6
/proc/16522/task/16567/stat : 16567 (plingeling) R 16520 16522 15826 0 -1 4202560 861567 1251 0 0 87587 138 0 0 25 0 5 0 520346888 1549090816 349608 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16522/tid=16568] ppid=16520 vsize=1512784 CPUtime=881.96 cores=0,2,4,6
/proc/16522/task/16568/stat : 16568 (plingeling) R 16520 16522 15826 0 -1 4202560 278104 1251 0 0 88151 45 0 0 25 0 5 0 520346888 1549090816 349608 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3523.59
Current children cumulated vsize (KiB) 1512784

[startup+942.301 s]
/proc/loadavg: 8.12 8.10 8.03 9/186 16629
/proc/meminfo: memFree=27374604/32950928 swapFree=67111428/67111528
[pid=16522] ppid=16520 vsize=1498204 CPUtime=3763.26 cores=0,2,4,6
/proc/16522/stat : 16522 (plingeling) S 16520 16522 15826 0 -1 4202496 2406256 1251 0 0 375966 360 0 0 23 0 5 0 520346862 1534160896 345995 33554432000 4194304 4417332 140734601858192 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16522/statm: 374551 345995 180 55 0 372437 0
[pid=16522/tid=16565] ppid=16520 vsize=1498204 CPUtime=942.05 cores=0,2,4,6
/proc/16522/task/16565/stat : 16565 (plingeling) R 16520 16522 15826 0 -1 4202560 599039 1251 0 0 94124 81 0 0 25 0 5 0 520346888 1534160896 345995 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=16522/tid=16566] ppid=16520 vsize=1498204 CPUtime=942.04 cores=0,2,4,6
/proc/16522/task/16566/stat : 16566 (plingeling) R 16520 16522 15826 0 -1 4202560 502566 1251 0 0 94129 75 0 0 25 0 5 0 520346888 1534160896 345995 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16522/tid=16567] ppid=16520 vsize=1498204 CPUtime=936.93 cores=0,2,4,6
/proc/16522/task/16567/stat : 16567 (plingeling) R 16520 16522 15826 0 -1 4202560 1000792 1251 0 0 93537 156 0 0 25 0 5 0 520346888 1534160896 345995 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16522/tid=16568] ppid=16520 vsize=1498204 CPUtime=941.95 cores=0,2,4,6
/proc/16522/task/16568/stat : 16568 (plingeling) R 16520 16522 15826 0 -1 4202560 290955 1251 0 0 94149 46 0 0 25 0 5 0 520346888 1534160896 345995 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3763.26
Current children cumulated vsize (KiB) 1498204

[startup+1002.3 s]
/proc/loadavg: 8.12 8.11 8.03 9/186 16631
/proc/meminfo: memFree=27388940/32950928 swapFree=67111428/67111528
[pid=16522] ppid=16520 vsize=1522620 CPUtime=4002.93 cores=0,2,4,6
/proc/16522/stat : 16522 (plingeling) S 16520 16522 15826 0 -1 4202496 2480023 1251 0 0 399924 369 0 0 23 0 5 0 520346862 1559162880 352067 33554432000 4194304 4417332 140734601858192 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16522/statm: 380655 352067 180 55 0 378541 0
[pid=16522/tid=16565] ppid=16520 vsize=1522620 CPUtime=1002.05 cores=0,2,4,6
/proc/16522/task/16565/stat : 16565 (plingeling) R 16520 16522 15826 0 -1 4202560 635718 1251 0 0 100120 85 0 0 25 0 5 0 520346888 1559162880 352067 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364526 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=16522/tid=16566] ppid=16520 vsize=1522620 CPUtime=1002.05 cores=0,2,4,6
/proc/16522/task/16566/stat : 16566 (plingeling) R 16520 16522 15826 0 -1 4202560 537624 1251 0 0 100126 79 0 0 25 0 5 0 520346888 1559162880 352067 33554432000 4194304 4417332 140734601858192 18446744073709551615 4249151 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16522/tid=16567] ppid=16520 vsize=1522620 CPUtime=996.6 cores=0,2,4,6
/proc/16522/task/16567/stat : 16567 (plingeling) R 16520 16522 15826 0 -1 4202560 1002822 1251 0 0 99504 156 0 0 25 0 5 0 520346888 1559162880 352067 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16522/tid=16568] ppid=16520 vsize=1522620 CPUtime=1001.94 cores=0,2,4,6
/proc/16522/task/16568/stat : 16568 (plingeling) R 16520 16522 15826 0 -1 4202560 290955 1251 0 0 100148 46 0 0 25 0 5 0 520346888 1559162880 352067 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364509 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4002.93
Current children cumulated vsize (KiB) 1522620

[startup+1062.3 s]
/proc/loadavg: 8.08 8.10 8.03 9/186 16632
/proc/meminfo: memFree=27504828/32950928 swapFree=67111428/67111528
[pid=16522] ppid=16520 vsize=1495632 CPUtime=4242.6 cores=0,2,4,6
/proc/16522/stat : 16522 (plingeling) S 16520 16522 15826 0 -1 4202496 2700712 1251 0 0 423859 401 0 0 23 0 5 0 520346862 1531527168 345320 33554432000 4194304 4417332 140734601858192 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16522/statm: 373908 345320 180 55 0 371794 0
[pid=16522/tid=16565] ppid=16520 vsize=1495632 CPUtime=1062.05 cores=0,2,4,6
/proc/16522/task/16565/stat : 16565 (plingeling) R 16520 16522 15826 0 -1 4202560 653947 1251 0 0 106117 88 0 0 25 0 5 0 520346888 1531527168 345320 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=16522/tid=16566] ppid=16520 vsize=1495632 CPUtime=1062.04 cores=0,2,4,6
/proc/16522/task/16566/stat : 16566 (plingeling) R 16520 16522 15826 0 -1 4202560 589682 1251 0 0 106118 86 0 0 25 0 5 0 520346888 1531527168 345320 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16522/tid=16567] ppid=16520 vsize=1495632 CPUtime=1056.27 cores=0,2,4,6
/proc/16522/task/16567/stat : 16567 (plingeling) R 16520 16522 15826 0 -1 4202560 1104974 1251 0 0 105456 171 0 0 25 0 5 0 520346888 1531527168 345320 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364470 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16522/tid=16568] ppid=16520 vsize=1495632 CPUtime=1061.94 cores=0,2,4,6
/proc/16522/task/16568/stat : 16568 (plingeling) R 16520 16522 15826 0 -1 4202560 339205 1251 0 0 106142 52 0 0 25 0 5 0 520346888 1531527168 345320 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364753 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4242.6
Current children cumulated vsize (KiB) 1495632

[startup+1122.3 s]
/proc/loadavg: 8.06 8.09 8.03 10/186 16634
/proc/meminfo: memFree=27088144/32950928 swapFree=67111428/67111528
[pid=16522] ppid=16520 vsize=1572048 CPUtime=4482.27 cores=0,2,4,6
/proc/16522/stat : 16522 (plingeling) S 16520 16522 15826 0 -1 4202496 2760981 1251 0 0 447818 409 0 0 23 0 5 0 520346862 1609777152 364424 33554432000 4194304 4417332 140734601858192 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16522/statm: 393012 364424 180 55 0 390898 0
[pid=16522/tid=16565] ppid=16520 vsize=1572048 CPUtime=1122.05 cores=0,2,4,6
/proc/16522/task/16565/stat : 16565 (plingeling) R 16520 16522 15826 0 -1 4202560 680245 1251 0 0 112113 92 0 0 25 0 5 0 520346888 1609777152 364424 33554432000 4194304 4417332 140734601858192 18446744073709551615 4232702 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=16522/tid=16566] ppid=16520 vsize=1572048 CPUtime=1122.04 cores=0,2,4,6
/proc/16522/task/16566/stat : 16566 (plingeling) R 16520 16522 15826 0 -1 4202560 618465 1251 0 0 112114 90 0 0 25 0 5 0 520346888 1609777152 364424 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16522/tid=16567] ppid=16520 vsize=1572048 CPUtime=1115.95 cores=0,2,4,6
/proc/16522/task/16567/stat : 16567 (plingeling) R 16520 16522 15826 0 -1 4202560 1104974 1251 0 0 111424 171 0 0 25 0 5 0 520346888 1609777152 364424 33554432000 4194304 4417332 140734601858192 18446744073709551615 4365003 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16522/tid=16568] ppid=16520 vsize=1572048 CPUtime=1121.94 cores=0,2,4,6
/proc/16522/task/16568/stat : 16568 (plingeling) R 16520 16522 15826 0 -1 4202560 344393 1251 0 0 112141 53 0 0 25 0 5 0 520346888 1609777152 364424 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364565 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4482.27
Current children cumulated vsize (KiB) 1572048

[startup+1182.3 s]
/proc/loadavg: 8.10 8.09 8.03 10/186 16635
/proc/meminfo: memFree=27754596/32950928 swapFree=67111428/67111528
[pid=16522] ppid=16520 vsize=1299072 CPUtime=4721.94 cores=0,2,4,6
/proc/16522/stat : 16522 (plingeling) S 16520 16522 15826 0 -1 4202496 2856314 1251 0 0 471770 424 0 0 23 0 5 0 520346862 1330249728 296180 33554432000 4194304 4417332 140734601858192 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16522/statm: 324768 296180 180 55 0 322654 0
[pid=16522/tid=16565] ppid=16520 vsize=1299072 CPUtime=1182.05 cores=0,2,4,6
/proc/16522/task/16565/stat : 16565 (plingeling) R 16520 16522 15826 0 -1 4202560 707200 1251 0 0 118110 95 0 0 25 0 5 0 520346888 1330249728 296180 33554432000 4194304 4417332 140734601858192 18446744073709551615 4277884 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=16522/tid=16566] ppid=16520 vsize=1299072 CPUtime=1182.03 cores=0,2,4,6
/proc/16522/task/16566/stat : 16566 (plingeling) R 16520 16522 15826 0 -1 4202560 645560 1251 0 0 118109 94 0 0 25 0 5 0 520346888 1330249728 296180 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16522/tid=16567] ppid=16520 vsize=1299072 CPUtime=1175.62 cores=0,2,4,6
/proc/16522/task/16567/stat : 16567 (plingeling) R 16520 16522 15826 0 -1 4202560 1146257 1251 0 0 117384 178 0 0 25 0 5 0 520346888 1330249728 296180 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16522/tid=16568] ppid=16520 vsize=1299072 CPUtime=1181.93 cores=0,2,4,6
/proc/16522/task/16568/stat : 16568 (plingeling) R 16520 16522 15826 0 -1 4202560 344393 1251 0 0 118140 53 0 0 25 0 5 0 520346888 1330249728 296180 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4721.94
Current children cumulated vsize (KiB) 1299072



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.9 s]
/proc/loadavg: 8.07 8.08 8.03 9/186 16636
/proc/meminfo: memFree=27872248/32950928 swapFree=67111428/67111528
[pid=16522] ppid=16520 vsize=1427252 CPUtime=4800.24 cores=0,2,4,6
/proc/16522/stat : 16522 (plingeling) S 16520 16522 15826 0 -1 4202496 2901514 1251 0 0 479594 430 0 0 23 0 5 0 520346862 1461506048 328225 33554432000 4194304 4417332 140734601858192 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16522/statm: 356813 328225 180 55 0 354699 0
[pid=16522/tid=16565] ppid=16520 vsize=1427252 CPUtime=1201.66 cores=0,2,4,6
/proc/16522/task/16565/stat : 16565 (plingeling) R 16520 16522 15826 0 -1 4202560 725482 1251 0 0 120069 97 0 0 25 0 5 0 520346888 1461506048 328225 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=16522/tid=16566] ppid=16520 vsize=1427252 CPUtime=1201.64 cores=0,2,4,6
/proc/16522/task/16566/stat : 16566 (plingeling) R 16520 16522 15826 0 -1 4202560 653892 1251 0 0 120069 95 0 0 25 0 5 0 520346888 1461506048 328225 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364732 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16522/tid=16567] ppid=16520 vsize=1427252 CPUtime=1195.11 cores=0,2,4,6
/proc/16522/task/16567/stat : 16567 (plingeling) R 16520 16522 15826 0 -1 4202560 1146905 1251 0 0 119333 178 0 0 25 0 5 0 520346888 1461506048 328225 33554432000 4194304 4417332 140734601858192 18446744073709551615 4366297 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16522/tid=16568] ppid=16520 vsize=1427252 CPUtime=1201.53 cores=0,2,4,6
/proc/16522/task/16568/stat : 16568 (plingeling) R 16520 16522 15826 0 -1 4202560 362331 1251 0 0 120097 56 0 0 25 0 5 0 520346888 1461506048 328225 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.24
Current children cumulated vsize (KiB) 1427252

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.9 s]
/proc/loadavg: 8.07 8.08 8.03 9/186 16636
/proc/meminfo: memFree=27872248/32950928 swapFree=67111428/67111528
[pid=16522] ppid=16520 vsize=1427252 CPUtime=4800.24 cores=0,2,4,6
/proc/16522/stat : 16522 (plingeling) S 16520 16522 15826 0 -1 4202496 2901514 1251 0 0 479594 430 0 0 23 0 5 0 520346862 1461506048 328225 33554432000 4194304 4417332 140734601858192 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16522/statm: 356813 328225 180 55 0 354699 0
[pid=16522/tid=16565] ppid=16520 vsize=1427252 CPUtime=1201.66 cores=0,2,4,6
/proc/16522/task/16565/stat : 16565 (plingeling) R 16520 16522 15826 0 -1 4202560 725482 1251 0 0 120069 97 0 0 25 0 5 0 520346888 1461506048 328225 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=16522/tid=16566] ppid=16520 vsize=1427252 CPUtime=1201.64 cores=0,2,4,6
/proc/16522/task/16566/stat : 16566 (plingeling) R 16520 16522 15826 0 -1 4202560 653892 1251 0 0 120069 95 0 0 25 0 5 0 520346888 1461506048 328225 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364732 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16522/tid=16567] ppid=16520 vsize=1427252 CPUtime=1195.11 cores=0,2,4,6
/proc/16522/task/16567/stat : 16567 (plingeling) R 16520 16522 15826 0 -1 4202560 1146905 1251 0 0 119333 178 0 0 25 0 5 0 520346888 1461506048 328225 33554432000 4194304 4417332 140734601858192 18446744073709551615 4366297 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16522/tid=16568] ppid=16520 vsize=1427252 CPUtime=1201.53 cores=0,2,4,6
/proc/16522/task/16568/stat : 16568 (plingeling) R 16520 16522 15826 0 -1 4202560 362331 1251 0 0 120097 56 0 0 25 0 5 0 520346888 1461506048 328225 33554432000 4194304 4417332 140734601858192 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.24
Current children cumulated vsize (KiB) 1427252

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1201.97
CPU time (s): 4800.34
CPU user time (s): 4795.97
CPU system time (s): 4.37133
CPU usage (%): 399.371
Max. virtual memory (cumulated for all children) (KiB): 1778072

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4795.97
system time used= 4.37134
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2902765
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 62
involuntary context switches= 25450

runsolver used 1.92971 second user time and 4.6163 second system time

The end

Launcher Data 

Begin job on node123 at 2011-04-29 16:31:11
IDJOB=3249712
IDBENCH=83182
IDSOLVER=1605
FILE ID=node123/3249712-1304087470
RUNJOBID= node123-1304087470-16493
PBS_JOBID= 13170535
Free space on /tmp= 72812 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT11/crafted/mosoi/289/289-unsat-14x34.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3249712-1304087470/watcher-3249712-1304087470 -o /tmp/evaluation-result-3249712-1304087470/solver-3249712-1304087470 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3249712-1304087470.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 72677fc46303044d3d9e832fa24a37b2
RANDOM SEED=1781401491

node123.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      30709832 kB
Buffers:        554960 kB
Cached:        1259684 kB
SwapCached:        100 kB
Active:         114176 kB
Inactive:      1764556 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      30709832 kB
SwapTotal:    67111528 kB
SwapFree:     67111428 kB
Dirty:            5220 kB
Writeback:           0 kB
AnonPages:       64684 kB
Mapped:          14884 kB
Slab:           298308 kB
PageTables:       4700 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   209228 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72796 MiB
End job on node123 at 2011-04-29 16:51:15