Trace number 3394458

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40000.7 5010.66

General information on the benchmark

Namecrafted/mosoi/289/
289-unsat-17x27.cnf
MD5SUMb9a7e3013b1b74b08c8e421f15cb16c4
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables1836
Number of clauses194157
Sum of the clauses size771120
Maximum clause length4
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 22754
Number of clauses of size 30
Number of clauses of size 4191403
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3394458-1306097094/watcher-3394458-1306097094 -o /tmp/evaluation-result-3394458-1306097094/solver-3394458-1306097094 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3394458-1306097094.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.50 7.94 7.46 1/457 29581
/proc/meminfo: memFree=10107520/32950928 swapFree=62427732/67111528
[pid=29581] ppid=29579 vsize=13128 CPUtime=0.01 cores=0-7
/proc/29581/stat : 29581 (plingeling) R 29579 29581 29387 0 -1 4202496 1344 1250 0 0 1 0 0 0 25 0 1 0 721309182 13443072 1274 33554432000 4194304 4417452 140734506391584 18446744073709551615 248089392109 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/29581/statm: 3282 1290 148 55 0 1168 0

[startup+0.0943029 s]
/proc/loadavg: 7.50 7.94 7.46 1/457 29581
/proc/meminfo: memFree=10107520/32950928 swapFree=62427732/67111528
[pid=29581] ppid=29579 vsize=34800 CPUtime=0.08 cores=0-7
/proc/29581/stat : 29581 (plingeling) R 29579 29581 29387 0 -1 4202496 7045 1250 0 0 8 0 0 0 25 0 1 0 721309182 35635200 6723 33554432000 4194304 4417452 140734506391584 18446744073709551615 248089316991 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/29581/statm: 8700 6723 148 55 0 6586 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 34800

[startup+0.100296 s]
/proc/loadavg: 7.50 7.94 7.46 1/457 29581
/proc/meminfo: memFree=10107520/32950928 swapFree=62427732/67111528
[pid=29581] ppid=29579 vsize=34800 CPUtime=0.08 cores=0-7
/proc/29581/stat : 29581 (plingeling) R 29579 29581 29387 0 -1 4202496 7045 1250 0 0 8 0 0 0 25 0 1 0 721309182 35635200 6723 33554432000 4194304 4417452 140734506391584 18446744073709551615 4259618 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/29581/statm: 8700 6723 148 55 0 6586 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 34800

[startup+0.30031 s]
/proc/loadavg: 7.50 7.94 7.46 1/457 29581
/proc/meminfo: memFree=10107520/32950928 swapFree=62427732/67111528
[pid=29581] ppid=29579 vsize=75760 CPUtime=0.29 cores=0-7
/proc/29581/stat : 29581 (plingeling) R 29579 29581 29387 0 -1 4202496 17285 1250 0 0 27 2 0 0 25 0 1 0 721309182 77578240 16963 33554432000 4194304 4417452 140734506391584 18446744073709551615 4209577 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/29581/statm: 18940 16963 148 55 0 16826 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 75760

[startup+0.700197 s]
/proc/loadavg: 7.50 7.94 7.46 1/457 29581
/proc/meminfo: memFree=10107520/32950928 swapFree=62427732/67111528
[pid=29581] ppid=29579 vsize=255576 CPUtime=2.52 cores=0-7
/proc/29581/stat : 29581 (plingeling) S 29579 29581 29387 0 -1 4202496 26494 1250 0 0 249 3 0 0 25 0 9 0 721309182 261709824 25115 33554432000 4194304 4417452 140734506391584 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/29581/statm: 63894 25115 176 55 0 61780 0
Current children cumulated CPU time (s) 2.52
Current children cumulated vsize (KiB) 255576

[startup+1.50106 s]
/proc/loadavg: 7.50 7.94 7.46 9/466 29595
/proc/meminfo: memFree=9983696/32950928 swapFree=62427732/67111528
[pid=29581] ppid=29579 vsize=305452 CPUtime=8.9 cores=0-7
/proc/29581/stat : 29581 (plingeling) S 29579 29581 29387 0 -1 4202496 41192 1250 0 0 883 7 0 0 25 0 9 0 721309182 312782848 37556 33554432000 4194304 4417452 140734506391584 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/29581/statm: 76363 37556 180 55 0 74249 0
[pid=29581/tid=29588] ppid=29579 vsize=305452 CPUtime=1.06 cores=0-7
/proc/29581/task/29588/stat : 29588 (plingeling) R 29579 29581 29387 0 -1 4202560 2292 1250 0 0 106 0 0 0 19 0 9 0 721309225 312782848 37556 33554432000 4194304 4417452 140734506391584 18446744073709551615 4277864 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29581/tid=29589] ppid=29579 vsize=305452 CPUtime=1.06 cores=0-7
/proc/29581/task/29589/stat : 29589 (plingeling) R 29579 29581 29387 0 -1 4202560 2244 1250 0 0 106 0 0 0 25 0 9 0 721309225 312782848 37556 33554432000 4194304 4417452 140734506391584 18446744073709551615 248089756074 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29581/tid=29590] ppid=29579 vsize=305452 CPUtime=1.06 cores=0-7
/proc/29581/task/29590/stat : 29590 (plingeling) R 29579 29581 29387 0 -1 4202560 972 1250 0 0 106 0 0 0 25 0 9 0 721309225 312782848 37556 33554432000 4194304 4417452 140734506391584 18446744073709551615 4239418 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29581/tid=29591] ppid=29579 vsize=305452 CPUtime=1.06 cores=0-7
/proc/29581/task/29591/stat : 29591 (plingeling) R 29579 29581 29387 0 -1 4202560 2489 1250 0 0 106 0 0 0 25 0 9 0 721309225 312782848 37556 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364616 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29581/tid=29592] ppid=29579 vsize=305452 CPUtime=1.05 cores=0-7
/proc/29581/task/29592/stat : 29592 (plingeling) R 29579 29581 29387 0 -1 4202560 2250 1250 0 0 105 0 0 0 25 0 9 0 721309225 312782848 37556 33554432000 4194304 4417452 140734506391584 18446744073709551615 4278246 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29581/tid=29593] ppid=29579 vsize=305452 CPUtime=1.03 cores=0-7
/proc/29581/task/29593/stat : 29593 (plingeling) R 29579 29581 29387 0 -1 4202560 2262 1250 0 0 103 0 0 0 25 0 9 0 721309225 312782848 37556 33554432000 4194304 4417452 140734506391584 18446744073709551615 4330858 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29581/tid=29594] ppid=29579 vsize=305452 CPUtime=1.03 cores=0-7
/proc/29581/task/29594/stat : 29594 (plingeling) R 29579 29581 29387 0 -1 4202560 950 1250 0 0 103 0 0 0 25 0 9 0 721309225 312782848 37556 33554432000 4194304 4417452 140734506391584 18446744073709551615 4260519 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29581/tid=29595] ppid=29579 vsize=305452 CPUtime=1.05 cores=0-7
/proc/29581/task/29595/stat : 29595 (plingeling) R 29579 29581 29387 0 -1 4202560 2239 1250 0 0 105 0 0 0 25 0 9 0 721309225 312782848 37556 33554432000 4194304 4417452 140734506391584 18446744073709551615 4331850 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 8.9
Current children cumulated vsize (KiB) 305452

[startup+3.10077 s]
/proc/loadavg: 7.50 7.94 7.46 9/466 29595
/proc/meminfo: memFree=9900668/32950928 swapFree=62427732/67111528
[pid=29581] ppid=29579 vsize=380568 CPUtime=21.66 cores=0-7
/proc/29581/stat : 29581 (plingeling) S 29579 29581 29387 0 -1 4202496 62765 1250 0 0 2154 12 0 0 25 0 9 0 721309182 389701632 56335 33554432000 4194304 4417452 140734506391584 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/29581/statm: 95142 56335 180 55 0 93028 0
[pid=29581/tid=29588] ppid=29579 vsize=380568 CPUtime=2.65 cores=0-7
/proc/29581/task/29588/stat : 29588 (plingeling) R 29579 29581 29387 0 -1 4202560 5148 1250 0 0 265 0 0 0 22 0 9 0 721309225 389701632 56335 33554432000 4194304 4417452 140734506391584 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29581/tid=29589] ppid=29579 vsize=380568 CPUtime=2.66 cores=0-7
/proc/29581/task/29589/stat : 29589 (plingeling) R 29579 29581 29387 0 -1 4202560 5048 1250 0 0 265 1 0 0 25 0 9 0 721309225 389701632 56335 33554432000 4194304 4417452 140734506391584 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29581/tid=29590] ppid=29579 vsize=380568 CPUtime=2.65 cores=0-7
/proc/29581/task/29590/stat : 29590 (plingeling) R 29579 29581 29387 0 -1 4202560 2841 1250 0 0 265 0 0 0 25 0 9 0 721309225 389701632 56335 33554432000 4194304 4417452 140734506391584 18446744073709551615 4249143 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29581/tid=29591] ppid=29579 vsize=380568 CPUtime=2.65 cores=0-7
/proc/29581/task/29591/stat : 29591 (plingeling) R 29579 29581 29387 0 -1 4202560 6116 1250 0 0 265 0 0 0 25 0 9 0 721309225 389701632 56335 33554432000 4194304 4417452 140734506391584 18446744073709551615 4251167 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29581/tid=29592] ppid=29579 vsize=380568 CPUtime=2.66 cores=0-7
/proc/29581/task/29592/stat : 29592 (plingeling) R 29579 29581 29387 0 -1 4202560 5106 1250 0 0 265 1 0 0 25 0 9 0 721309225 389701632 56335 33554432000 4194304 4417452 140734506391584 18446744073709551615 4258389 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29581/tid=29593] ppid=29579 vsize=380568 CPUtime=2.61 cores=0-7
/proc/29581/task/29593/stat : 29593 (plingeling) R 29579 29581 29387 0 -1 4202560 5121 1250 0 0 260 1 0 0 25 0 9 0 721309225 389701632 56335 33554432000 4194304 4417452 140734506391584 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29581/tid=29594] ppid=29579 vsize=380568 CPUtime=2.62 cores=0-7
/proc/29581/task/29594/stat : 29594 (plingeling) R 29579 29581 29387 0 -1 4202560 2819 1250 0 0 262 0 0 0 25 0 9 0 721309225 389701632 56335 33554432000 4194304 4417452 140734506391584 18446744073709551615 4249151 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29581/tid=29595] ppid=29579 vsize=380568 CPUtime=2.65 cores=0-7
/proc/29581/task/29595/stat : 29595 (plingeling) R 29579 29581 29387 0 -1 4202560 5072 1250 0 0 264 1 0 0 25 0 9 0 721309225 389701632 56335 33554432000 4194304 4417452 140734506391584 18446744073709551615 4249181 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 21.66
Current children cumulated vsize (KiB) 380568

[startup+6.30121 s]
/proc/loadavg: 7.54 7.94 7.46 9/466 29595
/proc/meminfo: memFree=9878224/32950928 swapFree=62427732/67111528
[pid=29581] ppid=29579 vsize=407864 CPUtime=47.22 cores=0-7
/proc/29581/stat : 29581 (plingeling) S 29579 29581 29387 0 -1 4202496 72190 1250 0 0 4708 14 0 0 25 0 9 0 721309182 417652736 63159 33554432000 4194304 4417452 140734506391584 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/29581/statm: 101966 63159 180 55 0 99852 0
[pid=29581/tid=29588] ppid=29579 vsize=407864 CPUtime=5.85 cores=0-7
/proc/29581/task/29588/stat : 29588 (plingeling) R 29579 29581 29387 0 -1 4202560 5148 1250 0 0 585 0 0 0 25 0 9 0 721309225 417652736 63159 33554432000 4194304 4417452 140734506391584 18446744073709551615 4249130 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29581/tid=29589] ppid=29579 vsize=407864 CPUtime=5.86 cores=0-7
/proc/29581/task/29589/stat : 29589 (plingeling) R 29579 29581 29387 0 -1 4202560 5048 1250 0 0 585 1 0 0 25 0 9 0 721309225 417652736 63159 33554432000 4194304 4417452 140734506391584 18446744073709551615 4302888 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29581/tid=29590] ppid=29579 vsize=407864 CPUtime=5.86 cores=0-7
/proc/29581/task/29590/stat : 29590 (plingeling) R 29579 29581 29387 0 -1 4202560 5415 1250 0 0 585 1 0 0 25 0 9 0 721309225 417652736 63159 33554432000 4194304 4417452 140734506391584 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29581/tid=29591] ppid=29579 vsize=407864 CPUtime=5.85 cores=0-7
/proc/29581/task/29591/stat : 29591 (plingeling) R 29579 29581 29387 0 -1 4202560 10815 1250 0 0 584 1 0 0 25 0 9 0 721309225 417652736 63159 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29581/tid=29592] ppid=29579 vsize=407864 CPUtime=5.86 cores=0-7
/proc/29581/task/29592/stat : 29592 (plingeling) R 29579 29581 29387 0 -1 4202560 5106 1250 0 0 585 1 0 0 25 0 9 0 721309225 417652736 63159 33554432000 4194304 4417452 140734506391584 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29581/tid=29593] ppid=29579 vsize=407864 CPUtime=5.76 cores=0-7
/proc/29581/task/29593/stat : 29593 (plingeling) R 29579 29581 29387 0 -1 4202560 5121 1250 0 0 575 1 0 0 25 0 9 0 721309225 417652736 63159 33554432000 4194304 4417452 140734506391584 18446744073709551615 4258161 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29581/tid=29594] ppid=29579 vsize=407864 CPUtime=5.82 cores=0-7
/proc/29581/task/29594/stat : 29594 (plingeling) R 29579 29581 29387 0 -1 4202560 4971 1250 0 0 582 0 0 0 25 0 9 0 721309225 417652736 63159 33554432000 4194304 4417452 140734506391584 18446744073709551615 4365078 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29581/tid=29595] ppid=29579 vsize=407864 CPUtime=5.85 cores=0-7
/proc/29581/task/29595/stat : 29595 (plingeling) R 29579 29581 29387 0 -1 4202560 5072 1250 0 0 584 1 0 0 25 0 9 0 721309225 417652736 63159 33554432000 4194304 4417452 140734506391584 18446744073709551615 4249110 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 47.22
Current children cumulated vsize (KiB) 407864

[startup+12.7011 s]
/proc/loadavg: 7.57 7.94 7.46 9/466 29595
/proc/meminfo: memFree=9848564/32950928 swapFree=62427732/67111528
[pid=29581] ppid=29579 vsize=414700 CPUtime=98.32 cores=0-7
/proc/29581/stat : 29581 (plingeling) S 29579 29581 29387 0 -1 4202496 125280 1250 0 0 9807 25 0 0 25 0 9 0 721309182 424652800 64875 33554432000 4194304 4417452 140734506391584 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/29581/statm: 103675 64875 180 55 0 101561 0
[pid=29581/tid=29588] ppid=29579 vsize=414700 CPUtime=12.26 cores=0-7
/proc/29581/task/29588/stat : 29588 (plingeling) R 29579 29581 29387 0 -1 4202560 7664 1250 0 0 1225 1 0 0 25 0 9 0 721309225 424652800 64875 33554432000 4194304 4417452 140734506391584 18446744073709551615 4249222 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29581/tid=29589] ppid=29579 vsize=414700 CPUtime=12.24 cores=0-7
/proc/29581/task/29589/stat : 29589 (plingeling) R 29579 29581 29387 0 -1 4202560 7750 1250 0 0 1223 1 0 0 25 0 9 0 721309225 424652800 64875 33554432000 4194304 4417452 140734506391584 18446744073709551615 4274646 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29581/tid=29590] ppid=29579 vsize=414700 CPUtime=12.25 cores=0-7
/proc/29581/task/29590/stat : 29590 (plingeling) R 29579 29581 29387 0 -1 4202560 18786 1250 0 0 1222 3 0 0 25 0 9 0 721309225 424652800 64875 33554432000 4194304 4417452 140734506391584 18446744073709551615 4249584 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29581/tid=29591] ppid=29579 vsize=414700 CPUtime=12.25 cores=0-7
/proc/29581/task/29591/stat : 29591 (plingeling) R 29579 29581 29387 0 -1 4202560 19076 1250 0 0 1222 3 0 0 25 0 9 0 721309225 424652800 64875 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29581/tid=29592] ppid=29579 vsize=414700 CPUtime=12.26 cores=0-7
/proc/29581/task/29592/stat : 29592 (plingeling) R 29579 29581 29387 0 -1 4202560 6507 1250 0 0 1225 1 0 0 25 0 9 0 721309225 424652800 64875 33554432000 4194304 4417452 140734506391584 18446744073709551615 4278271 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29581/tid=29593] ppid=29579 vsize=414700 CPUtime=12.08 cores=0-7
/proc/29581/task/29593/stat : 29593 (plingeling) R 29579 29581 29387 0 -1 4202560 18928 1250 0 0 1203 5 0 0 25 0 9 0 721309225 424652800 64875 33554432000 4194304 4417452 140734506391584 18446744073709551615 4249151 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29581/tid=29594] ppid=29579 vsize=414700 CPUtime=12.23 cores=0-7
/proc/29581/task/29594/stat : 29594 (plingeling) R 29579 29581 29387 0 -1 4202560 11497 1250 0 0 1221 2 0 0 25 0 9 0 721309225 424652800 64875 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29581/tid=29595] ppid=29579 vsize=414700 CPUtime=12.24 cores=0-7
/proc/29581/task/29595/stat : 29595 (plingeling) R 29579 29581 29387 0 -1 4202560 9578 1250 0 0 1223 1 0 0 25 0 9 0 721309225 424652800 64875 33554432000 4194304 4417452 140734506391584 18446744073709551615 4365111 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 98.32
Current children cumulated vsize (KiB) 414700

[startup+25.5008 s]
/proc/loadavg: 7.73 7.96 7.47 9/466 29595
/proc/meminfo: memFree=9714364/32950928 swapFree=62427732/67111528
[pid=29581] ppid=29579 vsize=518156 CPUtime=200.52 cores=0-7
/proc/29581/stat : 29581 (plingeling) S 29579 29581 29387 0 -1 4202496 236491 1250 0 0 20006 46 0 0 25 0 9 0 721309182 530591744 90837 33554432000 4194304 4417452 140734506391584 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/29581/statm: 129539 90837 180 55 0 127425 0
[pid=29581/tid=29588] ppid=29579 vsize=518156 CPUtime=25.05 cores=0-7
/proc/29581/task/29588/stat : 29588 (plingeling) R 29579 29581 29387 0 -1 4202560 28130 1250 0 0 2501 4 0 0 25 0 9 0 721309225 530591744 90837 33554432000 4194304 4417452 140734506391584 18446744073709551615 4248816 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29581/tid=29589] ppid=29579 vsize=518156 CPUtime=24.84 cores=0-7
/proc/29581/task/29589/stat : 29589 (plingeling) R 29579 29581 29387 0 -1 4202560 25949 1250 0 0 2480 4 0 0 25 0 9 0 721309225 530591744 90837 33554432000 4194304 4417452 140734506391584 18446744073709551615 4251148 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29581/tid=29590] ppid=29579 vsize=518156 CPUtime=25.06 cores=0-7
/proc/29581/task/29590/stat : 29590 (plingeling) R 29579 29581 29387 0 -1 4202560 33970 1250 0 0 2500 6 0 0 25 0 9 0 721309225 530591744 90837 33554432000 4194304 4417452 140734506391584 18446744073709551615 4236995 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29581/tid=29591] ppid=29579 vsize=518156 CPUtime=25.06 cores=0-7
/proc/29581/task/29591/stat : 29591 (plingeling) R 29579 29581 29387 0 -1 4202560 32663 1250 0 0 2499 7 0 0 25 0 9 0 721309225 530591744 90837 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29581/tid=29592] ppid=29579 vsize=518156 CPUtime=25.05 cores=0-7
/proc/29581/task/29592/stat : 29592 (plingeling) R 29579 29581 29387 0 -1 4202560 21584 1250 0 0 2501 4 0 0 25 0 9 0 721309225 530591744 90837 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29581/tid=29593] ppid=29579 vsize=518156 CPUtime=24.87 cores=0-7
/proc/29581/task/29593/stat : 29593 (plingeling) R 29579 29581 29387 0 -1 4202560 34943 1250 0 0 2479 8 0 0 25 0 9 0 721309225 530591744 90837 33554432000 4194304 4417452 140734506391584 18446744073709551615 4259603 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29581/tid=29594] ppid=29579 vsize=518156 CPUtime=25.02 cores=0-7
/proc/29581/task/29594/stat : 29594 (plingeling) R 29579 29581 29387 0 -1 4202560 22344 1250 0 0 2499 3 0 0 25 0 9 0 721309225 530591744 90837 33554432000 4194304 4417452 140734506391584 18446744073709551615 4249171 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29581/tid=29595] ppid=29579 vsize=518156 CPUtime=25.05 cores=0-7
/proc/29581/task/29595/stat : 29595 (plingeling) R 29579 29581 29387 0 -1 4202560 11414 1250 0 0 2503 2 0 0 25 0 9 0 721309225 530591744 90837 33554432000 4194304 4417452 140734506391584 18446744073709551615 4249171 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 200.52
Current children cumulated vsize (KiB) 518156

[startup+51.1013 s]
/proc/loadavg: 7.82 7.96 7.49 9/466 29596
/proc/meminfo: memFree=9570588/32950928 swapFree=62427732/67111528
[pid=29581] ppid=29579 vsize=680968 CPUtime=404.92 cores=0-7
/proc/29581/stat : 29581 (plingeling) S 29579 29581 29387 0 -1 4202496 378156 1250 0 0 40423 69 0 0 25 0 9 0 721309182 697311232 131444 33554432000 4194304 4417452 140734506391584 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/29581/statm: 170242 131444 180 55 0 168128 0
[pid=29581/tid=29588] ppid=29579 vsize=680968 CPUtime=50.65 cores=0-7
/proc/29581/task/29588/stat : 29588 (plingeling) R 29579 29581 29387 0 -1 4202560 64187 1250 0 0 5055 10 0 0 25 0 9 0 721309225 697311232 131444 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29581/tid=29589] ppid=29579 vsize=680968 CPUtime=50.06 cores=0-7
/proc/29581/task/29589/stat : 29589 (plingeling) R 29579 29581 29387 0 -1 4202560 40668 1250 0 0 5000 6 0 0 25 0 9 0 721309225 697311232 131444 33554432000 4194304 4417452 140734506391584 18446744073709551615 4249584 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29581/tid=29590] ppid=29579 vsize=680968 CPUtime=50.66 cores=0-7

################
# More data... #
################


[startup+4722.32 s]
/proc/loadavg: 8.12 8.08 8.01 9/466 29742
/proc/meminfo: memFree=3249144/32950928 swapFree=62427732/67111528
[pid=29581] ppid=29579 vsize=7002204 CPUtime=37700.8 cores=0-7
/proc/29581/stat : 29581 (plingeling) S 29579 29581 29387 0 -1 4202496 16151964 1250 0 0 3766487 3597 0 0 25 0 9 0 721309182 7170256896 1711753 33554432000 4194304 4417452 140734506391584 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/29581/statm: 1750551 1711753 180 55 0 1748437 0
[pid=29581/tid=29588] ppid=29579 vsize=7002204 CPUtime=4721.94 cores=0-7
/proc/29581/task/29588/stat : 29588 (plingeling) R 29579 29581 29387 0 -1 4202560 1757744 1250 0 0 471800 394 0 0 25 0 9 0 721309225 7170256896 1711753 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29581/tid=29589] ppid=29579 vsize=7002204 CPUtime=4649.21 cores=0-7
/proc/29581/task/29589/stat : 29589 (plingeling) R 29579 29581 29387 0 -1 4202560 1887642 1250 0 0 464540 381 0 0 25 0 9 0 721309225 7170256896 1711753 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29581/tid=29590] ppid=29579 vsize=7002204 CPUtime=4721.96 cores=0-7
/proc/29581/task/29590/stat : 29590 (plingeling) R 29579 29581 29387 0 -1 4202560 2390951 1250 0 0 471629 567 0 0 25 0 9 0 721309225 7170256896 1711753 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29581/tid=29591] ppid=29579 vsize=7002204 CPUtime=4721.94 cores=0-7
/proc/29581/task/29591/stat : 29591 (plingeling) R 29579 29581 29387 0 -1 4202560 632330 1250 0 0 472031 163 0 0 25 0 9 0 721309225 7170256896 1711753 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29581/tid=29592] ppid=29579 vsize=7002204 CPUtime=4721.92 cores=0-7
/proc/29581/task/29592/stat : 29592 (plingeling) R 29579 29581 29387 0 -1 4202560 2194041 1250 0 0 471608 584 0 0 25 0 9 0 721309225 7170256896 1711753 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29581/tid=29593] ppid=29579 vsize=7002204 CPUtime=4721.65 cores=0-7
/proc/29581/task/29593/stat : 29593 (plingeling) R 29579 29581 29387 0 -1 4202560 2477241 1250 0 0 471661 504 0 0 25 0 9 0 721309225 7170256896 1711753 33554432000 4194304 4417452 140734506391584 18446744073709551615 4357204 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29581/tid=29594] ppid=29579 vsize=7002204 CPUtime=4719.81 cores=0-7
/proc/29581/task/29594/stat : 29594 (plingeling) R 29579 29581 29387 0 -1 4202560 2690281 1250 0 0 471358 623 0 0 25 0 9 0 721309225 7170256896 1711753 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29581/tid=29595] ppid=29579 vsize=7002204 CPUtime=4721.91 cores=0-7
/proc/29581/task/29595/stat : 29595 (plingeling) R 29579 29581 29387 0 -1 4202560 2096240 1250 0 0 471816 375 0 0 25 0 9 0 721309225 7170256896 1711753 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 37700.8
Current children cumulated vsize (KiB) 7002204

[startup+4782.3 s]
/proc/loadavg: 8.30 8.12 8.02 9/466 29746
/proc/meminfo: memFree=2836284/32950928 swapFree=62427732/67111528
[pid=29581] ppid=29579 vsize=7414296 CPUtime=38178.9 cores=0-7
/proc/29581/stat : 29581 (plingeling) S 29579 29581 29387 0 -1 4202496 16286489 1250 0 0 3814260 3627 0 0 25 0 9 0 721309182 7592239104 1814776 33554432000 4194304 4417452 140734506391584 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/29581/statm: 1853574 1814776 180 55 0 1851460 0
[pid=29581/tid=29588] ppid=29579 vsize=7414296 CPUtime=4781.93 cores=0-7
/proc/29581/task/29588/stat : 29588 (plingeling) R 29579 29581 29387 0 -1 4202560 1757744 1250 0 0 477799 394 0 0 25 0 9 0 721309225 7592239104 1814776 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29581/tid=29589] ppid=29579 vsize=7414296 CPUtime=4708.28 cores=0-7
/proc/29581/task/29589/stat : 29589 (plingeling) R 29579 29581 29387 0 -1 4202560 1892800 1250 0 0 470446 382 0 0 25 0 9 0 721309225 7592239104 1814776 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29581/tid=29590] ppid=29579 vsize=7414296 CPUtime=4781.95 cores=0-7
/proc/29581/task/29590/stat : 29590 (plingeling) R 29579 29581 29387 0 -1 4202560 2390951 1250 0 0 477628 567 0 0 25 0 9 0 721309225 7592239104 1814776 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364614 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29581/tid=29591] ppid=29579 vsize=7414296 CPUtime=4781.93 cores=0-7
/proc/29581/task/29591/stat : 29591 (plingeling) R 29579 29581 29387 0 -1 4202560 632330 1250 0 0 478030 163 0 0 25 0 9 0 721309225 7592239104 1814776 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29581/tid=29592] ppid=29579 vsize=7414296 CPUtime=4781.19 cores=0-7
/proc/29581/task/29592/stat : 29592 (plingeling) R 29579 29581 29387 0 -1 4202560 2194041 1250 0 0 477535 584 0 0 25 0 9 0 721309225 7592239104 1814776 33554432000 4194304 4417452 140734506391584 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29581/tid=29593] ppid=29579 vsize=7414296 CPUtime=4781.63 cores=0-7
/proc/29581/task/29593/stat : 29593 (plingeling) R 29579 29581 29387 0 -1 4202560 2504468 1250 0 0 477655 508 0 0 25 0 9 0 721309225 7592239104 1814776 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29581/tid=29594] ppid=29579 vsize=7414296 CPUtime=4779.6 cores=0-7
/proc/29581/task/29594/stat : 29594 (plingeling) R 29579 29581 29387 0 -1 4202560 2734730 1250 0 0 477324 636 0 0 25 0 9 0 721309225 7592239104 1814776 33554432000 4194304 4417452 140734506391584 18446744073709551615 4357248 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29581/tid=29595] ppid=29579 vsize=7414296 CPUtime=4781.89 cores=0-7
/proc/29581/task/29595/stat : 29595 (plingeling) R 29579 29581 29387 0 -1 4202560 2153931 1250 0 0 477802 387 0 0 25 0 9 0 721309225 7592239104 1814776 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38178.9
Current children cumulated vsize (KiB) 7414296

[startup+4842.3 s]
/proc/loadavg: 8.17 8.12 8.02 9/466 29748
/proc/meminfo: memFree=2991088/32950928 swapFree=62427732/67111528
[pid=29581] ppid=29579 vsize=7260120 CPUtime=38657.9 cores=0-7
/proc/29581/stat : 29581 (plingeling) S 29579 29581 29387 0 -1 4202496 16354968 1250 0 0 3862156 3637 0 0 25 0 9 0 721309182 7434362880 1776232 33554432000 4194304 4417452 140734506391584 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/29581/statm: 1815030 1776232 180 55 0 1812916 0
[pid=29581/tid=29588] ppid=29579 vsize=7260120 CPUtime=4841.93 cores=0-7
/proc/29581/task/29588/stat : 29588 (plingeling) R 29579 29581 29387 0 -1 4202560 1757744 1250 0 0 483799 394 0 0 25 0 9 0 721309225 7434362880 1776232 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29581/tid=29589] ppid=29579 vsize=7260120 CPUtime=4767.34 cores=0-7
/proc/29581/task/29589/stat : 29589 (plingeling) R 29579 29581 29387 0 -1 4202560 1922678 1250 0 0 476349 385 0 0 25 0 9 0 721309225 7434362880 1776232 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29581/tid=29590] ppid=29579 vsize=7260120 CPUtime=4841.94 cores=0-7
/proc/29581/task/29590/stat : 29590 (plingeling) R 29579 29581 29387 0 -1 4202560 2407335 1250 0 0 483625 569 0 0 25 0 9 0 721309225 7434362880 1776232 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29581/tid=29591] ppid=29579 vsize=7260120 CPUtime=4841.93 cores=0-7
/proc/29581/task/29591/stat : 29591 (plingeling) R 29579 29581 29387 0 -1 4202560 632330 1250 0 0 484030 163 0 0 25 0 9 0 721309225 7434362880 1776232 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29581/tid=29592] ppid=29579 vsize=7260120 CPUtime=4841.19 cores=0-7
/proc/29581/task/29592/stat : 29592 (plingeling) R 29579 29581 29387 0 -1 4202560 2194041 1250 0 0 483535 584 0 0 25 0 9 0 721309225 7434362880 1776232 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29581/tid=29593] ppid=29579 vsize=7260120 CPUtime=4841.62 cores=0-7
/proc/29581/task/29593/stat : 29593 (plingeling) R 29579 29581 29387 0 -1 4202560 2509672 1250 0 0 483654 508 0 0 25 0 9 0 721309225 7434362880 1776232 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29581/tid=29594] ppid=29579 vsize=7260120 CPUtime=4839.58 cores=0-7
/proc/29581/task/29594/stat : 29594 (plingeling) R 29579 29581 29387 0 -1 4202560 2751743 1250 0 0 483318 640 0 0 25 0 9 0 721309225 7434362880 1776232 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29581/tid=29595] ppid=29579 vsize=7260120 CPUtime=4841.89 cores=0-7
/proc/29581/task/29595/stat : 29595 (plingeling) R 29579 29581 29387 0 -1 4202560 2153931 1250 0 0 483802 387 0 0 25 0 9 0 721309225 7434362880 1776232 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38657.9
Current children cumulated vsize (KiB) 7260120

[startup+4902.3 s]
/proc/loadavg: 8.13 8.11 8.02 9/466 29749
/proc/meminfo: memFree=3414808/32950928 swapFree=62427732/67111528
[pid=29581] ppid=29579 vsize=6837524 CPUtime=39137 cores=0-7
/proc/29581/stat : 29581 (plingeling) S 29579 29581 29387 0 -1 4202496 16506097 1250 0 0 3910031 3671 0 0 25 0 9 0 721309182 7001624576 1670583 33554432000 4194304 4417452 140734506391584 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/29581/statm: 1709381 1670583 180 55 0 1707267 0
[pid=29581/tid=29588] ppid=29579 vsize=6837524 CPUtime=4901.93 cores=0-7
/proc/29581/task/29588/stat : 29588 (plingeling) R 29579 29581 29387 0 -1 4202560 1793844 1250 0 0 489790 403 0 0 25 0 9 0 721309225 7001624576 1670583 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29581/tid=29589] ppid=29579 vsize=6837524 CPUtime=4826.43 cores=0-7
/proc/29581/task/29589/stat : 29589 (plingeling) R 29579 29581 29387 0 -1 4202560 1947079 1250 0 0 482255 388 0 0 25 0 9 0 721309225 7001624576 1670583 33554432000 4194304 4417452 140734506391584 18446744073709551615 4277864 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29581/tid=29590] ppid=29579 vsize=6837524 CPUtime=4901.95 cores=0-7
/proc/29581/task/29590/stat : 29590 (plingeling) R 29579 29581 29387 0 -1 4202560 2420487 1250 0 0 489623 572 0 0 25 0 9 0 721309225 7001624576 1670583 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364741 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29581/tid=29591] ppid=29579 vsize=6837524 CPUtime=4901.93 cores=0-7
/proc/29581/task/29591/stat : 29591 (plingeling) R 29579 29581 29387 0 -1 4202560 632330 1250 0 0 490030 163 0 0 25 0 9 0 721309225 7001624576 1670583 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29581/tid=29592] ppid=29579 vsize=6837524 CPUtime=4901.19 cores=0-7
/proc/29581/task/29592/stat : 29592 (plingeling) R 29579 29581 29387 0 -1 4202560 2214837 1250 0 0 489532 587 0 0 25 0 9 0 721309225 7001624576 1670583 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29581/tid=29593] ppid=29579 vsize=6837524 CPUtime=4901.63 cores=0-7
/proc/29581/task/29593/stat : 29593 (plingeling) R 29579 29581 29387 0 -1 4202560 2536899 1250 0 0 489645 518 0 0 25 0 9 0 721309225 7001624576 1670583 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29581/tid=29594] ppid=29579 vsize=6837524 CPUtime=4899.58 cores=0-7
/proc/29581/task/29594/stat : 29594 (plingeling) R 29579 29581 29387 0 -1 4202560 2780283 1250 0 0 489313 645 0 0 25 0 9 0 721309225 7001624576 1670583 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29581/tid=29595] ppid=29579 vsize=6837524 CPUtime=4901.91 cores=0-7
/proc/29581/task/29595/stat : 29595 (plingeling) R 29579 29581 29387 0 -1 4202560 2154844 1250 0 0 489802 389 0 0 25 0 9 0 721309225 7001624576 1670583 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364653 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39137
Current children cumulated vsize (KiB) 6837524

[startup+4962.3 s]
/proc/loadavg: 8.05 8.09 8.02 9/466 29751
/proc/meminfo: memFree=4253244/32950928 swapFree=62427732/67111528
[pid=29581] ppid=29579 vsize=5999872 CPUtime=39616.1 cores=0-7
/proc/29581/stat : 29581 (plingeling) S 29579 29581 29387 0 -1 4202496 16709644 1250 0 0 3957883 3724 0 0 25 0 9 0 721309182 6143868928 1461170 33554432000 4194304 4417452 140734506391584 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/29581/statm: 1499968 1461170 180 55 0 1497854 0
[pid=29581/tid=29588] ppid=29579 vsize=5999872 CPUtime=4961.92 cores=0-7
/proc/29581/task/29588/stat : 29588 (plingeling) R 29579 29581 29387 0 -1 4202560 1794684 1250 0 0 495789 403 0 0 25 0 9 0 721309225 6143868928 1461170 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29581/tid=29589] ppid=29579 vsize=5999872 CPUtime=4885.49 cores=0-7
/proc/29581/task/29589/stat : 29589 (plingeling) R 29579 29581 29387 0 -1 4202560 2031173 1250 0 0 488144 405 0 0 25 0 9 0 721309225 6143868928 1461170 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29581/tid=29590] ppid=29579 vsize=5999872 CPUtime=4961.95 cores=0-7
/proc/29581/task/29590/stat : 29590 (plingeling) R 29579 29581 29387 0 -1 4202560 2446983 1250 0 0 495613 582 0 0 25 0 9 0 721309225 6143868928 1461170 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29581/tid=29591] ppid=29579 vsize=5999872 CPUtime=4961.93 cores=0-7
/proc/29581/task/29591/stat : 29591 (plingeling) R 29579 29581 29387 0 -1 4202560 632330 1250 0 0 496030 163 0 0 25 0 9 0 721309225 6143868928 1461170 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29581/tid=29592] ppid=29579 vsize=5999872 CPUtime=4961.18 cores=0-7
/proc/29581/task/29592/stat : 29592 (plingeling) R 29579 29581 29387 0 -1 4202560 2243207 1250 0 0 495522 596 0 0 25 0 9 0 721309225 6143868928 1461170 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364572 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29581/tid=29593] ppid=29579 vsize=5999872 CPUtime=4961.63 cores=0-7
/proc/29581/task/29593/stat : 29593 (plingeling) R 29579 29581 29387 0 -1 4202560 2536899 1250 0 0 495645 518 0 0 25 0 9 0 721309225 6143868928 1461170 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29581/tid=29594] ppid=29579 vsize=5999872 CPUtime=4959.56 cores=0-7
/proc/29581/task/29594/stat : 29594 (plingeling) R 29579 29581 29387 0 -1 4202560 2829733 1250 0 0 495297 659 0 0 25 0 9 0 721309225 6143868928 1461170 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364735 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29581/tid=29595] ppid=29579 vsize=5999872 CPUtime=4961.91 cores=0-7
/proc/29581/task/29595/stat : 29595 (plingeling) R 29579 29581 29387 0 -1 4202560 2169141 1250 0 0 495800 391 0 0 25 0 9 0 721309225 6143868928 1461170 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39616.1
Current children cumulated vsize (KiB) 5999872



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5010.4 s]
/proc/loadavg: 8.09 8.09 8.02 10/466 29752
/proc/meminfo: memFree=3987148/32950928 swapFree=62427732/67111528
[pid=29581] ppid=29579 vsize=6265204 CPUtime=40000.4 cores=0-7
/proc/29581/stat : 29581 (plingeling) S 29579 29581 29387 0 -1 4202496 16852758 1250 0 0 3996290 3748 0 0 25 0 9 0 721309182 6415568896 1527503 33554432000 4194304 4417452 140734506391584 18446744073709551615 248101500901 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/29581/statm: 1566301 1527503 180 55 0 1564187 0
[pid=29581/tid=29588] ppid=29579 vsize=6265204 CPUtime=5010.06 cores=0-7
/proc/29581/task/29588/stat : 29588 (plingeling) R 29579 29581 29387 0 -1 4202560 1794684 1250 0 0 500603 403 0 0 25 0 9 0 721309225 6415568896 1527503 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29581/tid=29589] ppid=29579 vsize=6265204 CPUtime=4932.86 cores=0-7
/proc/29581/task/29589/stat : 29589 (plingeling) R 29579 29581 29387 0 -1 4202560 2102298 1250 0 0 492868 418 0 0 25 0 9 0 721309225 6415568896 1527503 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29581/tid=29590] ppid=29579 vsize=6265204 CPUtime=5010.08 cores=0-7
/proc/29581/task/29590/stat : 29590 (plingeling) R 29579 29581 29387 0 -1 4202560 2446983 1250 0 0 500426 582 0 0 25 0 9 0 721309225 6415568896 1527503 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=29581/tid=29591] ppid=29579 vsize=6265204 CPUtime=5010.07 cores=0-7
/proc/29581/task/29591/stat : 29591 (plingeling) R 29579 29581 29387 0 -1 4202560 632330 1250 0 0 500844 163 0 0 25 0 9 0 721309225 6415568896 1527503 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=29581/tid=29592] ppid=29579 vsize=6265204 CPUtime=5009.32 cores=0-7
/proc/29581/task/29592/stat : 29592 (plingeling) R 29579 29581 29387 0 -1 4202560 2253127 1250 0 0 500334 598 0 0 25 0 9 0 721309225 6415568896 1527503 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29581/tid=29593] ppid=29579 vsize=6265204 CPUtime=5009.76 cores=0-7
/proc/29581/task/29593/stat : 29593 (plingeling) R 29579 29581 29387 0 -1 4202560 2578124 1250 0 0 500452 524 0 0 25 0 9 0 721309225 6415568896 1527503 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29581/tid=29594] ppid=29579 vsize=6265204 CPUtime=5007.68 cores=0-7
/proc/29581/task/29594/stat : 29594 (plingeling) R 29579 29581 29387 0 -1 4202560 2829733 1250 0 0 500109 659 0 0 25 0 9 0 721309225 6415568896 1527503 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29581/tid=29595] ppid=29579 vsize=6265204 CPUtime=5010.03 cores=0-7
/proc/29581/task/29595/stat : 29595 (plingeling) R 29579 29581 29387 0 -1 4202560 2189985 1250 0 0 500610 393 0 0 25 0 9 0 721309225 6415568896 1527503 33554432000 4194304 4417452 140734506391584 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 6265204

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5010.6 s]
/proc/loadavg: 8.09 8.09 8.02 10/466 29752
/proc/meminfo: memFree=3987148/32950928 swapFree=62427732/67111528
[pid=29581] ppid=29579 vsize=0 CPUtime=40000.6 cores=0-7
/proc/29581/stat : 29581 (plingeling) Z 29579 29581 29387 0 -1 4203548 16852759 1250 0 0 3996295 3764 0 0 18 0 2 0 721309182 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 7 0 0 0
/proc/29581/statm: 0 0 0 0 0 0 0
[pid=29581/tid=29589] ppid=29579 vsize=0 CPUtime=4933.03 cores=0-7
/proc/29581/task/29589/stat : 29589 (plingeling) R 29579 29581 29387 0 -1 4203588 2102298 1250 0 0 492869 434 0 0 25 0 2 0 721309225 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 40000.6
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5010.66
CPU time (s): 40000.7
CPU user time (s): 39963
CPU system time (s): 37.7053
CPU usage (%): 798.311
Max. virtual memory (cumulated for all children) (KiB): 8304644

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39963
system time used= 37.7053
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 16854009
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 396
involuntary context switches= 193185

runsolver used 22.7335 second user time and 54.6907 second system time

The end

Launcher Data 

Begin job on node123 at 2011-05-22 22:44:54
IDJOB=3394458
IDBENCH=83173
IDSOLVER=1851
FILE ID=node123/3394458-1306097094
RUNJOBID= node123-1306094851-29401
PBS_JOBID= 13387962
Free space on /tmp= 73120 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/crafted/mosoi/289/289-unsat-17x27.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3394458-1306097094/watcher-3394458-1306097094 -o /tmp/evaluation-result-3394458-1306097094/solver-3394458-1306097094 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3394458-1306097094.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= b9a7e3013b1b74b08c8e421f15cb16c4
RANDOM SEED=598171352

node123.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      10108048 kB
Buffers:        183616 kB
Cached:        3569668 kB
SwapCached:      76116 kB
Active:        8184600 kB
Inactive:     14389840 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      10108048 kB
SwapTotal:    67111528 kB
SwapFree:     62427732 kB
Dirty:            4660 kB
Writeback:           0 kB
AnonPages:    18753688 kB
Mapped:          30916 kB
Slab:           118708 kB
PageTables:      77744 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 133350256 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73120 MiB
End job on node123 at 2011-05-23 00:08:26