Trace number 3372407

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40000.6 5010.57

General information on the benchmark

Nameapplication/rintanen/SATPlanning/
sokoban-sequential-p145-microban-sequential.060-NOTKNOWN.cnf
MD5SUMe44b288003a3eaf34673a2c0fb8c6f3e
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark1095.17
Satisfiable
(Un)Satisfiability was proved
Number of variables131484
Number of clauses2120376
Sum of the clauses size4841016
Maximum clause length78
Minimum clause length1
Number of clauses of size 1696
Number of clauses of size 22046240
Number of clauses of size 35760
Number of clauses of size 426640
Number of clauses of size 511520
Number of clauses of size over 529520

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3372407-1305437456/watcher-3372407-1305437456 -o /tmp/evaluation-result-3372407-1305437456/solver-3372407-1305437456 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3372407-1305437456.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.71 2.98 3.46 1/275 14657
/proc/meminfo: memFree=18767748/32951132 swapFree=67093824/67111528
[pid=14657] ppid=14655 vsize=8728 CPUtime=0 cores=0-7
/proc/14657/stat : 14657 (plingeling) S 14655 14657 14631 0 -1 4202496 215 0 0 0 0 0 0 0 18 0 1 0 655345347 8937472 146 33554432000 4194304 4417452 140734279657696 18446744073709551615 274664808192 0 0 4096 0 18446744071562362840 0 0 17 3 0 0 0
/proc/14657/statm: 2182 146 122 55 0 68 0

[startup+0.0266781 s]
/proc/loadavg: 2.71 2.98 3.46 1/275 14657
/proc/meminfo: memFree=18767748/32951132 swapFree=67093824/67111528
[pid=14657] ppid=14655 vsize=8728 CPUtime=0 cores=0-7
/proc/14657/stat : 14657 (plingeling) S 14655 14657 14631 0 -1 4202496 215 0 0 0 0 0 0 0 18 0 1 0 655345347 8937472 146 33554432000 4194304 4417452 140734279657696 18446744073709551615 274664808192 0 0 4096 0 18446744071562362840 0 0 17 3 0 0 0
/proc/14657/statm: 2182 146 122 55 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8728

[startup+0.100655 s]
/proc/loadavg: 2.71 2.98 3.46 1/275 14657
/proc/meminfo: memFree=18767748/32951132 swapFree=67093824/67111528
[pid=14657] ppid=14655 vsize=8728 CPUtime=0 cores=0-7
/proc/14657/stat : 14657 (plingeling) S 14655 14657 14631 0 -1 4202496 215 0 0 0 0 0 0 0 18 0 1 0 655345347 8937472 146 33554432000 4194304 4417452 140734279657696 18446744073709551615 274664808192 0 0 4096 0 18446744071562362840 0 0 17 3 0 0 0
/proc/14657/statm: 2182 146 122 55 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8728

[startup+0.300618 s]
/proc/loadavg: 2.71 2.98 3.46 1/275 14657
/proc/meminfo: memFree=18767748/32951132 swapFree=67093824/67111528
[pid=14657] ppid=14655 vsize=23220 CPUtime=0.2 cores=0-7
/proc/14657/stat : 14657 (plingeling) R 14655 14657 14631 0 -1 4202496 3887 1253 0 0 14 0 0 6 18 0 1 0 655345347 23777280 3813 33554432000 4194304 4417452 140734279657696 18446744073709551615 274664808192 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/14657/statm: 5805 3813 149 55 0 3691 0
Current children cumulated CPU time (s) 0.2
Current children cumulated vsize (KiB) 23220

[startup+0.700562 s]
/proc/loadavg: 2.71 2.98 3.46 1/275 14657
/proc/meminfo: memFree=18767748/32951132 swapFree=67093824/67111528
[pid=14657] ppid=14655 vsize=72216 CPUtime=0.59 cores=0-7
/proc/14657/stat : 14657 (plingeling) R 14655 14657 14631 0 -1 4202496 16160 1253 0 0 52 1 0 6 18 0 1 0 655345347 73949184 16078 33554432000 4194304 4417452 140734279657696 18446744073709551615 4231283 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/14657/statm: 18054 16078 149 55 0 15940 0
Current children cumulated CPU time (s) 0.59
Current children cumulated vsize (KiB) 72216

[startup+1.50042 s]
/proc/loadavg: 2.71 2.98 3.46 2/276 14663
/proc/meminfo: memFree=18670036/32951132 swapFree=67093824/67111528
[pid=14657] ppid=14655 vsize=127512 CPUtime=1.39 cores=0-7
/proc/14657/stat : 14657 (plingeling) R 14655 14657 14631 0 -1 4202496 29984 1253 0 0 130 3 0 6 19 0 1 0 655345347 130572288 29902 33554432000 4194304 4417452 140734279657696 18446744073709551615 4320705 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/14657/statm: 31878 29902 149 55 0 29764 0
Current children cumulated CPU time (s) 1.39
Current children cumulated vsize (KiB) 127512

[startup+3.10115 s]
/proc/loadavg: 2.71 2.98 3.46 3/276 14663
/proc/meminfo: memFree=18535628/32951132 swapFree=67093824/67111528
[pid=14657] ppid=14655 vsize=386084 CPUtime=3 cores=0-7
/proc/14657/stat : 14657 (plingeling) R 14655 14657 14631 0 -1 4202496 94627 1253 0 0 283 11 0 6 24 0 1 0 655345347 395350016 94545 33554432000 4194304 4417452 140734279657696 18446744073709551615 4231327 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/14657/statm: 96521 94545 149 55 0 94407 0
Current children cumulated CPU time (s) 3
Current children cumulated vsize (KiB) 386084

[startup+6.30055 s]
/proc/loadavg: 3.29 3.09 3.50 11/284 14671
/proc/meminfo: memFree=18357072/32951132 swapFree=67093824/67111528
[pid=14657] ppid=14655 vsize=545092 CPUtime=24.06 cores=0-7
/proc/14657/stat : 14657 (plingeling) S 14655 14657 14631 0 -1 4202496 164659 1253 0 0 2362 38 0 6 18 0 9 0 655345347 558174208 109297 33554432000 4194304 4417452 140734279657696 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/14657/statm: 136273 109297 177 55 0 134159 0
[pid=14657/tid=14664] ppid=14655 vsize=545092 CPUtime=2.84 cores=0-7
/proc/14657/task/14664/stat : 14664 (plingeling) R 14655 14657 14631 0 -1 4202560 6761 1253 0 0 276 2 0 6 23 0 9 0 655345687 558174208 109297 33554432000 4194304 4417452 140734279657696 18446744073709551615 4260714 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=14657/tid=14665] ppid=14655 vsize=545092 CPUtime=2.94 cores=0-7
/proc/14657/task/14665/stat : 14665 (plingeling) R 14655 14657 14631 0 -1 4202560 11052 1253 0 0 286 2 0 6 23 0 9 0 655345687 558174208 109297 33554432000 4194304 4417452 140734279657696 18446744073709551615 4260714 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=14657/tid=14666] ppid=14655 vsize=545092 CPUtime=2.88 cores=0-7
/proc/14657/task/14666/stat : 14666 (plingeling) R 14655 14657 14631 0 -1 4202560 2496 1253 0 0 282 0 0 6 24 0 9 0 655345687 558174208 109297 33554432000 4194304 4417452 140734279657696 18446744073709551615 4270920 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=14657/tid=14667] ppid=14655 vsize=545092 CPUtime=2.46 cores=0-7
/proc/14657/task/14667/stat : 14667 (plingeling) R 14655 14657 14631 0 -1 4202560 1791 1253 0 0 228 12 0 6 25 0 9 0 655345687 558174208 109297 33554432000 4194304 4417452 140734279657696 18446744073709551615 4364604 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14657/tid=14668] ppid=14655 vsize=545092 CPUtime=2.45 cores=0-7
/proc/14657/task/14668/stat : 14668 (plingeling) R 14655 14657 14631 0 -1 4202560 8288 1253 0 0 238 1 0 6 22 0 9 0 655345687 558174208 109297 33554432000 4194304 4417452 140734279657696 18446744073709551615 4255603 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=14657/tid=14669] ppid=14655 vsize=545092 CPUtime=2.77 cores=0-7
/proc/14657/task/14669/stat : 14669 (plingeling) R 14655 14657 14631 0 -1 4202560 9541 1253 0 0 269 2 0 6 23 0 9 0 655345687 558174208 109297 33554432000 4194304 4417452 140734279657696 18446744073709551615 4260137 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14657/tid=14670] ppid=14655 vsize=545092 CPUtime=1.99 cores=0-7
/proc/14657/task/14670/stat : 14670 (plingeling) R 14655 14657 14631 0 -1 4202560 3869 1253 0 0 192 1 0 6 25 0 9 0 655345687 558174208 109297 33554432000 4194304 4417452 140734279657696 18446744073709551615 4290926 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=14657/tid=14671] ppid=14655 vsize=545092 CPUtime=2.85 cores=0-7
/proc/14657/task/14671/stat : 14671 (plingeling) R 14655 14657 14631 0 -1 4202560 7462 1253 0 0 278 1 0 6 22 0 9 0 655345687 558174208 109297 33554432000 4194304 4417452 140734279657696 18446744073709551615 4271053 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 24.06
Current children cumulated vsize (KiB) 545092

[startup+12.7004 s]
/proc/loadavg: 3.83 3.21 3.53 9/284 14671
/proc/meminfo: memFree=18352500/32951132 swapFree=67093824/67111528
[pid=14657] ppid=14655 vsize=566748 CPUtime=70.9 cores=0-7
/proc/14657/stat : 14657 (plingeling) S 14655 14657 14631 0 -1 4202496 192114 1253 0 0 7028 56 0 6 18 0 9 0 655345347 580349952 103619 33554432000 4194304 4417452 140734279657696 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/14657/statm: 141687 103619 180 55 0 139573 0
[pid=14657/tid=14664] ppid=14655 vsize=566748 CPUtime=8.04 cores=0-7
/proc/14657/task/14664/stat : 14664 (plingeling) R 14655 14657 14631 0 -1 4202560 11324 1253 0 0 796 2 0 6 25 0 9 0 655345687 580349952 103619 33554432000 4194304 4417452 140734279657696 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=14657/tid=14665] ppid=14655 vsize=566748 CPUtime=9.34 cores=0-7
/proc/14657/task/14665/stat : 14665 (plingeling) R 14655 14657 14631 0 -1 4202560 11237 1253 0 0 926 2 0 6 25 0 9 0 655345687 580349952 103619 33554432000 4194304 4417452 140734279657696 18446744073709551615 4264940 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=14657/tid=14666] ppid=14655 vsize=566748 CPUtime=6.36 cores=0-7
/proc/14657/task/14666/stat : 14666 (plingeling) R 14655 14657 14631 0 -1 4202560 5201 1253 0 0 616 14 0 6 25 0 9 0 655345687 580349952 103619 33554432000 4194304 4417452 140734279657696 18446744073709551615 4236239 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=14657/tid=14667] ppid=14655 vsize=566748 CPUtime=8.75 cores=0-7
/proc/14657/task/14667/stat : 14667 (plingeling) R 14655 14657 14631 0 -1 4202560 1791 1253 0 0 857 12 0 6 25 0 9 0 655345687 580349952 103619 33554432000 4194304 4417452 140734279657696 18446744073709551615 4320867 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14657/tid=14668] ppid=14655 vsize=566748 CPUtime=8.7 cores=0-7
/proc/14657/task/14668/stat : 14668 (plingeling) R 14655 14657 14631 0 -1 4202560 14723 1253 0 0 862 2 0 6 25 0 9 0 655345687 580349952 103619 33554432000 4194304 4417452 140734279657696 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=14657/tid=14669] ppid=14655 vsize=566748 CPUtime=9.16 cores=0-7
/proc/14657/task/14669/stat : 14669 (plingeling) R 14655 14657 14631 0 -1 4202560 14621 1253 0 0 908 2 0 6 25 0 9 0 655345687 580349952 103619 33554432000 4194304 4417452 140734279657696 18446744073709551615 4364657 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14657/tid=14670] ppid=14655 vsize=566748 CPUtime=8.38 cores=0-7
/proc/14657/task/14670/stat : 14670 (plingeling) R 14655 14657 14631 0 -1 4202560 8254 1253 0 0 831 1 0 6 25 0 9 0 655345687 580349952 103619 33554432000 4194304 4417452 140734279657696 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=14657/tid=14671] ppid=14655 vsize=566748 CPUtime=9.25 cores=0-7
/proc/14657/task/14671/stat : 14671 (plingeling) R 14655 14657 14631 0 -1 4202560 11564 1253 0 0 917 2 0 6 25 0 9 0 655345687 580349952 103619 33554432000 4194304 4417452 140734279657696 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 70.9
Current children cumulated vsize (KiB) 566748

[startup+25.5012 s]
/proc/loadavg: 4.75 3.44 3.60 9/284 14671
/proc/meminfo: memFree=18336940/32951132 swapFree=67093824/67111528
[pid=14657] ppid=14655 vsize=577020 CPUtime=173.17 cores=0-7
/proc/14657/stat : 14657 (plingeling) S 14655 14657 14631 0 -1 4202496 196558 1253 0 0 17255 56 0 6 18 0 9 0 655345347 590868480 108058 33554432000 4194304 4417452 140734279657696 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/14657/statm: 144255 108058 180 55 0 142141 0
[pid=14657/tid=14664] ppid=14655 vsize=577020 CPUtime=20.73 cores=0-7
/proc/14657/task/14664/stat : 14664 (plingeling) R 14655 14657 14631 0 -1 4202560 11324 1253 0 0 2065 2 0 6 25 0 9 0 655345687 590868480 108058 33554432000 4194304 4417452 140734279657696 18446744073709551615 4260039 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=14657/tid=14665] ppid=14655 vsize=577020 CPUtime=22.14 cores=0-7
/proc/14657/task/14665/stat : 14665 (plingeling) R 14655 14657 14631 0 -1 4202560 11237 1253 0 0 2206 2 0 6 25 0 9 0 655345687 590868480 108058 33554432000 4194304 4417452 140734279657696 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=14657/tid=14666] ppid=14655 vsize=577020 CPUtime=19.16 cores=0-7
/proc/14657/task/14666/stat : 14666 (plingeling) R 14655 14657 14631 0 -1 4202560 5584 1253 0 0 1896 14 0 6 25 0 9 0 655345687 590868480 108058 33554432000 4194304 4417452 140734279657696 18446744073709551615 4364466 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=14657/tid=14667] ppid=14655 vsize=577020 CPUtime=21.56 cores=0-7
/proc/14657/task/14667/stat : 14667 (plingeling) R 14655 14657 14631 0 -1 4202560 4984 1253 0 0 2137 13 0 6 25 0 9 0 655345687 590868480 108058 33554432000 4194304 4417452 140734279657696 18446744073709551615 4364466 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14657/tid=14668] ppid=14655 vsize=577020 CPUtime=21.5 cores=0-7
/proc/14657/task/14668/stat : 14668 (plingeling) R 14655 14657 14631 0 -1 4202560 14979 1253 0 0 2142 2 0 6 25 0 9 0 655345687 590868480 108058 33554432000 4194304 4417452 140734279657696 18446744073709551615 4366543 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=14657/tid=14669] ppid=14655 vsize=577020 CPUtime=21.97 cores=0-7
/proc/14657/task/14669/stat : 14669 (plingeling) R 14655 14657 14631 0 -1 4202560 14815 1253 0 0 2188 3 0 6 25 0 9 0 655345687 590868480 108058 33554432000 4194304 4417452 140734279657696 18446744073709551615 4364446 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14657/tid=14670] ppid=14655 vsize=577020 CPUtime=21.18 cores=0-7
/proc/14657/task/14670/stat : 14670 (plingeling) R 14655 14657 14631 0 -1 4202560 8544 1253 0 0 2111 1 0 6 25 0 9 0 655345687 590868480 108058 33554432000 4194304 4417452 140734279657696 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=14657/tid=14671] ppid=14655 vsize=577020 CPUtime=22.05 cores=0-7
/proc/14657/task/14671/stat : 14671 (plingeling) R 14655 14657 14631 0 -1 4202560 11692 1253 0 0 2197 2 0 6 25 0 9 0 655345687 590868480 108058 33554432000 4194304 4417452 140734279657696 18446744073709551615 4366063 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 173.17
Current children cumulated vsize (KiB) 577020

[startup+51.1008 s]
/proc/loadavg: 5.86 3.81 3.72 9/284 14672
/proc/meminfo: memFree=18277316/32951132 swapFree=67093824/67111528
[pid=14657] ppid=14655 vsize=644732 CPUtime=377.67 cores=0-7
/proc/14657/stat : 14657 (plingeling) S 14655 14657 14631 0 -1 4202496 213968 1253 0 0 37699 62 0 6 18 0 9 0 655345347 660205568 122823 33554432000 4194304 4417452 140734279657696 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/14657/statm: 161183 122823 180 55 0 159069 0
[pid=14657/tid=14664] ppid=14655 vsize=644732 CPUtime=46.09 cores=0-7
/proc/14657/task/14664/stat : 14664 (plingeling) R 14655 14657 14631 0 -1 4202560 12849 1253 0 0 4601 2 0 6 25 0 9 0 655345687 660205568 122823 33554432000 4194304 4417452 140734279657696 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=14657/tid=14665] ppid=14655 vsize=644732 CPUtime=47.69 cores=0-7
/proc/14657/task/14665/stat : 14665 (plingeling) R 14655 14657 14631 0 -1 4202560 12496 1253 0 0 4757 6 0 6 25 0 9 0 655345687 660205568 122823 33554432000 4194304 4417452 140734279657696 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=14657/tid=14666] ppid=14655 vsize=644732 CPUtime=44.76 cores=0-7
/proc/14657/task/14666/stat : 14666 (plingeling) R 14655 14657 14631 0 -1 4202560 5584 1253 0 0 4456 14 0 6 18 0 9 0 655345687 660205568 122823 33554432000 4194304 4417452 140734279657696 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=14657/tid=14667] ppid=14655 vsize=644732 CPUtime=47.13 cores=0-7
/proc/14657/task/14667/stat : 14667 (plingeling) R 14655 14657 14631 0 -1 4202560 9014 1253 0 0 4694 13 0 6 25 0 9 0 655345687 660205568 122823 33554432000 4194304 4417452 140734279657696 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14657/tid=14668] ppid=14655 vsize=644732 CPUtime=47.1 cores=0-7
/proc/14657/task/14668/stat : 14668 (plingeling) R 14655 14657 14631 0 -1 4202560 20312 1253 0 0 4701 3 0 6 25 0 9 0 655345687 660205568 122823 33554432000 4194304 4417452 140734279657696 18446744073709551615 4364427 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=14657/tid=14669] ppid=14655 vsize=644732 CPUtime=47.57 cores=0-7
/proc/14657/task/14669/stat : 14669 (plingeling) R 14655 14657 14631 0 -1 4202560 16214 1253 0 0 4748 3 0 6 25 0 9 0 655345687 660205568 122823 33554432000 4194304 4417452 140734279657696 18446744073709551615 4353557 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14657/tid=14670] ppid=14655 vsize=644732 CPUtime=46.77 cores=0-7
/proc/14657/task/14670/stat : 14670 (plingeling) R 14655 14657 14631 0 -1 4202560 10111 1253 0 0 4670 1 0 6 25 0 9 0 655345687 660205568 122823 33554432000 4194304 4417452 140734279657696 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=14657/tid=14671] ppid=14655 vsize=644732 CPUtime=47.65 cores=0-7
/proc/14657/task/14671/stat : 14671 (plingeling) R 14655 14657 14631 0 -1 4202560 13989 1253 0 0 4757 2 0 6 25 0 9 0 655345687 660205568 122823 33554432000 4194304 4417452 140734279657696 18446744073709551615 4353830 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 377.67
Current children cumulated vsize (KiB) 644732

[startup+102.311 s]
/proc/loadavg: 7.07 4.45 3.94 9/284 14673
/proc/meminfo: memFree=18212400/32951132 swapFree=67093824/67111528
[pid=14657] ppid=14655 vsize=707052 CPUtime=786.85 cores=0-7
/proc/14657/stat : 14657 (plingeling) S 14655 14657 14631 0 -1 4202496 232463 1253 0 0 78614 65 0 6 18 0 9 0 655345347 724021248 139370 33554432000 4194304 4417452 140734279657696 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/14657/statm: 176763 139370 180 55 0 174649 0
[pid=14657/tid=14664] ppid=14655 vsize=707052 CPUtime=96.84 cores=0-7
/proc/14657/task/14664/stat : 14664 (plingeling) R 14655 14657 14631 0 -1 4202560 13567 1253 0 0 9675 3 0 6 25 0 9 0 655345687 724021248 139370 33554432000 4194304 4417452 140734279657696 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=14657/tid=14665] ppid=14655 vsize=707052 CPUtime=98.89 cores=0-7
/proc/14657/task/14665/stat : 14665 (plingeling) R 14655 14657 14631 0 -1 4202560 13822 1253 0 0 9877 6 0 6 25 0 9 0 655345687 724021248 139370 33554432000 4194304 4417452 140734279657696 18446744073709551615 4364653 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=14657/tid=14666] ppid=14655 vsize=707052 CPUtime=95.98 cores=0-7
/proc/14657/task/14666/stat : 14666 (plingeling) R 14655 14657 14631 0 -1 4202560 10971 1253 0 0 9577 15 0 6 25 0 9 0 655345687 724021248 139370 33554432000 4194304 4417452 140734279657696 18446744073709551615 4353557 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=14657/tid=14667] ppid=14655 vsize=707052 CPUtime=98.34 cores=0-7
/proc/14657/task/14667/stat : 14667 (plingeling) R 14655 14657 14631 0 -1 4202560 15885 1253 0 0 9814 14 0 6 25 0 9 0 655345687 724021248 139370 33554432000 4194304 4417452 140734279657696 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14657/tid=14668] ppid=14655 vsize=707052 CPUtime=98.32 cores=0-7
/proc/14657/task/14668/stat : 14668 (plingeling) R 14655 14657 14631 0 -1 4202560 24022 1253 0 0 9822 4 0 6 25 0 9 0 655345687 724021248 139370 33554432000 4194304 4417452 140734279657696 18446744073709551615 4275628 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=14657/tid=14669] ppid=14655 vsize=707052 CPUtime=98.78 cores=0-7
/proc/14657/task/14669/stat : 14669 (plingeling) R 14655 14657 14631 0 -1 4202560 16697 1253 0 0 9869 3 0 6 25 0 9 0 655345687 724021248 139370 33554432000 4194304 4417452 140734279657696 18446744073709551615 4366523 0 0 4096 17442 0 0 0 -1 0 0 0 0

################
# More data... #
################

[pid=14657/tid=14666] ppid=14655 vsize=9156368 CPUtime=4714.39 cores=0-7
/proc/14657/task/14666/stat : 14666 (plingeling) R 14655 14657 14631 0 -1 4202560 1398190 1253 0 0 471145 288 0 6 25 0 9 0 655345687 9376120832 2250738 33554432000 4194304 4417452 140734279657696 18446744073709551615 4357207 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14657/tid=14667] ppid=14655 vsize=9156368 CPUtime=4716.87 cores=0-7
/proc/14657/task/14667/stat : 14667 (plingeling) R 14655 14657 14631 0 -1 4202560 2581967 1253 0 0 471115 566 0 6 25 0 9 0 655345687 9376120832 2250738 33554432000 4194304 4417452 140734279657696 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14657/tid=14668] ppid=14655 vsize=9156368 CPUtime=4692.14 cores=0-7
/proc/14657/task/14668/stat : 14668 (plingeling) R 14655 14657 14631 0 -1 4202560 1129312 1253 0 0 469017 191 0 6 25 0 9 0 655345687 9376120832 2250738 33554432000 4194304 4417452 140734279657696 18446744073709551615 4236975 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=14657/tid=14669] ppid=14655 vsize=9156368 CPUtime=4718.72 cores=0-7
/proc/14657/task/14669/stat : 14669 (plingeling) R 14655 14657 14631 0 -1 4202560 1418599 1253 0 0 471612 254 0 6 25 0 9 0 655345687 9376120832 2250738 33554432000 4194304 4417452 140734279657696 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14657/tid=14670] ppid=14655 vsize=9156368 CPUtime=4717.9 cores=0-7
/proc/14657/task/14670/stat : 14670 (plingeling) R 14655 14657 14631 0 -1 4202560 1350805 1253 0 0 471549 235 0 6 25 0 9 0 655345687 9376120832 2250738 33554432000 4194304 4417452 140734279657696 18446744073709551615 4263127 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=14657/tid=14671] ppid=14655 vsize=9156368 CPUtime=4717.14 cores=0-7
/proc/14657/task/14671/stat : 14671 (plingeling) R 14655 14657 14631 0 -1 4202560 1284911 1253 0 0 471446 262 0 6 25 0 9 0 655345687 9376120832 2250738 33554432000 4194304 4417452 140734279657696 18446744073709551615 4364548 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 37699.8
Current children cumulated vsize (KiB) 9156368

[startup+4782.3 s]
/proc/loadavg: 8.03 8.05 8.00 9/284 14810
/proc/meminfo: memFree=10564600/32951132 swapFree=67093824/67111528
[pid=14657] ppid=14655 vsize=8343468 CPUtime=38179.1 cores=0-7
/proc/14657/stat : 14657 (plingeling) S 14655 14657 14631 0 -1 4202496 12442679 1253 0 0 3815464 2442 0 6 18 0 9 0 655345347 8543711232 2047513 33554432000 4194304 4417452 140734279657696 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/14657/statm: 2085867 2047513 180 55 0 2083753 0
[pid=14657/tid=14664] ppid=14655 vsize=8343468 CPUtime=4760.93 cores=0-7
/proc/14657/task/14664/stat : 14664 (plingeling) R 14655 14657 14631 0 -1 4202560 1504011 1253 0 0 475808 279 0 6 25 0 9 0 655345687 8543711232 2047513 33554432000 4194304 4417452 140734279657696 18446744073709551615 4357283 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=14657/tid=14665] ppid=14655 vsize=8343468 CPUtime=4778.75 cores=0-7
/proc/14657/task/14665/stat : 14665 (plingeling) R 14655 14657 14631 0 -1 4202560 1575943 1253 0 0 477544 325 0 6 25 0 9 0 655345687 8543711232 2047513 33554432000 4194304 4417452 140734279657696 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=14657/tid=14666] ppid=14655 vsize=8343468 CPUtime=4774.36 cores=0-7
/proc/14657/task/14666/stat : 14666 (plingeling) R 14655 14657 14631 0 -1 4202560 1408415 1253 0 0 477136 294 0 6 25 0 9 0 655345687 8543711232 2047513 33554432000 4194304 4417452 140734279657696 18446744073709551615 4358748 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14657/tid=14667] ppid=14655 vsize=8343468 CPUtime=4776.83 cores=0-7
/proc/14657/task/14667/stat : 14667 (plingeling) R 14655 14657 14631 0 -1 4202560 2581967 1253 0 0 477111 566 0 6 25 0 9 0 655345687 8543711232 2047513 33554432000 4194304 4417452 140734279657696 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14657/tid=14668] ppid=14655 vsize=8343468 CPUtime=4751.6 cores=0-7
/proc/14657/task/14668/stat : 14668 (plingeling) R 14655 14657 14631 0 -1 4202560 1132706 1253 0 0 474961 193 0 6 25 0 9 0 655345687 8543711232 2047513 33554432000 4194304 4417452 140734279657696 18446744073709551615 18446744073699065863 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=14657/tid=14669] ppid=14655 vsize=8343468 CPUtime=4778.72 cores=0-7
/proc/14657/task/14669/stat : 14669 (plingeling) R 14655 14657 14631 0 -1 4202560 1440686 1253 0 0 477609 257 0 6 25 0 9 0 655345687 8543711232 2047513 33554432000 4194304 4417452 140734279657696 18446744073709551615 4364604 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14657/tid=14670] ppid=14655 vsize=8343468 CPUtime=4777.89 cores=0-7
/proc/14657/task/14670/stat : 14670 (plingeling) R 14655 14657 14631 0 -1 4202560 1400641 1253 0 0 477536 247 0 6 25 0 9 0 655345687 8543711232 2047513 33554432000 4194304 4417452 140734279657696 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=14657/tid=14671] ppid=14655 vsize=8343468 CPUtime=4777.13 cores=0-7
/proc/14657/task/14671/stat : 14671 (plingeling) R 14655 14657 14631 0 -1 4202560 1284911 1253 0 0 477445 262 0 6 25 0 9 0 655345687 8543711232 2047513 33554432000 4194304 4417452 140734279657696 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38179.1
Current children cumulated vsize (KiB) 8343468

[startup+4842.3 s]
/proc/loadavg: 8.01 8.04 8.00 9/284 14812
/proc/meminfo: memFree=11781528/32951132 swapFree=67093824/67111528
[pid=14657] ppid=14655 vsize=7128424 CPUtime=38658.5 cores=0-7
/proc/14657/stat : 14657 (plingeling) S 14655 14657 14631 0 -1 4202496 12696030 1253 0 0 3863335 2511 0 6 18 0 9 0 655345347 7299506176 1743752 33554432000 4194304 4417452 140734279657696 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/14657/statm: 1782106 1743752 180 55 0 1779992 0
[pid=14657/tid=14664] ppid=14655 vsize=7128424 CPUtime=4820.93 cores=0-7
/proc/14657/task/14664/stat : 14664 (plingeling) R 14655 14657 14631 0 -1 4202560 1525558 1253 0 0 481801 286 0 6 25 0 9 0 655345687 7299506176 1743752 33554432000 4194304 4417452 140734279657696 18446744073709551615 4281668 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=14657/tid=14665] ppid=14655 vsize=7128424 CPUtime=4838.76 cores=0-7
/proc/14657/task/14665/stat : 14665 (plingeling) R 14655 14657 14631 0 -1 4202560 1666596 1253 0 0 483521 349 0 6 25 0 9 0 655345687 7299506176 1743752 33554432000 4194304 4417452 140734279657696 18446744073709551615 4265443 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=14657/tid=14666] ppid=14655 vsize=7128424 CPUtime=4834.34 cores=0-7
/proc/14657/task/14666/stat : 14666 (plingeling) R 14655 14657 14631 0 -1 4202560 1419191 1253 0 0 483132 296 0 6 25 0 9 0 655345687 7299506176 1743752 33554432000 4194304 4417452 140734279657696 18446744073709551615 4364711 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14657/tid=14667] ppid=14655 vsize=7128424 CPUtime=4836.81 cores=0-7
/proc/14657/task/14667/stat : 14667 (plingeling) R 14655 14657 14631 0 -1 4202560 2670767 1253 0 0 483083 592 0 6 25 0 9 0 655345687 7299506176 1743752 33554432000 4194304 4417452 140734279657696 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14657/tid=14668] ppid=14655 vsize=7128424 CPUtime=4811.05 cores=0-7
/proc/14657/task/14668/stat : 14668 (plingeling) R 14655 14657 14631 0 -1 4202560 1136101 1253 0 0 480905 194 0 6 25 0 9 0 655345687 7299506176 1743752 33554432000 4194304 4417452 140734279657696 18446744073709551615 4357207 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=14657/tid=14669] ppid=14655 vsize=7128424 CPUtime=4838.71 cores=0-7
/proc/14657/task/14669/stat : 14669 (plingeling) R 14655 14657 14631 0 -1 4202560 1453486 1253 0 0 483607 258 0 6 25 0 9 0 655345687 7299506176 1743752 33554432000 4194304 4417452 140734279657696 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14657/tid=14670] ppid=14655 vsize=7128424 CPUtime=4837.89 cores=0-7
/proc/14657/task/14670/stat : 14670 (plingeling) R 14655 14657 14631 0 -1 4202560 1409533 1253 0 0 483533 250 0 6 25 0 9 0 655345687 7299506176 1743752 33554432000 4194304 4417452 140734279657696 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=14657/tid=14671] ppid=14655 vsize=7128424 CPUtime=4837.14 cores=0-7
/proc/14657/task/14671/stat : 14671 (plingeling) R 14655 14657 14631 0 -1 4202560 1301399 1253 0 0 483440 268 0 6 25 0 9 0 655345687 7299506176 1743752 33554432000 4194304 4417452 140734279657696 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38658.5
Current children cumulated vsize (KiB) 7128424

[startup+4902.3 s]
/proc/loadavg: 8.05 8.04 8.00 9/284 14813
/proc/meminfo: memFree=10276264/32951132 swapFree=67093824/67111528
[pid=14657] ppid=14655 vsize=8630032 CPUtime=39137.9 cores=0-7
/proc/14657/stat : 14657 (plingeling) S 14655 14657 14631 0 -1 4202496 13141397 1253 0 0 3911184 2602 0 6 18 0 9 0 655345347 8837152768 2119155 33554432000 4194304 4417452 140734279657696 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/14657/statm: 2157508 2119155 180 55 0 2155394 0
[pid=14657/tid=14664] ppid=14655 vsize=8630032 CPUtime=4880.94 cores=0-7
/proc/14657/task/14664/stat : 14664 (plingeling) R 14655 14657 14631 0 -1 4202560 1552821 1253 0 0 487796 292 0 6 25 0 9 0 655345687 8837152768 2119155 33554432000 4194304 4417452 140734279657696 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=14657/tid=14665] ppid=14655 vsize=8630032 CPUtime=4898.76 cores=0-7
/proc/14657/task/14665/stat : 14665 (plingeling) R 14655 14657 14631 0 -1 4202560 1666596 1253 0 0 489521 349 0 6 25 0 9 0 655345687 8837152768 2119155 33554432000 4194304 4417452 140734279657696 18446744073709551615 4270130 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=14657/tid=14666] ppid=14655 vsize=8630032 CPUtime=4893.83 cores=0-7
/proc/14657/task/14666/stat : 14666 (plingeling) R 14655 14657 14631 0 -1 4202560 1508192 1253 0 0 489064 313 0 6 25 0 9 0 655345687 8837152768 2119155 33554432000 4194304 4417452 140734279657696 18446744073709551615 4357283 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=14657/tid=14667] ppid=14655 vsize=8630032 CPUtime=4896.78 cores=0-7
/proc/14657/task/14667/stat : 14667 (plingeling) R 14655 14657 14631 0 -1 4202560 2690582 1253 0 0 489074 598 0 6 25 0 9 0 655345687 8837152768 2119155 33554432000 4194304 4417452 140734279657696 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14657/tid=14668] ppid=14655 vsize=8630032 CPUtime=4870.99 cores=0-7
/proc/14657/task/14668/stat : 14668 (plingeling) R 14655 14657 14631 0 -1 4202560 1187568 1253 0 0 486893 200 0 6 25 0 9 0 655345687 8837152768 2119155 33554432000 4194304 4417452 140734279657696 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14657/tid=14669] ppid=14655 vsize=8630032 CPUtime=4898.71 cores=0-7
/proc/14657/task/14669/stat : 14669 (plingeling) R 14655 14657 14631 0 -1 4202560 1502464 1253 0 0 489598 267 0 6 25 0 9 0 655345687 8837152768 2119155 33554432000 4194304 4417452 140734279657696 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14657/tid=14670] ppid=14655 vsize=8630032 CPUtime=4897.88 cores=0-7
/proc/14657/task/14670/stat : 14670 (plingeling) R 14655 14657 14631 0 -1 4202560 1583615 1253 0 0 489491 291 0 6 25 0 9 0 655345687 8837152768 2119155 33554432000 4194304 4417452 140734279657696 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=14657/tid=14671] ppid=14655 vsize=8630032 CPUtime=4897.13 cores=0-7
/proc/14657/task/14671/stat : 14671 (plingeling) R 14655 14657 14631 0 -1 4202560 1336160 1253 0 0 489433 274 0 6 25 0 9 0 655345687 8837152768 2119155 33554432000 4194304 4417452 140734279657696 18446744073709551615 4357283 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39137.9
Current children cumulated vsize (KiB) 8630032

[startup+4962.3 s]
/proc/loadavg: 8.06 8.05 8.00 9/284 14815
/proc/meminfo: memFree=9832996/32951132 swapFree=67093824/67111528
[pid=14657] ppid=14655 vsize=9072392 CPUtime=39617.3 cores=0-7
/proc/14657/stat : 14657 (plingeling) S 14655 14657 14631 0 -1 4202496 13399557 1253 0 0 3959064 2663 0 6 18 0 9 0 655345347 9290129408 2229745 33554432000 4194304 4417452 140734279657696 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/14657/statm: 2268098 2229745 180 55 0 2265984 0
[pid=14657/tid=14664] ppid=14655 vsize=9072392 CPUtime=4940.93 cores=0-7
/proc/14657/task/14664/stat : 14664 (plingeling) R 14655 14657 14631 0 -1 4202560 1578554 1253 0 0 493790 297 0 6 25 0 9 0 655345687 9290129408 2229745 33554432000 4194304 4417452 140734279657696 18446744073709551615 274664866218 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=14657/tid=14665] ppid=14655 vsize=9072392 CPUtime=4958.76 cores=0-7
/proc/14657/task/14665/stat : 14665 (plingeling) R 14655 14657 14631 0 -1 4202560 1808603 1253 0 0 495483 387 0 6 25 0 9 0 655345687 9290129408 2229745 33554432000 4194304 4417452 140734279657696 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=14657/tid=14666] ppid=14655 vsize=9072392 CPUtime=4953.28 cores=0-7
/proc/14657/task/14666/stat : 14666 (plingeling) R 14655 14657 14631 0 -1 4202560 1508192 1253 0 0 495009 313 0 6 25 0 9 0 655345687 9290129408 2229745 33554432000 4194304 4417452 140734279657696 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=14657/tid=14667] ppid=14655 vsize=9072392 CPUtime=4956.76 cores=0-7
/proc/14657/task/14667/stat : 14667 (plingeling) R 14655 14657 14631 0 -1 4202560 2690582 1253 0 0 495072 598 0 6 25 0 9 0 655345687 9290129408 2229745 33554432000 4194304 4417452 140734279657696 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14657/tid=14668] ppid=14655 vsize=9072392 CPUtime=4930.97 cores=0-7
/proc/14657/task/14668/stat : 14668 (plingeling) R 14655 14657 14631 0 -1 4202560 1225981 1253 0 0 492884 207 0 6 25 0 9 0 655345687 9290129408 2229745 33554432000 4194304 4417452 140734279657696 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14657/tid=14669] ppid=14655 vsize=9072392 CPUtime=4958.71 cores=0-7
/proc/14657/task/14669/stat : 14669 (plingeling) R 14655 14657 14631 0 -1 4202560 1513292 1253 0 0 495595 270 0 6 25 0 9 0 655345687 9290129408 2229745 33554432000 4194304 4417452 140734279657696 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14657/tid=14670] ppid=14655 vsize=9072392 CPUtime=4957.88 cores=0-7
/proc/14657/task/14670/stat : 14670 (plingeling) R 14655 14657 14631 0 -1 4202560 1583615 1253 0 0 495491 291 0 6 25 0 9 0 655345687 9290129408 2229745 33554432000 4194304 4417452 140734279657696 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=14657/tid=14671] ppid=14655 vsize=9072392 CPUtime=4957.13 cores=0-7
/proc/14657/task/14671/stat : 14671 (plingeling) R 14655 14657 14631 0 -1 4202560 1377339 1253 0 0 495426 281 0 6 25 0 9 0 655345687 9290129408 2229745 33554432000 4194304 4417452 140734279657696 18446744073709551615 4357902 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39617.3
Current children cumulated vsize (KiB) 9072392



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5010.2 s]
/proc/loadavg: 8.07 8.05 8.00 9/284 14816
/proc/meminfo: memFree=10027256/32951132 swapFree=67093824/67111528
[pid=14657] ppid=14655 vsize=8877964 CPUtime=40000.1 cores=0-7
/proc/14657/stat : 14657 (plingeling) S 14655 14657 14631 0 -1 4202496 13507442 1253 0 0 3997317 2689 0 6 18 0 9 0 655345347 9091035136 2181138 33554432000 4194304 4417452 140734279657696 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/14657/statm: 2219491 2181138 180 55 0 2217377 0
[pid=14657/tid=14664] ppid=14655 vsize=8877964 CPUtime=4988.85 cores=0-7
/proc/14657/task/14664/stat : 14664 (plingeling) R 14655 14657 14631 0 -1 4202560 1649752 1253 0 0 498566 313 0 6 25 0 9 0 655345687 9091035136 2181138 33554432000 4194304 4417452 140734279657696 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=14657/tid=14665] ppid=14655 vsize=8877964 CPUtime=5006.67 cores=0-7
/proc/14657/task/14665/stat : 14665 (plingeling) R 14655 14657 14631 0 -1 4202560 1808603 1253 0 0 500272 389 0 6 25 0 9 0 655345687 9091035136 2181138 33554432000 4194304 4417452 140734279657696 18446744073709551615 4305871 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=14657/tid=14666] ppid=14655 vsize=8877964 CPUtime=5000.76 cores=0-7
/proc/14657/task/14666/stat : 14666 (plingeling) R 14655 14657 14631 0 -1 4202560 1508192 1253 0 0 499756 314 0 6 25 0 9 0 655345687 9091035136 2181138 33554432000 4194304 4417452 140734279657696 18446744073709551615 4265465 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=14657/tid=14667] ppid=14655 vsize=8877964 CPUtime=5004.65 cores=0-7
/proc/14657/task/14667/stat : 14667 (plingeling) R 14655 14657 14631 0 -1 4202560 2690582 1253 0 0 499861 598 0 6 25 0 9 0 655345687 9091035136 2181138 33554432000 4194304 4417452 140734279657696 18446744073709551615 4236906 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14657/tid=14668] ppid=14655 vsize=8877964 CPUtime=4978.87 cores=0-7
/proc/14657/task/14668/stat : 14668 (plingeling) R 14655 14657 14631 0 -1 4202560 1242173 1253 0 0 497669 212 0 6 25 0 9 0 655345687 9091035136 2181138 33554432000 4194304 4417452 140734279657696 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14657/tid=14669] ppid=14655 vsize=8877964 CPUtime=5006.63 cores=0-7
/proc/14657/task/14669/stat : 14669 (plingeling) R 14655 14657 14631 0 -1 4202560 1533787 1253 0 0 500383 274 0 6 25 0 9 0 655345687 9091035136 2181138 33554432000 4194304 4417452 140734279657696 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14657/tid=14670] ppid=14655 vsize=8877964 CPUtime=5005.79 cores=0-7
/proc/14657/task/14670/stat : 14670 (plingeling) R 14655 14657 14631 0 -1 4202560 1583615 1253 0 0 500282 291 0 6 25 0 9 0 655345687 9091035136 2181138 33554432000 4194304 4417452 140734279657696 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=14657/tid=14671] ppid=14655 vsize=8877964 CPUtime=5005.04 cores=0-7
/proc/14657/task/14671/stat : 14671 (plingeling) R 14655 14657 14631 0 -1 4202560 1377339 1253 0 0 500217 281 0 6 25 0 9 0 655345687 9091035136 2181138 33554432000 4194304 4417452 140734279657696 18446744073709551615 4236649 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 40000.1
Current children cumulated vsize (KiB) 8877964

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5010.4 s]
/proc/loadavg: 8.07 8.05 8.00 2/278 14817
/proc/meminfo: memFree=14310760/32951132 swapFree=67093824/67111528
[pid=14657] ppid=14655 vsize=0 CPUtime=40000.4 cores=0-7
/proc/14657/stat : 14657 (plingeling) Z 14655 14657 14631 0 -1 4203548 13507443 1253 0 0 3997324 2708 0 6 18 0 2 0 655345347 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/14657/statm: 0 0 0 0 0 0 0
[pid=14657/tid=14666] ppid=14655 vsize=0 CPUtime=5000.96 cores=0-7
/proc/14657/task/14666/stat : 14666 (plingeling) R 14655 14657 14631 0 -1 4203588 1508192 1253 0 0 499757 333 0 6 25 0 2 0 655345687 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 0

[startup+5010.51 s]
/proc/loadavg: 8.07 8.05 8.00 2/278 14817
/proc/meminfo: memFree=14310760/32951132 swapFree=67093824/67111528
[pid=14657] ppid=14655 vsize=0 CPUtime=40000.5 cores=0-7
/proc/14657/stat : 14657 (plingeling) Z 14655 14657 14631 0 -1 4203548 13507443 1253 0 0 3997324 2718 0 6 18 0 2 0 655345347 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/14657/statm: 0 0 0 0 0 0 0
[pid=14657/tid=14666] ppid=14655 vsize=0 CPUtime=5001.06 cores=0-7
/proc/14657/task/14666/stat : 14666 (plingeling) R 14655 14657 14631 0 -1 4203588 1508192 1253 0 0 499757 343 0 6 25 0 2 0 655345687 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40000.5
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5010.57
CPU time (s): 40000.6
CPU user time (s): 39973.2
CPU system time (s): 27.3148
CPU usage (%): 798.323
Max. virtual memory (cumulated for all children) (KiB): 9332728

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39973.2
system time used= 27.3148
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 13508696
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 685
involuntary context switches= 192686

runsolver used 12.909 second user time and 32.654 second system time

The end

Launcher Data 

Begin job on node131 at 2011-05-15 07:30:57
IDJOB=3372407
IDBENCH=82366
IDSOLVER=1851
FILE ID=node131/3372407-1305437456
RUNJOBID= node131-1305437456-14645
PBS_JOBID= 13324777
Free space on /tmp= 73676 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/application/rintanen/SATPlanning/sokoban-sequential-p145-microban-sequential.060-NOTKNOWN.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3372407-1305437456/watcher-3372407-1305437456 -o /tmp/evaluation-result-3372407-1305437456/solver-3372407-1305437456 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3372407-1305437456.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= e44b288003a3eaf34673a2c0fb8c6f3e
RANDOM SEED=1982126493

node131.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951132 kB
MemFree:      18768028 kB
Buffers:        282036 kB
Cached:        3924396 kB
SwapCached:       1504 kB
Active:       10754056 kB
Inactive:      3169160 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951132 kB
LowFree:      18768028 kB
SwapTotal:    67111528 kB
SwapFree:     67093824 kB
Dirty:           37092 kB
Writeback:           0 kB
AnonPages:     9715248 kB
Mapped:          26472 kB
Slab:           165580 kB
PageTables:      30436 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587092 kB
Committed_AS: 27992352 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296000 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73672 MiB
End job on node131 at 2011-05-15 08:54:29