Trace number 3372455

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40001.8 5017.15

General information on the benchmark

Nameapplication/rintanen/SATPlanning/
openstacks-sequencedstrips-nonadl-nonnegated-os-sequencedstrips-p30_1.025-NOTKNOWN.cnf
MD5SUM732b9c9b0d44d6a9e75489b2f5004165
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark2632.81
Satisfiable
(Un)Satisfiability was proved
Number of variables95456
Number of clauses477186
Sum of the clauses size1137161
Maximum clause length63
Minimum clause length1
Number of clauses of size 1211
Number of clauses of size 2470925
Number of clauses of size 31500
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 54550

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3372455-1305437663/watcher-3372455-1305437663 -o /tmp/evaluation-result-3372455-1305437663/solver-3372455-1305437663 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3372455-1305437663.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.48 7.86 7.15 1/310 18168
/proc/meminfo: memFree=11085648/32951124 swapFree=67093864/67111528
[pid=18168] ppid=18166 vsize=11768 CPUtime=0 cores=0-7
/proc/18168/stat : 18168 (plingeling) R 18166 18168 18068 0 -1 4202496 1000 1257 0 0 0 0 0 0 25 0 1 0 655368733 12050432 930 33554432000 4194304 4417452 140735831322192 18446744073709551615 212436016896 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/18168/statm: 2942 930 144 55 0 828 0

[startup+0.046077 s]
/proc/loadavg: 7.48 7.86 7.15 1/310 18168
/proc/meminfo: memFree=11085648/32951124 swapFree=67093864/67111528
[pid=18168] ppid=18166 vsize=17864 CPUtime=0.04 cores=0-7
/proc/18168/stat : 18168 (plingeling) R 18166 18168 18068 0 -1 4202496 2561 1257 0 0 4 0 0 0 25 0 1 0 655368733 18292736 2491 33554432000 4194304 4417452 140735831322192 18446744073709551615 4231283 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/18168/statm: 4466 2491 149 55 0 2352 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 17864

[startup+0.101062 s]
/proc/loadavg: 7.48 7.86 7.15 1/310 18168
/proc/meminfo: memFree=11085648/32951124 swapFree=67093864/67111528
[pid=18168] ppid=18166 vsize=26968 CPUtime=0.09 cores=0-7
/proc/18168/stat : 18168 (plingeling) R 18166 18168 18068 0 -1 4202496 4837 1257 0 0 9 0 0 0 25 0 1 0 655368733 27615232 4767 33554432000 4194304 4417452 140735831322192 18446744073709551615 4209566 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/18168/statm: 6742 4767 149 55 0 4628 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 26968

[startup+0.301026 s]
/proc/loadavg: 7.48 7.86 7.15 1/310 18168
/proc/meminfo: memFree=11085648/32951124 swapFree=67093864/67111528
[pid=18168] ppid=18166 vsize=62496 CPUtime=0.28 cores=0-7
/proc/18168/stat : 18168 (plingeling) R 18166 18168 18068 0 -1 4202496 13719 1257 0 0 27 1 0 0 25 0 1 0 655368733 63995904 13649 33554432000 4194304 4417452 140735831322192 18446744073709551615 212436016896 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/18168/statm: 15624 13649 149 55 0 13510 0
Current children cumulated CPU time (s) 0.28
Current children cumulated vsize (KiB) 62496

[startup+0.700952 s]
/proc/loadavg: 7.48 7.86 7.15 1/310 18168
/proc/meminfo: memFree=11085648/32951124 swapFree=67093864/67111528
[pid=18168] ppid=18166 vsize=128128 CPUtime=0.69 cores=0-7
/proc/18168/stat : 18168 (plingeling) R 18166 18168 18068 0 -1 4202496 30143 1257 0 0 66 3 0 0 25 0 1 0 655368733 131203072 30057 33554432000 4194304 4417452 140735831322192 18446744073709551615 4231321 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/18168/statm: 32032 30057 149 55 0 29918 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 128128

[startup+1.50077 s]
/proc/loadavg: 7.48 7.86 7.15 9/319 18182
/proc/meminfo: memFree=10969512/32951124 swapFree=67093864/67111528
[pid=18168] ppid=18166 vsize=271324 CPUtime=6.12 cores=0-7
/proc/18168/stat : 18168 (plingeling) S 18166 18168 18068 0 -1 4202496 50030 1257 0 0 606 6 0 0 18 0 9 0 655368733 277835776 30832 33554432000 4194304 4417452 140735831322192 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/18168/statm: 67831 30832 175 55 0 65717 0
[pid=18168/tid=18175] ppid=18166 vsize=271324 CPUtime=0.73 cores=0-7
/proc/18168/task/18175/stat : 18175 (plingeling) R 18166 18168 18068 0 -1 4202560 7032 1257 0 0 73 0 0 0 18 0 9 0 655368809 277835776 30832 33554432000 4194304 4417452 140735831322192 18446744073709551615 4284624 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=18168/tid=18176] ppid=18166 vsize=271324 CPUtime=0.52 cores=0-7
/proc/18168/task/18176/stat : 18176 (plingeling) R 18166 18168 18068 0 -1 4202560 524 1257 0 0 52 0 0 0 25 0 9 0 655368809 277835776 30832 33554432000 4194304 4417452 140735831322192 18446744073709551615 4284192 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=18168/tid=18177] ppid=18166 vsize=271324 CPUtime=0.64 cores=0-7
/proc/18168/task/18177/stat : 18177 (plingeling) R 18166 18168 18068 0 -1 4202560 1433 1257 0 0 64 0 0 0 25 0 9 0 655368809 277835776 30832 33554432000 4194304 4417452 140735831322192 18446744073709551615 4306602 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18168/tid=18178] ppid=18166 vsize=271324 CPUtime=0.74 cores=0-7
/proc/18168/task/18178/stat : 18178 (plingeling) R 18166 18168 18068 0 -1 4202560 1272 1257 0 0 74 0 0 0 21 0 9 0 655368809 277835776 30832 33554432000 4194304 4417452 140735831322192 18446744073709551615 212435711049 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=18168/tid=18179] ppid=18166 vsize=271324 CPUtime=0.53 cores=0-7
/proc/18168/task/18179/stat : 18179 (plingeling) R 18166 18168 18068 0 -1 4202560 1761 1257 0 0 53 0 0 0 20 0 9 0 655368809 277835776 30832 33554432000 4194304 4417452 140735831322192 18446744073709551615 4284784 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=18168/tid=18180] ppid=18166 vsize=271324 CPUtime=0.68 cores=0-7
/proc/18168/task/18180/stat : 18180 (plingeling) R 18166 18168 18068 0 -1 4202560 1697 1257 0 0 68 0 0 0 21 0 9 0 655368809 277835776 30832 33554432000 4194304 4417452 140735831322192 18446744073709551615 4309168 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=18168/tid=18181] ppid=18166 vsize=271324 CPUtime=0.73 cores=0-7
/proc/18168/task/18181/stat : 18181 (plingeling) R 18166 18168 18068 0 -1 4202560 2347 1257 0 0 73 0 0 0 18 0 9 0 655368809 277835776 30832 33554432000 4194304 4417452 140735831322192 18446744073709551615 4290490 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18168/tid=18182] ppid=18166 vsize=271324 CPUtime=0.72 cores=0-7
/proc/18168/task/18182/stat : 18182 (plingeling) R 18166 18168 18068 0 -1 4202560 3805 1257 0 0 72 0 0 0 21 0 9 0 655368809 277835776 30832 33554432000 4194304 4417452 140735831322192 18446744073709551615 4284784 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 6.12
Current children cumulated vsize (KiB) 271324

[startup+3.10046 s]
/proc/loadavg: 7.48 7.86 7.15 9/319 18182
/proc/meminfo: memFree=10972736/32951124 swapFree=67093864/67111528
[pid=18168] ppid=18166 vsize=274980 CPUtime=18.9 cores=0-7
/proc/18168/stat : 18168 (plingeling) S 18166 18168 18068 0 -1 4202496 62787 1257 0 0 1882 8 0 0 18 0 9 0 655368733 281579520 30075 33554432000 4194304 4417452 140735831322192 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/18168/statm: 68745 30075 180 55 0 66631 0
[pid=18168/tid=18175] ppid=18166 vsize=274980 CPUtime=2.33 cores=0-7
/proc/18168/task/18175/stat : 18175 (plingeling) R 18166 18168 18068 0 -1 4202560 8373 1257 0 0 233 0 0 0 21 0 9 0 655368809 281579520 30075 33554432000 4194304 4417452 140735831322192 18446744073709551615 4271763 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=18168/tid=18176] ppid=18166 vsize=274980 CPUtime=2.11 cores=0-7
/proc/18168/task/18176/stat : 18176 (plingeling) R 18166 18168 18068 0 -1 4202560 3091 1257 0 0 211 0 0 0 25 0 9 0 655368809 281579520 30075 33554432000 4194304 4417452 140735831322192 18446744073709551615 4331729 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=18168/tid=18177] ppid=18166 vsize=274980 CPUtime=2.24 cores=0-7
/proc/18168/task/18177/stat : 18177 (plingeling) R 18166 18168 18068 0 -1 4202560 3450 1257 0 0 224 0 0 0 25 0 9 0 655368809 281579520 30075 33554432000 4194304 4417452 140735831322192 18446744073709551615 4249597 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18168/tid=18178] ppid=18166 vsize=274980 CPUtime=2.34 cores=0-7
/proc/18168/task/18178/stat : 18178 (plingeling) R 18166 18168 18068 0 -1 4202560 1557 1257 0 0 234 0 0 0 25 0 9 0 655368809 281579520 30075 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=18168/tid=18179] ppid=18166 vsize=274980 CPUtime=2.13 cores=0-7
/proc/18168/task/18179/stat : 18179 (plingeling) R 18166 18168 18068 0 -1 4202560 3570 1257 0 0 213 0 0 0 25 0 9 0 655368809 281579520 30075 33554432000 4194304 4417452 140735831322192 18446744073709551615 4331796 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=18168/tid=18180] ppid=18166 vsize=274980 CPUtime=2.27 cores=0-7
/proc/18168/task/18180/stat : 18180 (plingeling) R 18166 18168 18068 0 -1 4202560 2965 1257 0 0 227 0 0 0 25 0 9 0 655368809 281579520 30075 33554432000 4194304 4417452 140735831322192 18446744073709551615 4254855 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=18168/tid=18181] ppid=18166 vsize=274980 CPUtime=2.33 cores=0-7
/proc/18168/task/18181/stat : 18181 (plingeling) R 18166 18168 18068 0 -1 4202560 4401 1257 0 0 233 0 0 0 21 0 9 0 655368809 281579520 30075 33554432000 4194304 4417452 140735831322192 18446744073709551615 4262465 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18168/tid=18182] ppid=18166 vsize=274980 CPUtime=2.32 cores=0-7
/proc/18168/task/18182/stat : 18182 (plingeling) R 18166 18168 18068 0 -1 4202560 5221 1257 0 0 232 0 0 0 25 0 9 0 655368809 281579520 30075 33554432000 4194304 4417452 140735831322192 18446744073709551615 4272109 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 18.9
Current children cumulated vsize (KiB) 274980

[startup+6.30086 s]
/proc/loadavg: 7.60 7.87 7.16 9/319 18182
/proc/meminfo: memFree=10956212/32951124 swapFree=67093864/67111528
[pid=18168] ppid=18166 vsize=302136 CPUtime=44.46 cores=0-7
/proc/18168/stat : 18168 (plingeling) S 18166 18168 18068 0 -1 4202496 73829 1257 0 0 4437 9 0 0 18 0 9 0 655368733 309387264 38573 33554432000 4194304 4417452 140735831322192 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/18168/statm: 75534 38573 180 55 0 73420 0
[pid=18168/tid=18175] ppid=18166 vsize=302136 CPUtime=5.53 cores=0-7
/proc/18168/task/18175/stat : 18175 (plingeling) R 18166 18168 18068 0 -1 4202560 8709 1257 0 0 553 0 0 0 25 0 9 0 655368809 309387264 38573 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364587 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=18168/tid=18176] ppid=18166 vsize=302136 CPUtime=5.31 cores=0-7
/proc/18168/task/18176/stat : 18176 (plingeling) R 18166 18168 18068 0 -1 4202560 3920 1257 0 0 531 0 0 0 25 0 9 0 655368809 309387264 38573 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=18168/tid=18177] ppid=18166 vsize=302136 CPUtime=5.44 cores=0-7
/proc/18168/task/18177/stat : 18177 (plingeling) R 18166 18168 18068 0 -1 4202560 3794 1257 0 0 544 0 0 0 25 0 9 0 655368809 309387264 38573 33554432000 4194304 4417452 140735831322192 18446744073709551615 4357970 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18168/tid=18178] ppid=18166 vsize=302136 CPUtime=5.54 cores=0-7
/proc/18168/task/18178/stat : 18178 (plingeling) R 18166 18168 18068 0 -1 4202560 7420 1257 0 0 553 1 0 0 25 0 9 0 655368809 309387264 38573 33554432000 4194304 4417452 140735831322192 18446744073709551615 4264978 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=18168/tid=18179] ppid=18166 vsize=302136 CPUtime=5.33 cores=0-7
/proc/18168/task/18179/stat : 18179 (plingeling) R 18166 18168 18068 0 -1 4202560 3910 1257 0 0 533 0 0 0 25 0 9 0 655368809 309387264 38573 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=18168/tid=18180] ppid=18166 vsize=302136 CPUtime=5.43 cores=0-7
/proc/18168/task/18180/stat : 18180 (plingeling) R 18166 18168 18068 0 -1 4202560 4511 1257 0 0 543 0 0 0 25 0 9 0 655368809 309387264 38573 33554432000 4194304 4417452 140735831322192 18446744073709551615 4250972 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=18168/tid=18181] ppid=18166 vsize=302136 CPUtime=5.53 cores=0-7
/proc/18168/task/18181/stat : 18181 (plingeling) R 18166 18168 18068 0 -1 4202560 5285 1257 0 0 553 0 0 0 25 0 9 0 655368809 309387264 38573 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364629 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18168/tid=18182] ppid=18166 vsize=302136 CPUtime=5.53 cores=0-7
/proc/18168/task/18182/stat : 18182 (plingeling) R 18166 18168 18068 0 -1 4202560 6121 1257 0 0 552 1 0 0 25 0 9 0 655368809 309387264 38573 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364476 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 44.46
Current children cumulated vsize (KiB) 302136

[startup+12.7006 s]
/proc/loadavg: 7.63 7.88 7.16 9/319 18182
/proc/meminfo: memFree=10842060/32951124 swapFree=67093864/67111528
[pid=18168] ppid=18166 vsize=389368 CPUtime=95.6 cores=0-7
/proc/18168/stat : 18168 (plingeling) S 18166 18168 18068 0 -1 4202496 104141 1257 0 0 9547 13 0 0 18 0 9 0 655368733 398712832 61582 33554432000 4194304 4417452 140735831322192 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/18168/statm: 97342 61582 180 55 0 95228 0
[pid=18168/tid=18175] ppid=18166 vsize=389368 CPUtime=11.94 cores=0-7
/proc/18168/task/18175/stat : 18175 (plingeling) R 18166 18168 18068 0 -1 4202560 11667 1257 0 0 1193 1 0 0 25 0 9 0 655368809 398712832 61582 33554432000 4194304 4417452 140735831322192 18446744073709551615 4354497 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=18168/tid=18176] ppid=18166 vsize=389368 CPUtime=11.71 cores=0-7
/proc/18168/task/18176/stat : 18176 (plingeling) R 18166 18168 18068 0 -1 4202560 4789 1257 0 0 1171 0 0 0 25 0 9 0 655368809 398712832 61582 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=18168/tid=18177] ppid=18166 vsize=389368 CPUtime=11.84 cores=0-7
/proc/18168/task/18177/stat : 18177 (plingeling) R 18166 18168 18068 0 -1 4202560 6817 1257 0 0 1184 0 0 0 25 0 9 0 655368809 398712832 61582 33554432000 4194304 4417452 140735831322192 18446744073709551615 4357106 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18168/tid=18178] ppid=18166 vsize=389368 CPUtime=11.93 cores=0-7
/proc/18168/task/18178/stat : 18178 (plingeling) R 18166 18168 18068 0 -1 4202560 18344 1257 0 0 1191 2 0 0 25 0 9 0 655368809 398712832 61582 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=18168/tid=18179] ppid=18166 vsize=389368 CPUtime=11.73 cores=0-7
/proc/18168/task/18179/stat : 18179 (plingeling) R 18166 18168 18068 0 -1 4202560 4911 1257 0 0 1173 0 0 0 25 0 9 0 655368809 398712832 61582 33554432000 4194304 4417452 140735831322192 18446744073709551615 4331729 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=18168/tid=18180] ppid=18166 vsize=389368 CPUtime=11.76 cores=0-7
/proc/18168/task/18180/stat : 18180 (plingeling) R 18166 18168 18068 0 -1 4202560 9384 1257 0 0 1176 0 0 0 25 0 9 0 655368809 398712832 61582 33554432000 4194304 4417452 140735831322192 18446744073709551615 4236577 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=18168/tid=18181] ppid=18166 vsize=389368 CPUtime=11.93 cores=0-7
/proc/18168/task/18181/stat : 18181 (plingeling) R 18166 18168 18068 0 -1 4202560 10292 1257 0 0 1192 1 0 0 25 0 9 0 655368809 398712832 61582 33554432000 4194304 4417452 140735831322192 18446744073709551615 4237290 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18168/tid=18182] ppid=18166 vsize=389368 CPUtime=11.93 cores=0-7
/proc/18168/task/18182/stat : 18182 (plingeling) R 18166 18168 18068 0 -1 4202560 7778 1257 0 0 1192 1 0 0 23 0 9 0 655368809 398712832 61582 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 95.6
Current children cumulated vsize (KiB) 389368

[startup+25.5012 s]
/proc/loadavg: 7.71 7.88 7.17 9/319 18183
/proc/meminfo: memFree=10730540/32951124 swapFree=67093864/67111528
[pid=18168] ppid=18166 vsize=510708 CPUtime=197.86 cores=0-7
/proc/18168/stat : 18168 (plingeling) S 18166 18168 18068 0 -1 4202496 181775 1257 0 0 19762 24 0 0 18 0 9 0 655368733 522964992 89356 33554432000 4194304 4417452 140735831322192 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/18168/statm: 127677 89356 180 55 0 125563 0
[pid=18168/tid=18175] ppid=18166 vsize=510708 CPUtime=24.73 cores=0-7
/proc/18168/task/18175/stat : 18175 (plingeling) R 18166 18168 18068 0 -1 4202560 22700 1257 0 0 2471 2 0 0 25 0 9 0 655368809 522964992 89356 33554432000 4194304 4417452 140735831322192 18446744073709551615 4232609 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=18168/tid=18176] ppid=18166 vsize=510708 CPUtime=24.51 cores=0-7
/proc/18168/task/18176/stat : 18176 (plingeling) R 18166 18168 18068 0 -1 4202560 17055 1257 0 0 2449 2 0 0 25 0 9 0 655368809 522964992 89356 33554432000 4194304 4417452 140735831322192 18446744073709551615 4357554 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=18168/tid=18177] ppid=18166 vsize=510708 CPUtime=24.63 cores=0-7
/proc/18168/task/18177/stat : 18177 (plingeling) R 18166 18168 18068 0 -1 4202560 8079 1257 0 0 2463 0 0 0 25 0 9 0 655368809 522964992 89356 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364696 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=18168/tid=18178] ppid=18166 vsize=510708 CPUtime=24.73 cores=0-7
/proc/18168/task/18178/stat : 18178 (plingeling) R 18166 18168 18068 0 -1 4202560 47517 1257 0 0 2467 6 0 0 25 0 9 0 655368809 522964992 89356 33554432000 4194304 4417452 140735831322192 18446744073709551615 4356898 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=18168/tid=18179] ppid=18166 vsize=510708 CPUtime=24.53 cores=0-7
/proc/18168/task/18179/stat : 18179 (plingeling) R 18166 18168 18068 0 -1 4202560 11488 1257 0 0 2452 1 0 0 25 0 9 0 655368809 522964992 89356 33554432000 4194304 4417452 140735831322192 18446744073709551615 4250633 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=18168/tid=18180] ppid=18166 vsize=510708 CPUtime=24.43 cores=0-7
/proc/18168/task/18180/stat : 18180 (plingeling) R 18166 18168 18068 0 -1 4202560 14727 1257 0 0 2442 1 0 0 25 0 9 0 655368809 522964992 89356 33554432000 4194304 4417452 140735831322192 18446744073709551615 4357106 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=18168/tid=18181] ppid=18166 vsize=510708 CPUtime=24.73 cores=0-7
/proc/18168/task/18181/stat : 18181 (plingeling) R 18166 18168 18068 0 -1 4202560 19076 1257 0 0 2471 2 0 0 25 0 9 0 655368809 522964992 89356 33554432000 4194304 4417452 140735831322192 18446744073709551615 4250447 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=18168/tid=18182] ppid=18166 vsize=510708 CPUtime=24.72 cores=0-7
/proc/18168/task/18182/stat : 18182 (plingeling) R 18166 18168 18068 0 -1 4202560 10974 1257 0 0 2471 1 0 0 25 0 9 0 655368809 522964992 89356 33554432000 4194304 4417452 140735831322192 18446744073709551615 4358118 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 197.86
Current children cumulated vsize (KiB) 510708

[startup+51.1014 s]
/proc/loadavg: 7.88 7.91 7.20 9/319 18183
/proc/meminfo: memFree=10567316/32951124 swapFree=67093864/67111528
[pid=18168] ppid=18166 vsize=640956 CPUtime=402.37 cores=0-7
/proc/18168/stat : 18168 (plingeling) S 18166 18168 18068 0 -1 4202496 329750 1257 0 0 40189 48 0 0 18 0 9 0 655368733 656338944 129777 33554432000 4194304 4417452 140735831322192 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/18168/statm: 160239 129777 180 55 0 158125 0
[pid=18168/tid=18175] ppid=18166 vsize=640956 CPUtime=50.33 cores=0-7
/proc/18168/task/18175/stat : 18175 (plingeling) R 18166 18168 18068 0 -1 4202560 36983 1257 0 0 5029 4 0 0 25 0 9 0 655368809 656338944 129777 33554432000 4194304 4417452 140735831322192 18446744073709551615 4263206 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=18168/tid=18176] ppid=18166 vsize=640956 CPUtime=50.11 cores=0-7
/proc/18168/task/18176/stat : 18176 (plingeling) R 18166 18168 18068 0 -1 4202560 37302 1257 0 0 5005 6 0 0 25 0 9 0 655368809 656338944 129777 33554432000 4194304 4417452 140735831322192 18446744073709551615 4320992 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=18168/tid=18177] ppid=18166 vsize=640956 CPUtime=50.23 cores=0-7

################
# More data... #
################

[pid=18168/tid=18177] ppid=18166 vsize=23762000 CPUtime=4705.7 cores=0-7
/proc/18168/task/18177/stat : 18177 (plingeling) R 18166 18168 18068 0 -1 4202560 1805634 1257 0 0 470143 427 0 0 25 0 9 0 655368809 24332288000 5899463 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 5 0 0 143
[pid=18168/tid=18178] ppid=18166 vsize=23762000 CPUtime=4702.9 cores=0-7
/proc/18168/task/18178/stat : 18178 (plingeling) R 18166 18168 18068 0 -1 4202560 5105498 1257 255 0 469031 1259 0 0 25 0 9 0 655368809 24332288000 5899463 33554432000 4194304 4417452 140735831322192 18446744073709551615 4354506 0 0 4096 17442 0 0 0 -1 7 0 0 1222
[pid=18168/tid=18179] ppid=18166 vsize=23762000 CPUtime=4708.19 cores=0-7
/proc/18168/task/18179/stat : 18179 (plingeling) R 18166 18168 18068 0 -1 4202560 2593103 1257 539 0 470290 529 0 0 25 0 9 0 655368809 24332288000 5899463 33554432000 4194304 4417452 140735831322192 18446744073709551615 4365058 0 0 4096 17442 0 0 0 -1 6 0 0 319
[pid=18168/tid=18180] ppid=18166 vsize=23762000 CPUtime=4707.42 cores=0-7
/proc/18168/task/18180/stat : 18180 (plingeling) R 18166 18168 18068 0 -1 4202560 4215267 1257 0 0 469829 913 0 0 25 0 9 0 655368809 24332288000 5899463 33554432000 4194304 4417452 140735831322192 18446744073709551615 4355860 0 0 4096 17442 0 0 0 -1 3 0 0 543
[pid=18168/tid=18181] ppid=18166 vsize=23762000 CPUtime=4713.21 cores=0-7
/proc/18168/task/18181/stat : 18181 (plingeling) R 18166 18168 18068 0 -1 4202560 4273749 1257 4 0 470332 989 0 0 25 0 9 0 655368809 24332288000 5899463 33554432000 4194304 4417452 140735831322192 18446744073709551615 4263232 0 0 4096 17442 0 0 0 -1 0 0 0 279
[pid=18168/tid=18182] ppid=18166 vsize=23762000 CPUtime=4706.16 cores=0-7
/proc/18168/task/18182/stat : 18182 (plingeling) R 18166 18168 18068 0 -1 4202560 3394857 1257 0 0 469794 822 0 0 25 0 9 0 655368809 24332288000 5899463 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 503
Current children cumulated CPU time (s) 37655
Current children cumulated vsize (KiB) 23762000

[startup+4782.3 s]
/proc/loadavg: 8.13 8.12 8.08 9/319 18326
/proc/meminfo: memFree=98316/32951124 swapFree=57615440/67111528
[pid=18168] ppid=18166 vsize=23771612 CPUtime=38134.4 cores=0-7
/proc/18168/stat : 18168 (plingeling) S 18166 18168 18068 0 -1 4202496 27903938 1257 802 0 3806872 6567 0 0 18 0 9 0 655368733 24342130688 5901884 33554432000 4194304 4417452 140735831322192 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/18168/statm: 5942903 5901884 175 55 0 5940789 0
[pid=18168/tid=18175] ppid=18166 vsize=23771612 CPUtime=4767.24 cores=0-7
/proc/18168/task/18175/stat : 18175 (plingeling) R 18166 18168 18068 0 -1 4202560 2669566 1257 0 0 476121 603 0 0 25 0 9 0 655368809 24342130688 5901884 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364629 0 0 4096 17442 0 0 0 -1 2 0 0 964
[pid=18168/tid=18176] ppid=18166 vsize=23771612 CPUtime=4763.38 cores=0-7
/proc/18168/task/18176/stat : 18176 (plingeling) R 18166 18168 18068 0 -1 4202560 3797296 1257 0 0 475327 1011 0 0 25 0 9 0 655368809 24342130688 5901884 33554432000 4194304 4417452 140735831322192 18446744073709551615 4310641 0 0 4096 17442 0 0 0 -1 4 0 0 698
[pid=18168/tid=18177] ppid=18166 vsize=23771612 CPUtime=4765.71 cores=0-7
/proc/18168/task/18177/stat : 18177 (plingeling) R 18166 18168 18068 0 -1 4202560 1805634 1257 0 0 476144 427 0 0 25 0 9 0 655368809 24342130688 5901884 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 143
[pid=18168/tid=18178] ppid=18166 vsize=23771612 CPUtime=4762.24 cores=0-7
/proc/18168/task/18178/stat : 18178 (plingeling) R 18166 18168 18068 0 -1 4202560 5105520 1257 259 0 474965 1259 0 0 25 0 9 0 655368809 24342130688 5901884 33554432000 4194304 4417452 140735831322192 18446744073709551615 4250623 0 0 4096 17442 0 0 0 -1 7 0 0 1228
[pid=18168/tid=18179] ppid=18166 vsize=23771612 CPUtime=4768.18 cores=0-7
/proc/18168/task/18179/stat : 18179 (plingeling) R 18166 18168 18068 0 -1 4202560 2593103 1257 539 0 476289 529 0 0 25 0 9 0 655368809 24342130688 5901884 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 319
[pid=18168/tid=18180] ppid=18166 vsize=23771612 CPUtime=4767.43 cores=0-7
/proc/18168/task/18180/stat : 18180 (plingeling) R 18166 18168 18068 0 -1 4202560 4215267 1257 0 0 475830 913 0 0 25 0 9 0 655368809 24342130688 5901884 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 543
[pid=18168/tid=18181] ppid=18166 vsize=23771612 CPUtime=4773.22 cores=0-7
/proc/18168/task/18181/stat : 18181 (plingeling) R 18166 18168 18068 0 -1 4202560 4292536 1257 4 0 476327 995 0 0 25 0 9 0 655368809 24342130688 5901884 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 279
[pid=18168/tid=18182] ppid=18166 vsize=23771612 CPUtime=4766.16 cores=0-7
/proc/18168/task/18182/stat : 18182 (plingeling) R 18166 18168 18068 0 -1 4202560 3394857 1257 0 0 475794 822 0 0 25 0 9 0 655368809 24342130688 5901884 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 503
Current children cumulated CPU time (s) 38134.4
Current children cumulated vsize (KiB) 23771612

[startup+4842.3 s]
/proc/loadavg: 8.04 8.10 8.07 9/319 18328
/proc/meminfo: memFree=647964/32951124 swapFree=57380312/67111528
[pid=18168] ppid=18166 vsize=23382776 CPUtime=38612.8 cores=0-7
/proc/18168/stat : 18168 (plingeling) S 18166 18168 18068 0 -1 4202496 28291537 1257 804 0 3854634 6649 0 0 18 0 9 0 655368733 23943962624 5804647 33554432000 4194304 4417452 140735831322192 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/18168/statm: 5845694 5804647 175 55 0 5843580 0
[pid=18168/tid=18175] ppid=18166 vsize=23382776 CPUtime=4827.23 cores=0-7
/proc/18168/task/18175/stat : 18175 (plingeling) R 18166 18168 18068 0 -1 4202560 2669566 1257 0 0 482120 603 0 0 25 0 9 0 655368809 23943962624 5804647 33554432000 4194304 4417452 140735831322192 18446744073709551615 4366074 0 0 4096 17442 0 0 0 -1 2 0 0 964
[pid=18168/tid=18176] ppid=18166 vsize=23382776 CPUtime=4823.11 cores=0-7
/proc/18168/task/18176/stat : 18176 (plingeling) R 18166 18168 18068 0 -1 4202560 4097695 1257 0 0 481259 1052 0 0 25 0 9 0 655368809 23943962624 5804647 33554432000 4194304 4417452 140735831322192 18446744073709551615 4366094 0 0 4096 17442 0 0 0 -1 1 0 0 698
[pid=18168/tid=18177] ppid=18166 vsize=23382776 CPUtime=4825.71 cores=0-7
/proc/18168/task/18177/stat : 18177 (plingeling) R 18166 18168 18068 0 -1 4202560 1805634 1257 0 0 482144 427 0 0 25 0 9 0 655368809 23943962624 5804647 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 143
[pid=18168/tid=18178] ppid=18166 vsize=23382776 CPUtime=4822.1 cores=0-7
/proc/18168/task/18178/stat : 18178 (plingeling) R 18166 18168 18068 0 -1 4202560 5138306 1257 261 0 480941 1269 0 0 25 0 9 0 655368809 23943962624 5804647 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 1232
[pid=18168/tid=18179] ppid=18166 vsize=23382776 CPUtime=4828.17 cores=0-7
/proc/18168/task/18179/stat : 18179 (plingeling) R 18166 18168 18068 0 -1 4202560 2593103 1257 539 0 482288 529 0 0 25 0 9 0 655368809 23943962624 5804647 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 319
[pid=18168/tid=18180] ppid=18166 vsize=23382776 CPUtime=4827.43 cores=0-7
/proc/18168/task/18180/stat : 18180 (plingeling) R 18166 18168 18068 0 -1 4202560 4215267 1257 0 0 481829 914 0 0 25 0 9 0 655368809 23943962624 5804647 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 543
[pid=18168/tid=18181] ppid=18166 vsize=23382776 CPUtime=4832.31 cores=0-7
/proc/18168/task/18181/stat : 18181 (plingeling) R 18166 18168 18068 0 -1 4202560 4346950 1257 4 0 482208 1023 0 0 25 0 9 0 655368809 23943962624 5804647 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 306
[pid=18168/tid=18182] ppid=18166 vsize=23382776 CPUtime=4825.95 cores=0-7
/proc/18168/task/18182/stat : 18182 (plingeling) R 18166 18168 18068 0 -1 4202560 3394857 1257 0 0 481771 824 0 0 25 0 9 0 655368809 23943962624 5804647 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 503
Current children cumulated CPU time (s) 38612.8
Current children cumulated vsize (KiB) 23382776

[startup+4902.3 s]
/proc/loadavg: 8.08 8.09 8.07 9/319 18330
/proc/meminfo: memFree=200808/32951124 swapFree=57380312/67111528
[pid=18168] ppid=18166 vsize=23829480 CPUtime=39092.3 cores=0-7
/proc/18168/stat : 18168 (plingeling) S 18166 18168 18068 0 -1 4202496 28403278 1257 809 0 3902544 6684 0 0 18 0 9 0 655368733 24401387520 5916353 33554432000 4194304 4417452 140735831322192 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/18168/statm: 5957370 5916353 175 55 0 5955256 0
[pid=18168/tid=18175] ppid=18166 vsize=23829480 CPUtime=4887.24 cores=0-7
/proc/18168/task/18175/stat : 18175 (plingeling) R 18166 18168 18068 0 -1 4202560 2669566 1257 0 0 488121 603 0 0 25 0 9 0 655368809 24401387520 5916353 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 964
[pid=18168/tid=18176] ppid=18166 vsize=23829480 CPUtime=4883.12 cores=0-7
/proc/18168/task/18176/stat : 18176 (plingeling) R 18166 18168 18068 0 -1 4202560 4097710 1257 0 0 487260 1052 0 0 25 0 9 0 655368809 24401387520 5916353 33554432000 4194304 4417452 140735831322192 18446744073709551615 4355860 0 0 4096 17442 0 0 0 -1 1 0 0 698
[pid=18168/tid=18177] ppid=18166 vsize=23829480 CPUtime=4885.72 cores=0-7
/proc/18168/task/18177/stat : 18177 (plingeling) R 18166 18168 18068 0 -1 4202560 1805634 1257 0 0 488145 427 0 0 25 0 9 0 655368809 24401387520 5916353 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 5 0 0 143
[pid=18168/tid=18178] ppid=18166 vsize=23829480 CPUtime=4882.08 cores=0-7
/proc/18168/task/18178/stat : 18178 (plingeling) R 18166 18168 18068 0 -1 4202560 5250004 1257 266 0 486904 1304 0 0 25 0 9 0 655368809 24401387520 5916353 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 0 0 0 1235
[pid=18168/tid=18179] ppid=18166 vsize=23829480 CPUtime=4888.17 cores=0-7
/proc/18168/task/18179/stat : 18179 (plingeling) R 18166 18168 18068 0 -1 4202560 2593117 1257 539 0 488288 529 0 0 25 0 9 0 655368809 24401387520 5916353 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 319
[pid=18168/tid=18180] ppid=18166 vsize=23829480 CPUtime=4887.44 cores=0-7
/proc/18168/task/18180/stat : 18180 (plingeling) R 18166 18168 18068 0 -1 4202560 4215267 1257 0 0 487830 914 0 0 25 0 9 0 655368809 24401387520 5916353 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 543
[pid=18168/tid=18181] ppid=18166 vsize=23829480 CPUtime=4891.72 cores=0-7
/proc/18168/task/18181/stat : 18181 (plingeling) R 18166 18168 18068 0 -1 4202560 4346950 1257 4 0 488149 1023 0 0 25 0 9 0 655368809 24401387520 5916353 33554432000 4194304 4417452 140735831322192 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 7 0 0 306
[pid=18168/tid=18182] ppid=18166 vsize=23829480 CPUtime=4885.96 cores=0-7
/proc/18168/task/18182/stat : 18182 (plingeling) R 18166 18168 18068 0 -1 4202560 3394871 1257 0 0 487772 824 0 0 25 0 9 0 655368809 24401387520 5916353 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 4 0 0 503
Current children cumulated CPU time (s) 39092.3
Current children cumulated vsize (KiB) 23829480

[startup+4962.3 s]
/proc/loadavg: 8.03 8.07 8.07 9/319 18331
/proc/meminfo: memFree=1740012/32951124 swapFree=57302448/67111528
[pid=18168] ppid=18166 vsize=22387248 CPUtime=39571.2 cores=0-7
/proc/18168/stat : 18168 (plingeling) S 18166 18168 18068 0 -1 4202496 28484915 1257 815 0 3950409 6714 0 0 18 0 9 0 655368733 22924541952 5555818 33554432000 4194304 4417452 140735831322192 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/18168/statm: 5596812 5555818 175 55 0 5594698 0
[pid=18168/tid=18175] ppid=18166 vsize=22387248 CPUtime=4947.23 cores=0-7
/proc/18168/task/18175/stat : 18175 (plingeling) R 18166 18168 18068 0 -1 4202560 2669566 1257 0 0 494120 603 0 0 25 0 9 0 655368809 22924541952 5555818 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 964
[pid=18168/tid=18176] ppid=18166 vsize=22387248 CPUtime=4942.95 cores=0-7
/proc/18168/task/18176/stat : 18176 (plingeling) R 18166 18168 18068 0 -1 4202560 4114094 1257 0 0 493238 1057 0 0 25 0 9 0 655368809 22924541952 5555818 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 698
[pid=18168/tid=18177] ppid=18166 vsize=22387248 CPUtime=4945.71 cores=0-7
/proc/18168/task/18177/stat : 18177 (plingeling) R 18166 18168 18068 0 -1 4202560 1805634 1257 0 0 494144 427 0 0 25 0 9 0 655368809 22924541952 5555818 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364629 0 0 4096 17442 0 0 0 -1 5 0 0 143
[pid=18168/tid=18178] ppid=18166 vsize=22387248 CPUtime=4941.98 cores=0-7
/proc/18168/task/18178/stat : 18178 (plingeling) R 18166 18168 18068 0 -1 4202560 5282806 1257 272 0 492884 1314 0 0 25 0 9 0 655368809 22924541952 5555818 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 1243
[pid=18168/tid=18179] ppid=18166 vsize=22387248 CPUtime=4948.15 cores=0-7
/proc/18168/task/18179/stat : 18179 (plingeling) R 18166 18168 18068 0 -1 4202560 2625568 1257 539 0 494280 535 0 0 25 0 9 0 655368809 22924541952 5555818 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 319
[pid=18168/tid=18180] ppid=18166 vsize=22387248 CPUtime=4947.43 cores=0-7
/proc/18168/task/18180/stat : 18180 (plingeling) R 18166 18168 18068 0 -1 4202560 4215267 1257 0 0 493829 914 0 0 25 0 9 0 655368809 22924541952 5555818 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 3 0 0 543
[pid=18168/tid=18181] ppid=18166 vsize=22387248 CPUtime=4951.11 cores=0-7
/proc/18168/task/18181/stat : 18181 (plingeling) R 18166 18168 18068 0 -1 4202560 4346950 1257 4 0 494084 1027 0 0 25 0 9 0 655368809 22924541952 5555818 33554432000 4194304 4417452 140735831322192 18446744073709551615 4309862 0 0 4096 17442 0 0 0 -1 7 0 0 306
[pid=18168/tid=18182] ppid=18166 vsize=22387248 CPUtime=4945.83 cores=0-7
/proc/18168/task/18182/stat : 18182 (plingeling) R 18166 18168 18068 0 -1 4202560 3394871 1257 0 0 493754 829 0 0 25 0 9 0 655368809 22924541952 5555818 33554432000 4194304 4417452 140735831322192 18446744073709551615 4311347 0 0 4096 17442 0 0 0 -1 4 0 0 503
Current children cumulated CPU time (s) 39571.2
Current children cumulated vsize (KiB) 22387248



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5016 s]
/proc/loadavg: 8.07 8.07 8.07 9/319 18333
/proc/meminfo: memFree=1352228/32951124 swapFree=57302448/67111528
[pid=18168] ppid=18166 vsize=22775836 CPUtime=40000.3 cores=0-7
/proc/18168/stat : 18168 (plingeling) S 18166 18168 18068 0 -1 4202496 28582079 1257 818 0 3993300 6731 0 0 18 0 9 0 655368733 23322456064 5652975 33554432000 4194304 4417452 140735831322192 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/18168/statm: 5693959 5652975 175 55 0 5691845 0
[pid=18168/tid=18175] ppid=18166 vsize=22775836 CPUtime=5000.95 cores=0-7
/proc/18168/task/18175/stat : 18175 (plingeling) R 18166 18168 18068 0 -1 4202560 2669566 1257 0 0 499492 603 0 0 25 0 9 0 655368809 23322456064 5652975 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 2 0 0 964
[pid=18168/tid=18176] ppid=18166 vsize=22775836 CPUtime=4996.67 cores=0-7
/proc/18168/task/18176/stat : 18176 (plingeling) R 18166 18168 18068 0 -1 4202560 4146862 1257 0 0 498605 1062 0 0 25 0 9 0 655368809 23322456064 5652975 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 698
[pid=18168/tid=18177] ppid=18166 vsize=22775836 CPUtime=4999.42 cores=0-7
/proc/18168/task/18177/stat : 18177 (plingeling) R 18166 18168 18068 0 -1 4202560 1805634 1257 0 0 499515 427 0 0 25 0 9 0 655368809 23322456064 5652975 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 143
[pid=18168/tid=18178] ppid=18166 vsize=22775836 CPUtime=4995.62 cores=0-7
/proc/18168/task/18178/stat : 18178 (plingeling) R 18166 18168 18068 0 -1 4202560 5282823 1257 275 0 498248 1314 0 0 25 0 9 0 655368809 23322456064 5652975 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 1248
[pid=18168/tid=18179] ppid=18166 vsize=22775836 CPUtime=5001.86 cores=0-7
/proc/18168/task/18179/stat : 18179 (plingeling) R 18166 18168 18068 0 -1 4202560 2657179 1257 539 0 499641 545 0 0 25 0 9 0 655368809 23322456064 5652975 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 6 0 0 319
[pid=18168/tid=18180] ppid=18166 vsize=22775836 CPUtime=5001.15 cores=0-7
/proc/18168/task/18180/stat : 18180 (plingeling) R 18166 18168 18068 0 -1 4202560 4248035 1257 0 0 499198 917 0 0 25 0 9 0 655368809 23322456064 5652975 33554432000 4194304 4417452 140735831322192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 543
[pid=18168/tid=18181] ppid=18166 vsize=22775836 CPUtime=5004.28 cores=0-7
/proc/18168/task/18181/stat : 18181 (plingeling) R 18166 18168 18068 0 -1 4202560 4346950 1257 4 0 499401 1027 0 0 25 0 9 0 655368809 23322456064 5652975 33554432000 4194304 4417452 140735831322192 18446744073709551615 4308806 0 0 4096 17442 0 0 0 -1 7 0 0 306
[pid=18168/tid=18182] ppid=18166 vsize=22775836 CPUtime=4999.54 cores=0-7
/proc/18168/task/18182/stat : 18182 (plingeling) R 18166 18168 18068 0 -1 4202560 3394871 1257 0 0 499125 829 0 0 25 0 9 0 655368809 23322456064 5652975 33554432000 4194304 4417452 140735831322192 18446744073709551615 4311409 0 0 4096 17442 0 0 0 -1 4 0 0 503
Current children cumulated CPU time (s) 40000.3
Current children cumulated vsize (KiB) 22775836

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5016.81 s]
/proc/loadavg: 8.07 8.07 8.07 2/313 18334
/proc/meminfo: memFree=16052648/32951124 swapFree=57328676/67111528
[pid=18168] ppid=18166 vsize=0 CPUtime=40001.5 cores=0-7
/proc/18168/stat : 18168 (plingeling) Z 18166 18168 18068 0 -1 4203548 28582081 1257 820 0 3993346 6805 0 0 18 0 2 0 655368733 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/18168/statm: 0 0 0 0 0 0 0
[pid=18168/tid=18175] ppid=18166 vsize=0 CPUtime=5001.74 cores=0-7
/proc/18168/task/18175/stat : 18175 (plingeling) R 18166 18168 18068 0 -1 4203588 2669566 1257 0 0 499497 677 0 0 25 0 2 0 655368809 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 2 0 0 964
Current children cumulated CPU time (s) 40001.5
Current children cumulated vsize (KiB) 0

[startup+5017 s]
/proc/loadavg: 8.07 8.07 8.07 2/313 18334
/proc/meminfo: memFree=16052648/32951124 swapFree=57328676/67111528
[pid=18168] ppid=18166 vsize=0 CPUtime=40001.7 cores=0-7
/proc/18168/stat : 18168 (plingeling) Z 18166 18168 18068 0 -1 4203548 28582081 1257 820 0 3993346 6824 0 0 18 0 2 0 655368733 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/18168/statm: 0 0 0 0 0 0 0
[pid=18168/tid=18175] ppid=18166 vsize=0 CPUtime=5001.93 cores=0-7
/proc/18168/task/18175/stat : 18175 (plingeling) R 18166 18168 18068 0 -1 4203588 2669566 1257 0 0 499497 696 0 0 25 0 2 0 655368809 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 2 0 0 964
Current children cumulated CPU time (s) 40001.7
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5017.15
CPU time (s): 40001.8
CPU user time (s): 39933.5
CPU system time (s): 68.3356
CPU usage (%): 797.302
Max. virtual memory (cumulated for all children) (KiB): 24024820

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39933.5
system time used= 68.3356
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 28583338
page faults= 820
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2627
involuntary context switches= 194726

runsolver used 14.1478 second user time and 36.1475 second system time

The end

Launcher Data 

Begin job on node102 at 2011-05-15 07:34:24
IDJOB=3372455
IDBENCH=82390
IDSOLVER=1851
FILE ID=node102/3372455-1305437663
RUNJOBID= node102-1305436390-18082
PBS_JOBID= 13324748
Free space on /tmp= 73656 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/application/rintanen/SATPlanning/openstacks-sequencedstrips-nonadl-nonnegated-os-sequencedstrips-p30_1.025-NOTKNOWN.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3372455-1305437663/watcher-3372455-1305437663 -o /tmp/evaluation-result-3372455-1305437663/solver-3372455-1305437663 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3372455-1305437663.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 732b9c9b0d44d6a9e75489b2f5004165
RANDOM SEED=265228574

node102.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      11085936 kB
Buffers:        271988 kB
Cached:        2863408 kB
SwapCached:       1128 kB
Active:       19583760 kB
Inactive:      1991184 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      11085936 kB
SwapTotal:    67111528 kB
SwapFree:     67093864 kB
Dirty:            8180 kB
Writeback:           0 kB
AnonPages:    18438320 kB
Mapped:          26316 kB
Slab:           170172 kB
PageTables:      52532 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 59037868 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73652 MiB
End job on node102 at 2011-05-15 08:58:02