Trace number 3371118

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.36 1203.13

General information on the benchmark

Namecrafted/kullmann/VanDerWaerden/VanderWaerden_pd_3k/
VanDerWaerden_pd_2-3-21_406.cnf
MD5SUM906590ef62b3db2841b2d529fd1636f7
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark698.76
Satisfiable
(Un)Satisfiability was proved
Number of variables203
Number of clauses22241
Sum of the clauses size99837
Maximum clause length21
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 268
Number of clauses of size 320324
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 51849

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3371118-1305428289/watcher-3371118-1305428289 -o /tmp/evaluation-result-3371118-1305428289/solver-3371118-1305428289 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3371118-1305428289.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.14 7.89 8.17 3/315 26692
/proc/meminfo: memFree=15643068/32950928 swapFree=67085488/67111528
[pid=26690] ppid=26687 vsize=9948 CPUtime=0 cores=1,3,5,7
/proc/26690/stat : 26690 (plingeling) R 26687 26690 26655 0 -1 4202496 533 1251 0 0 0 0 0 0 18 0 1 0 733059588 10186752 463 33554432000 4194304 4417452 140736125795472 18446744073709551615 231895187664 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/26690/statm: 2487 479 149 55 0 373 0

[startup+0.023783 s]
/proc/loadavg: 7.14 7.89 8.17 3/315 26692
/proc/meminfo: memFree=15643068/32950928 swapFree=67085488/67111528
[pid=26690] ppid=26687 vsize=17156 CPUtime=0.01 cores=1,3,5,7
/proc/26690/stat : 26690 (plingeling) R 26687 26690 26655 0 -1 4202496 2405 1251 0 0 1 0 0 0 18 0 1 0 733059588 17567744 2313 33554432000 4194304 4417452 140736125795472 18446744073709551615 4274358 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/26690/statm: 4289 2313 149 55 0 2175 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 17156

[startup+0.154669 s]
/proc/loadavg: 7.14 7.89 8.17 3/315 26692
/proc/meminfo: memFree=15643068/32950928 swapFree=67085488/67111528
[pid=26690] ppid=26687 vsize=61608 CPUtime=0.5 cores=1,3,5,7
/proc/26690/stat : 26690 (plingeling) S 26687 26690 26655 0 -1 4202496 3579 1251 0 0 50 0 0 0 18 0 5 0 733059588 63086592 3206 33554432000 4194304 4417452 140736125795472 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/26690/statm: 15402 3206 179 55 0 13288 0
Current children cumulated CPU time (s) 0.5
Current children cumulated vsize (KiB) 61608

[startup+0.300641 s]
/proc/loadavg: 7.14 7.89 8.17 3/315 26692
/proc/meminfo: memFree=15643068/32950928 swapFree=67085488/67111528
[pid=26690] ppid=26687 vsize=126648 CPUtime=1.08 cores=1,3,5,7
/proc/26690/stat : 26690 (plingeling) S 26687 26690 26655 0 -1 4202496 3857 1251 0 0 108 0 0 0 18 0 5 0 733059588 129687552 3006 33554432000 4194304 4417452 140736125795472 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/26690/statm: 31662 3006 180 55 0 29548 0
Current children cumulated CPU time (s) 1.08
Current children cumulated vsize (KiB) 126648

[startup+0.700569 s]
/proc/loadavg: 7.14 7.89 8.17 3/315 26692
/proc/meminfo: memFree=15643068/32950928 swapFree=67085488/67111528
[pid=26690] ppid=26687 vsize=127220 CPUtime=2.68 cores=1,3,5,7
/proc/26690/stat : 26690 (plingeling) S 26687 26690 26655 0 -1 4202496 4063 1251 0 0 268 0 0 0 18 0 5 0 733059588 130273280 3212 33554432000 4194304 4417452 140736125795472 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/26690/statm: 31805 3212 180 55 0 29691 0
Current children cumulated CPU time (s) 2.68
Current children cumulated vsize (KiB) 127220

[startup+1.50042 s]
/proc/loadavg: 7.14 7.89 8.17 10/325 26712
/proc/meminfo: memFree=15614724/32950928 swapFree=67085488/67111528
[pid=26690] ppid=26687 vsize=132616 CPUtime=5.87 cores=1,3,5,7
/proc/26690/stat : 26690 (plingeling) S 26687 26690 26655 0 -1 4202496 5412 1251 0 0 587 0 0 0 18 0 5 0 733059588 135798784 4561 33554432000 4194304 4417452 140736125795472 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/26690/statm: 33154 4561 180 55 0 31040 0
[pid=26690/tid=26705] ppid=26687 vsize=132616 CPUtime=1.46 cores=1,3,5,7
/proc/26690/task/26705/stat : 26705 (plingeling) R 26687 26690 26655 0 -1 4202560 691 1251 0 0 146 0 0 0 23 0 5 0 733059591 135798784 4561 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26690/tid=26706] ppid=26687 vsize=132616 CPUtime=1.45 cores=1,3,5,7
/proc/26690/task/26706/stat : 26706 (plingeling) R 26687 26690 26655 0 -1 4202560 319 1251 0 0 145 0 0 0 23 0 5 0 733059591 135798784 4561 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364794 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26690/tid=26707] ppid=26687 vsize=132616 CPUtime=1.46 cores=1,3,5,7
/proc/26690/task/26707/stat : 26707 (plingeling) R 26687 26690 26655 0 -1 4202560 395 1251 0 0 146 0 0 0 23 0 5 0 733059591 135798784 4561 33554432000 4194304 4417452 140736125795472 18446744073709551615 4275319 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26690/tid=26708] ppid=26687 vsize=132616 CPUtime=1.46 cores=1,3,5,7
/proc/26690/task/26708/stat : 26708 (plingeling) R 26687 26690 26655 0 -1 4202560 1282 1251 0 0 146 0 0 0 23 0 5 0 733059591 135798784 4561 33554432000 4194304 4417452 140736125795472 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 5.87
Current children cumulated vsize (KiB) 132616

[startup+3.10112 s]
/proc/loadavg: 7.21 7.89 8.16 10/325 26712
/proc/meminfo: memFree=15605184/32950928 swapFree=67085488/67111528
[pid=26690] ppid=26687 vsize=136244 CPUtime=12.26 cores=1,3,5,7
/proc/26690/stat : 26690 (plingeling) S 26687 26690 26655 0 -1 4202496 6780 1251 0 0 1226 0 0 0 18 0 5 0 733059588 139513856 5468 33554432000 4194304 4417452 140736125795472 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/26690/statm: 34061 5468 180 55 0 31947 0
[pid=26690/tid=26705] ppid=26687 vsize=136244 CPUtime=3.06 cores=1,3,5,7
/proc/26690/task/26705/stat : 26705 (plingeling) R 26687 26690 26655 0 -1 4202560 1129 1251 0 0 306 0 0 0 25 0 5 0 733059591 139513856 5468 33554432000 4194304 4417452 140736125795472 18446744073709551615 4232664 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26690/tid=26706] ppid=26687 vsize=136244 CPUtime=3.04 cores=1,3,5,7
/proc/26690/task/26706/stat : 26706 (plingeling) R 26687 26690 26655 0 -1 4202560 497 1251 0 0 304 0 0 0 25 0 5 0 733059591 139513856 5468 33554432000 4194304 4417452 140736125795472 18446744073709551615 4365946 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26690/tid=26707] ppid=26687 vsize=136244 CPUtime=3.06 cores=1,3,5,7
/proc/26690/task/26707/stat : 26707 (plingeling) R 26687 26690 26655 0 -1 4202560 395 1251 0 0 306 0 0 0 25 0 5 0 733059591 139513856 5468 33554432000 4194304 4417452 140736125795472 18446744073709551615 4366447 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26690/tid=26708] ppid=26687 vsize=136244 CPUtime=3.06 cores=1,3,5,7
/proc/26690/task/26708/stat : 26708 (plingeling) R 26687 26690 26655 0 -1 4202560 2034 1251 0 0 306 0 0 0 25 0 5 0 733059591 139513856 5468 33554432000 4194304 4417452 140736125795472 18446744073709551615 18446744073699065939 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 12.26
Current children cumulated vsize (KiB) 136244

[startup+6.30052 s]
/proc/loadavg: 7.21 7.89 8.16 10/325 26712
/proc/meminfo: memFree=15596512/32950928 swapFree=67085488/67111528
[pid=26690] ppid=26687 vsize=140492 CPUtime=25.03 cores=1,3,5,7
/proc/26690/stat : 26690 (plingeling) S 26687 26690 26655 0 -1 4202496 10938 1251 0 0 2502 1 0 0 18 0 5 0 733059588 143863808 6530 33554432000 4194304 4417452 140736125795472 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/26690/statm: 35123 6530 180 55 0 33009 0
[pid=26690/tid=26705] ppid=26687 vsize=140492 CPUtime=6.26 cores=1,3,5,7
/proc/26690/task/26705/stat : 26705 (plingeling) R 26687 26690 26655 0 -1 4202560 2014 1251 0 0 626 0 0 0 25 0 5 0 733059591 143863808 6530 33554432000 4194304 4417452 140736125795472 18446744073709551615 4366391 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26690/tid=26706] ppid=26687 vsize=140492 CPUtime=6.2 cores=1,3,5,7
/proc/26690/task/26706/stat : 26706 (plingeling) R 26687 26690 26655 0 -1 4202560 1276 1251 0 0 620 0 0 0 25 0 5 0 733059591 143863808 6530 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26690/tid=26707] ppid=26687 vsize=140492 CPUtime=6.26 cores=1,3,5,7
/proc/26690/task/26707/stat : 26707 (plingeling) R 26687 26690 26655 0 -1 4202560 779 1251 0 0 626 0 0 0 25 0 5 0 733059591 143863808 6530 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26690/tid=26708] ppid=26687 vsize=140492 CPUtime=6.26 cores=1,3,5,7
/proc/26690/task/26708/stat : 26708 (plingeling) R 26687 26690 26655 0 -1 4202560 4144 1251 0 0 626 0 0 0 25 0 5 0 733059591 143863808 6530 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 25.03
Current children cumulated vsize (KiB) 140492

[startup+12.7003 s]
/proc/loadavg: 7.33 7.90 8.16 10/325 26712
/proc/meminfo: memFree=15576344/32950928 swapFree=67085488/67111528
[pid=26690] ppid=26687 vsize=145820 CPUtime=50.56 cores=1,3,5,7
/proc/26690/stat : 26690 (plingeling) S 26687 26690 26655 0 -1 4202496 23640 1251 0 0 5053 3 0 0 18 0 5 0 733059588 149319680 7862 33554432000 4194304 4417452 140736125795472 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/26690/statm: 36455 7862 180 55 0 34341 0
[pid=26690/tid=26705] ppid=26687 vsize=145820 CPUtime=12.66 cores=1,3,5,7
/proc/26690/task/26705/stat : 26705 (plingeling) R 26687 26690 26655 0 -1 4202560 5601 1251 0 0 1266 0 0 0 25 0 5 0 733059591 149319680 7862 33554432000 4194304 4417452 140736125795472 18446744073709551615 4366385 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26690/tid=26706] ppid=26687 vsize=145820 CPUtime=12.53 cores=1,3,5,7
/proc/26690/task/26706/stat : 26706 (plingeling) R 26687 26690 26655 0 -1 4202560 3752 1251 0 0 1253 0 0 0 25 0 5 0 733059591 149319680 7862 33554432000 4194304 4417452 140736125795472 18446744073709551615 4343760 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26690/tid=26707] ppid=26687 vsize=145820 CPUtime=12.65 cores=1,3,5,7
/proc/26690/task/26707/stat : 26707 (plingeling) R 26687 26690 26655 0 -1 4202560 2814 1251 0 0 1265 0 0 0 25 0 5 0 733059591 149319680 7862 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26690/tid=26708] ppid=26687 vsize=145820 CPUtime=12.66 cores=1,3,5,7
/proc/26690/task/26708/stat : 26708 (plingeling) R 26687 26690 26655 0 -1 4202560 8748 1251 0 0 1265 1 0 0 25 0 5 0 733059591 149319680 7862 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 50.56
Current children cumulated vsize (KiB) 145820

[startup+25.5009 s]
/proc/loadavg: 7.44 7.90 8.16 10/325 26712
/proc/meminfo: memFree=15557064/32950928 swapFree=67085488/67111528
[pid=26690] ppid=26687 vsize=160176 CPUtime=101.63 cores=1,3,5,7
/proc/26690/stat : 26690 (plingeling) S 26687 26690 26655 0 -1 4202496 55067 1251 0 0 10154 9 0 0 18 0 5 0 733059588 164020224 11464 33554432000 4194304 4417452 140736125795472 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/26690/statm: 40044 11464 180 55 0 37930 0
[pid=26690/tid=26705] ppid=26687 vsize=160176 CPUtime=25.46 cores=1,3,5,7
/proc/26690/task/26705/stat : 26705 (plingeling) R 26687 26690 26655 0 -1 4202560 13527 1251 0 0 2544 2 0 0 25 0 5 0 733059591 164020224 11464 33554432000 4194304 4417452 140736125795472 18446744073709551615 4366385 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26690/tid=26706] ppid=26687 vsize=160176 CPUtime=25.2 cores=1,3,5,7
/proc/26690/task/26706/stat : 26706 (plingeling) R 26687 26690 26655 0 -1 4202560 8898 1251 0 0 2519 1 0 0 25 0 5 0 733059591 164020224 11464 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364877 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26690/tid=26707] ppid=26687 vsize=160176 CPUtime=25.45 cores=1,3,5,7
/proc/26690/task/26707/stat : 26707 (plingeling) R 26687 26690 26655 0 -1 4202560 9457 1251 0 0 2544 1 0 0 25 0 5 0 733059591 164020224 11464 33554432000 4194304 4417452 140736125795472 18446744073709551615 4366113 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26690/tid=26708] ppid=26687 vsize=160176 CPUtime=25.46 cores=1,3,5,7
/proc/26690/task/26708/stat : 26708 (plingeling) R 26687 26690 26655 0 -1 4202560 20460 1251 0 0 2543 3 0 0 25 0 5 0 733059591 164020224 11464 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 101.63
Current children cumulated vsize (KiB) 160176

[startup+51.1011 s]
/proc/loadavg: 7.75 7.94 8.17 9/325 26713
/proc/meminfo: memFree=15543660/32950928 swapFree=67085488/67111528
[pid=26690] ppid=26687 vsize=158608 CPUtime=203.78 cores=1,3,5,7
/proc/26690/stat : 26690 (plingeling) S 26687 26690 26655 0 -1 4202496 115102 1251 0 0 20358 20 0 0 18 0 5 0 733059588 162414592 11074 33554432000 4194304 4417452 140736125795472 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/26690/statm: 39652 11074 180 55 0 37538 0
[pid=26690/tid=26705] ppid=26687 vsize=158608 CPUtime=51.06 cores=1,3,5,7
/proc/26690/task/26705/stat : 26705 (plingeling) R 26687 26690 26655 0 -1 4202560 27898 1251 0 0 5102 4 0 0 25 0 5 0 733059591 162414592 11074 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26690/tid=26706] ppid=26687 vsize=158608 CPUtime=50.54 cores=1,3,5,7
/proc/26690/task/26706/stat : 26706 (plingeling) R 26687 26690 26655 0 -1 4202560 23711 1251 0 0 5051 3 0 0 25 0 5 0 733059591 162414592 11074 33554432000 4194304 4417452 140736125795472 18446744073709551615 4310096 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26690/tid=26707] ppid=26687 vsize=158608 CPUtime=51.06 cores=1,3,5,7
/proc/26690/task/26707/stat : 26707 (plingeling) R 26687 26690 26655 0 -1 4202560 21522 1251 0 0 5102 4 0 0 25 0 5 0 733059591 162414592 11074 33554432000 4194304 4417452 140736125795472 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26690/tid=26708] ppid=26687 vsize=158608 CPUtime=51.05 cores=1,3,5,7
/proc/26690/task/26708/stat : 26708 (plingeling) R 26687 26690 26655 0 -1 4202560 39246 1251 0 0 5099 6 0 0 25 0 5 0 733059591 162414592 11074 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 203.78
Current children cumulated vsize (KiB) 158608

[startup+102.313 s]
/proc/loadavg: 7.96 7.96 8.16 10/325 26714
/proc/meminfo: memFree=15507556/32950928 swapFree=67085488/67111528
[pid=26690] ppid=26687 vsize=185424 CPUtime=408.1 cores=1,3,5,7
/proc/26690/stat : 26690 (plingeling) S 26687 26690 26655 0 -1 4202496 220802 1251 0 0 40771 39 0 0 18 0 5 0 733059588 189874176 17778 33554432000 4194304 4417452 140736125795472 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/26690/statm: 46356 17778 180 55 0 44242 0
[pid=26690/tid=26705] ppid=26687 vsize=185424 CPUtime=102.27 cores=1,3,5,7
/proc/26690/task/26705/stat : 26705 (plingeling) R 26687 26690 26655 0 -1 4202560 54344 1251 0 0 10218 9 0 0 25 0 5 0 733059591 189874176 17778 33554432000 4194304 4417452 140736125795472 18446744073709551615 4312183 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26690/tid=26706] ppid=26687 vsize=185424 CPUtime=101.22 cores=1,3,5,7
/proc/26690/task/26706/stat : 26706 (plingeling) R 26687 26690 26655 0 -1 4202560 47374 1251 0 0 10115 7 0 0 25 0 5 0 733059591 189874176 17778 33554432000 4194304 4417452 140736125795472 18446744073709551615 4365959 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26690/tid=26707] ppid=26687 vsize=185424 CPUtime=102.26 cores=1,3,5,7
/proc/26690/task/26707/stat : 26707 (plingeling) R 26687 26690 26655 0 -1 4202560 48820 1251 0 0 10218 8 0 0 25 0 5 0 733059591 189874176 17778 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26690/tid=26708] ppid=26687 vsize=185424 CPUtime=102.28 cores=1,3,5,7
/proc/26690/task/26708/stat : 26708 (plingeling) R 26687 26690 26655 0 -1 4202560 67539 1251 0 0 10216 12 0 0 25 0 5 0 733059591 189874176 17778 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 408.1
Current children cumulated vsize (KiB) 185424

[startup+162.3 s]
/proc/loadavg: 8.11 8.00 8.16 10/325 26716
/proc/meminfo: memFree=15486944/32950928 swapFree=67085488/67111528
[pid=26690] ppid=26687 vsize=199268 CPUtime=647.44 cores=1,3,5,7
/proc/26690/stat : 26690 (plingeling) S 26687 26690 26655 0 -1 4202496 343723 1251 0 0 64685 59 0 0 18 0 5 0 733059588 204050432 21224 33554432000 4194304 4417452 140736125795472 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/26690/statm: 49817 21224 180 55 0 47703 0
[pid=26690/tid=26705] ppid=26687 vsize=199268 CPUtime=162.26 cores=1,3,5,7
/proc/26690/task/26705/stat : 26705 (plingeling) R 26687 26690 26655 0 -1 4202560 84965 1251 0 0 16212 14 0 0 25 0 5 0 733059591 204050432 21224 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364629 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26690/tid=26706] ppid=26687 vsize=199268 CPUtime=160.6 cores=1,3,5,7
/proc/26690/task/26706/stat : 26706 (plingeling) R 26687 26690 26655 0 -1 4202560 75470 1251 0 0 16048 12 0 0 25 0 5 0 733059591 204050432 21224 33554432000 4194304 4417452 140736125795472 18446744073709551615 4343771 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26690/tid=26707] ppid=26687 vsize=199268 CPUtime=162.26 cores=1,3,5,7
/proc/26690/task/26707/stat : 26707 (plingeling) R 26687 26690 26655 0 -1 4202560 78302 1251 0 0 16213 13 0 0 25 0 5 0 733059591 204050432 21224 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26690/tid=26708] ppid=26687 vsize=199268 CPUtime=162.27 cores=1,3,5,7
/proc/26690/task/26708/stat : 26708 (plingeling) R 26687 26690 26655 0 -1 4202560 102261 1251 0 0 16209 18 0 0 25 0 5 0 733059591 204050432 21224 33554432000 4194304 4417452 140736125795472 18446744073709551615 4365973 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 647.44
Current children cumulated vsize (KiB) 199268


################
# More data... #
################

[pid=26690/tid=26706] ppid=26687 vsize=247220 CPUtime=754.56 cores=1,3,5,7
/proc/26690/task/26706/stat : 26706 (plingeling) R 26687 26690 26655 0 -1 4202560 295000 1251 0 0 75412 44 0 0 25 0 5 0 733059591 253153280 33212 33554432000 4194304 4417452 140736125795472 18446744073709551615 4259600 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26690/tid=26707] ppid=26687 vsize=247220 CPUtime=762.28 cores=1,3,5,7
/proc/26690/task/26707/stat : 26707 (plingeling) R 26687 26690 26655 0 -1 4202560 310496 1251 0 0 76180 48 0 0 25 0 5 0 733059591 253153280 33212 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26690/tid=26708] ppid=26687 vsize=247220 CPUtime=762.29 cores=1,3,5,7
/proc/26690/task/26708/stat : 26708 (plingeling) R 26687 26690 26655 0 -1 4202560 321223 1251 0 0 76177 52 0 0 25 0 5 0 733059591 253153280 33212 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3041.48
Current children cumulated vsize (KiB) 247220

[startup+822.301 s]
/proc/loadavg: 8.13 8.10 8.12 9/325 26778
/proc/meminfo: memFree=15359496/32950928 swapFree=67085488/67111528
[pid=26690] ppid=26687 vsize=278296 CPUtime=3280.85 cores=1,3,5,7
/proc/26690/stat : 26690 (plingeling) S 26687 26690 26655 0 -1 4202496 1337024 1251 0 0 327873 212 0 0 18 0 5 0 733059588 284975104 40981 33554432000 4194304 4417452 140736125795472 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/26690/statm: 69574 40981 180 55 0 67460 0
[pid=26690/tid=26705] ppid=26687 vsize=278296 CPUtime=822.28 cores=1,3,5,7
/proc/26690/task/26705/stat : 26705 (plingeling) R 26687 26690 26655 0 -1 4202560 333869 1251 0 0 82175 53 0 0 25 0 5 0 733059591 284975104 40981 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26690/tid=26706] ppid=26687 vsize=278296 CPUtime=813.97 cores=1,3,5,7
/proc/26690/task/26706/stat : 26706 (plingeling) R 26687 26690 26655 0 -1 4202560 319586 1251 0 0 81349 48 0 0 25 0 5 0 733059591 284975104 40981 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26690/tid=26707] ppid=26687 vsize=278296 CPUtime=822.26 cores=1,3,5,7
/proc/26690/task/26707/stat : 26707 (plingeling) R 26687 26690 26655 0 -1 4202560 335378 1251 0 0 82174 52 0 0 25 0 5 0 733059591 284975104 40981 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26690/tid=26708] ppid=26687 vsize=278296 CPUtime=822.27 cores=1,3,5,7
/proc/26690/task/26708/stat : 26708 (plingeling) R 26687 26690 26655 0 -1 4202560 345466 1251 0 0 82171 56 0 0 25 0 5 0 733059591 284975104 40981 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3280.85
Current children cumulated vsize (KiB) 278296

[startup+882.3 s]
/proc/loadavg: 8.12 8.09 8.11 9/325 26780
/proc/meminfo: memFree=15342212/32950928 swapFree=67085488/67111528
[pid=26690] ppid=26687 vsize=279900 CPUtime=3520.25 cores=1,3,5,7
/proc/26690/stat : 26690 (plingeling) S 26687 26690 26655 0 -1 4202496 1416429 1251 0 0 351800 225 0 0 18 0 5 0 733059588 286617600 41382 33554432000 4194304 4417452 140736125795472 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/26690/statm: 69975 41382 180 55 0 67861 0
[pid=26690/tid=26705] ppid=26687 vsize=279900 CPUtime=882.28 cores=1,3,5,7
/proc/26690/task/26705/stat : 26705 (plingeling) R 26687 26690 26655 0 -1 4202560 353362 1251 0 0 88172 56 0 0 25 0 5 0 733059591 286617600 41382 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26690/tid=26706] ppid=26687 vsize=279900 CPUtime=873.36 cores=1,3,5,7
/proc/26690/task/26706/stat : 26706 (plingeling) R 26687 26690 26655 0 -1 4202560 334862 1251 0 0 87286 50 0 0 25 0 5 0 733059591 286617600 41382 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364838 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26690/tid=26707] ppid=26687 vsize=279900 CPUtime=882.26 cores=1,3,5,7
/proc/26690/task/26707/stat : 26707 (plingeling) R 26687 26690 26655 0 -1 4202560 358020 1251 0 0 88170 56 0 0 25 0 5 0 733059591 286617600 41382 33554432000 4194304 4417452 140736125795472 18446744073709551615 4247490 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26690/tid=26708] ppid=26687 vsize=279900 CPUtime=882.28 cores=1,3,5,7
/proc/26690/task/26708/stat : 26708 (plingeling) R 26687 26690 26655 0 -1 4202560 367460 1251 0 0 88168 60 0 0 25 0 5 0 733059591 286617600 41382 33554432000 4194304 4417452 140736125795472 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3520.25
Current children cumulated vsize (KiB) 279900

[startup+942.3 s]
/proc/loadavg: 8.12 8.09 8.11 9/325 26782
/proc/meminfo: memFree=15391776/32950928 swapFree=67085488/67111528
[pid=26690] ppid=26687 vsize=250560 CPUtime=3759.67 cores=1,3,5,7
/proc/26690/stat : 26690 (plingeling) S 26687 26690 26655 0 -1 4202496 1487769 1251 0 0 375730 237 0 0 18 0 5 0 733059588 256573440 34058 33554432000 4194304 4417452 140736125795472 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/26690/statm: 62640 34058 180 55 0 60526 0
[pid=26690/tid=26705] ppid=26687 vsize=250560 CPUtime=942.29 cores=1,3,5,7
/proc/26690/task/26705/stat : 26705 (plingeling) R 26687 26690 26655 0 -1 4202560 369025 1251 0 0 94170 59 0 0 25 0 5 0 733059591 256573440 34058 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26690/tid=26706] ppid=26687 vsize=250560 CPUtime=932.77 cores=1,3,5,7
/proc/26690/task/26706/stat : 26706 (plingeling) R 26687 26690 26655 0 -1 4202560 349422 1251 0 0 93225 52 0 0 25 0 5 0 733059591 256573440 34058 33554432000 4194304 4417452 140736125795472 18446744073709551615 4365984 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26690/tid=26707] ppid=26687 vsize=250560 CPUtime=942.27 cores=1,3,5,7
/proc/26690/task/26707/stat : 26707 (plingeling) R 26687 26690 26655 0 -1 4202560 370224 1251 0 0 94169 58 0 0 25 0 5 0 733059591 256573440 34058 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26690/tid=26708] ppid=26687 vsize=250560 CPUtime=942.28 cores=1,3,5,7
/proc/26690/task/26708/stat : 26708 (plingeling) R 26687 26690 26655 0 -1 4202560 396373 1251 0 0 94163 65 0 0 25 0 5 0 733059591 256573440 34058 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3759.67
Current children cumulated vsize (KiB) 250560

[startup+1002.3 s]
/proc/loadavg: 8.10 8.09 8.10 9/325 26783
/proc/meminfo: memFree=15368168/32950928 swapFree=67085488/67111528
[pid=26690] ppid=26687 vsize=268060 CPUtime=3999.07 cores=1,3,5,7
/proc/26690/stat : 26690 (plingeling) S 26687 26690 26655 0 -1 4202496 1565195 1251 0 0 399659 248 0 0 18 0 5 0 733059588 274493440 38427 33554432000 4194304 4417452 140736125795472 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/26690/statm: 67015 38427 180 55 0 64901 0
[pid=26690/tid=26705] ppid=26687 vsize=268060 CPUtime=1002.29 cores=1,3,5,7
/proc/26690/task/26705/stat : 26705 (plingeling) R 26687 26690 26655 0 -1 4202560 386375 1251 0 0 100168 61 0 0 25 0 5 0 733059591 274493440 38427 33554432000 4194304 4417452 140736125795472 18446744073709551615 4344164 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26690/tid=26706] ppid=26687 vsize=268060 CPUtime=992.17 cores=1,3,5,7
/proc/26690/task/26706/stat : 26706 (plingeling) R 26687 26690 26655 0 -1 4202560 363684 1251 0 0 99162 55 0 0 25 0 5 0 733059591 274493440 38427 33554432000 4194304 4417452 140736125795472 18446744073709551615 4265399 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26690/tid=26707] ppid=26687 vsize=268060 CPUtime=1002.27 cores=1,3,5,7
/proc/26690/task/26707/stat : 26707 (plingeling) R 26687 26690 26655 0 -1 4202560 389912 1251 0 0 100166 61 0 0 25 0 5 0 733059591 274493440 38427 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364653 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26690/tid=26708] ppid=26687 vsize=268060 CPUtime=1002.28 cores=1,3,5,7
/proc/26690/task/26708/stat : 26708 (plingeling) R 26687 26690 26655 0 -1 4202560 422499 1251 0 0 100159 69 0 0 25 0 5 0 733059591 274493440 38427 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3999.07
Current children cumulated vsize (KiB) 268060

[startup+1062.3 s]
/proc/loadavg: 8.10 8.08 8.09 9/325 26785
/proc/meminfo: memFree=15344192/32950928 swapFree=67085488/67111528
[pid=26690] ppid=26687 vsize=304000 CPUtime=4238.48 cores=1,3,5,7
/proc/26690/stat : 26690 (plingeling) S 26687 26690 26655 0 -1 4202496 1652143 1251 0 0 423585 263 0 0 18 0 5 0 733059588 311296000 47407 33554432000 4194304 4417452 140736125795472 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/26690/statm: 76000 47407 180 55 0 73886 0
[pid=26690/tid=26705] ppid=26687 vsize=304000 CPUtime=1062.29 cores=1,3,5,7
/proc/26690/task/26705/stat : 26705 (plingeling) R 26687 26690 26655 0 -1 4202560 413041 1251 0 0 106163 66 0 0 25 0 5 0 733059591 311296000 47407 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26690/tid=26706] ppid=26687 vsize=304000 CPUtime=1051.57 cores=1,3,5,7
/proc/26690/task/26706/stat : 26706 (plingeling) R 26687 26690 26655 0 -1 4202560 385857 1251 0 0 105099 58 0 0 25 0 5 0 733059591 311296000 47407 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364829 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26690/tid=26707] ppid=26687 vsize=304000 CPUtime=1062.26 cores=1,3,5,7
/proc/26690/task/26707/stat : 26707 (plingeling) R 26687 26690 26655 0 -1 4202560 405999 1251 0 0 106163 63 0 0 25 0 5 0 733059591 311296000 47407 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26690/tid=26708] ppid=26687 vsize=304000 CPUtime=1062.29 cores=1,3,5,7
/proc/26690/task/26708/stat : 26708 (plingeling) R 26687 26690 26655 0 -1 4202560 444521 1251 0 0 106156 73 0 0 25 0 5 0 733059591 311296000 47407 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4238.48
Current children cumulated vsize (KiB) 304000

[startup+1122.3 s]
/proc/loadavg: 8.08 8.08 8.09 9/325 26786
/proc/meminfo: memFree=15312784/32950928 swapFree=67085488/67111528
[pid=26690] ppid=26687 vsize=288776 CPUtime=4477.88 cores=1,3,5,7
/proc/26690/stat : 26690 (plingeling) S 26687 26690 26655 0 -1 4202496 1740213 1251 0 0 447511 277 0 0 18 0 5 0 733059588 295706624 43601 33554432000 4194304 4417452 140736125795472 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/26690/statm: 72194 43601 180 55 0 70080 0
[pid=26690/tid=26705] ppid=26687 vsize=288776 CPUtime=1122.3 cores=1,3,5,7
/proc/26690/task/26705/stat : 26705 (plingeling) R 26687 26690 26655 0 -1 4202560 438333 1251 0 0 112160 70 0 0 25 0 5 0 733059591 295706624 43601 33554432000 4194304 4417452 140736125795472 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26690/tid=26706] ppid=26687 vsize=288776 CPUtime=1110.98 cores=1,3,5,7
/proc/26690/task/26706/stat : 26706 (plingeling) R 26687 26690 26655 0 -1 4202560 409415 1251 0 0 111035 63 0 0 25 0 5 0 733059591 295706624 43601 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26690/tid=26707] ppid=26687 vsize=288776 CPUtime=1122.26 cores=1,3,5,7
/proc/26690/task/26707/stat : 26707 (plingeling) R 26687 26690 26655 0 -1 4202560 428040 1251 0 0 112160 66 0 0 25 0 5 0 733059591 295706624 43601 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26690/tid=26708] ppid=26687 vsize=288776 CPUtime=1122.29 cores=1,3,5,7
/proc/26690/task/26708/stat : 26708 (plingeling) R 26687 26690 26655 0 -1 4202560 461700 1251 0 0 112153 76 0 0 25 0 5 0 733059591 295706624 43601 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364690 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4477.88
Current children cumulated vsize (KiB) 288776

[startup+1182.3 s]
/proc/loadavg: 8.14 8.10 8.09 9/325 26788
/proc/meminfo: memFree=15327756/32950928 swapFree=67085488/67111528
[pid=26690] ppid=26687 vsize=286624 CPUtime=4717.29 cores=1,3,5,7
/proc/26690/stat : 26690 (plingeling) S 26687 26690 26655 0 -1 4202496 1819302 1251 0 0 471441 288 0 0 18 0 5 0 733059588 293502976 43063 33554432000 4194304 4417452 140736125795472 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/26690/statm: 71656 43063 180 55 0 69542 0
[pid=26690/tid=26705] ppid=26687 vsize=286624 CPUtime=1182.3 cores=1,3,5,7
/proc/26690/task/26705/stat : 26705 (plingeling) R 26687 26690 26655 0 -1 4202560 458804 1251 0 0 118157 73 0 0 25 0 5 0 733059591 293502976 43063 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364765 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26690/tid=26706] ppid=26687 vsize=286624 CPUtime=1170.38 cores=1,3,5,7
/proc/26690/task/26706/stat : 26706 (plingeling) R 26687 26690 26655 0 -1 4202560 427679 1251 0 0 116973 65 0 0 25 0 5 0 733059591 293502976 43063 33554432000 4194304 4417452 140736125795472 18446744073709551615 4250795 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26690/tid=26707] ppid=26687 vsize=286624 CPUtime=1182.26 cores=1,3,5,7
/proc/26690/task/26707/stat : 26707 (plingeling) R 26687 26690 26655 0 -1 4202560 447272 1251 0 0 118157 69 0 0 25 0 5 0 733059591 293502976 43063 33554432000 4194304 4417452 140736125795472 18446744073709551615 4365075 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26690/tid=26708] ppid=26687 vsize=286624 CPUtime=1182.29 cores=1,3,5,7
/proc/26690/task/26708/stat : 26708 (plingeling) R 26687 26690 26655 0 -1 4202560 482822 1251 0 0 118150 79 0 0 25 0 5 0 733059591 293502976 43063 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4717.29
Current children cumulated vsize (KiB) 286624



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.1 s]
/proc/loadavg: 8.09 8.09 8.09 9/325 26789
/proc/meminfo: memFree=15325512/32950928 swapFree=67085488/67111528
[pid=26690] ppid=26687 vsize=284476 CPUtime=4800.3 cores=1,3,5,7
/proc/26690/stat : 26690 (plingeling) S 26687 26690 26655 0 -1 4202496 1845738 1251 0 0 479739 291 0 0 18 0 5 0 733059588 291303424 42526 33554432000 4194304 4417452 140736125795472 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/26690/statm: 71119 42526 180 55 0 69005 0
[pid=26690/tid=26705] ppid=26687 vsize=284476 CPUtime=1203.11 cores=1,3,5,7
/proc/26690/task/26705/stat : 26705 (plingeling) R 26687 26690 26655 0 -1 4202560 461909 1251 0 0 120238 73 0 0 25 0 5 0 733059591 291303424 42526 33554432000 4194304 4417452 140736125795472 18446744073709551615 4232383 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26690/tid=26706] ppid=26687 vsize=284476 CPUtime=1190.97 cores=1,3,5,7
/proc/26690/task/26706/stat : 26706 (plingeling) R 26687 26690 26655 0 -1 4202560 435740 1251 0 0 119031 66 0 0 25 0 5 0 733059591 291303424 42526 33554432000 4194304 4417452 140736125795472 18446744073709551615 4344164 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26690/tid=26707] ppid=26687 vsize=284476 CPUtime=1203.07 cores=1,3,5,7
/proc/26690/task/26707/stat : 26707 (plingeling) R 26687 26690 26655 0 -1 4202560 453657 1251 0 0 120237 70 0 0 25 0 5 0 733059591 291303424 42526 33554432000 4194304 4417452 140736125795472 18446744073709551615 4343760 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26690/tid=26708] ppid=26687 vsize=284476 CPUtime=1203.1 cores=1,3,5,7
/proc/26690/task/26708/stat : 26708 (plingeling) R 26687 26690 26655 0 -1 4202560 491707 1251 0 0 120230 80 0 0 25 0 5 0 733059591 291303424 42526 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4800.3
Current children cumulated vsize (KiB) 284476

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1203.1 s]
/proc/loadavg: 8.09 8.09 8.09 9/325 26789
/proc/meminfo: memFree=15325512/32950928 swapFree=67085488/67111528
[pid=26690] ppid=26687 vsize=284476 CPUtime=4800.3 cores=1,3,5,7
/proc/26690/stat : 26690 (plingeling) S 26687 26690 26655 0 -1 4202496 1845738 1251 0 0 479739 291 0 0 18 0 5 0 733059588 291303424 42526 33554432000 4194304 4417452 140736125795472 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/26690/statm: 71119 42526 180 55 0 69005 0
[pid=26690/tid=26705] ppid=26687 vsize=284476 CPUtime=1203.11 cores=1,3,5,7
/proc/26690/task/26705/stat : 26705 (plingeling) R 26687 26690 26655 0 -1 4202560 461909 1251 0 0 120238 73 0 0 25 0 5 0 733059591 291303424 42526 33554432000 4194304 4417452 140736125795472 18446744073709551615 4232383 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26690/tid=26706] ppid=26687 vsize=284476 CPUtime=1190.97 cores=1,3,5,7
/proc/26690/task/26706/stat : 26706 (plingeling) R 26687 26690 26655 0 -1 4202560 435740 1251 0 0 119031 66 0 0 25 0 5 0 733059591 291303424 42526 33554432000 4194304 4417452 140736125795472 18446744073709551615 4344164 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26690/tid=26707] ppid=26687 vsize=284476 CPUtime=1203.07 cores=1,3,5,7
/proc/26690/task/26707/stat : 26707 (plingeling) R 26687 26690 26655 0 -1 4202560 453657 1251 0 0 120237 70 0 0 25 0 5 0 733059591 291303424 42526 33554432000 4194304 4417452 140736125795472 18446744073709551615 4343760 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26690/tid=26708] ppid=26687 vsize=284476 CPUtime=1203.1 cores=1,3,5,7
/proc/26690/task/26708/stat : 26708 (plingeling) R 26687 26690 26655 0 -1 4202560 491707 1251 0 0 120230 80 0 0 25 0 5 0 733059591 291303424 42526 33554432000 4194304 4417452 140736125795472 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4800.3
Current children cumulated vsize (KiB) 284476

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1203.13
CPU time (s): 4800.36
CPU user time (s): 4797.43
CPU system time (s): 2.92955
CPU usage (%): 398.991
Max. virtual memory (cumulated for all children) (KiB): 320184

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4797.43
system time used= 2.92955
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1846989
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 259
involuntary context switches= 39308

runsolver used 3.41148 second user time and 8.72867 second system time

The end

Launcher Data 

Begin job on node114 at 2011-05-15 04:58:09
IDJOB=3371118
IDBENCH=83298
IDSOLVER=1848
FILE ID=node114/3371118-1305428289
RUNJOBID= node114-1305428289-26674
PBS_JOBID= 13324611
Free space on /tmp= 73000 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/crafted/kullmann/VanDerWaerden/VanderWaerden_pd_3k/VanDerWaerden_pd_2-3-21_406.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3371118-1305428289/watcher-3371118-1305428289 -o /tmp/evaluation-result-3371118-1305428289/solver-3371118-1305428289 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3371118-1305428289.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 906590ef62b3db2841b2d529fd1636f7
RANDOM SEED=1392788503

node114.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.73
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      15643504 kB
Buffers:        286652 kB
Cached:        6259996 kB
SwapCached:         96 kB
Active:       11888584 kB
Inactive:      5010552 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      15643504 kB
SwapTotal:    67111528 kB
SwapFree:     67085488 kB
Dirty:            1352 kB
Writeback:           0 kB
AnonPages:    10352836 kB
Mapped:          25888 kB
Slab:           305956 kB
PageTables:      37232 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 52431160 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73000 MiB
End job on node114 at 2011-05-15 05:18:14