Trace number 3370918

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.37 1201.72

General information on the benchmark

Namecrafted/kullmann/GreenTao/
GreenTaok1k2/GreenTao_2-3-6_2071.cnf
MD5SUM337bdbc5d8bf0013ca73aeffa06cffe8
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark95.7414
Satisfiable
(Un)Satisfiability was proved
Number of variables2070
Number of clauses158884
Sum of the clauses size478584
Maximum clause length6
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3158240
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 5644

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3370918-1307643626/watcher-3370918-1307643626 -o /tmp/evaluation-result-3370918-1307643626/solver-3370918-1307643626 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3370918-1307643626.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 5.92 3.14 2.71 5/181 4953
/proc/meminfo: memFree=32372200/32951124 swapFree=67056964/67111528
[pid=4953] ppid=4951 vsize=9460 CPUtime=0 cores=0,2,4,6
/proc/4953/stat : 4953 (plingeling) R 4951 4953 4755 0 -1 4202496 459 1252 0 0 0 0 0 0 18 0 1 0 856778491 9687040 388 33554432000 4194304 4417452 140735148580288 18446744073709551615 233510297344 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/4953/statm: 2365 388 148 55 0 251 0

[startup+0.071626 s]
/proc/loadavg: 5.92 3.14 2.71 5/181 4953
/proc/meminfo: memFree=32372200/32951124 swapFree=67056964/67111528
[pid=4953] ppid=4951 vsize=25812 CPUtime=0.06 cores=0,2,4,6
/proc/4953/stat : 4953 (plingeling) R 4951 4953 4755 0 -1 4202496 4563 1252 0 0 6 0 0 0 18 0 1 0 856778491 26431488 4461 33554432000 4194304 4417452 140735148580288 18446744073709551615 4259618 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/4953/statm: 6453 4461 149 55 0 4339 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 25812

[startup+0.100612 s]
/proc/loadavg: 5.92 3.14 2.71 5/181 4953
/proc/meminfo: memFree=32372200/32951124 swapFree=67056964/67111528
[pid=4953] ppid=4951 vsize=25812 CPUtime=0.09 cores=0,2,4,6
/proc/4953/stat : 4953 (plingeling) R 4951 4953 4755 0 -1 4202496 4563 1252 0 0 9 0 0 0 18 0 1 0 856778491 26431488 4461 33554432000 4194304 4417452 140735148580288 18446744073709551615 233510297344 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/4953/statm: 6453 4461 149 55 0 4339 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 25812

[startup+0.305535 s]
/proc/loadavg: 5.92 3.14 2.71 5/181 4953
/proc/meminfo: memFree=32372200/32951124 swapFree=67056964/67111528
[pid=4953] ppid=4951 vsize=155548 CPUtime=0.61 cores=0,2,4,6
/proc/4953/stat : 4953 (plingeling) S 4951 4953 4755 0 -1 4202496 15108 1252 0 0 60 1 0 0 18 0 5 0 856778491 159281152 9817 33554432000 4194304 4417452 140735148580288 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/4953/statm: 38887 9817 177 55 0 36773 0
Current children cumulated CPU time (s) 0.61
Current children cumulated vsize (KiB) 155548

[startup+0.700465 s]
/proc/loadavg: 5.92 3.14 2.71 5/181 4953
/proc/meminfo: memFree=32372200/32951124 swapFree=67056964/67111528
[pid=4953] ppid=4951 vsize=163732 CPUtime=2.2 cores=0,2,4,6
/proc/4953/stat : 4953 (plingeling) S 4951 4953 4755 0 -1 4202496 17669 1252 0 0 218 2 0 0 18 0 5 0 856778491 167661568 12344 33554432000 4194304 4417452 140735148580288 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/4953/statm: 40933 12344 179 55 0 38819 0
Current children cumulated CPU time (s) 2.2
Current children cumulated vsize (KiB) 163732

[startup+1.50032 s]
/proc/loadavg: 5.92 3.14 2.71 9/186 4963
/proc/meminfo: memFree=32308944/32951124 swapFree=67056964/67111528
[pid=4953] ppid=4951 vsize=172820 CPUtime=5.38 cores=0,2,4,6
/proc/4953/stat : 4953 (plingeling) S 4951 4953 4755 0 -1 4202496 19941 1252 0 0 536 2 0 0 18 0 5 0 856778491 176967680 14616 33554432000 4194304 4417452 140735148580288 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/4953/statm: 43205 14616 179 55 0 41091 0
[pid=4953/tid=4960] ppid=4951 vsize=172820 CPUtime=1.3 cores=0,2,4,6
/proc/4953/task/4960/stat : 4960 (plingeling) R 4951 4953 4755 0 -1 4202560 3641 1252 0 0 130 0 0 0 19 0 5 0 856778510 176967680 14616 33554432000 4194304 4417452 140735148580288 18446744073709551615 4249222 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4953/tid=4961] ppid=4951 vsize=172820 CPUtime=1.3 cores=0,2,4,6
/proc/4953/task/4961/stat : 4961 (plingeling) R 4951 4953 4755 0 -1 4202560 4140 1252 0 0 130 0 0 0 25 0 5 0 856778510 176967680 14616 33554432000 4194304 4417452 140735148580288 18446744073709551615 4249222 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4953/tid=4962] ppid=4951 vsize=172820 CPUtime=1.29 cores=0,2,4,6
/proc/4953/task/4962/stat : 4962 (plingeling) R 4951 4953 4755 0 -1 4202560 3238 1252 0 0 129 0 0 0 25 0 5 0 856778510 176967680 14616 33554432000 4194304 4417452 140735148580288 18446744073709551615 4249110 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4953/tid=4963] ppid=4951 vsize=172820 CPUtime=1.28 cores=0,2,4,6
/proc/4953/task/4963/stat : 4963 (plingeling) R 4951 4953 4755 0 -1 4202560 71 1252 0 0 128 0 0 0 25 0 5 0 856778510 176967680 14616 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364583 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 5.38
Current children cumulated vsize (KiB) 172820

[startup+3.10103 s]
/proc/loadavg: 5.92 3.14 2.71 9/186 4963
/proc/meminfo: memFree=32298768/32951124 swapFree=67056964/67111528
[pid=4953] ppid=4951 vsize=176916 CPUtime=11.78 cores=0,2,4,6
/proc/4953/stat : 4953 (plingeling) S 4951 4953 4755 0 -1 4202496 20965 1252 0 0 1176 2 0 0 18 0 5 0 856778491 181161984 15640 33554432000 4194304 4417452 140735148580288 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/4953/statm: 44229 15640 179 55 0 42115 0
[pid=4953/tid=4960] ppid=4951 vsize=176916 CPUtime=2.9 cores=0,2,4,6
/proc/4953/task/4960/stat : 4960 (plingeling) R 4951 4953 4755 0 -1 4202560 4665 1252 0 0 290 0 0 0 24 0 5 0 856778510 181161984 15640 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364604 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4953/tid=4961] ppid=4951 vsize=176916 CPUtime=2.9 cores=0,2,4,6
/proc/4953/task/4961/stat : 4961 (plingeling) R 4951 4953 4755 0 -1 4202560 4140 1252 0 0 290 0 0 0 25 0 5 0 856778510 181161984 15640 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4953/tid=4962] ppid=4951 vsize=176916 CPUtime=2.89 cores=0,2,4,6
/proc/4953/task/4962/stat : 4962 (plingeling) R 4951 4953 4755 0 -1 4202560 3238 1252 0 0 289 0 0 0 25 0 5 0 856778510 181161984 15640 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364537 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4953/tid=4963] ppid=4951 vsize=176916 CPUtime=2.88 cores=0,2,4,6
/proc/4953/task/4963/stat : 4963 (plingeling) R 4951 4953 4755 0 -1 4202560 71 1252 0 0 288 0 0 0 25 0 5 0 856778510 181161984 15640 33554432000 4194304 4417452 140735148580288 18446744073709551615 4357207 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 11.78
Current children cumulated vsize (KiB) 176916

[startup+6.30046 s]
/proc/loadavg: 6.16 3.23 2.75 9/186 4963
/proc/meminfo: memFree=32294428/32951124 swapFree=67056964/67111528
[pid=4953] ppid=4951 vsize=182032 CPUtime=24.57 cores=0,2,4,6
/proc/4953/stat : 4953 (plingeling) S 4951 4953 4755 0 -1 4202496 22244 1252 0 0 2454 3 0 0 18 0 5 0 856778491 186400768 16919 33554432000 4194304 4417452 140735148580288 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/4953/statm: 45508 16919 179 55 0 43394 0
[pid=4953/tid=4960] ppid=4951 vsize=182032 CPUtime=6.1 cores=0,2,4,6
/proc/4953/task/4960/stat : 4960 (plingeling) R 4951 4953 4755 0 -1 4202560 5176 1252 0 0 610 0 0 0 25 0 5 0 856778510 186400768 16919 33554432000 4194304 4417452 140735148580288 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4953/tid=4961] ppid=4951 vsize=182032 CPUtime=6.1 cores=0,2,4,6
/proc/4953/task/4961/stat : 4961 (plingeling) R 4951 4953 4755 0 -1 4202560 4346 1252 0 0 610 0 0 0 25 0 5 0 856778510 186400768 16919 33554432000 4194304 4417452 140735148580288 18446744073709551615 4249978 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4953/tid=4962] ppid=4951 vsize=182032 CPUtime=6.09 cores=0,2,4,6
/proc/4953/task/4962/stat : 4962 (plingeling) R 4951 4953 4755 0 -1 4202560 3757 1252 0 0 609 0 0 0 25 0 5 0 856778510 186400768 16919 33554432000 4194304 4417452 140735148580288 18446744073709551615 4265429 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4953/tid=4963] ppid=4951 vsize=182032 CPUtime=6.06 cores=0,2,4,6
/proc/4953/task/4963/stat : 4963 (plingeling) R 4951 4953 4755 0 -1 4202560 114 1252 0 0 606 0 0 0 25 0 5 0 856778510 186400768 16919 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 24.57
Current children cumulated vsize (KiB) 182032

[startup+12.7003 s]
/proc/loadavg: 6.31 3.31 2.78 9/186 4963
/proc/meminfo: memFree=32299560/32951124 swapFree=67056964/67111528
[pid=4953] ppid=4951 vsize=188688 CPUtime=50.14 cores=0,2,4,6
/proc/4953/stat : 4953 (plingeling) S 4951 4953 4755 0 -1 4202496 23908 1252 0 0 5011 3 0 0 18 0 5 0 856778491 193216512 18583 33554432000 4194304 4417452 140735148580288 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/4953/statm: 47172 18583 179 55 0 45058 0
[pid=4953/tid=4960] ppid=4951 vsize=188688 CPUtime=12.5 cores=0,2,4,6
/proc/4953/task/4960/stat : 4960 (plingeling) R 4951 4953 4755 0 -1 4202560 5528 1252 0 0 1250 0 0 0 25 0 5 0 856778510 193216512 18583 33554432000 4194304 4417452 140735148580288 18446744073709551615 4249222 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4953/tid=4961] ppid=4951 vsize=188688 CPUtime=12.5 cores=0,2,4,6
/proc/4953/task/4961/stat : 4961 (plingeling) R 4951 4953 4755 0 -1 4202560 4677 1252 0 0 1250 0 0 0 25 0 5 0 856778510 193216512 18583 33554432000 4194304 4417452 140735148580288 18446744073709551615 4249110 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4953/tid=4962] ppid=4951 vsize=188688 CPUtime=12.49 cores=0,2,4,6
/proc/4953/task/4962/stat : 4962 (plingeling) R 4951 4953 4755 0 -1 4202560 4157 1252 0 0 1249 0 0 0 25 0 5 0 856778510 193216512 18583 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364650 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4953/tid=4963] ppid=4951 vsize=188688 CPUtime=12.43 cores=0,2,4,6
/proc/4953/task/4963/stat : 4963 (plingeling) R 4951 4953 4755 0 -1 4202560 695 1252 0 0 1243 0 0 0 25 0 5 0 856778510 193216512 18583 33554432000 4194304 4417452 140735148580288 18446744073709551615 4357947 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 50.14
Current children cumulated vsize (KiB) 188688

[startup+25.501 s]
/proc/loadavg: 6.68 3.54 2.86 9/186 4963
/proc/meminfo: memFree=32242376/32951124 swapFree=67056964/67111528
[pid=4953] ppid=4951 vsize=213372 CPUtime=101.28 cores=0,2,4,6
/proc/4953/stat : 4953 (plingeling) S 4951 4953 4755 0 -1 4202496 30079 1252 0 0 10124 4 0 0 18 0 5 0 856778491 218492928 24754 33554432000 4194304 4417452 140735148580288 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/4953/statm: 53343 24754 179 55 0 51229 0
[pid=4953/tid=4960] ppid=4951 vsize=213372 CPUtime=25.3 cores=0,2,4,6
/proc/4953/task/4960/stat : 4960 (plingeling) R 4951 4953 4755 0 -1 4202560 5528 1252 0 0 2530 0 0 0 25 0 5 0 856778510 218492928 24754 33554432000 4194304 4417452 140735148580288 18446744073709551615 4232654 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4953/tid=4961] ppid=4951 vsize=213372 CPUtime=25.3 cores=0,2,4,6
/proc/4953/task/4961/stat : 4961 (plingeling) R 4951 4953 4755 0 -1 4202560 5032 1252 0 0 2530 0 0 0 25 0 5 0 856778510 218492928 24754 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4953/tid=4962] ppid=4951 vsize=213372 CPUtime=25.29 cores=0,2,4,6
/proc/4953/task/4962/stat : 4962 (plingeling) R 4951 4953 4755 0 -1 4202560 4398 1252 0 0 2529 0 0 0 25 0 5 0 856778510 218492928 24754 33554432000 4194304 4417452 140735148580288 18446744073709551615 4366402 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4953/tid=4963] ppid=4951 vsize=213372 CPUtime=25.16 cores=0,2,4,6
/proc/4953/task/4963/stat : 4963 (plingeling) R 4951 4953 4755 0 -1 4202560 6270 1252 0 0 2516 0 0 0 25 0 5 0 856778510 218492928 24754 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364537 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 101.28
Current children cumulated vsize (KiB) 213372

[startup+51.1004 s]
/proc/loadavg: 7.19 3.91 3.00 9/186 4964
/proc/meminfo: memFree=32194728/32951124 swapFree=67056964/67111528
[pid=4953] ppid=4951 vsize=221932 CPUtime=203.55 cores=0,2,4,6
/proc/4953/stat : 4953 (plingeling) S 4951 4953 4755 0 -1 4202496 32221 1252 0 0 20351 4 0 0 18 0 5 0 856778491 227258368 26896 33554432000 4194304 4417452 140735148580288 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/4953/statm: 55483 26896 179 55 0 53369 0
[pid=4953/tid=4960] ppid=4951 vsize=221932 CPUtime=50.9 cores=0,2,4,6
/proc/4953/task/4960/stat : 4960 (plingeling) R 4951 4953 4755 0 -1 4202560 5528 1252 0 0 5090 0 0 0 25 0 5 0 856778510 227258368 26896 33554432000 4194304 4417452 140735148580288 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4953/tid=4961] ppid=4951 vsize=221932 CPUtime=50.9 cores=0,2,4,6
/proc/4953/task/4961/stat : 4961 (plingeling) R 4951 4953 4755 0 -1 4202560 5067 1252 0 0 5090 0 0 0 25 0 5 0 856778510 227258368 26896 33554432000 4194304 4417452 140735148580288 18446744073709551615 4249130 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4953/tid=4962] ppid=4951 vsize=221932 CPUtime=50.89 cores=0,2,4,6
/proc/4953/task/4962/stat : 4962 (plingeling) R 4951 4953 4755 0 -1 4202560 4398 1252 0 0 5089 0 0 0 25 0 5 0 856778510 227258368 26896 33554432000 4194304 4417452 140735148580288 18446744073709551615 4249186 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4953/tid=4963] ppid=4951 vsize=221932 CPUtime=50.64 cores=0,2,4,6
/proc/4953/task/4963/stat : 4963 (plingeling) R 4951 4953 4755 0 -1 4202560 8377 1252 0 0 5063 1 0 0 25 0 5 0 856778510 227258368 26896 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364669 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 203.55
Current children cumulated vsize (KiB) 221932

[startup+102.306 s]
/proc/loadavg: 7.73 4.62 3.29 9/186 4965
/proc/meminfo: memFree=32154252/32951124 swapFree=67056964/67111528
[pid=4953] ppid=4951 vsize=245792 CPUtime=408.13 cores=0,2,4,6
/proc/4953/stat : 4953 (plingeling) S 4951 4953 4755 0 -1 4202496 38184 1252 0 0 40808 5 0 0 18 0 5 0 856778491 251691008 32859 33554432000 4194304 4417452 140735148580288 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/4953/statm: 61448 32859 179 55 0 59334 0
[pid=4953/tid=4960] ppid=4951 vsize=245792 CPUtime=102.11 cores=0,2,4,6
/proc/4953/task/4960/stat : 4960 (plingeling) R 4951 4953 4755 0 -1 4202560 7047 1252 0 0 10211 0 0 0 25 0 5 0 856778510 251691008 32859 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4953/tid=4961] ppid=4951 vsize=245792 CPUtime=102.11 cores=0,2,4,6
/proc/4953/task/4961/stat : 4961 (plingeling) R 4951 4953 4755 0 -1 4202560 5067 1252 0 0 10210 1 0 0 25 0 5 0 856778510 251691008 32859 33554432000 4194304 4417452 140735148580288 18446744073709551615 4276028 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4953/tid=4962] ppid=4951 vsize=245792 CPUtime=102.1 cores=0,2,4,6
/proc/4953/task/4962/stat : 4962 (plingeling) R 4951 4953 4755 0 -1 4202560 5344 1252 0 0 10210 0 0 0 25 0 5 0 856778510 251691008 32859 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364861 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4953/tid=4963] ppid=4951 vsize=245792 CPUtime=101.6 cores=0,2,4,6
/proc/4953/task/4963/stat : 4963 (plingeling) R 4951 4953 4755 0 -1 4202560 11875 1252 0 0 10159 1 0 0 25 0 5 0 856778510 251691008 32859 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 408.13
Current children cumulated vsize (KiB) 245792

[startup+162.3 s]
/proc/loadavg: 7.95 5.20 3.56 9/186 4967
/proc/meminfo: memFree=32054744/32951124 swapFree=67056964/67111528
[pid=4953] ppid=4951 vsize=266024 CPUtime=647.82 cores=0,2,4,6
/proc/4953/stat : 4953 (plingeling) S 4951 4953 4755 0 -1 4202496 43242 1252 0 0 64776 6 0 0 18 0 5 0 856778491 272408576 37917 33554432000 4194304 4417452 140735148580288 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/4953/statm: 66506 37917 179 55 0 64392 0
[pid=4953/tid=4960] ppid=4951 vsize=266024 CPUtime=162.1 cores=0,2,4,6
/proc/4953/task/4960/stat : 4960 (plingeling) R 4951 4953 4755 0 -1 4202560 7047 1252 0 0 16210 0 0 0 25 0 5 0 856778510 272408576 37917 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4953/tid=4961] ppid=4951 vsize=266024 CPUtime=162.1 cores=0,2,4,6
/proc/4953/task/4961/stat : 4961 (plingeling) R 4951 4953 4755 0 -1 4202560 6549 1252 0 0 16209 1 0 0 25 0 5 0 856778510 272408576 37917 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4953/tid=4962] ppid=4951 vsize=266024 CPUtime=162.1 cores=0,2,4,6
/proc/4953/task/4962/stat : 4962 (plingeling) R 4951 4953 4755 0 -1 4202560 7312 1252 0 0 16209 1 0 0 25 0 5 0 856778510 272408576 37917 33554432000 4194304 4417452 140735148580288 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4953/tid=4963] ppid=4951 vsize=266024 CPUtime=161.3 cores=0,2,4,6
/proc/4953/task/4963/stat : 4963 (plingeling) R 4951 4953 4755 0 -1 4202560 13483 1252 0 0 16129 1 0 0 25 0 5 0 856778510 272408576 37917 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364821 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 647.82
Current children cumulated vsize (KiB) 266024


################
# More data... #
################

[pid=4953/tid=4961] ppid=4951 vsize=361060 CPUtime=762.12 cores=0,2,4,6
/proc/4953/task/4961/stat : 4961 (plingeling) R 4951 4953 4755 0 -1 4202560 30833 1252 0 0 76207 5 0 0 25 0 5 0 856778510 369725440 61676 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364597 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4953/tid=4962] ppid=4951 vsize=361060 CPUtime=762.1 cores=0,2,4,6
/proc/4953/task/4962/stat : 4962 (plingeling) R 4951 4953 4755 0 -1 4202560 64979 1252 0 0 76202 8 0 0 25 0 5 0 856778510 369725440 61676 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4953/tid=4963] ppid=4951 vsize=361060 CPUtime=758.34 cores=0,2,4,6
/proc/4953/task/4963/stat : 4963 (plingeling) R 4951 4953 4755 0 -1 4202560 135266 1252 0 0 75814 20 0 0 25 0 5 0 856778510 369725440 61676 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3044.91
Current children cumulated vsize (KiB) 361060

[startup+822.301 s]
/proc/loadavg: 8.18 7.80 5.87 9/186 4987
/proc/meminfo: memFree=31022996/32951124 swapFree=67056964/67111528
[pid=4953] ppid=4951 vsize=382724 CPUtime=3284.59 cores=0,2,4,6
/proc/4953/stat : 4953 (plingeling) S 4951 4953 4755 0 -1 4202496 320194 1252 0 0 328413 46 0 0 18 0 5 0 856778491 391909376 67092 33554432000 4194304 4417452 140735148580288 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/4953/statm: 95681 67092 179 55 0 93567 0
[pid=4953/tid=4960] ppid=4951 vsize=382724 CPUtime=822.12 cores=0,2,4,6
/proc/4953/task/4960/stat : 4960 (plingeling) R 4951 4953 4755 0 -1 4202560 57598 1252 0 0 82206 6 0 0 25 0 5 0 856778510 391909376 67092 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364537 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4953/tid=4961] ppid=4951 vsize=382724 CPUtime=822.12 cores=0,2,4,6
/proc/4953/task/4961/stat : 4961 (plingeling) R 4951 4953 4755 0 -1 4202560 30833 1252 0 0 82207 5 0 0 25 0 5 0 856778510 391909376 67092 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364984 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4953/tid=4962] ppid=4951 vsize=382724 CPUtime=822.1 cores=0,2,4,6
/proc/4953/task/4962/stat : 4962 (plingeling) R 4951 4953 4755 0 -1 4202560 69831 1252 0 0 82201 9 0 0 25 0 5 0 856778510 391909376 67092 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4953/tid=4963] ppid=4951 vsize=382724 CPUtime=818.03 cores=0,2,4,6
/proc/4953/task/4963/stat : 4963 (plingeling) R 4951 4953 4755 0 -1 4202560 153081 1252 0 0 81780 23 0 0 25 0 5 0 856778510 391909376 67092 33554432000 4194304 4417452 140735148580288 18446744073709551615 4357159 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3284.59
Current children cumulated vsize (KiB) 382724

[startup+882.3 s]
/proc/loadavg: 8.12 7.85 6.02 9/186 4989
/proc/meminfo: memFree=31135164/32951124 swapFree=67056964/67111528
[pid=4953] ppid=4951 vsize=401472 CPUtime=3524.29 cores=0,2,4,6
/proc/4953/stat : 4953 (plingeling) S 4951 4953 4755 0 -1 4202496 365198 1252 0 0 352376 53 0 0 18 0 5 0 856778491 411107328 71779 33554432000 4194304 4417452 140735148580288 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/4953/statm: 100368 71779 179 55 0 98254 0
[pid=4953/tid=4960] ppid=4951 vsize=401472 CPUtime=882.12 cores=0,2,4,6
/proc/4953/task/4960/stat : 4960 (plingeling) R 4951 4953 4755 0 -1 4202560 66233 1252 0 0 88204 8 0 0 25 0 5 0 856778510 411107328 71779 33554432000 4194304 4417452 140735148580288 18446744073709551615 4236930 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4953/tid=4961] ppid=4951 vsize=401472 CPUtime=882.12 cores=0,2,4,6
/proc/4953/task/4961/stat : 4961 (plingeling) R 4951 4953 4755 0 -1 4202560 48911 1252 0 0 88204 8 0 0 25 0 5 0 856778510 411107328 71779 33554432000 4194304 4417452 140735148580288 18446744073709551615 4250961 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4953/tid=4962] ppid=4951 vsize=401472 CPUtime=882.09 cores=0,2,4,6
/proc/4953/task/4962/stat : 4962 (plingeling) R 4951 4953 4755 0 -1 4202560 75173 1252 0 0 88200 9 0 0 25 0 5 0 856778510 411107328 71779 33554432000 4194304 4417452 140735148580288 18446744073709551615 4237278 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4953/tid=4963] ppid=4951 vsize=401472 CPUtime=877.73 cores=0,2,4,6
/proc/4953/task/4963/stat : 4963 (plingeling) R 4951 4953 4755 0 -1 4202560 166030 1252 0 0 87748 25 0 0 25 0 5 0 856778510 411107328 71779 33554432000 4194304 4417452 140735148580288 18446744073709551615 4365042 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3524.29
Current children cumulated vsize (KiB) 401472

[startup+942.3 s]
/proc/loadavg: 8.12 7.89 6.13 9/186 4991
/proc/meminfo: memFree=31140684/32951124 swapFree=67056964/67111528
[pid=4953] ppid=4951 vsize=373848 CPUtime=3764 cores=0,2,4,6
/proc/4953/stat : 4953 (plingeling) S 4951 4953 4755 0 -1 4202496 404449 1252 0 0 376342 58 0 0 18 0 5 0 856778491 382820352 64873 33554432000 4194304 4417452 140735148580288 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/4953/statm: 93462 64873 179 55 0 91348 0
[pid=4953/tid=4960] ppid=4951 vsize=373848 CPUtime=942.13 cores=0,2,4,6
/proc/4953/task/4960/stat : 4960 (plingeling) R 4951 4953 4755 0 -1 4202560 73788 1252 0 0 94204 9 0 0 25 0 5 0 856778510 382820352 64873 33554432000 4194304 4417452 140735148580288 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4953/tid=4961] ppid=4951 vsize=373848 CPUtime=942.12 cores=0,2,4,6
/proc/4953/task/4961/stat : 4961 (plingeling) R 4951 4953 4755 0 -1 4202560 65025 1252 0 0 94202 10 0 0 25 0 5 0 856778510 382820352 64873 33554432000 4194304 4417452 140735148580288 18446744073709551615 4365512 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4953/tid=4962] ppid=4951 vsize=373848 CPUtime=942.1 cores=0,2,4,6
/proc/4953/task/4962/stat : 4962 (plingeling) R 4951 4953 4755 0 -1 4202560 79565 1252 0 0 94200 10 0 0 25 0 5 0 856778510 382820352 64873 33554432000 4194304 4417452 140735148580288 18446744073709551615 4366563 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4953/tid=4963] ppid=4951 vsize=373848 CPUtime=937.45 cores=0,2,4,6
/proc/4953/task/4963/stat : 4963 (plingeling) R 4951 4953 4755 0 -1 4202560 177220 1252 0 0 93718 27 0 0 25 0 5 0 856778510 382820352 64873 33554432000 4194304 4417452 140735148580288 18446744073709551615 4365954 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3764
Current children cumulated vsize (KiB) 373848

[startup+1002.3 s]
/proc/loadavg: 8.13 7.94 6.27 9/186 4992
/proc/meminfo: memFree=31131940/32951124 swapFree=67056964/67111528
[pid=4953] ppid=4951 vsize=392408 CPUtime=4003.7 cores=0,2,4,6
/proc/4953/stat : 4953 (plingeling) S 4951 4953 4755 0 -1 4202496 449728 1252 0 0 400305 65 0 0 18 0 5 0 856778491 401825792 69513 33554432000 4194304 4417452 140735148580288 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/4953/statm: 98102 69513 179 55 0 95988 0
[pid=4953/tid=4960] ppid=4951 vsize=392408 CPUtime=1002.12 cores=0,2,4,6
/proc/4953/task/4960/stat : 4960 (plingeling) R 4951 4953 4755 0 -1 4202560 80883 1252 0 0 100203 9 0 0 25 0 5 0 856778510 401825792 69513 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364838 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4953/tid=4961] ppid=4951 vsize=392408 CPUtime=1002.12 cores=0,2,4,6
/proc/4953/task/4961/stat : 4961 (plingeling) R 4951 4953 4755 0 -1 4202560 74465 1252 0 0 100201 11 0 0 25 0 5 0 856778510 401825792 69513 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4953/tid=4962] ppid=4951 vsize=392408 CPUtime=1002.1 cores=0,2,4,6
/proc/4953/task/4962/stat : 4962 (plingeling) R 4951 4953 4755 0 -1 4202560 90411 1252 0 0 100198 12 0 0 25 0 5 0 856778510 401825792 69513 33554432000 4194304 4417452 140735148580288 18446744073709551615 4264940 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4953/tid=4963] ppid=4951 vsize=392408 CPUtime=997.13 cores=0,2,4,6
/proc/4953/task/4963/stat : 4963 (plingeling) R 4951 4953 4755 0 -1 4202560 195118 1252 0 0 99684 29 0 0 25 0 5 0 856778510 401825792 69513 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4003.7
Current children cumulated vsize (KiB) 392408

[startup+1062.3 s]
/proc/loadavg: 8.15 7.98 6.37 9/186 4994
/proc/meminfo: memFree=30938196/32951124 swapFree=67056964/67111528
[pid=4953] ppid=4951 vsize=370116 CPUtime=4243.41 cores=0,2,4,6
/proc/4953/stat : 4953 (plingeling) S 4951 4953 4755 0 -1 4202496 481712 1252 0 0 424271 70 0 0 18 0 5 0 856778491 378998784 63940 33554432000 4194304 4417452 140735148580288 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/4953/statm: 92529 63940 179 55 0 90415 0
[pid=4953/tid=4960] ppid=4951 vsize=370116 CPUtime=1062.13 cores=0,2,4,6
/proc/4953/task/4960/stat : 4960 (plingeling) R 4951 4953 4755 0 -1 4202560 86354 1252 0 0 106203 10 0 0 25 0 5 0 856778510 378998784 63940 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4953/tid=4961] ppid=4951 vsize=370116 CPUtime=1062.13 cores=0,2,4,6
/proc/4953/task/4961/stat : 4961 (plingeling) R 4951 4953 4755 0 -1 4202560 83053 1252 0 0 106200 13 0 0 25 0 5 0 856778510 378998784 63940 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4953/tid=4962] ppid=4951 vsize=370116 CPUtime=1062.09 cores=0,2,4,6
/proc/4953/task/4962/stat : 4962 (plingeling) R 4951 4953 4755 0 -1 4202560 94393 1252 0 0 106197 12 0 0 25 0 5 0 856778510 378998784 63940 33554432000 4194304 4417452 140735148580288 18446744073709551615 4366469 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4953/tid=4963] ppid=4951 vsize=370116 CPUtime=1056.84 cores=0,2,4,6
/proc/4953/task/4963/stat : 4963 (plingeling) R 4951 4953 4755 0 -1 4202560 209061 1252 0 0 105653 31 0 0 25 0 5 0 856778510 378998784 63940 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4243.41
Current children cumulated vsize (KiB) 370116

[startup+1122.3 s]
/proc/loadavg: 8.14 8.01 6.48 9/186 4995
/proc/meminfo: memFree=31320428/32951124 swapFree=67056964/67111528
[pid=4953] ppid=4951 vsize=391200 CPUtime=4483.1 cores=0,2,4,6
/proc/4953/stat : 4953 (plingeling) S 4951 4953 4755 0 -1 4202496 523698 1252 0 0 448234 76 0 0 18 0 5 0 856778491 400588800 69211 33554432000 4194304 4417452 140735148580288 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/4953/statm: 97800 69211 179 55 0 95686 0
[pid=4953/tid=4960] ppid=4951 vsize=391200 CPUtime=1122.13 cores=0,2,4,6
/proc/4953/task/4960/stat : 4960 (plingeling) R 4951 4953 4755 0 -1 4202560 92744 1252 0 0 112202 11 0 0 25 0 5 0 856778510 400588800 69211 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4953/tid=4961] ppid=4951 vsize=391200 CPUtime=1122.13 cores=0,2,4,6
/proc/4953/task/4961/stat : 4961 (plingeling) R 4951 4953 4755 0 -1 4202560 96289 1252 0 0 112198 15 0 0 25 0 5 0 856778510 400588800 69211 33554432000 4194304 4417452 140735148580288 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4953/tid=4962] ppid=4951 vsize=391200 CPUtime=1122.1 cores=0,2,4,6
/proc/4953/task/4962/stat : 4962 (plingeling) R 4951 4953 4755 0 -1 4202560 101579 1252 0 0 112196 14 0 0 25 0 5 0 856778510 400588800 69211 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4953/tid=4963] ppid=4951 vsize=391200 CPUtime=1116.54 cores=0,2,4,6
/proc/4953/task/4963/stat : 4963 (plingeling) R 4951 4953 4755 0 -1 4202560 224235 1252 0 0 111621 33 0 0 25 0 5 0 856778510 400588800 69211 33554432000 4194304 4417452 140735148580288 18446744073709551615 4321595 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4483.1
Current children cumulated vsize (KiB) 391200

[startup+1182.3 s]
/proc/loadavg: 7.96 7.97 6.56 9/186 5045
/proc/meminfo: memFree=32148704/32951124 swapFree=67056964/67111528
[pid=4953] ppid=4951 vsize=382456 CPUtime=4722.8 cores=0,2,4,6
/proc/4953/stat : 4953 (plingeling) S 4951 4953 4755 0 -1 4202496 569041 1252 0 0 472197 83 0 0 18 0 5 0 856778491 391634944 67025 33554432000 4194304 4417452 140735148580288 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/4953/statm: 95614 67025 179 55 0 93500 0
[pid=4953/tid=4960] ppid=4951 vsize=382456 CPUtime=1182.13 cores=0,2,4,6
/proc/4953/task/4960/stat : 4960 (plingeling) R 4951 4953 4755 0 -1 4202560 98640 1252 0 0 118201 12 0 0 25 0 5 0 856778510 391634944 67025 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4953/tid=4961] ppid=4951 vsize=382456 CPUtime=1182.13 cores=0,2,4,6
/proc/4953/task/4961/stat : 4961 (plingeling) R 4951 4953 4755 0 -1 4202560 111056 1252 0 0 118196 17 0 0 25 0 5 0 856778510 391634944 67025 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364537 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4953/tid=4962] ppid=4951 vsize=382456 CPUtime=1182.1 cores=0,2,4,6
/proc/4953/task/4962/stat : 4962 (plingeling) R 4951 4953 4755 0 -1 4202560 111817 1252 0 0 118193 17 0 0 25 0 5 0 856778510 391634944 67025 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364893 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4953/tid=4963] ppid=4951 vsize=382456 CPUtime=1176.24 cores=0,2,4,6
/proc/4953/task/4963/stat : 4963 (plingeling) R 4951 4953 4755 0 -1 4202560 238677 1252 0 0 117589 35 0 0 25 0 5 0 856778510 391634944 67025 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4722.8
Current children cumulated vsize (KiB) 382456



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.7 s]
/proc/loadavg: 8.04 7.99 6.60 9/186 5046
/proc/meminfo: memFree=32113964/32951124 swapFree=67056964/67111528
[pid=4953] ppid=4951 vsize=394192 CPUtime=4800.32 cores=0,2,4,6
/proc/4953/stat : 4953 (plingeling) S 4951 4953 4755 0 -1 4202496 580752 1252 0 0 479947 85 0 0 18 0 5 0 856778491 403652608 69959 33554432000 4194304 4417452 140735148580288 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/4953/statm: 98548 69959 179 55 0 96434 0
[pid=4953/tid=4960] ppid=4951 vsize=394192 CPUtime=1201.54 cores=0,2,4,6
/proc/4953/task/4960/stat : 4960 (plingeling) R 4951 4953 4755 0 -1 4202560 103259 1252 0 0 120141 13 0 0 25 0 5 0 856778510 403652608 69959 33554432000 4194304 4417452 140735148580288 18446744073709551615 4277545 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4953/tid=4961] ppid=4951 vsize=394192 CPUtime=1201.53 cores=0,2,4,6
/proc/4953/task/4961/stat : 4961 (plingeling) R 4951 4953 4755 0 -1 4202560 113087 1252 0 0 120136 17 0 0 25 0 5 0 856778510 403652608 69959 33554432000 4194304 4417452 140735148580288 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4953/tid=4962] ppid=4951 vsize=394192 CPUtime=1201.5 cores=0,2,4,6
/proc/4953/task/4962/stat : 4962 (plingeling) R 4951 4953 4755 0 -1 4202560 113777 1252 0 0 120133 17 0 0 25 0 5 0 856778510 403652608 69959 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4953/tid=4963] ppid=4951 vsize=394192 CPUtime=1195.54 cores=0,2,4,6
/proc/4953/task/4963/stat : 4963 (plingeling) R 4951 4953 4755 0 -1 4202560 241778 1252 0 0 119519 35 0 0 25 0 5 0 856778510 403652608 69959 33554432000 4194304 4417452 140735148580288 18446744073709551615 4352463 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4800.32
Current children cumulated vsize (KiB) 394192

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.7 s]
/proc/loadavg: 8.04 7.99 6.60 9/186 5046
/proc/meminfo: memFree=32113964/32951124 swapFree=67056964/67111528
[pid=4953] ppid=4951 vsize=394192 CPUtime=4800.32 cores=0,2,4,6
/proc/4953/stat : 4953 (plingeling) S 4951 4953 4755 0 -1 4202496 580752 1252 0 0 479947 85 0 0 18 0 5 0 856778491 403652608 69959 33554432000 4194304 4417452 140735148580288 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/4953/statm: 98548 69959 179 55 0 96434 0
[pid=4953/tid=4960] ppid=4951 vsize=394192 CPUtime=1201.54 cores=0,2,4,6
/proc/4953/task/4960/stat : 4960 (plingeling) R 4951 4953 4755 0 -1 4202560 103259 1252 0 0 120141 13 0 0 25 0 5 0 856778510 403652608 69959 33554432000 4194304 4417452 140735148580288 18446744073709551615 4277545 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4953/tid=4961] ppid=4951 vsize=394192 CPUtime=1201.53 cores=0,2,4,6
/proc/4953/task/4961/stat : 4961 (plingeling) R 4951 4953 4755 0 -1 4202560 113087 1252 0 0 120136 17 0 0 25 0 5 0 856778510 403652608 69959 33554432000 4194304 4417452 140735148580288 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4953/tid=4962] ppid=4951 vsize=394192 CPUtime=1201.5 cores=0,2,4,6
/proc/4953/task/4962/stat : 4962 (plingeling) R 4951 4953 4755 0 -1 4202560 113777 1252 0 0 120133 17 0 0 25 0 5 0 856778510 403652608 69959 33554432000 4194304 4417452 140735148580288 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4953/tid=4963] ppid=4951 vsize=394192 CPUtime=1195.54 cores=0,2,4,6
/proc/4953/task/4963/stat : 4963 (plingeling) R 4951 4953 4755 0 -1 4202560 241778 1252 0 0 119519 35 0 0 25 0 5 0 856778510 403652608 69959 33554432000 4194304 4417452 140735148580288 18446744073709551615 4352463 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4800.32
Current children cumulated vsize (KiB) 394192

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1201.72
CPU time (s): 4800.37
CPU user time (s): 4799.5
CPU system time (s): 0.866868
CPU usage (%): 399.457
Max. virtual memory (cumulated for all children) (KiB): 423372

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4799.5
system time used= 0.866868
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 582004
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 82
involuntary context switches= 24272

runsolver used 1.82172 second user time and 4.05738 second system time

The end

Launcher Data 

Begin job on node145 at 2011-06-09 20:20:26
IDJOB=3370918
IDBENCH=83253
IDSOLVER=1848
FILE ID=node145/3370918-1307643626
RUNJOBID= node145-1307643561-4773
PBS_JOBID= 13549405
Free space on /tmp= 74196 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/crafted/kullmann/GreenTao/GreenTaok1k2/GreenTao_2-3-6_2071.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3370918-1307643626/watcher-3370918-1307643626 -o /tmp/evaluation-result-3370918-1307643626/solver-3370918-1307643626 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3370918-1307643626.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 337bdbc5d8bf0013ca73aeffa06cffe8
RANDOM SEED=1002747623

node145.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.57
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      32372496 kB
Buffers:         47480 kB
Cached:         214732 kB
SwapCached:       7944 kB
Active:         252580 kB
Inactive:       186500 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      32372496 kB
SwapTotal:    67111528 kB
SwapFree:     67056964 kB
Dirty:            2644 kB
Writeback:           0 kB
AnonPages:      174764 kB
Mapped:          13452 kB
Slab:            74284 kB
PageTables:       5272 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   414040 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 74196 MiB
End job on node145 at 2011-06-09 20:40:30