Trace number 3371071

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.13 1203.05

General information on the benchmark

Namecrafted/anton/SRHD-SGI/
srhd-sgi-m27-q255-n25-p15-s2076598.cnf
MD5SUMf5a22d1154b549fd98215f1710a15f46
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark0.357944
Satisfiable
(Un)Satisfiability was proved
Number of variables545
Number of clauses29734
Sum of the clauses size59963
Maximum clause length27
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 229709
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 525

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3371071-1305433105/watcher-3371071-1305433105 -o /tmp/evaluation-result-3371071-1305433105/solver-3371071-1305433105 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3371071-1305433105.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.85 0.97 0.99 3/288 5960
/proc/meminfo: memFree=24276432/32951124 swapFree=63485424/67111528
[pid=5960] ppid=5958 vsize=9284 CPUtime=0 cores=0,2,4,6
/proc/5960/stat : 5960 (plingeling) R 5958 5960 28842 0 -1 4202496 382 1262 0 0 0 0 0 0 22 0 1 0 635726405 9506816 311 33554432000 4194304 4417452 140735226205728 18446744073709551615 220816236288 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/5960/statm: 2321 311 147 55 0 207 0

[startup+0.0520711 s]
/proc/loadavg: 0.85 0.97 0.99 3/288 5960
/proc/meminfo: memFree=24276432/32951124 swapFree=63485424/67111528
[pid=5960] ppid=5958 vsize=55072 CPUtime=0.09 cores=0,2,4,6
/proc/5960/stat : 5960 (plingeling) S 5958 5960 28842 0 -1 4202496 1713 1262 0 0 9 0 0 0 22 0 5 0 635726405 56393728 1525 33554432000 4194304 4417452 140735226205728 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/5960/statm: 13768 1525 175 55 0 11654 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 55072

[startup+0.100917 s]
/proc/loadavg: 0.85 0.97 0.99 3/288 5960
/proc/meminfo: memFree=24276432/32951124 swapFree=63485424/67111528
[pid=5960] ppid=5958 vsize=122304 CPUtime=0.24 cores=0,2,4,6
/proc/5960/stat : 5960 (plingeling) S 5958 5960 28842 0 -1 4202496 2368 1262 0 0 24 0 0 0 22 0 5 0 635726405 125239296 1922 33554432000 4194304 4417452 140735226205728 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/5960/statm: 30576 1922 177 55 0 28462 0
Current children cumulated CPU time (s) 0.24
Current children cumulated vsize (KiB) 122304

[startup+0.300881 s]
/proc/loadavg: 0.85 0.97 0.99 3/288 5960
/proc/meminfo: memFree=24276432/32951124 swapFree=63485424/67111528
[pid=5960] ppid=5958 vsize=124408 CPUtime=1.03 cores=0,2,4,6
/proc/5960/stat : 5960 (plingeling) S 5958 5960 28842 0 -1 4202496 2991 1262 0 0 103 0 0 0 22 0 5 0 635726405 127393792 2489 33554432000 4194304 4417452 140735226205728 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/5960/statm: 31102 2489 180 55 0 28988 0
Current children cumulated CPU time (s) 1.03
Current children cumulated vsize (KiB) 124408

[startup+0.700776 s]
/proc/loadavg: 0.85 0.97 0.99 3/288 5960
/proc/meminfo: memFree=24276432/32951124 swapFree=63485424/67111528
[pid=5960] ppid=5958 vsize=132544 CPUtime=2.62 cores=0,2,4,6
/proc/5960/stat : 5960 (plingeling) S 5958 5960 28842 0 -1 4202496 5882 1262 0 0 262 0 0 0 22 0 5 0 635726405 135725056 4548 33554432000 4194304 4417452 140735226205728 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/5960/statm: 33136 4548 180 55 0 31022 0
Current children cumulated CPU time (s) 2.62
Current children cumulated vsize (KiB) 132544

[startup+1.50063 s]
/proc/loadavg: 0.85 0.97 0.99 9/300 5985
/proc/meminfo: memFree=24198892/32951124 swapFree=63485424/67111528
[pid=5960] ppid=5958 vsize=139876 CPUtime=5.81 cores=0,2,4,6
/proc/5960/stat : 5960 (plingeling) S 5958 5960 28842 0 -1 4202496 10797 1262 0 0 580 1 0 0 22 0 5 0 635726405 143233024 6381 33554432000 4194304 4417452 140735226205728 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/5960/statm: 34969 6381 180 55 0 32855 0
[pid=5960/tid=5967] ppid=5958 vsize=139876 CPUtime=1.43 cores=0,2,4,6
/proc/5960/task/5967/stat : 5967 (plingeling) R 5958 5960 28842 0 -1 4202560 1332 1262 0 0 143 0 0 0 25 0 5 0 635726408 143233024 6381 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364780 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5960/tid=5968] ppid=5958 vsize=139876 CPUtime=1.43 cores=0,2,4,6
/proc/5960/task/5968/stat : 5968 (plingeling) R 5958 5960 28842 0 -1 4202560 1691 1262 0 0 143 0 0 0 25 0 5 0 635726408 143233024 6381 33554432000 4194304 4417452 140735226205728 18446744073709551615 4365047 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5960/tid=5969] ppid=5958 vsize=139876 CPUtime=1.45 cores=0,2,4,6
/proc/5960/task/5969/stat : 5969 (plingeling) R 5958 5960 28842 0 -1 4202560 2281 1262 0 0 145 0 0 0 25 0 5 0 635726408 143233024 6381 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5960/tid=5970] ppid=5958 vsize=139876 CPUtime=1.46 cores=0,2,4,6
/proc/5960/task/5970/stat : 5970 (plingeling) R 5958 5960 28842 0 -1 4202560 3990 1262 0 0 146 0 0 0 25 0 5 0 635726408 143233024 6381 33554432000 4194304 4417452 140735226205728 18446744073709551615 4356980 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 5.81
Current children cumulated vsize (KiB) 139876

[startup+3.10034 s]
/proc/loadavg: 0.85 0.97 0.99 9/300 5985
/proc/meminfo: memFree=24183516/32951124 swapFree=63485424/67111528
[pid=5960] ppid=5958 vsize=155908 CPUtime=12.2 cores=0,2,4,6
/proc/5960/stat : 5960 (plingeling) S 5958 5960 28842 0 -1 4202496 26483 1262 0 0 1217 3 0 0 22 0 5 0 635726405 159649792 10421 33554432000 4194304 4417452 140735226205728 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/5960/statm: 38977 10421 180 55 0 36863 0
[pid=5960/tid=5967] ppid=5958 vsize=155908 CPUtime=3.02 cores=0,2,4,6
/proc/5960/task/5967/stat : 5967 (plingeling) R 5958 5960 28842 0 -1 4202560 5070 1262 0 0 302 0 0 0 25 0 5 0 635726408 159649792 10421 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364912 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5960/tid=5968] ppid=5958 vsize=155908 CPUtime=3.02 cores=0,2,4,6
/proc/5960/task/5968/stat : 5968 (plingeling) R 5958 5960 28842 0 -1 4202560 5672 1262 0 0 302 0 0 0 25 0 5 0 635726408 159649792 10421 33554432000 4194304 4417452 140735226205728 18446744073709551615 4232787 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5960/tid=5969] ppid=5958 vsize=155908 CPUtime=3.04 cores=0,2,4,6
/proc/5960/task/5969/stat : 5969 (plingeling) R 5958 5960 28842 0 -1 4202560 7437 1262 0 0 304 0 0 0 25 0 5 0 635726408 159649792 10421 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364838 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5960/tid=5970] ppid=5958 vsize=155908 CPUtime=3.07 cores=0,2,4,6
/proc/5960/task/5970/stat : 5970 (plingeling) R 5958 5960 28842 0 -1 4202560 6801 1262 0 0 306 1 0 0 25 0 5 0 635726408 159649792 10421 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 12.2
Current children cumulated vsize (KiB) 155908

[startup+6.30075 s]
/proc/loadavg: 1.42 1.08 1.03 9/300 5985
/proc/meminfo: memFree=24125484/32951124 swapFree=63485424/67111528
[pid=5960] ppid=5958 vsize=165732 CPUtime=24.97 cores=0,2,4,6
/proc/5960/stat : 5960 (plingeling) S 5958 5960 28842 0 -1 4202496 46664 1262 0 0 2491 6 0 0 22 0 5 0 635726405 169709568 12865 33554432000 4194304 4417452 140735226205728 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/5960/statm: 41433 12865 180 55 0 39319 0
[pid=5960/tid=5967] ppid=5958 vsize=165732 CPUtime=6.22 cores=0,2,4,6
/proc/5960/task/5967/stat : 5967 (plingeling) R 5958 5960 28842 0 -1 4202560 6785 1262 0 0 622 0 0 0 25 0 5 0 635726408 169709568 12865 33554432000 4194304 4417452 140735226205728 18446744073709551615 4310248 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5960/tid=5968] ppid=5958 vsize=165732 CPUtime=6.23 cores=0,2,4,6
/proc/5960/task/5968/stat : 5968 (plingeling) R 5958 5960 28842 0 -1 4202560 11042 1262 0 0 622 1 0 0 25 0 5 0 635726408 169709568 12865 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364456 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5960/tid=5969] ppid=5958 vsize=165732 CPUtime=6.21 cores=0,2,4,6
/proc/5960/task/5969/stat : 5969 (plingeling) R 5958 5960 28842 0 -1 4202560 14179 1262 0 0 620 1 0 0 25 0 5 0 635726408 169709568 12865 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364903 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5960/tid=5970] ppid=5958 vsize=165732 CPUtime=6.27 cores=0,2,4,6
/proc/5960/task/5970/stat : 5970 (plingeling) R 5958 5960 28842 0 -1 4202560 13155 1262 0 0 625 2 0 0 25 0 5 0 635726408 169709568 12865 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 24.97
Current children cumulated vsize (KiB) 165732

[startup+12.7006 s]
/proc/loadavg: 1.95 1.20 1.06 9/300 5985
/proc/meminfo: memFree=24081088/32951124 swapFree=63485424/67111528
[pid=5960] ppid=5958 vsize=191112 CPUtime=50.51 cores=0,2,4,6
/proc/5960/stat : 5960 (plingeling) S 5958 5960 28842 0 -1 4202496 113559 1262 0 0 5035 16 0 0 22 0 5 0 635726405 195698688 19190 33554432000 4194304 4417452 140735226205728 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/5960/statm: 47778 19190 180 55 0 45664 0
[pid=5960/tid=5967] ppid=5958 vsize=191112 CPUtime=12.63 cores=0,2,4,6
/proc/5960/task/5967/stat : 5967 (plingeling) R 5958 5960 28842 0 -1 4202560 33108 1262 0 0 1258 5 0 0 25 0 5 0 635726408 195698688 19190 33554432000 4194304 4417452 140735226205728 18446744073709551615 4250660 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5960/tid=5968] ppid=5958 vsize=191112 CPUtime=12.62 cores=0,2,4,6
/proc/5960/task/5968/stat : 5968 (plingeling) R 5958 5960 28842 0 -1 4202560 29894 1262 0 0 1259 3 0 0 25 0 5 0 635726408 195698688 19190 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5960/tid=5969] ppid=5958 vsize=191112 CPUtime=12.55 cores=0,2,4,6
/proc/5960/task/5969/stat : 5969 (plingeling) R 5958 5960 28842 0 -1 4202560 29232 1262 0 0 1252 3 0 0 25 0 5 0 635726408 195698688 19190 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5960/tid=5970] ppid=5958 vsize=191112 CPUtime=12.66 cores=0,2,4,6
/proc/5960/task/5970/stat : 5970 (plingeling) R 5958 5960 28842 0 -1 4202560 19822 1262 0 0 1263 3 0 0 25 0 5 0 635726408 195698688 19190 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364587 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 50.51
Current children cumulated vsize (KiB) 191112

[startup+25.5012 s]
/proc/loadavg: 3.43 1.56 1.18 9/300 5985
/proc/meminfo: memFree=23925104/32951124 swapFree=63485424/67111528
[pid=5960] ppid=5958 vsize=249676 CPUtime=101.6 cores=0,2,4,6
/proc/5960/stat : 5960 (plingeling) S 5958 5960 28842 0 -1 4202496 208032 1262 0 0 10130 30 0 0 22 0 5 0 635726405 255668224 33831 33554432000 4194304 4417452 140735226205728 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/5960/statm: 62419 33831 180 55 0 60305 0
[pid=5960/tid=5967] ppid=5958 vsize=249676 CPUtime=25.43 cores=0,2,4,6
/proc/5960/task/5967/stat : 5967 (plingeling) R 5958 5960 28842 0 -1 4202560 52969 1262 0 0 2535 8 0 0 25 0 5 0 635726408 255668224 33831 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5960/tid=5968] ppid=5958 vsize=249676 CPUtime=25.43 cores=0,2,4,6
/proc/5960/task/5968/stat : 5968 (plingeling) R 5958 5960 28842 0 -1 4202560 59518 1262 0 0 2535 8 0 0 25 0 5 0 635726408 255668224 33831 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5960/tid=5969] ppid=5958 vsize=249676 CPUtime=25.24 cores=0,2,4,6
/proc/5960/task/5969/stat : 5969 (plingeling) R 5958 5960 28842 0 -1 4202560 60829 1262 0 0 2517 7 0 0 25 0 5 0 635726408 255668224 33831 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5960/tid=5970] ppid=5958 vsize=249676 CPUtime=25.45 cores=0,2,4,6
/proc/5960/task/5970/stat : 5970 (plingeling) R 5958 5960 28842 0 -1 4202560 33213 1262 0 0 2540 5 0 0 25 0 5 0 635726408 255668224 33831 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364877 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 101.6
Current children cumulated vsize (KiB) 249676

[startup+51.1005 s]
/proc/loadavg: 5.04 2.09 1.37 9/300 5986
/proc/meminfo: memFree=23761232/32951124 swapFree=63485424/67111528
[pid=5960] ppid=5958 vsize=285688 CPUtime=203.76 cores=0,2,4,6
/proc/5960/stat : 5960 (plingeling) S 5958 5960 28842 0 -1 4202496 369078 1262 0 0 20324 52 0 0 22 0 5 0 635726405 292544512 42834 33554432000 4194304 4417452 140735226205728 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/5960/statm: 71422 42834 180 55 0 69308 0
[pid=5960/tid=5967] ppid=5958 vsize=285688 CPUtime=51.03 cores=0,2,4,6
/proc/5960/task/5967/stat : 5967 (plingeling) R 5958 5960 28842 0 -1 4202560 90885 1262 0 0 5089 14 0 0 25 0 5 0 635726408 292544512 42834 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5960/tid=5968] ppid=5958 vsize=285688 CPUtime=51.03 cores=0,2,4,6
/proc/5960/task/5968/stat : 5968 (plingeling) R 5958 5960 28842 0 -1 4202560 111056 1262 0 0 5089 14 0 0 25 0 5 0 635726408 292544512 42834 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364724 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5960/tid=5969] ppid=5958 vsize=285688 CPUtime=50.6 cores=0,2,4,6
/proc/5960/task/5969/stat : 5969 (plingeling) R 5958 5960 28842 0 -1 4202560 112706 1262 0 0 5046 14 0 0 25 0 5 0 635726408 292544512 42834 33554432000 4194304 4417452 140735226205728 18446744073709551615 4365058 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5960/tid=5970] ppid=5958 vsize=285688 CPUtime=51.06 cores=0,2,4,6
/proc/5960/task/5970/stat : 5970 (plingeling) R 5958 5960 28842 0 -1 4202560 52928 1262 0 0 5097 9 0 0 25 0 5 0 635726408 292544512 42834 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364780 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 203.76
Current children cumulated vsize (KiB) 285688

[startup+102.311 s]
/proc/loadavg: 6.83 3.03 1.73 9/300 5987
/proc/meminfo: memFree=23447736/32951124 swapFree=63485424/67111528
[pid=5960] ppid=5958 vsize=380288 CPUtime=408.12 cores=0,2,4,6
/proc/5960/stat : 5960 (plingeling) S 5958 5960 28842 0 -1 4202496 604534 1262 0 0 40728 84 0 0 22 0 5 0 635726405 389414912 66484 33554432000 4194304 4417452 140735226205728 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/5960/statm: 95072 66484 180 55 0 92958 0
[pid=5960/tid=5967] ppid=5958 vsize=380288 CPUtime=102.23 cores=0,2,4,6
/proc/5960/task/5967/stat : 5967 (plingeling) R 5958 5960 28842 0 -1 4202560 152398 1262 0 0 10201 22 0 0 25 0 5 0 635726408 389414912 66484 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364838 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5960/tid=5968] ppid=5958 vsize=380288 CPUtime=102.24 cores=0,2,4,6
/proc/5960/task/5968/stat : 5968 (plingeling) R 5958 5960 28842 0 -1 4202560 176527 1262 0 0 10201 23 0 0 25 0 5 0 635726408 389414912 66484 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5960/tid=5969] ppid=5958 vsize=380288 CPUtime=101.35 cores=0,2,4,6
/proc/5960/task/5969/stat : 5969 (plingeling) R 5958 5960 28842 0 -1 4202560 190941 1262 0 0 10111 24 0 0 25 0 5 0 635726408 389414912 66484 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364597 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5960/tid=5970] ppid=5958 vsize=380288 CPUtime=102.26 cores=0,2,4,6
/proc/5960/task/5970/stat : 5970 (plingeling) R 5958 5960 28842 0 -1 4202560 83165 1262 0 0 10212 14 0 0 25 0 5 0 635726408 389414912 66484 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 408.12
Current children cumulated vsize (KiB) 380288

[startup+162.301 s]
/proc/loadavg: 7.66 3.96 2.13 9/300 5989
/proc/meminfo: memFree=23200212/32951124 swapFree=63485424/67111528
[pid=5960] ppid=5958 vsize=495600 CPUtime=647.54 cores=0,2,4,6
/proc/5960/stat : 5960 (plingeling) S 5958 5960 28842 0 -1 4202496 916370 1262 0 0 64630 124 0 0 22 0 5 0 635726405 507494400 95312 33554432000 4194304 4417452 140735226205728 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/5960/statm: 123900 95312 180 55 0 121786 0
[pid=5960/tid=5967] ppid=5958 vsize=495600 CPUtime=162.23 cores=0,2,4,6
/proc/5960/task/5967/stat : 5967 (plingeling) R 5958 5960 28842 0 -1 4202560 288917 1262 0 0 16182 41 0 0 25 0 5 0 635726408 507494400 95312 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5960/tid=5968] ppid=5958 vsize=495600 CPUtime=162.23 cores=0,2,4,6
/proc/5960/task/5968/stat : 5968 (plingeling) R 5958 5960 28842 0 -1 4202560 251185 1262 0 0 16191 32 0 0 25 0 5 0 635726408 507494400 95312 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5960/tid=5969] ppid=5958 vsize=495600 CPUtime=160.8 cores=0,2,4,6
/proc/5960/task/5969/stat : 5969 (plingeling) R 5958 5960 28842 0 -1 4202560 266320 1262 0 0 16047 33 0 0 25 0 5 0 635726408 507494400 95312 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5960/tid=5970] ppid=5958 vsize=495600 CPUtime=162.24 cores=0,2,4,6
/proc/5960/task/5970/stat : 5970 (plingeling) R 5958 5960 28842 0 -1 4202560 108445 1262 0 0 16206 18 0 0 25 0 5 0 635726408 507494400 95312 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 647.54
Current children cumulated vsize (KiB) 495600


################
# More data... #
################

[pid=5960/tid=5968] ppid=5958 vsize=792192 CPUtime=702.24 cores=0,2,4,6
/proc/5960/task/5968/stat : 5968 (plingeling) R 5958 5960 28842 0 -1 4202560 696796 1262 0 0 70130 94 0 0 25 0 5 0 635726408 811204608 169460 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5960/tid=5969] ppid=5958 vsize=792192 CPUtime=695.76 cores=0,2,4,6
/proc/5960/task/5969/stat : 5969 (plingeling) R 5958 5960 28842 0 -1 4202560 710033 1262 0 0 69483 93 0 0 25 0 5 0 635726408 811204608 169460 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364873 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5960/tid=5970] ppid=5958 vsize=792192 CPUtime=701.51 cores=0,2,4,6
/proc/5960/task/5970/stat : 5970 (plingeling) R 5958 5960 28842 0 -1 4202560 242221 1262 0 0 70112 39 0 0 25 0 5 0 635726408 811204608 169460 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 2801.8
Current children cumulated vsize (KiB) 792192

[startup+762.311 s]
/proc/loadavg: 8.13 7.59 4.98 9/300 6008
/proc/meminfo: memFree=21337480/32951124 swapFree=63485424/67111528
[pid=5960] ppid=5958 vsize=933800 CPUtime=3041.29 cores=0,2,4,6
/proc/5960/stat : 5960 (plingeling) S 5958 5960 28842 0 -1 4202496 2913161 1262 0 0 303737 392 0 0 22 0 5 0 635726405 956211200 204862 33554432000 4194304 4417452 140735226205728 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/5960/statm: 233450 204862 180 55 0 231336 0
[pid=5960/tid=5967] ppid=5958 vsize=933800 CPUtime=762.25 cores=0,2,4,6
/proc/5960/task/5967/stat : 5967 (plingeling) R 5958 5960 28842 0 -1 4202560 1149243 1262 0 0 76076 149 0 0 25 0 5 0 635726408 956211200 204862 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5960/tid=5968] ppid=5958 vsize=933800 CPUtime=762.26 cores=0,2,4,6
/proc/5960/task/5968/stat : 5968 (plingeling) R 5958 5960 28842 0 -1 4202560 757657 1262 0 0 76125 101 0 0 25 0 5 0 635726408 956211200 204862 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5960/tid=5969] ppid=5958 vsize=933800 CPUtime=755.22 cores=0,2,4,6
/proc/5960/task/5969/stat : 5969 (plingeling) R 5958 5960 28842 0 -1 4202560 742801 1262 0 0 75425 97 0 0 25 0 5 0 635726408 956211200 204862 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5960/tid=5970] ppid=5958 vsize=933800 CPUtime=761.51 cores=0,2,4,6
/proc/5960/task/5970/stat : 5970 (plingeling) R 5958 5960 28842 0 -1 4202560 261957 1262 0 0 76108 43 0 0 25 0 5 0 635726408 956211200 204862 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3041.29
Current children cumulated vsize (KiB) 933800

[startup+822.3 s]
/proc/loadavg: 8.23 7.71 5.18 9/300 6009
/proc/meminfo: memFree=21474856/32951124 swapFree=63485424/67111528
[pid=5960] ppid=5958 vsize=827088 CPUtime=3280.7 cores=0,2,4,6
/proc/5960/stat : 5960 (plingeling) S 5958 5960 28842 0 -1 4202496 3036539 1262 0 0 327660 410 0 0 22 0 5 0 635726405 846938112 177691 33554432000 4194304 4417452 140735226205728 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/5960/statm: 206772 177691 180 55 0 204658 0
[pid=5960/tid=5967] ppid=5958 vsize=827088 CPUtime=822.24 cores=0,2,4,6
/proc/5960/task/5967/stat : 5967 (plingeling) R 5958 5960 28842 0 -1 4202560 1189033 1262 0 0 82068 156 0 0 25 0 5 0 635726408 846938112 177691 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5960/tid=5968] ppid=5958 vsize=827088 CPUtime=822.24 cores=0,2,4,6
/proc/5960/task/5968/stat : 5968 (plingeling) R 5958 5960 28842 0 -1 4202560 804381 1262 0 0 82118 106 0 0 25 0 5 0 635726408 846938112 177691 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5960/tid=5969] ppid=5958 vsize=827088 CPUtime=814.68 cores=0,2,4,6
/proc/5960/task/5969/stat : 5969 (plingeling) R 5958 5960 28842 0 -1 4202560 771473 1262 0 0 81367 101 0 0 25 0 5 0 635726408 846938112 177691 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5960/tid=5970] ppid=5958 vsize=827088 CPUtime=821.49 cores=0,2,4,6
/proc/5960/task/5970/stat : 5970 (plingeling) R 5958 5960 28842 0 -1 4202560 270149 1262 0 0 82104 45 0 0 25 0 5 0 635726408 846938112 177691 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3280.7
Current children cumulated vsize (KiB) 827088

[startup+882.3 s]
/proc/loadavg: 8.16 7.77 5.36 9/300 6011
/proc/meminfo: memFree=21340552/32951124 swapFree=63485424/67111528
[pid=5960] ppid=5958 vsize=876560 CPUtime=3520.15 cores=0,2,4,6
/proc/5960/stat : 5960 (plingeling) S 5958 5960 28842 0 -1 4202496 3175883 1262 0 0 351586 429 0 0 22 0 5 0 635726405 897597440 190059 33554432000 4194304 4417452 140735226205728 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/5960/statm: 219140 190059 180 55 0 217026 0
[pid=5960/tid=5967] ppid=5958 vsize=876560 CPUtime=882.24 cores=0,2,4,6
/proc/5960/task/5967/stat : 5967 (plingeling) R 5958 5960 28842 0 -1 4202560 1257711 1262 0 0 88060 164 0 0 25 0 5 0 635726408 897597440 190059 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5960/tid=5968] ppid=5958 vsize=876560 CPUtime=882.25 cores=0,2,4,6
/proc/5960/task/5968/stat : 5968 (plingeling) R 5958 5960 28842 0 -1 4202560 837149 1262 0 0 88114 111 0 0 25 0 5 0 635726408 897597440 190059 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5960/tid=5969] ppid=5958 vsize=876560 CPUtime=874.13 cores=0,2,4,6
/proc/5960/task/5969/stat : 5969 (plingeling) R 5958 5960 28842 0 -1 4202560 796049 1262 0 0 87309 104 0 0 25 0 5 0 635726408 897597440 190059 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5960/tid=5970] ppid=5958 vsize=876560 CPUtime=881.48 cores=0,2,4,6
/proc/5960/task/5970/stat : 5970 (plingeling) R 5958 5960 28842 0 -1 4202560 283471 1262 0 0 88100 48 0 0 25 0 5 0 635726408 897597440 190059 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3520.15
Current children cumulated vsize (KiB) 876560

[startup+942.301 s]
/proc/loadavg: 8.16 7.84 5.54 9/300 6013
/proc/meminfo: memFree=21103200/32951124 swapFree=63485424/67111528
[pid=5960] ppid=5958 vsize=920708 CPUtime=3759.62 cores=0,2,4,6
/proc/5960/stat : 5960 (plingeling) S 5958 5960 28842 0 -1 4202496 3363048 1262 0 0 375510 452 0 0 22 0 5 0 635726405 942804992 201096 33554432000 4194304 4417452 140735226205728 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/5960/statm: 230177 201096 180 55 0 228063 0
[pid=5960/tid=5967] ppid=5958 vsize=920708 CPUtime=942.25 cores=0,2,4,6
/proc/5960/task/5967/stat : 5967 (plingeling) R 5958 5960 28842 0 -1 4202560 1356015 1262 0 0 94048 177 0 0 25 0 5 0 635726408 942804992 201096 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5960/tid=5968] ppid=5958 vsize=920708 CPUtime=942.25 cores=0,2,4,6
/proc/5960/task/5968/stat : 5968 (plingeling) R 5958 5960 28842 0 -1 4202560 876858 1262 0 0 94109 116 0 0 25 0 5 0 635726408 942804992 201096 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5960/tid=5969] ppid=5958 vsize=920708 CPUtime=933.59 cores=0,2,4,6
/proc/5960/task/5969/stat : 5969 (plingeling) R 5958 5960 28842 0 -1 4202560 828817 1262 0 0 93252 107 0 0 25 0 5 0 635726408 942804992 201096 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5960/tid=5970] ppid=5958 vsize=920708 CPUtime=941.48 cores=0,2,4,6
/proc/5960/task/5970/stat : 5970 (plingeling) R 5958 5960 28842 0 -1 4202560 299855 1262 0 0 94098 50 0 0 25 0 5 0 635726408 942804992 201096 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3759.62
Current children cumulated vsize (KiB) 920708

[startup+1002.3 s]
/proc/loadavg: 8.12 7.88 5.69 9/300 6014
/proc/meminfo: memFree=20841296/32951124 swapFree=63485424/67111528
[pid=5960] ppid=5958 vsize=881468 CPUtime=3999.06 cores=0,2,4,6
/proc/5960/stat : 5960 (plingeling) S 5958 5960 28842 0 -1 4202496 3581488 1262 0 0 399427 479 0 0 22 0 5 0 635726405 902623232 191811 33554432000 4194304 4417452 140735226205728 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/5960/statm: 220367 191811 180 55 0 218253 0
[pid=5960/tid=5967] ppid=5958 vsize=881468 CPUtime=1002.24 cores=0,2,4,6
/proc/5960/task/5967/stat : 5967 (plingeling) R 5958 5960 28842 0 -1 4202560 1478335 1262 0 0 100033 191 0 0 25 0 5 0 635726408 902623232 191811 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5960/tid=5968] ppid=5958 vsize=881468 CPUtime=1002.26 cores=0,2,4,6
/proc/5960/task/5968/stat : 5968 (plingeling) R 5958 5960 28842 0 -1 4202560 921914 1262 0 0 100104 122 0 0 25 0 5 0 635726408 902623232 191811 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5960/tid=5969] ppid=5958 vsize=881468 CPUtime=993.04 cores=0,2,4,6
/proc/5960/task/5969/stat : 5969 (plingeling) R 5958 5960 28842 0 -1 4202560 875581 1262 0 0 99191 113 0 0 25 0 5 0 635726408 902623232 191811 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364724 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5960/tid=5970] ppid=5958 vsize=881468 CPUtime=1001.46 cores=0,2,4,6
/proc/5960/task/5970/stat : 5970 (plingeling) R 5958 5960 28842 0 -1 4202560 304155 1262 0 0 100095 51 0 0 25 0 5 0 635726408 902623232 191811 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3999.06
Current children cumulated vsize (KiB) 881468

[startup+1062.3 s]
/proc/loadavg: 8.13 7.93 5.84 9/300 6016
/proc/meminfo: memFree=20523708/32951124 swapFree=63485424/67111528
[pid=5960] ppid=5958 vsize=927020 CPUtime=4238.52 cores=0,2,4,6
/proc/5960/stat : 5960 (plingeling) S 5958 5960 28842 0 -1 4202496 3646124 1262 0 0 423362 490 0 0 22 0 5 0 635726405 949268480 203199 33554432000 4194304 4417452 140735226205728 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/5960/statm: 231755 203199 180 55 0 229641 0
[pid=5960/tid=5967] ppid=5958 vsize=927020 CPUtime=1062.25 cores=0,2,4,6
/proc/5960/task/5967/stat : 5967 (plingeling) R 5958 5960 28842 0 -1 4202560 1490881 1262 0 0 106031 194 0 0 25 0 5 0 635726408 949268480 203199 33554432000 4194304 4417452 140735226205728 18446744073709551615 4251160 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5960/tid=5968] ppid=5958 vsize=927020 CPUtime=1062.25 cores=0,2,4,6
/proc/5960/task/5968/stat : 5968 (plingeling) R 5958 5960 28842 0 -1 4202560 957620 1262 0 0 106098 127 0 0 25 0 5 0 635726408 949268480 203199 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5960/tid=5969] ppid=5958 vsize=927020 CPUtime=1052.5 cores=0,2,4,6
/proc/5960/task/5969/stat : 5969 (plingeling) R 5958 5960 28842 0 -1 4202560 883773 1262 0 0 105136 114 0 0 25 0 5 0 635726408 949268480 203199 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5960/tid=5970] ppid=5958 vsize=927020 CPUtime=1061.46 cores=0,2,4,6
/proc/5960/task/5970/stat : 5970 (plingeling) R 5958 5960 28842 0 -1 4202560 312347 1262 0 0 106094 52 0 0 25 0 5 0 635726408 949268480 203199 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4238.52
Current children cumulated vsize (KiB) 927020

[startup+1122.3 s]
/proc/loadavg: 8.17 7.97 5.99 9/300 6017
/proc/meminfo: memFree=20063652/32951124 swapFree=63485424/67111528
[pid=5960] ppid=5958 vsize=922352 CPUtime=4477.96 cores=0,2,4,6
/proc/5960/stat : 5960 (plingeling) S 5958 5960 28842 0 -1 4202496 3830898 1262 0 0 447285 511 0 0 22 0 5 0 635726405 944488448 202000 33554432000 4194304 4417452 140735226205728 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/5960/statm: 230588 202000 180 55 0 228474 0
[pid=5960/tid=5967] ppid=5958 vsize=922352 CPUtime=1122.25 cores=0,2,4,6
/proc/5960/task/5967/stat : 5967 (plingeling) R 5958 5960 28842 0 -1 4202560 1572801 1262 0 0 112020 205 0 0 25 0 5 0 635726408 944488448 202000 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5960/tid=5968] ppid=5958 vsize=922352 CPUtime=1122.26 cores=0,2,4,6
/proc/5960/task/5968/stat : 5968 (plingeling) R 5958 5960 28842 0 -1 4202560 998580 1262 0 0 112094 132 0 0 25 0 5 0 635726408 944488448 202000 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5960/tid=5969] ppid=5958 vsize=922352 CPUtime=1111.96 cores=0,2,4,6
/proc/5960/task/5969/stat : 5969 (plingeling) R 5958 5960 28842 0 -1 4202560 907503 1262 0 0 111079 117 0 0 25 0 5 0 635726408 944488448 202000 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5960/tid=5970] ppid=5958 vsize=922352 CPUtime=1121.45 cores=0,2,4,6
/proc/5960/task/5970/stat : 5970 (plingeling) R 5958 5960 28842 0 -1 4202560 350511 1262 0 0 112089 56 0 0 25 0 5 0 635726408 944488448 202000 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4477.96
Current children cumulated vsize (KiB) 922352

[startup+1182.3 s]
/proc/loadavg: 8.11 7.99 6.11 9/300 6019
/proc/meminfo: memFree=19543580/32951124 swapFree=63485424/67111528
[pid=5960] ppid=5958 vsize=965700 CPUtime=4717.41 cores=0,2,4,6
/proc/5960/stat : 5960 (plingeling) S 5958 5960 28842 0 -1 4202496 3997383 1262 0 0 471208 533 0 0 22 0 5 0 635726405 988876800 212837 33554432000 4194304 4417452 140735226205728 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/5960/statm: 241425 212837 180 55 0 239311 0
[pid=5960/tid=5967] ppid=5958 vsize=965700 CPUtime=1182.25 cores=0,2,4,6
/proc/5960/task/5967/stat : 5967 (plingeling) R 5958 5960 28842 0 -1 4202560 1658817 1262 0 0 118009 216 0 0 25 0 5 0 635726408 988876800 212837 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5960/tid=5968] ppid=5958 vsize=965700 CPUtime=1182.26 cores=0,2,4,6
/proc/5960/task/5968/stat : 5968 (plingeling) R 5958 5960 28842 0 -1 4202560 1047732 1262 0 0 118088 138 0 0 25 0 5 0 635726408 988876800 212837 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5960/tid=5969] ppid=5958 vsize=965700 CPUtime=1171.41 cores=0,2,4,6
/proc/5960/task/5969/stat : 5969 (plingeling) R 5958 5960 28842 0 -1 4202560 938820 1262 0 0 117020 121 0 0 25 0 5 0 635726408 988876800 212837 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5960/tid=5970] ppid=5958 vsize=965700 CPUtime=1181.44 cores=0,2,4,6
/proc/5960/task/5970/stat : 5970 (plingeling) R 5958 5960 28842 0 -1 4202560 350511 1262 0 0 118088 56 0 0 25 0 5 0 635726408 988876800 212837 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4717.41
Current children cumulated vsize (KiB) 965700



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203 s]
/proc/loadavg: 8.08 7.99 6.15 9/300 6020
/proc/meminfo: memFree=19472892/32951124 swapFree=63485424/67111528
[pid=5960] ppid=5958 vsize=932932 CPUtime=4800.04 cores=0,2,4,6
/proc/5960/stat : 5960 (plingeling) S 5958 5960 28842 0 -1 4202496 4021959 1262 0 0 479468 536 0 0 22 0 5 0 635726405 955322368 204645 33554432000 4194304 4417452 140735226205728 18446744073709551615 220828039141 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/5960/statm: 233233 204645 180 55 0 231119 0
[pid=5960/tid=5967] ppid=5958 vsize=932932 CPUtime=1202.96 cores=0,2,4,6
/proc/5960/task/5967/stat : 5967 (plingeling) R 5958 5960 28842 0 -1 4202560 1667009 1262 0 0 120079 217 0 0 25 0 5 0 635726408 955322368 204645 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5960/tid=5968] ppid=5958 vsize=932932 CPUtime=1202.97 cores=0,2,4,6
/proc/5960/task/5968/stat : 5968 (plingeling) R 5958 5960 28842 0 -1 4202560 1055924 1262 0 0 120158 139 0 0 25 0 5 0 635726408 955322368 204645 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5960/tid=5969] ppid=5958 vsize=932932 CPUtime=1191.92 cores=0,2,4,6
/proc/5960/task/5969/stat : 5969 (plingeling) R 5958 5960 28842 0 -1 4202560 947012 1262 0 0 119070 122 0 0 25 0 5 0 635726408 955322368 204645 33554432000 4194304 4417452 140735226205728 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5960/tid=5970] ppid=5958 vsize=932932 CPUtime=1202.15 cores=0,2,4,6
/proc/5960/task/5970/stat : 5970 (plingeling) R 5958 5960 28842 0 -1 4202560 350511 1262 0 0 120159 56 0 0 25 0 5 0 635726408 955322368 204645 33554432000 4194304 4417452 140735226205728 18446744073709551615 4366099 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.04
Current children cumulated vsize (KiB) 932932

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1203.05
CPU time (s): 4800.13
CPU user time (s): 4794.72
CPU system time (s): 5.40218
CPU usage (%): 398.996
Max. virtual memory (cumulated for all children) (KiB): 1004272

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4794.72
system time used= 5.40218
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 4023221
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 149
involuntary context switches= 22710

runsolver used 3.13652 second user time and 7.81381 second system time

The end

Launcher Data 

Begin job on node147 at 2011-05-15 06:18:25
IDJOB=3371071
IDBENCH=88220
IDSOLVER=1848
FILE ID=node147/3371071-1305433105
RUNJOBID= node147-1305433105-5946
PBS_JOBID= 13324574
Free space on /tmp= 73160 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/crafted/anton/SRHD-SGI/srhd-sgi-m27-q255-n25-p15-s2076598.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3371071-1305433105/watcher-3371071-1305433105 -o /tmp/evaluation-result-3371071-1305433105/solver-3371071-1305433105 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3371071-1305433105.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= f5a22d1154b549fd98215f1710a15f46
RANDOM SEED=1313653420

node147.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      24276836 kB
Buffers:        299440 kB
Cached:        5109196 kB
SwapCached:      13132 kB
Active:        1993576 kB
Inactive:      6447672 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      24276836 kB
SwapTotal:    67111528 kB
SwapFree:     63485424 kB
Dirty:             432 kB
Writeback:           0 kB
AnonPages:     3027784 kB
Mapped:          14444 kB
Slab:           141384 kB
PageTables:      28872 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 44834184 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73156 MiB
End job on node147 at 2011-05-15 06:38:30