Trace number 3251208

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 4800.4 1202.02

General information on the benchmark

Namecrafted/anton/SRHD-SGI/
srhd-sgi-m47-q841.5-n45-p30-s84954709.cnf
MD5SUM556c58061360a1b195d9b2f28888af77
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark2.03469
Satisfiable
(Un)Satisfiability was proved
Number of variables2115
Number of clauses356340
Sum of the clauses size714705
Maximum clause length47
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2356295
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 545

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3251208-1304116748/watcher-3251208-1304116748 -o /tmp/evaluation-result-3251208-1304116748/solver-3251208-1304116748 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3251208-1304116748.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.92 6.12 5.56 5/181 7597
/proc/meminfo: memFree=30413508/32951124 swapFree=67107188/67111528
[pid=7597] ppid=7595 vsize=10132 CPUtime=0 cores=1,3,5,7
/proc/7597/stat : 7597 (plingeling) R 7595 7597 6934 0 -1 4202496 625 1258 0 0 0 0 0 0 25 0 1 0 523273359 10375168 554 33554432000 4194304 4417332 140734608330144 18446744073709551615 210204647168 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/7597/statm: 2533 554 146 55 0 419 0

[startup+0.0118519 s]
/proc/loadavg: 7.92 6.12 5.56 5/181 7597
/proc/meminfo: memFree=30413508/32951124 swapFree=67107188/67111528
[pid=7597] ppid=7595 vsize=11104 CPUtime=0 cores=1,3,5,7
/proc/7597/stat : 7597 (plingeling) R 7595 7597 6934 0 -1 4202496 869 1258 0 0 0 0 0 0 25 0 1 0 523273359 11370496 798 33554432000 4194304 4417332 140734608330144 18446744073709551615 4262059 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/7597/statm: 2776 798 147 55 0 662 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 11104

[startup+0.100838 s]
/proc/loadavg: 7.92 6.12 5.56 5/181 7597
/proc/meminfo: memFree=30413508/32951124 swapFree=67107188/67111528
[pid=7597] ppid=7595 vsize=26596 CPUtime=0.09 cores=1,3,5,7
/proc/7597/stat : 7597 (plingeling) R 7595 7597 6934 0 -1 4202496 4774 1258 0 0 9 0 0 0 25 0 1 0 523273359 27234304 4671 33554432000 4194304 4417332 140734608330144 18446744073709551615 210204647168 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/7597/statm: 6649 4671 147 55 0 4535 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 26596

[startup+0.300759 s]
/proc/loadavg: 7.92 6.12 5.56 5/181 7597
/proc/meminfo: memFree=30413508/32951124 swapFree=67107188/67111528
[pid=7597] ppid=7595 vsize=85872 CPUtime=0.49 cores=1,3,5,7
/proc/7597/stat : 7597 (plingeling) S 7595 7597 6934 0 -1 4202496 9474 1258 0 0 48 1 0 0 25 0 5 0 523273359 87932928 9277 33554432000 4194304 4417332 140734608330144 18446744073709551615 210216450021 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7597/statm: 21468 9277 170 55 0 19354 0
Current children cumulated CPU time (s) 0.49
Current children cumulated vsize (KiB) 85872

[startup+0.700691 s]
/proc/loadavg: 7.92 6.12 5.56 5/181 7597
/proc/meminfo: memFree=30413508/32951124 swapFree=67107188/67111528
[pid=7597] ppid=7595 vsize=105028 CPUtime=2.09 cores=1,3,5,7
/proc/7597/stat : 7597 (plingeling) S 7595 7597 6934 0 -1 4202496 26176 1258 0 0 206 3 0 0 25 0 5 0 523273359 107548672 14074 33554432000 4194304 4417332 140734608330144 18446744073709551615 210216450021 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7597/statm: 26257 14074 178 55 0 24143 0
Current children cumulated CPU time (s) 2.09
Current children cumulated vsize (KiB) 105028

[startup+1.50056 s]
/proc/loadavg: 7.92 6.16 5.58 9/186 7607
/proc/meminfo: memFree=30355976/32951124 swapFree=67107188/67111528
[pid=7597] ppid=7595 vsize=103636 CPUtime=5.28 cores=1,3,5,7
/proc/7597/stat : 7597 (plingeling) S 7595 7597 6934 0 -1 4202496 30470 1258 0 0 525 3 0 0 25 0 5 0 523273359 106123264 13728 33554432000 4194304 4417332 140734608330144 18446744073709551615 210216450021 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7597/statm: 25909 13728 180 55 0 23795 0
[pid=7597/tid=7604] ppid=7595 vsize=103636 CPUtime=1.26 cores=1,3,5,7
/proc/7597/task/7604/stat : 7604 (plingeling) R 7595 7597 6934 0 -1 4202560 5138 1258 0 0 126 0 0 0 25 0 5 0 523273382 106123264 13728 33554432000 4194304 4417332 140734608330144 18446744073709551615 4211469 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7597/tid=7605] ppid=7595 vsize=103636 CPUtime=1.25 cores=1,3,5,7
/proc/7597/task/7605/stat : 7605 (plingeling) R 7595 7597 6934 0 -1 4202560 4154 1258 0 0 125 0 0 0 25 0 5 0 523273382 106123264 13728 33554432000 4194304 4417332 140734608330144 18446744073709551615 4258377 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7597/tid=7606] ppid=7595 vsize=103636 CPUtime=1.26 cores=1,3,5,7
/proc/7597/task/7606/stat : 7606 (plingeling) R 7595 7597 6934 0 -1 4202560 4101 1258 0 0 126 0 0 0 25 0 5 0 523273382 106123264 13728 33554432000 4194304 4417332 140734608330144 18446744073709551615 4250072 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7597/tid=7607] ppid=7595 vsize=103636 CPUtime=1.26 cores=1,3,5,7
/proc/7597/task/7607/stat : 7607 (plingeling) R 7595 7597 6934 0 -1 4202560 8195 1258 0 0 125 1 0 0 25 0 5 0 523273382 106123264 13728 33554432000 4194304 4417332 140734608330144 18446744073709551615 4353524 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 5.28
Current children cumulated vsize (KiB) 103636

[startup+3.10029 s]
/proc/loadavg: 7.92 6.16 5.58 9/186 7607
/proc/meminfo: memFree=30346800/32951124 swapFree=67107188/67111528
[pid=7597] ppid=7595 vsize=134200 CPUtime=11.67 cores=1,3,5,7
/proc/7597/stat : 7597 (plingeling) S 7595 7597 6934 0 -1 4202496 40817 1258 0 0 1163 4 0 0 25 0 5 0 523273359 137420800 21337 33554432000 4194304 4417332 140734608330144 18446744073709551615 210216450021 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7597/statm: 33550 21337 180 55 0 31436 0
[pid=7597/tid=7604] ppid=7595 vsize=134200 CPUtime=2.86 cores=1,3,5,7
/proc/7597/task/7604/stat : 7604 (plingeling) R 7595 7597 6934 0 -1 4202560 5138 1258 0 0 286 0 0 0 25 0 5 0 523273382 137420800 21337 33554432000 4194304 4417332 140734608330144 18446744073709551615 4258392 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7597/tid=7605] ppid=7595 vsize=134200 CPUtime=2.85 cores=1,3,5,7
/proc/7597/task/7605/stat : 7605 (plingeling) R 7595 7597 6934 0 -1 4202560 4154 1258 0 0 285 0 0 0 25 0 5 0 523273382 137420800 21337 33554432000 4194304 4417332 140734608330144 18446744073709551615 4249171 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7597/tid=7606] ppid=7595 vsize=134200 CPUtime=2.86 cores=1,3,5,7
/proc/7597/task/7606/stat : 7606 (plingeling) R 7595 7597 6934 0 -1 4202560 4101 1258 0 0 286 0 0 0 25 0 5 0 523273382 137420800 21337 33554432000 4194304 4417332 140734608330144 18446744073709551615 4250047 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7597/tid=7607] ppid=7595 vsize=134200 CPUtime=2.85 cores=1,3,5,7
/proc/7597/task/7607/stat : 7607 (plingeling) R 7595 7597 6934 0 -1 4202560 18542 1258 0 0 283 2 0 0 25 0 5 0 523273382 137420800 21337 33554432000 4194304 4417332 140734608330144 18446744073709551615 4357167 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 11.67
Current children cumulated vsize (KiB) 134200

[startup+6.30076 s]
/proc/loadavg: 7.92 6.16 5.58 9/186 7607
/proc/meminfo: memFree=30316916/32951124 swapFree=67107188/67111528
[pid=7597] ppid=7595 vsize=155984 CPUtime=24.46 cores=1,3,5,7
/proc/7597/stat : 7597 (plingeling) S 7595 7597 6934 0 -1 4202496 53248 1258 0 0 2440 6 0 0 25 0 5 0 523273359 159727616 25851 33554432000 4194304 4417332 140734608330144 18446744073709551615 210216450021 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7597/statm: 38996 25851 181 55 0 36882 0
[pid=7597/tid=7604] ppid=7595 vsize=155984 CPUtime=6.06 cores=1,3,5,7
/proc/7597/task/7604/stat : 7604 (plingeling) R 7595 7597 6934 0 -1 4202560 5138 1258 0 0 606 0 0 0 25 0 5 0 523273382 159727616 25851 33554432000 4194304 4417332 140734608330144 18446744073709551615 4248923 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7597/tid=7605] ppid=7595 vsize=155984 CPUtime=6.05 cores=1,3,5,7
/proc/7597/task/7605/stat : 7605 (plingeling) R 7595 7597 6934 0 -1 4202560 4154 1258 0 0 605 0 0 0 25 0 5 0 523273382 159727616 25851 33554432000 4194304 4417332 140734608330144 18446744073709551615 4248888 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7597/tid=7606] ppid=7595 vsize=155984 CPUtime=6.06 cores=1,3,5,7
/proc/7597/task/7606/stat : 7606 (plingeling) R 7595 7597 6934 0 -1 4202560 4102 1258 0 0 606 0 0 0 25 0 5 0 523273382 159727616 25851 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364519 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7597/tid=7607] ppid=7595 vsize=155984 CPUtime=6.03 cores=1,3,5,7
/proc/7597/task/7607/stat : 7607 (plingeling) R 7595 7597 6934 0 -1 4202560 30972 1258 0 0 599 4 0 0 25 0 5 0 523273382 159727616 25851 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364565 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 24.46
Current children cumulated vsize (KiB) 155984

[startup+12.7007 s]
/proc/loadavg: 7.93 6.22 5.60 9/186 7607
/proc/meminfo: memFree=30247240/32951124 swapFree=67107188/67111528
[pid=7597] ppid=7595 vsize=222364 CPUtime=50.02 cores=1,3,5,7
/proc/7597/stat : 7597 (plingeling) S 7595 7597 6934 0 -1 4202496 82031 1258 0 0 4993 9 0 0 25 0 5 0 523273359 227700736 42665 33554432000 4194304 4417332 140734608330144 18446744073709551615 210216450021 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7597/statm: 55591 42665 181 55 0 53477 0
[pid=7597/tid=7604] ppid=7595 vsize=222364 CPUtime=12.46 cores=1,3,5,7
/proc/7597/task/7604/stat : 7604 (plingeling) R 7595 7597 6934 0 -1 4202560 5138 1258 0 0 1246 0 0 0 25 0 5 0 523273382 227700736 42665 33554432000 4194304 4417332 140734608330144 18446744073709551615 4258418 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7597/tid=7605] ppid=7595 vsize=222364 CPUtime=12.45 cores=1,3,5,7
/proc/7597/task/7605/stat : 7605 (plingeling) R 7595 7597 6934 0 -1 4202560 4154 1258 0 0 1245 0 0 0 25 0 5 0 523273382 227700736 42665 33554432000 4194304 4417332 140734608330144 18446744073709551615 4249186 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7597/tid=7606] ppid=7595 vsize=222364 CPUtime=12.46 cores=1,3,5,7
/proc/7597/task/7606/stat : 7606 (plingeling) R 7595 7597 6934 0 -1 4202560 4102 1258 0 0 1246 0 0 0 25 0 5 0 523273382 227700736 42665 33554432000 4194304 4417332 140734608330144 18446744073709551615 4277545 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7597/tid=7607] ppid=7595 vsize=222364 CPUtime=12.4 cores=1,3,5,7
/proc/7597/task/7607/stat : 7607 (plingeling) R 7595 7597 6934 0 -1 4202560 59755 1258 0 0 1233 7 0 0 25 0 5 0 523273382 227700736 42665 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 50.02
Current children cumulated vsize (KiB) 222364

[startup+25.5006 s]
/proc/loadavg: 7.94 6.27 5.63 9/186 7607
/proc/meminfo: memFree=30093740/32951124 swapFree=67107188/67111528
[pid=7597] ppid=7595 vsize=289088 CPUtime=101.14 cores=1,3,5,7
/proc/7597/stat : 7597 (plingeling) S 7595 7597 6934 0 -1 4202496 125608 1258 0 0 10098 16 0 0 25 0 5 0 523273359 296026112 43706 33554432000 4194304 4417332 140734608330144 18446744073709551615 210216450021 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7597/statm: 72272 43706 181 55 0 70158 0
[pid=7597/tid=7604] ppid=7595 vsize=289088 CPUtime=25.26 cores=1,3,5,7
/proc/7597/task/7604/stat : 7604 (plingeling) R 7595 7597 6934 0 -1 4202560 6192 1258 0 0 2526 0 0 0 25 0 5 0 523273382 296026112 43706 33554432000 4194304 4417332 140734608330144 18446744073709551615 4259180 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7597/tid=7605] ppid=7595 vsize=289088 CPUtime=25.25 cores=1,3,5,7
/proc/7597/task/7605/stat : 7605 (plingeling) R 7595 7597 6934 0 -1 4202560 4154 1258 0 0 2525 0 0 0 25 0 5 0 523273382 296026112 43706 33554432000 4194304 4417332 140734608330144 18446744073709551615 4358652 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7597/tid=7606] ppid=7595 vsize=289088 CPUtime=25.26 cores=1,3,5,7
/proc/7597/task/7606/stat : 7606 (plingeling) R 7595 7597 6934 0 -1 4202560 4944 1258 0 0 2526 0 0 0 25 0 5 0 523273382 296026112 43706 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364796 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7597/tid=7607] ppid=7595 vsize=289088 CPUtime=25.13 cores=1,3,5,7
/proc/7597/task/7607/stat : 7607 (plingeling) R 7595 7597 6934 0 -1 4202560 101436 1258 0 0 2499 14 0 0 25 0 5 0 523273382 296026112 43706 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 101.14
Current children cumulated vsize (KiB) 289088

[startup+51.1003 s]
/proc/loadavg: 8.09 6.44 5.70 9/186 7608
/proc/meminfo: memFree=29889388/32951124 swapFree=67107188/67111528
[pid=7597] ppid=7595 vsize=431096 CPUtime=203.41 cores=1,3,5,7
/proc/7597/stat : 7597 (plingeling) S 7595 7597 6934 0 -1 4202496 207296 1258 0 0 20312 29 0 0 25 0 5 0 523273359 441442304 78605 33554432000 4194304 4417332 140734608330144 18446744073709551615 210216450021 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7597/statm: 107774 78605 181 55 0 105660 0
[pid=7597/tid=7604] ppid=7595 vsize=431096 CPUtime=50.86 cores=1,3,5,7
/proc/7597/task/7604/stat : 7604 (plingeling) R 7595 7597 6934 0 -1 4202560 16595 1258 0 0 5085 1 0 0 25 0 5 0 523273382 441442304 78605 33554432000 4194304 4417332 140734608330144 18446744073709551615 4251104 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7597/tid=7605] ppid=7595 vsize=431096 CPUtime=50.85 cores=1,3,5,7
/proc/7597/task/7605/stat : 7605 (plingeling) R 7595 7597 6934 0 -1 4202560 5354 1258 0 0 5085 0 0 0 25 0 5 0 523273382 441442304 78605 33554432000 4194304 4417332 140734608330144 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7597/tid=7606] ppid=7595 vsize=431096 CPUtime=50.86 cores=1,3,5,7
/proc/7597/task/7606/stat : 7606 (plingeling) R 7595 7597 6934 0 -1 4202560 4944 1258 0 0 5086 0 0 0 25 0 5 0 523273382 441442304 78605 33554432000 4194304 4417332 140734608330144 18446744073709551615 4248225 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7597/tid=7607] ppid=7595 vsize=431096 CPUtime=50.58 cores=1,3,5,7
/proc/7597/task/7607/stat : 7607 (plingeling) R 7595 7597 6934 0 -1 4202560 171521 1258 0 0 5033 25 0 0 25 0 5 0 523273382 441442304 78605 33554432000 4194304 4417332 140734608330144 18446744073709551615 4354385 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 203.41
Current children cumulated vsize (KiB) 431096

[startup+102.307 s]
/proc/loadavg: 8.16 6.73 5.84 9/186 7609
/proc/meminfo: memFree=29691472/32951124 swapFree=67107188/67111528
[pid=7597] ppid=7595 vsize=694716 CPUtime=407.95 cores=1,3,5,7
/proc/7597/stat : 7597 (plingeling) S 7595 7597 6934 0 -1 4202496 354921 1258 0 0 40743 52 0 0 25 0 5 0 523273359 711389184 145116 33554432000 4194304 4417332 140734608330144 18446744073709551615 210216450021 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7597/statm: 173679 145116 181 55 0 171565 0
[pid=7597/tid=7604] ppid=7595 vsize=694716 CPUtime=102.06 cores=1,3,5,7
/proc/7597/task/7604/stat : 7604 (plingeling) R 7595 7597 6934 0 -1 4202560 28814 1258 0 0 10203 3 0 0 25 0 5 0 523273382 711389184 145116 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364742 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7597/tid=7605] ppid=7595 vsize=694716 CPUtime=102.06 cores=1,3,5,7
/proc/7597/task/7605/stat : 7605 (plingeling) R 7595 7597 6934 0 -1 4202560 16457 1258 0 0 10204 2 0 0 25 0 5 0 523273382 711389184 145116 33554432000 4194304 4417332 140734608330144 18446744073709551615 4279626 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7597/tid=7606] ppid=7595 vsize=694716 CPUtime=102.07 cores=1,3,5,7
/proc/7597/task/7606/stat : 7606 (plingeling) R 7595 7597 6934 0 -1 4202560 22209 1258 0 0 10204 3 0 0 25 0 5 0 523273382 711389184 145116 33554432000 4194304 4417332 140734608330144 18446744073709551615 4250651 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7597/tid=7607] ppid=7595 vsize=694716 CPUtime=101.51 cores=1,3,5,7
/proc/7597/task/7607/stat : 7607 (plingeling) R 7595 7597 6934 0 -1 4202560 278559 1258 0 0 10109 42 0 0 25 0 5 0 523273382 711389184 145116 33554432000 4194304 4417332 140734608330144 18446744073709551615 4365977 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 407.95
Current children cumulated vsize (KiB) 694716

[startup+162.301 s]
/proc/loadavg: 8.12 6.98 5.98 9/186 7611
/proc/meminfo: memFree=29813092/32951124 swapFree=67107188/67111528
[pid=7597] ppid=7595 vsize=735412 CPUtime=647.59 cores=1,3,5,7
/proc/7597/stat : 7597 (plingeling) S 7595 7597 6934 0 -1 4202496 600983 1258 0 0 64671 88 0 0 25 0 5 0 523273359 753061888 155290 33554432000 4194304 4417332 140734608330144 18446744073709551615 210216450021 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7597/statm: 183853 155290 181 55 0 181739 0
[pid=7597/tid=7604] ppid=7595 vsize=735412 CPUtime=162.04 cores=1,3,5,7
/proc/7597/task/7604/stat : 7604 (plingeling) R 7595 7597 6934 0 -1 4202560 51314 1258 0 0 16199 5 0 0 25 0 5 0 523273382 753061888 155290 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364734 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7597/tid=7605] ppid=7595 vsize=735412 CPUtime=162.05 cores=1,3,5,7
/proc/7597/task/7605/stat : 7605 (plingeling) R 7595 7597 6934 0 -1 4202560 48842 1258 0 0 16199 6 0 0 25 0 5 0 523273382 753061888 155290 33554432000 4194304 4417332 140734608330144 18446744073709551615 4259849 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7597/tid=7606] ppid=7595 vsize=735412 CPUtime=162.06 cores=1,3,5,7
/proc/7597/task/7606/stat : 7606 (plingeling) R 7595 7597 6934 0 -1 4202560 72279 1258 0 0 16198 8 0 0 25 0 5 0 523273382 753061888 155290 33554432000 4194304 4417332 140734608330144 18446744073709551615 4365977 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7597/tid=7607] ppid=7595 vsize=735412 CPUtime=161.18 cores=1,3,5,7
/proc/7597/task/7607/stat : 7607 (plingeling) R 7595 7597 6934 0 -1 4202560 419666 1258 0 0 16052 66 0 0 25 0 5 0 523273382 753061888 155290 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364482 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 647.59
Current children cumulated vsize (KiB) 735412


################
# More data... #
################

/proc/meminfo: memFree=26968804/32951124 swapFree=67107188/67111528
[pid=7597] ppid=7595 vsize=2625768 CPUtime=3044.29 cores=1,3,5,7
/proc/7597/stat : 7597 (plingeling) S 7595 7597 6934 0 -1 4202496 1650690 1258 0 0 304185 244 0 0 25 0 5 0 523273359 2688786432 627911 33554432000 4194304 4417332 140734608330144 18446744073709551615 210216450021 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7597/statm: 656442 627911 181 55 0 654328 0
[pid=7597/tid=7604] ppid=7595 vsize=2625768 CPUtime=762 cores=1,3,5,7
/proc/7597/task/7604/stat : 7604 (plingeling) R 7595 7597 6934 0 -1 4202560 251820 1258 0 0 76168 32 0 0 25 0 5 0 523273382 2688786432 627911 33554432000 4194304 4417332 140734608330144 18446744073709551615 4277567 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7597/tid=7605] ppid=7595 vsize=2625768 CPUtime=762.05 cores=1,3,5,7
/proc/7597/task/7605/stat : 7605 (plingeling) R 7595 7597 6934 0 -1 4202560 212732 1258 0 0 76176 29 0 0 25 0 5 0 523273382 2688786432 627911 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7597/tid=7606] ppid=7595 vsize=2625768 CPUtime=762.07 cores=1,3,5,7
/proc/7597/task/7606/stat : 7606 (plingeling) R 7595 7597 6934 0 -1 4202560 285061 1258 0 0 76169 38 0 0 25 0 5 0 523273382 2688786432 627911 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7597/tid=7607] ppid=7595 vsize=2625768 CPUtime=757.91 cores=1,3,5,7
/proc/7597/task/7607/stat : 7607 (plingeling) R 7595 7597 6934 0 -1 4202560 892195 1258 0 0 75648 143 0 0 25 0 5 0 523273382 2688786432 627911 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3044.29
Current children cumulated vsize (KiB) 2625768

[startup+822.3 s]
/proc/loadavg: 8.07 7.97 7.04 9/186 7629
/proc/meminfo: memFree=27559988/32951124 swapFree=67107188/67111528
[pid=7597] ppid=7595 vsize=2232416 CPUtime=3283.94 cores=1,3,5,7
/proc/7597/stat : 7597 (plingeling) S 7595 7597 6934 0 -1 4202496 1782106 1258 0 0 328130 264 0 0 25 0 5 0 523273359 2285993984 529541 33554432000 4194304 4417332 140734608330144 18446744073709551615 210216450021 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7597/statm: 558104 529541 181 55 0 555990 0
[pid=7597/tid=7604] ppid=7595 vsize=2232416 CPUtime=821.99 cores=1,3,5,7
/proc/7597/task/7604/stat : 7604 (plingeling) R 7595 7597 6934 0 -1 4202560 295997 1258 0 0 82162 37 0 0 25 0 5 0 523273382 2285993984 529541 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7597/tid=7605] ppid=7595 vsize=2232416 CPUtime=822.03 cores=1,3,5,7
/proc/7597/task/7605/stat : 7605 (plingeling) R 7595 7597 6934 0 -1 4202560 239102 1258 0 0 82171 32 0 0 25 0 5 0 523273382 2285993984 529541 33554432000 4194304 4417332 140734608330144 18446744073709551615 4308183 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7597/tid=7606] ppid=7595 vsize=2232416 CPUtime=822.07 cores=1,3,5,7
/proc/7597/task/7606/stat : 7606 (plingeling) R 7595 7597 6934 0 -1 4202560 315256 1258 0 0 82164 43 0 0 25 0 5 0 523273382 2285993984 529541 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364722 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7597/tid=7607] ppid=7595 vsize=2232416 CPUtime=817.59 cores=1,3,5,7
/proc/7597/task/7607/stat : 7607 (plingeling) R 7595 7597 6934 0 -1 4202560 922869 1258 0 0 81610 149 0 0 25 0 5 0 523273382 2285993984 529541 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3283.94
Current children cumulated vsize (KiB) 2232416

[startup+882.3 s]
/proc/loadavg: 8.13 8.00 7.11 10/186 7630
/proc/meminfo: memFree=27518908/32951124 swapFree=67107188/67111528
[pid=7597] ppid=7595 vsize=2290700 CPUtime=3523.6 cores=1,3,5,7
/proc/7597/stat : 7597 (plingeling) S 7595 7597 6934 0 -1 4202496 1976399 1258 0 0 352070 290 0 0 25 0 5 0 523273359 2345676800 544112 33554432000 4194304 4417332 140734608330144 18446744073709551615 210216450021 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7597/statm: 572675 544112 181 55 0 570561 0
[pid=7597/tid=7604] ppid=7595 vsize=2290700 CPUtime=881.98 cores=1,3,5,7
/proc/7597/task/7604/stat : 7604 (plingeling) R 7595 7597 6934 0 -1 4202560 361533 1258 0 0 88153 45 0 0 25 0 5 0 523273382 2345676800 544112 33554432000 4194304 4417332 140734608330144 18446744073709551615 4199406 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7597/tid=7605] ppid=7595 vsize=2290700 CPUtime=882.03 cores=1,3,5,7
/proc/7597/task/7605/stat : 7605 (plingeling) R 7595 7597 6934 0 -1 4202560 317846 1258 0 0 88159 44 0 0 25 0 5 0 523273382 2345676800 544112 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364671 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7597/tid=7606] ppid=7595 vsize=2290700 CPUtime=882.07 cores=1,3,5,7
/proc/7597/task/7606/stat : 7606 (plingeling) R 7595 7597 6934 0 -1 4202560 331640 1258 0 0 88162 45 0 0 25 0 5 0 523273382 2345676800 544112 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7597/tid=7607] ppid=7595 vsize=2290700 CPUtime=877.26 cores=1,3,5,7
/proc/7597/task/7607/stat : 7607 (plingeling) R 7595 7597 6934 0 -1 4202560 956498 1258 0 0 87573 153 0 0 25 0 5 0 523273382 2345676800 544112 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3523.6
Current children cumulated vsize (KiB) 2290700

[startup+942.3 s]
/proc/loadavg: 8.10 8.01 7.17 9/186 7632
/proc/meminfo: memFree=27176480/32951124 swapFree=67107188/67111528
[pid=7597] ppid=7595 vsize=2465944 CPUtime=3763.28 cores=1,3,5,7
/proc/7597/stat : 7597 (plingeling) S 7595 7597 6934 0 -1 4202496 2143090 1258 0 0 376015 313 0 0 25 0 5 0 523273359 2525126656 587923 33554432000 4194304 4417332 140734608330144 18446744073709551615 210216450021 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7597/statm: 616486 587923 181 55 0 614372 0
[pid=7597/tid=7604] ppid=7595 vsize=2465944 CPUtime=941.98 cores=1,3,5,7
/proc/7597/task/7604/stat : 7604 (plingeling) R 7595 7597 6934 0 -1 4202560 418877 1258 0 0 94146 52 0 0 25 0 5 0 523273382 2525126656 587923 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7597/tid=7605] ppid=7595 vsize=2465944 CPUtime=942.04 cores=1,3,5,7
/proc/7597/task/7605/stat : 7605 (plingeling) R 7595 7597 6934 0 -1 4202560 322526 1258 0 0 94159 45 0 0 25 0 5 0 523273382 2525126656 587923 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364750 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7597/tid=7606] ppid=7595 vsize=2465944 CPUtime=942.06 cores=1,3,5,7
/proc/7597/task/7606/stat : 7606 (plingeling) R 7595 7597 6934 0 -1 4202560 397176 1258 0 0 94154 52 0 0 25 0 5 0 523273382 2525126656 587923 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7597/tid=7607] ppid=7595 vsize=2465944 CPUtime=936.94 cores=1,3,5,7
/proc/7597/task/7607/stat : 7607 (plingeling) R 7595 7597 6934 0 -1 4202560 995629 1258 0 0 93532 162 0 0 25 0 5 0 523273382 2525126656 587923 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364572 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3763.28
Current children cumulated vsize (KiB) 2465944

[startup+1002.3 s]
/proc/loadavg: 8.04 8.01 7.22 9/186 7670
/proc/meminfo: memFree=28242568/32951124 swapFree=67107188/67111528
[pid=7597] ppid=7595 vsize=2754228 CPUtime=4002.95 cores=1,3,5,7
/proc/7597/stat : 7597 (plingeling) S 7595 7597 6934 0 -1 4202496 2305273 1258 0 0 399959 336 0 0 25 0 5 0 523273359 2820329472 659994 33554432000 4194304 4417332 140734608330144 18446744073709551615 210216450021 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7597/statm: 688557 659994 181 55 0 686443 0
[pid=7597/tid=7604] ppid=7595 vsize=2754228 CPUtime=1001.98 cores=1,3,5,7
/proc/7597/task/7604/stat : 7604 (plingeling) R 7595 7597 6934 0 -1 4202560 492605 1258 0 0 100138 60 0 0 25 0 5 0 523273382 2820329472 659994 33554432000 4194304 4417332 140734608330144 18446744073709551615 4263092 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7597/tid=7605] ppid=7595 vsize=2754228 CPUtime=1002.03 cores=1,3,5,7
/proc/7597/task/7605/stat : 7605 (plingeling) R 7595 7597 6934 0 -1 4202560 360803 1258 0 0 100153 50 0 0 25 0 5 0 523273382 2820329472 659994 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7597/tid=7606] ppid=7595 vsize=2754228 CPUtime=1002.07 cores=1,3,5,7
/proc/7597/task/7606/stat : 7606 (plingeling) R 7595 7597 6934 0 -1 4202560 416116 1258 0 0 100153 54 0 0 25 0 5 0 523273382 2820329472 659994 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7597/tid=7607] ppid=7595 vsize=2754228 CPUtime=996.61 cores=1,3,5,7
/proc/7597/task/7607/stat : 7607 (plingeling) R 7595 7597 6934 0 -1 4202560 1026867 1258 0 0 99492 169 0 0 25 0 5 0 523273382 2820329472 659994 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364656 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4002.95
Current children cumulated vsize (KiB) 2754228

[startup+1062.3 s]
/proc/loadavg: 8.01 8.00 7.26 9/186 7672
/proc/meminfo: memFree=27974660/32951124 swapFree=67107188/67111528
[pid=7597] ppid=7595 vsize=2812668 CPUtime=4242.62 cores=1,3,5,7
/proc/7597/stat : 7597 (plingeling) S 7595 7597 6934 0 -1 4202496 2407675 1258 0 0 423913 349 0 0 25 0 5 0 523273359 2880172032 674604 33554432000 4194304 4417332 140734608330144 18446744073709551615 210216450021 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7597/statm: 703167 674604 181 55 0 701053 0
[pid=7597/tid=7604] ppid=7595 vsize=2812668 CPUtime=1061.97 cores=1,3,5,7
/proc/7597/task/7604/stat : 7604 (plingeling) R 7595 7597 6934 0 -1 4202560 545347 1258 0 0 106131 66 0 0 25 0 5 0 523273382 2880172032 674604 33554432000 4194304 4417332 140734608330144 18446744073709551615 4311298 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7597/tid=7605] ppid=7595 vsize=2812668 CPUtime=1062.03 cores=1,3,5,7
/proc/7597/task/7605/stat : 7605 (plingeling) R 7595 7597 6934 0 -1 4202560 379891 1258 0 0 106151 52 0 0 25 0 5 0 523273382 2880172032 674604 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7597/tid=7606] ppid=7595 vsize=2812668 CPUtime=1062.07 cores=1,3,5,7
/proc/7597/task/7606/stat : 7606 (plingeling) R 7595 7597 6934 0 -1 4202560 430304 1258 0 0 106151 56 0 0 25 0 5 0 523273382 2880172032 674604 33554432000 4194304 4417332 140734608330144 18446744073709551615 4365977 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7597/tid=7607] ppid=7595 vsize=2812668 CPUtime=1056.28 cores=1,3,5,7
/proc/7597/task/7607/stat : 7607 (plingeling) R 7595 7597 6934 0 -1 4202560 1043251 1258 0 0 105456 172 0 0 25 0 5 0 523273382 2880172032 674604 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4242.62
Current children cumulated vsize (KiB) 2812668

[startup+1122.3 s]
/proc/loadavg: 8.04 8.01 7.31 9/186 7674
/proc/meminfo: memFree=27299416/32951124 swapFree=67107188/67111528
[pid=7597] ppid=7595 vsize=3112360 CPUtime=4482.28 cores=1,3,5,7
/proc/7597/stat : 7597 (plingeling) S 7595 7597 6934 0 -1 4202496 2563448 1258 0 0 447854 374 0 0 25 0 5 0 523273359 3187056640 749527 33554432000 4194304 4417332 140734608330144 18446744073709551615 210216450021 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7597/statm: 778090 749527 181 55 0 775976 0
[pid=7597/tid=7604] ppid=7595 vsize=3112360 CPUtime=1121.97 cores=1,3,5,7
/proc/7597/task/7604/stat : 7604 (plingeling) R 7595 7597 6934 0 -1 4202560 604930 1258 0 0 112122 75 0 0 25 0 5 0 523273382 3187056640 749527 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364479 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7597/tid=7605] ppid=7595 vsize=3112360 CPUtime=1122.03 cores=1,3,5,7
/proc/7597/task/7605/stat : 7605 (plingeling) R 7595 7597 6934 0 -1 4202560 460566 1258 0 0 112138 65 0 0 25 0 5 0 523273382 3187056640 749527 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7597/tid=7606] ppid=7595 vsize=3112360 CPUtime=1122.07 cores=1,3,5,7
/proc/7597/task/7606/stat : 7606 (plingeling) R 7595 7597 6934 0 -1 4202560 433696 1258 0 0 112151 56 0 0 25 0 5 0 523273382 3187056640 749527 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364599 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7597/tid=7607] ppid=7595 vsize=3112360 CPUtime=1115.96 cores=1,3,5,7
/proc/7597/task/7607/stat : 7607 (plingeling) R 7595 7597 6934 0 -1 4202560 1055374 1258 0 0 111421 175 0 0 25 0 5 0 523273382 3187056640 749527 33554432000 4194304 4417332 140734608330144 18446744073709551615 4251063 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4482.28
Current children cumulated vsize (KiB) 3112360

[startup+1182.3 s]
/proc/loadavg: 8.01 8.01 7.35 9/186 7675
/proc/meminfo: memFree=27309016/32951124 swapFree=67107188/67111528
[pid=7597] ppid=7595 vsize=3172264 CPUtime=4721.95 cores=1,3,5,7
/proc/7597/stat : 7597 (plingeling) S 7595 7597 6934 0 -1 4202496 2603000 1258 0 0 471816 379 0 0 25 0 5 0 523273359 3248398336 764503 33554432000 4194304 4417332 140734608330144 18446744073709551615 210216450021 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7597/statm: 793066 764503 181 55 0 790952 0
[pid=7597/tid=7604] ppid=7595 vsize=3172264 CPUtime=1181.97 cores=1,3,5,7
/proc/7597/task/7604/stat : 7604 (plingeling) R 7595 7597 6934 0 -1 4202560 604930 1258 0 0 118122 75 0 0 25 0 5 0 523273382 3248398336 764503 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7597/tid=7605] ppid=7595 vsize=3172264 CPUtime=1182.03 cores=1,3,5,7
/proc/7597/task/7605/stat : 7605 (plingeling) R 7595 7597 6934 0 -1 4202560 468758 1258 0 0 118137 66 0 0 25 0 5 0 523273382 3248398336 764503 33554432000 4194304 4417332 140734608330144 18446744073709551615 4302921 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7597/tid=7606] ppid=7595 vsize=3172264 CPUtime=1182.07 cores=1,3,5,7
/proc/7597/task/7606/stat : 7606 (plingeling) R 7595 7597 6934 0 -1 4202560 465056 1258 0 0 118147 60 0 0 25 0 5 0 523273382 3248398336 764503 33554432000 4194304 4417332 140734608330144 18446744073709551615 4311363 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7597/tid=7607] ppid=7595 vsize=3172264 CPUtime=1175.63 cores=1,3,5,7
/proc/7597/task/7607/stat : 7607 (plingeling) R 7595 7597 6934 0 -1 4202560 1055374 1258 0 0 117388 175 0 0 25 0 5 0 523273382 3248398336 764503 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364734 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4721.95
Current children cumulated vsize (KiB) 3172264



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.9 s]
/proc/loadavg: 8.01 8.01 7.37 9/186 7676
/proc/meminfo: memFree=27052828/32951124 swapFree=67107188/67111528
[pid=7597] ppid=7595 vsize=3323116 CPUtime=4800.26 cores=1,3,5,7
/proc/7597/stat : 7597 (plingeling) S 7595 7597 6934 0 -1 4202496 2695743 1258 0 0 479632 394 0 0 25 0 5 0 523273359 3402870784 802216 33554432000 4194304 4417332 140734608330144 18446744073709551615 210216450021 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7597/statm: 830779 802216 181 55 0 828665 0
[pid=7597/tid=7604] ppid=7595 vsize=3323116 CPUtime=1201.57 cores=1,3,5,7
/proc/7597/task/7604/stat : 7604 (plingeling) R 7595 7597 6934 0 -1 4202560 604930 1258 0 0 120082 75 0 0 25 0 5 0 523273382 3402870784 802216 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7597/tid=7605] ppid=7595 vsize=3323116 CPUtime=1201.64 cores=1,3,5,7
/proc/7597/task/7605/stat : 7605 (plingeling) R 7595 7597 6934 0 -1 4202560 515624 1258 0 0 120089 75 0 0 25 0 5 0 523273382 3402870784 802216 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364818 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7597/tid=7606] ppid=7595 vsize=3323116 CPUtime=1201.68 cores=1,3,5,7
/proc/7597/task/7606/stat : 7606 (plingeling) R 7595 7597 6934 0 -1 4202560 510933 1258 0 0 120101 67 0 0 25 0 5 0 523273382 3402870784 802216 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364796 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7597/tid=7607] ppid=7595 vsize=3323116 CPUtime=1195.13 cores=1,3,5,7
/proc/7597/task/7607/stat : 7607 (plingeling) R 7595 7597 6934 0 -1 4202560 1055374 1258 0 0 119338 175 0 0 25 0 5 0 523273382 3402870784 802216 33554432000 4194304 4417332 140734608330144 18446744073709551615 4364683 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4800.26
Current children cumulated vsize (KiB) 3323116

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1202.01 s]
/proc/loadavg: 8.01 8.01 7.37 9/186 7676
/proc/meminfo: memFree=27052828/32951124 swapFree=67107188/67111528
[pid=7597] ppid=7595 vsize=0 CPUtime=4800.38 cores=1,3,5,7
/proc/7597/stat : 7597 (plingeling) Z 7595 7597 6934 0 -1 4203548 2695743 1258 0 0 479634 404 0 0 18 0 2 0 523273359 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/7597/statm: 0 0 0 0 0 0 0
[pid=7597/tid=7607] ppid=7595 vsize=0 CPUtime=1195.23 cores=1,3,5,7
/proc/7597/task/7607/stat : 7607 (plingeling) R 7595 7597 6934 0 -1 4203588 1055374 1258 0 0 119338 185 0 0 25 0 2 0 523273382 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4800.38
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1202.02
CPU time (s): 4800.4
CPU user time (s): 4796.35
CPU system time (s): 4.04738
CPU usage (%): 399.361
Max. virtual memory (cumulated for all children) (KiB): 3323116

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4796.35
system time used= 4.04738
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2697001
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 107
involuntary context switches= 35917

runsolver used 2.01769 second user time and 4.54331 second system time

The end

Launcher Data 

Begin job on node130 at 2011-04-30 00:39:09
IDJOB=3251208
IDBENCH=88216
IDSOLVER=1605
FILE ID=node130/3251208-1304116748
RUNJOBID= node130-1304116518-7455
PBS_JOBID= 13170686
Free space on /tmp= 71988 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT11/crafted/anton/SRHD-SGI/srhd-sgi-m47-q841.5-n45-p30-s84954709.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3251208-1304116748/watcher-3251208-1304116748 -o /tmp/evaluation-result-3251208-1304116748/solver-3251208-1304116748 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3251208-1304116748.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 556c58061360a1b195d9b2f28888af77
RANDOM SEED=1674430533

node130.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.81
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.81
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      30413664 kB
Buffers:        158288 kB
Cached:        1213072 kB
SwapCached:       1448 kB
Active:         906356 kB
Inactive:      1354876 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      30413664 kB
SwapTotal:    67111528 kB
SwapFree:     67107188 kB
Dirty:            4684 kB
Writeback:           0 kB
AnonPages:      888596 kB
Mapped:          15580 kB
Slab:           212376 kB
PageTables:       6484 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:  1080556 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71976 MiB
End job on node130 at 2011-04-30 00:59:13