Trace number 3395002

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40000.5 5012.3

General information on the benchmark

Namecrafted/anton/SRHD-SGI/
srhd-sgi-m47-q742.5-n45-p15-s28972035.cnf
MD5SUMbfa662ff57088c54e0febd091ce11e5c
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark0.268958
Satisfiable
(Un)Satisfiability was proved
Number of variables1985
Number of clauses409171
Sum of the clauses size820237
Maximum clause length47
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2409126
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 545

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3395002-1306110361/watcher-3395002-1306110361 -o /tmp/evaluation-result-3395002-1306110361/solver-3395002-1306110361 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3395002-1306110361.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.56 7.98 7.98 3/528 9671
/proc/meminfo: memFree=15520824/32951124 swapFree=51873512/67111528
[pid=9671] ppid=9669 vsize=10140 CPUtime=0 cores=0-7
/proc/9671/stat : 9671 (plingeling) R 9669 9671 9645 0 -1 4202496 595 1252 0 0 0 0 0 0 21 0 1 0 703452351 10383360 524 33554432000 4194304 4417452 140733959500672 18446744073709551615 212039352075 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/9671/statm: 2535 532 147 55 0 421 0

[startup+0.0320279 s]
/proc/loadavg: 7.56 7.98 7.98 3/528 9671
/proc/meminfo: memFree=15520824/32951124 swapFree=51873512/67111528
[pid=9671] ppid=9669 vsize=14260 CPUtime=0.02 cores=0-7
/proc/9671/stat : 9671 (plingeling) R 9669 9671 9645 0 -1 4202496 1777 1252 0 0 2 0 0 0 21 0 1 0 703452351 14602240 1587 33554432000 4194304 4417452 140733959500672 18446744073709551615 4262125 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/9671/statm: 3565 1587 148 55 0 1451 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 14260

[startup+0.101013 s]
/proc/loadavg: 7.56 7.98 7.98 3/528 9671
/proc/meminfo: memFree=15520824/32951124 swapFree=51873512/67111528
[pid=9671] ppid=9669 vsize=26548 CPUtime=0.09 cores=0-7
/proc/9671/stat : 9671 (plingeling) R 9669 9671 9645 0 -1 4202496 4849 1252 0 0 9 0 0 0 21 0 1 0 703452351 27185152 4659 33554432000 4194304 4417452 140733959500672 18446744073709551615 212039274076 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/9671/statm: 6637 4659 148 55 0 4523 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 26548

[startup+0.300975 s]
/proc/loadavg: 7.56 7.98 7.98 3/528 9671
/proc/meminfo: memFree=15520824/32951124 swapFree=51873512/67111528
[pid=9671] ppid=9669 vsize=42932 CPUtime=0.29 cores=0-7
/proc/9671/stat : 9671 (plingeling) R 9669 9671 9645 0 -1 4202496 8945 1252 0 0 28 1 0 0 22 0 1 0 703452351 43962368 8755 33554432000 4194304 4417452 140733959500672 18446744073709551615 4262019 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/9671/statm: 10733 8755 148 55 0 8619 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 42932

[startup+0.700872 s]
/proc/loadavg: 7.56 7.98 7.98 3/528 9671
/proc/meminfo: memFree=15520824/32951124 swapFree=51873512/67111528
[pid=9671] ppid=9669 vsize=226632 CPUtime=1.94 cores=0-7
/proc/9671/stat : 9671 (plingeling) S 9669 9671 9645 0 -1 4202496 18494 1252 0 0 192 2 0 0 25 0 9 0 703452351 232071168 17879 33554432000 4194304 4417452 140733959500672 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/9671/statm: 56658 17879 175 55 0 54544 0
Current children cumulated CPU time (s) 1.94
Current children cumulated vsize (KiB) 226632

[startup+1.50073 s]
/proc/loadavg: 7.56 7.98 7.98 9/537 9685
/proc/meminfo: memFree=15434192/32951124 swapFree=51873512/67111528
[pid=9671] ppid=9669 vsize=251172 CPUtime=8.26 cores=0-7
/proc/9671/stat : 9671 (plingeling) S 9669 9671 9645 0 -1 4202496 54103 1252 0 0 816 10 0 0 25 0 9 0 703452351 257200128 23992 33554432000 4194304 4417452 140733959500672 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/9671/statm: 62793 23992 180 55 0 60679 0
[pid=9671/tid=9678] ppid=9669 vsize=251172 CPUtime=0.96 cores=0-7
/proc/9671/task/9678/stat : 9678 (plingeling) R 9669 9671 9645 0 -1 4202560 4596 1252 0 0 96 0 0 0 18 0 9 0 703452403 257200128 23992 33554432000 4194304 4417452 140733959500672 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=9671/tid=9679] ppid=9669 vsize=251172 CPUtime=0.96 cores=0-7
/proc/9671/task/9679/stat : 9679 (plingeling) R 9669 9671 9645 0 -1 4202560 2852 1252 0 0 96 0 0 0 25 0 9 0 703452403 257200128 23992 33554432000 4194304 4417452 140733959500672 18446744073709551615 4236224 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9671/tid=9680] ppid=9669 vsize=251172 CPUtime=0.96 cores=0-7
/proc/9671/task/9680/stat : 9680 (plingeling) R 9669 9671 9645 0 -1 4202560 4475 1252 0 0 96 0 0 0 25 0 9 0 703452403 257200128 23992 33554432000 4194304 4417452 140733959500672 18446744073709551615 4250017 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9671/tid=9681] ppid=9669 vsize=251172 CPUtime=0.97 cores=0-7
/proc/9671/task/9681/stat : 9681 (plingeling) R 9669 9671 9645 0 -1 4202560 6014 1252 0 0 96 1 0 0 25 0 9 0 703452403 257200128 23992 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9671/tid=9682] ppid=9669 vsize=251172 CPUtime=0.96 cores=0-7
/proc/9671/task/9682/stat : 9682 (plingeling) R 9669 9671 9645 0 -1 4202560 6182 1252 0 0 95 1 0 0 25 0 9 0 703452403 257200128 23992 33554432000 4194304 4417452 140733959500672 18446744073709551615 4250053 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9671/tid=9683] ppid=9669 vsize=251172 CPUtime=0.96 cores=0-7
/proc/9671/task/9683/stat : 9683 (plingeling) R 9669 9671 9645 0 -1 4202560 4465 1252 0 0 95 1 0 0 25 0 9 0 703452404 257200128 23992 33554432000 4194304 4417452 140733959500672 18446744073709551615 4236255 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9671/tid=9684] ppid=9669 vsize=251172 CPUtime=0.96 cores=0-7
/proc/9671/task/9684/stat : 9684 (plingeling) R 9669 9671 9645 0 -1 4202560 5980 1252 0 0 95 1 0 0 18 0 9 0 703452404 257200128 23992 33554432000 4194304 4417452 140733959500672 18446744073709551615 4249130 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9671/tid=9685] ppid=9669 vsize=251172 CPUtime=0.94 cores=0-7
/proc/9671/task/9685/stat : 9685 (plingeling) R 9669 9671 9645 0 -1 4202560 2387 1252 0 0 94 0 0 0 25 0 9 0 703452404 257200128 23992 33554432000 4194304 4417452 140733959500672 18446744073709551615 4336723 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 8.26
Current children cumulated vsize (KiB) 251172

[startup+3.10044 s]
/proc/loadavg: 7.56 7.98 7.98 9/537 9685
/proc/meminfo: memFree=15417968/32951124 swapFree=51873512/67111528
[pid=9671] ppid=9669 vsize=270372 CPUtime=21.04 cores=0-7
/proc/9671/stat : 9671 (plingeling) S 9669 9671 9645 0 -1 4202496 67335 1252 0 0 2092 12 0 0 25 0 9 0 703452351 276860928 28792 33554432000 4194304 4417452 140733959500672 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/9671/statm: 67593 28792 180 55 0 65479 0
[pid=9671/tid=9678] ppid=9669 vsize=270372 CPUtime=2.56 cores=0-7
/proc/9671/task/9678/stat : 9678 (plingeling) R 9669 9671 9645 0 -1 4202560 4596 1252 0 0 256 0 0 0 21 0 9 0 703452403 276860928 28792 33554432000 4194304 4417452 140733959500672 18446744073709551615 4250017 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=9671/tid=9679] ppid=9669 vsize=270372 CPUtime=2.56 cores=0-7
/proc/9671/task/9679/stat : 9679 (plingeling) R 9669 9671 9645 0 -1 4202560 4198 1252 0 0 256 0 0 0 25 0 9 0 703452403 276860928 28792 33554432000 4194304 4417452 140733959500672 18446744073709551615 4258620 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9671/tid=9680] ppid=9669 vsize=270372 CPUtime=2.56 cores=0-7
/proc/9671/task/9680/stat : 9680 (plingeling) R 9669 9671 9645 0 -1 4202560 4983 1252 0 0 256 0 0 0 25 0 9 0 703452403 276860928 28792 33554432000 4194304 4417452 140733959500672 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9671/tid=9681] ppid=9669 vsize=270372 CPUtime=2.57 cores=0-7
/proc/9671/task/9681/stat : 9681 (plingeling) R 9669 9671 9645 0 -1 4202560 14348 1252 0 0 254 3 0 0 25 0 9 0 703452403 276860928 28792 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9671/tid=9682] ppid=9669 vsize=270372 CPUtime=2.56 cores=0-7
/proc/9671/task/9682/stat : 9682 (plingeling) R 9669 9671 9645 0 -1 4202560 7197 1252 0 0 255 1 0 0 25 0 9 0 703452403 276860928 28792 33554432000 4194304 4417452 140733959500672 18446744073709551615 4249130 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9671/tid=9683] ppid=9669 vsize=270372 CPUtime=2.56 cores=0-7
/proc/9671/task/9683/stat : 9683 (plingeling) R 9669 9671 9645 0 -1 4202560 5987 1252 0 0 255 1 0 0 25 0 9 0 703452404 276860928 28792 33554432000 4194304 4417452 140733959500672 18446744073709551615 4258587 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9671/tid=9684] ppid=9669 vsize=270372 CPUtime=2.56 cores=0-7
/proc/9671/task/9684/stat : 9684 (plingeling) R 9669 9671 9645 0 -1 4202560 6487 1252 0 0 255 1 0 0 21 0 9 0 703452404 276860928 28792 33554432000 4194304 4417452 140733959500672 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9671/tid=9685] ppid=9669 vsize=270372 CPUtime=2.52 cores=0-7
/proc/9671/task/9685/stat : 9685 (plingeling) R 9669 9671 9645 0 -1 4202560 2387 1252 0 0 252 0 0 0 25 0 9 0 703452404 276860928 28792 33554432000 4194304 4417452 140733959500672 18446744073709551615 4336723 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 21.04
Current children cumulated vsize (KiB) 270372

[startup+6.30087 s]
/proc/loadavg: 7.60 7.98 7.98 9/537 9685
/proc/meminfo: memFree=15394292/32951124 swapFree=51873512/67111528
[pid=9671] ppid=9669 vsize=288500 CPUtime=46.59 cores=0-7
/proc/9671/stat : 9671 (plingeling) S 9669 9671 9645 0 -1 4202496 77975 1252 0 0 4645 14 0 0 25 0 9 0 703452351 295424000 33324 33554432000 4194304 4417452 140733959500672 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/9671/statm: 72125 33324 180 55 0 70011 0
[pid=9671/tid=9678] ppid=9669 vsize=288500 CPUtime=5.76 cores=0-7
/proc/9671/task/9678/stat : 9678 (plingeling) R 9669 9671 9645 0 -1 4202560 4596 1252 0 0 576 0 0 0 25 0 9 0 703452403 295424000 33324 33554432000 4194304 4417452 140733959500672 18446744073709551615 4249222 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=9671/tid=9679] ppid=9669 vsize=288500 CPUtime=5.76 cores=0-7
/proc/9671/task/9679/stat : 9679 (plingeling) R 9669 9671 9645 0 -1 4202560 4198 1252 0 0 576 0 0 0 25 0 9 0 703452403 295424000 33324 33554432000 4194304 4417452 140733959500672 18446744073709551615 4249222 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9671/tid=9680] ppid=9669 vsize=288500 CPUtime=5.76 cores=0-7
/proc/9671/task/9680/stat : 9680 (plingeling) R 9669 9671 9645 0 -1 4202560 4983 1252 0 0 576 0 0 0 25 0 9 0 703452403 295424000 33324 33554432000 4194304 4417452 140733959500672 18446744073709551615 4248895 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9671/tid=9681] ppid=9669 vsize=288500 CPUtime=5.77 cores=0-7
/proc/9671/task/9681/stat : 9681 (plingeling) R 9669 9671 9645 0 -1 4202560 24988 1252 0 0 573 4 0 0 25 0 9 0 703452403 295424000 33324 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9671/tid=9682] ppid=9669 vsize=288500 CPUtime=5.76 cores=0-7
/proc/9671/task/9682/stat : 9682 (plingeling) R 9669 9671 9645 0 -1 4202560 7197 1252 0 0 575 1 0 0 25 0 9 0 703452403 295424000 33324 33554432000 4194304 4417452 140733959500672 18446744073709551615 4250017 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9671/tid=9683] ppid=9669 vsize=288500 CPUtime=5.76 cores=0-7
/proc/9671/task/9683/stat : 9683 (plingeling) R 9669 9671 9645 0 -1 4202560 5987 1252 0 0 575 1 0 0 25 0 9 0 703452404 295424000 33324 33554432000 4194304 4417452 140733959500672 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9671/tid=9684] ppid=9669 vsize=288500 CPUtime=5.76 cores=0-7
/proc/9671/task/9684/stat : 9684 (plingeling) R 9669 9671 9645 0 -1 4202560 6487 1252 0 0 575 1 0 0 25 0 9 0 703452404 295424000 33324 33554432000 4194304 4417452 140733959500672 18446744073709551615 4250080 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9671/tid=9685] ppid=9669 vsize=288500 CPUtime=5.66 cores=0-7
/proc/9671/task/9685/stat : 9685 (plingeling) R 9669 9671 9645 0 -1 4202560 2387 1252 0 0 566 0 0 0 25 0 9 0 703452404 295424000 33324 33554432000 4194304 4417452 140733959500672 18446744073709551615 4249563 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 46.59
Current children cumulated vsize (KiB) 288500

[startup+12.7007 s]
/proc/loadavg: 7.63 7.98 7.98 9/537 9686
/proc/meminfo: memFree=15338540/32951124 swapFree=51873512/67111528
[pid=9671] ppid=9669 vsize=317708 CPUtime=97.68 cores=0-7
/proc/9671/stat : 9671 (plingeling) S 9669 9671 9645 0 -1 4202496 100856 1252 0 0 9750 18 0 0 25 0 9 0 703452351 325332992 40144 33554432000 4194304 4417452 140733959500672 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/9671/statm: 79427 40144 180 55 0 77313 0
[pid=9671/tid=9678] ppid=9669 vsize=317708 CPUtime=12.16 cores=0-7
/proc/9671/task/9678/stat : 9678 (plingeling) R 9669 9671 9645 0 -1 4202560 4596 1252 0 0 1216 0 0 0 25 0 9 0 703452403 325332992 40144 33554432000 4194304 4417452 140733959500672 18446744073709551615 4248242 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=9671/tid=9679] ppid=9669 vsize=317708 CPUtime=12.16 cores=0-7
/proc/9671/task/9679/stat : 9679 (plingeling) R 9669 9671 9645 0 -1 4202560 4198 1252 0 0 1216 0 0 0 25 0 9 0 703452403 325332992 40144 33554432000 4194304 4417452 140733959500672 18446744073709551615 4249584 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9671/tid=9680] ppid=9669 vsize=317708 CPUtime=12.16 cores=0-7
/proc/9671/task/9680/stat : 9680 (plingeling) R 9669 9671 9645 0 -1 4202560 7456 1252 0 0 1216 0 0 0 25 0 9 0 703452403 325332992 40144 33554432000 4194304 4417452 140733959500672 18446744073709551615 4250010 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9671/tid=9681] ppid=9669 vsize=317708 CPUtime=12.17 cores=0-7
/proc/9671/task/9681/stat : 9681 (plingeling) R 9669 9671 9645 0 -1 4202560 44719 1252 0 0 1209 8 0 0 25 0 9 0 703452403 325332992 40144 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9671/tid=9682] ppid=9669 vsize=317708 CPUtime=12.16 cores=0-7
/proc/9671/task/9682/stat : 9682 (plingeling) R 9669 9671 9645 0 -1 4202560 7197 1252 0 0 1215 1 0 0 25 0 9 0 703452403 325332992 40144 33554432000 4194304 4417452 140733959500672 18446744073709551615 4249151 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9671/tid=9683] ppid=9669 vsize=317708 CPUtime=12.16 cores=0-7
/proc/9671/task/9683/stat : 9683 (plingeling) R 9669 9671 9645 0 -1 4202560 5987 1252 0 0 1215 1 0 0 25 0 9 0 703452404 325332992 40144 33554432000 4194304 4417452 140733959500672 18446744073709551615 4249151 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9671/tid=9684] ppid=9669 vsize=317708 CPUtime=12.16 cores=0-7
/proc/9671/task/9684/stat : 9684 (plingeling) R 9669 9671 9645 0 -1 4202560 7164 1252 0 0 1215 1 0 0 25 0 9 0 703452404 325332992 40144 33554432000 4194304 4417452 140733959500672 18446744073709551615 4249209 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9671/tid=9685] ppid=9669 vsize=317708 CPUtime=11.95 cores=0-7
/proc/9671/task/9685/stat : 9685 (plingeling) R 9669 9671 9645 0 -1 4202560 2387 1252 0 0 1195 0 0 0 25 0 9 0 703452404 325332992 40144 33554432000 4194304 4417452 140733959500672 18446744073709551615 4249584 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 97.68
Current children cumulated vsize (KiB) 317708

[startup+25.5004 s]
/proc/loadavg: 7.71 7.98 7.98 9/537 9686
/proc/meminfo: memFree=15317196/32951124 swapFree=51873512/67111528
[pid=9671] ppid=9669 vsize=360048 CPUtime=199.83 cores=0-7
/proc/9671/stat : 9671 (plingeling) S 9669 9671 9645 0 -1 4202496 131377 1252 0 0 19962 21 0 0 25 0 9 0 703452351 368689152 51211 33554432000 4194304 4417452 140733959500672 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/9671/statm: 90012 51211 180 55 0 87898 0
[pid=9671/tid=9678] ppid=9669 vsize=360048 CPUtime=24.97 cores=0-7
/proc/9671/task/9678/stat : 9678 (plingeling) R 9669 9671 9645 0 -1 4202560 5839 1252 0 0 2496 1 0 0 25 0 9 0 703452403 368689152 51211 33554432000 4194304 4417452 140733959500672 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=9671/tid=9679] ppid=9669 vsize=360048 CPUtime=24.96 cores=0-7
/proc/9671/task/9679/stat : 9679 (plingeling) R 9669 9671 9645 0 -1 4202560 6495 1252 0 0 2495 1 0 0 25 0 9 0 703452403 368689152 51211 33554432000 4194304 4417452 140733959500672 18446744073709551615 4259017 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9671/tid=9680] ppid=9669 vsize=360048 CPUtime=24.95 cores=0-7
/proc/9671/task/9680/stat : 9680 (plingeling) R 9669 9671 9645 0 -1 4202560 8549 1252 0 0 2495 0 0 0 25 0 9 0 703452403 368689152 51211 33554432000 4194304 4417452 140733959500672 18446744073709551615 4250017 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9671/tid=9681] ppid=9669 vsize=360048 CPUtime=24.98 cores=0-7
/proc/9671/task/9681/stat : 9681 (plingeling) R 9669 9671 9645 0 -1 4202560 59643 1252 0 0 2488 10 0 0 25 0 9 0 703452403 368689152 51211 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9671/tid=9682] ppid=9669 vsize=360048 CPUtime=24.96 cores=0-7
/proc/9671/task/9682/stat : 9682 (plingeling) R 9669 9671 9645 0 -1 4202560 8329 1252 0 0 2495 1 0 0 25 0 9 0 703452403 368689152 51211 33554432000 4194304 4417452 140733959500672 18446744073709551615 4250907 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9671/tid=9683] ppid=9669 vsize=360048 CPUtime=24.96 cores=0-7
/proc/9671/task/9683/stat : 9683 (plingeling) R 9669 9671 9645 0 -1 4202560 10719 1252 0 0 2494 2 0 0 25 0 9 0 703452404 368689152 51211 33554432000 4194304 4417452 140733959500672 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9671/tid=9684] ppid=9669 vsize=360048 CPUtime=24.96 cores=0-7
/proc/9671/task/9684/stat : 9684 (plingeling) R 9669 9671 9645 0 -1 4202560 9341 1252 0 0 2495 1 0 0 25 0 9 0 703452404 368689152 51211 33554432000 4194304 4417452 140733959500672 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9671/tid=9685] ppid=9669 vsize=360048 CPUtime=24.52 cores=0-7
/proc/9671/task/9685/stat : 9685 (plingeling) R 9669 9671 9645 0 -1 4202560 5310 1252 0 0 2452 0 0 0 25 0 9 0 703452404 368689152 51211 33554432000 4194304 4417452 140733959500672 18446744073709551615 4236302 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 199.83
Current children cumulated vsize (KiB) 360048

[startup+51.1009 s]
/proc/loadavg: 7.89 8.00 7.99 9/537 9687
/proc/meminfo: memFree=15170516/32951124 swapFree=51873512/67111528
[pid=9671] ppid=9669 vsize=506880 CPUtime=404.19 cores=0-7
/proc/9671/stat : 9671 (plingeling) S 9669 9671 9645 0 -1 4202496 196613 1252 0 0 40390 29 0 0 25 0 9 0 703452351 519045120 87919 33554432000 4194304 4417452 140733959500672 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/9671/statm: 126720 87919 180 55 0 124606 0
[pid=9671/tid=9678] ppid=9669 vsize=506880 CPUtime=50.56 cores=0-7
/proc/9671/task/9678/stat : 9678 (plingeling) R 9669 9671 9645 0 -1 4202560 8871 1252 0 0 5055 1 0 0 25 0 9 0 703452403 519045120 87919 33554432000 4194304 4417452 140733959500672 18446744073709551615 4236761 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=9671/tid=9679] ppid=9669 vsize=506880 CPUtime=50.56 cores=0-7
/proc/9671/task/9679/stat : 9679 (plingeling) R 9669 9671 9645 0 -1 4202560 11535 1252 0 0 5055 1 0 0 25 0 9 0 703452403 519045120 87919 33554432000 4194304 4417452 140733959500672 18446744073709551615 4259029 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9671/tid=9680] ppid=9669 vsize=506880 CPUtime=50.56 cores=0-7

################
# More data... #
################

[pid=9671/tid=9680] ppid=9669 vsize=8062152 CPUtime=4720.5 cores=0-7
/proc/9671/task/9680/stat : 9680 (plingeling) R 9669 9671 9645 0 -1 4202560 1731202 1252 0 0 471765 285 0 0 25 0 9 0 703452403 8255643648 1976739 33554432000 4194304 4417452 140733959500672 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9671/tid=9681] ppid=9669 vsize=8062152 CPUtime=4721.1 cores=0-7
/proc/9671/task/9681/stat : 9681 (plingeling) R 9669 9671 9645 0 -1 4202560 1141986 1252 0 0 471834 276 0 0 25 0 9 0 703452403 8255643648 1976739 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9671/tid=9682] ppid=9669 vsize=8062152 CPUtime=4721.87 cores=0-7
/proc/9671/task/9682/stat : 9682 (plingeling) R 9669 9671 9645 0 -1 4202560 1455738 1252 0 0 471942 245 0 0 25 0 9 0 703452403 8255643648 1976739 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9671/tid=9683] ppid=9669 vsize=8062152 CPUtime=4721.88 cores=0-7
/proc/9671/task/9683/stat : 9683 (plingeling) R 9669 9671 9645 0 -1 4202560 1410760 1252 0 0 471959 229 0 0 25 0 9 0 703452404 8255643648 1976739 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9671/tid=9684] ppid=9669 vsize=8062152 CPUtime=4721.89 cores=0-7
/proc/9671/task/9684/stat : 9684 (plingeling) R 9669 9671 9645 0 -1 4202560 1321100 1252 0 0 471963 226 0 0 25 0 9 0 703452404 8255643648 1976739 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9671/tid=9685] ppid=9669 vsize=8062152 CPUtime=4708.71 cores=0-7
/proc/9671/task/9685/stat : 9685 (plingeling) R 9669 9671 9645 0 -1 4202560 1449219 1252 0 0 470648 223 0 0 25 0 9 0 703452404 8255643648 1976739 33554432000 4194304 4417452 140733959500672 18446744073709551615 18446744073699065989 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 37688
Current children cumulated vsize (KiB) 8062152

[startup+4782.3 s]
/proc/loadavg: 8.08 8.06 8.02 9/537 9825
/proc/meminfo: memFree=7663928/32951124 swapFree=51873512/67111528
[pid=9671] ppid=9669 vsize=7996616 CPUtime=38166.7 cores=0-7
/proc/9671/stat : 9671 (plingeling) S 9669 9671 9645 0 -1 4202496 11258516 1252 0 0 3814711 1959 0 0 25 0 9 0 703452351 8188534784 1960355 33554432000 4194304 4417452 140733959500672 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/9671/statm: 1999154 1960355 180 55 0 1997040 0
[pid=9671/tid=9678] ppid=9669 vsize=7996616 CPUtime=4708.52 cores=0-7
/proc/9671/task/9678/stat : 9678 (plingeling) R 9669 9671 9645 0 -1 4202560 1260835 1252 0 0 470640 212 0 0 25 0 9 0 703452403 8188534784 1960355 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9671/tid=9679] ppid=9669 vsize=7996616 CPUtime=4781.75 cores=0-7
/proc/9671/task/9679/stat : 9679 (plingeling) R 9669 9671 9645 0 -1 4202560 1404988 1252 0 0 477928 247 0 0 25 0 9 0 703452403 8188534784 1960355 33554432000 4194304 4417452 140733959500672 18446744073709551615 4307843 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9671/tid=9680] ppid=9669 vsize=7996616 CPUtime=4780.46 cores=0-7
/proc/9671/task/9680/stat : 9680 (plingeling) R 9669 9671 9645 0 -1 4202560 1763970 1252 0 0 477757 289 0 0 25 0 9 0 703452403 8188534784 1960355 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9671/tid=9681] ppid=9669 vsize=7996616 CPUtime=4781.08 cores=0-7
/proc/9671/task/9681/stat : 9681 (plingeling) R 9669 9671 9645 0 -1 4202560 1141986 1252 0 0 477832 276 0 0 25 0 9 0 703452403 8188534784 1960355 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364780 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9671/tid=9682] ppid=9669 vsize=7996616 CPUtime=4781.86 cores=0-7
/proc/9671/task/9682/stat : 9682 (plingeling) R 9669 9671 9645 0 -1 4202560 1472122 1252 0 0 477939 247 0 0 25 0 9 0 703452403 8188534784 1960355 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9671/tid=9683] ppid=9669 vsize=7996616 CPUtime=4781.86 cores=0-7
/proc/9671/task/9683/stat : 9683 (plingeling) R 9669 9671 9645 0 -1 4202560 1410760 1252 0 0 477956 230 0 0 25 0 9 0 703452404 8188534784 1960355 33554432000 4194304 4417452 140733959500672 18446744073709551615 4199406 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9671/tid=9684] ppid=9669 vsize=7996616 CPUtime=4781.87 cores=0-7
/proc/9671/task/9684/stat : 9684 (plingeling) R 9669 9671 9645 0 -1 4202560 1337484 1252 0 0 477959 228 0 0 25 0 9 0 703452404 8188534784 1960355 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364807 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9671/tid=9685] ppid=9669 vsize=7996616 CPUtime=4768.69 cores=0-7
/proc/9671/task/9685/stat : 9685 (plingeling) R 9669 9671 9645 0 -1 4202560 1449219 1252 0 0 476646 223 0 0 25 0 9 0 703452404 8188534784 1960355 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38166.7
Current children cumulated vsize (KiB) 7996616

[startup+4842.3 s]
/proc/loadavg: 8.12 8.08 8.03 9/537 9826
/proc/meminfo: memFree=7274768/32951124 swapFree=51873512/67111528
[pid=9671] ppid=9669 vsize=8385468 CPUtime=38645.6 cores=0-7
/proc/9671/stat : 9671 (plingeling) S 9669 9671 9645 0 -1 4202496 11525844 1252 0 0 3862555 2002 0 0 25 0 9 0 703452351 8586719232 2057568 33554432000 4194304 4417452 140733959500672 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/9671/statm: 2096367 2057568 180 55 0 2094253 0
[pid=9671/tid=9678] ppid=9669 vsize=8385468 CPUtime=4767.42 cores=0-7
/proc/9671/task/9678/stat : 9678 (plingeling) R 9669 9671 9645 0 -1 4202560 1260835 1252 0 0 476530 212 0 0 25 0 9 0 703452403 8586719232 2057568 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9671/tid=9679] ppid=9669 vsize=8385468 CPUtime=4841.75 cores=0-7
/proc/9671/task/9679/stat : 9679 (plingeling) R 9669 9671 9645 0 -1 4202560 1508550 1252 0 0 483908 267 0 0 25 0 9 0 703452403 8586719232 2057568 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9671/tid=9680] ppid=9669 vsize=8385468 CPUtime=4840.43 cores=0-7
/proc/9671/task/9680/stat : 9680 (plingeling) R 9669 9671 9645 0 -1 4202560 1796738 1252 0 0 483750 293 0 0 25 0 9 0 703452403 8586719232 2057568 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9671/tid=9681] ppid=9669 vsize=8385468 CPUtime=4841.08 cores=0-7
/proc/9671/task/9681/stat : 9681 (plingeling) R 9669 9671 9645 0 -1 4202560 1141986 1252 0 0 483832 276 0 0 25 0 9 0 703452403 8586719232 2057568 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9671/tid=9682] ppid=9669 vsize=8385468 CPUtime=4841.86 cores=0-7
/proc/9671/task/9682/stat : 9682 (plingeling) R 9669 9671 9645 0 -1 4202560 1553968 1252 0 0 483926 260 0 0 25 0 9 0 703452403 8586719232 2057568 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9671/tid=9683] ppid=9669 vsize=8385468 CPUtime=4841.86 cores=0-7
/proc/9671/task/9683/stat : 9683 (plingeling) R 9669 9671 9645 0 -1 4202560 1443528 1252 0 0 483952 234 0 0 25 0 9 0 703452404 8586719232 2057568 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9671/tid=9684] ppid=9669 vsize=8385468 CPUtime=4841.88 cores=0-7
/proc/9671/task/9684/stat : 9684 (plingeling) R 9669 9671 9645 0 -1 4202560 1353868 1252 0 0 483957 231 0 0 25 0 9 0 703452404 8586719232 2057568 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9671/tid=9685] ppid=9669 vsize=8385468 CPUtime=4828.69 cores=0-7
/proc/9671/task/9685/stat : 9685 (plingeling) R 9669 9671 9645 0 -1 4202560 1449219 1252 0 0 482646 223 0 0 25 0 9 0 703452404 8586719232 2057568 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364616 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38645.6
Current children cumulated vsize (KiB) 8385468

[startup+4902.3 s]
/proc/loadavg: 8.08 8.08 8.03 9/537 9828
/proc/meminfo: memFree=7144420/32951124 swapFree=51873512/67111528
[pid=9671] ppid=9669 vsize=8516540 CPUtime=39124.5 cores=0-7
/proc/9671/stat : 9671 (plingeling) S 9669 9671 9645 0 -1 4202496 11591380 1252 0 0 3910433 2017 0 0 25 0 9 0 703452351 8720936960 2090336 33554432000 4194304 4417452 140733959500672 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/9671/statm: 2129135 2090336 180 55 0 2127021 0
[pid=9671/tid=9678] ppid=9669 vsize=8516540 CPUtime=4826.35 cores=0-7
/proc/9671/task/9678/stat : 9678 (plingeling) R 9669 9671 9645 0 -1 4202560 1293603 1252 0 0 482412 223 0 0 25 0 9 0 703452403 8720936960 2090336 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9671/tid=9679] ppid=9669 vsize=8516540 CPUtime=4901.75 cores=0-7
/proc/9671/task/9679/stat : 9679 (plingeling) R 9669 9671 9645 0 -1 4202560 1508550 1252 0 0 489908 267 0 0 25 0 9 0 703452403 8720936960 2090336 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9671/tid=9680] ppid=9669 vsize=8516540 CPUtime=4900.42 cores=0-7
/proc/9671/task/9680/stat : 9680 (plingeling) R 9669 9671 9645 0 -1 4202560 1829506 1252 0 0 489745 297 0 0 25 0 9 0 703452403 8720936960 2090336 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9671/tid=9681] ppid=9669 vsize=8516540 CPUtime=4901.08 cores=0-7
/proc/9671/task/9681/stat : 9681 (plingeling) R 9669 9671 9645 0 -1 4202560 1141986 1252 0 0 489832 276 0 0 25 0 9 0 703452403 8720936960 2090336 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9671/tid=9682] ppid=9669 vsize=8516540 CPUtime=4901.86 cores=0-7
/proc/9671/task/9682/stat : 9682 (plingeling) R 9669 9671 9645 0 -1 4202560 1553968 1252 0 0 489926 260 0 0 25 0 9 0 703452403 8720936960 2090336 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9671/tid=9683] ppid=9669 vsize=8516540 CPUtime=4901.86 cores=0-7
/proc/9671/task/9683/stat : 9683 (plingeling) R 9669 9671 9645 0 -1 4202560 1443528 1252 0 0 489952 234 0 0 25 0 9 0 703452404 8720936960 2090336 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9671/tid=9684] ppid=9669 vsize=8516540 CPUtime=4901.88 cores=0-7
/proc/9671/task/9684/stat : 9684 (plingeling) R 9669 9671 9645 0 -1 4202560 1353868 1252 0 0 489957 231 0 0 25 0 9 0 703452404 8720936960 2090336 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9671/tid=9685] ppid=9669 vsize=8516540 CPUtime=4888.7 cores=0-7
/proc/9671/task/9685/stat : 9685 (plingeling) R 9669 9671 9645 0 -1 4202560 1449219 1252 0 0 488647 223 0 0 25 0 9 0 703452404 8720936960 2090336 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364829 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39124.5
Current children cumulated vsize (KiB) 8516540

[startup+4962.3 s]
/proc/loadavg: 8.03 8.06 8.02 9/537 9830
/proc/meminfo: memFree=7405888/32951124 swapFree=51873512/67111528
[pid=9671] ppid=9669 vsize=8254396 CPUtime=39603.4 cores=0-7
/proc/9671/stat : 9671 (plingeling) S 9669 9671 9645 0 -1 4202496 11591380 1252 0 0 3958321 2018 0 0 25 0 9 0 703452351 8452501504 2024800 33554432000 4194304 4417452 140733959500672 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/9671/statm: 2063599 2024800 180 55 0 2061485 0
[pid=9671/tid=9678] ppid=9669 vsize=8254396 CPUtime=4885.25 cores=0-7
/proc/9671/task/9678/stat : 9678 (plingeling) R 9669 9671 9645 0 -1 4202560 1293603 1252 0 0 488302 223 0 0 25 0 9 0 703452403 8452501504 2024800 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9671/tid=9679] ppid=9669 vsize=8254396 CPUtime=4961.75 cores=0-7
/proc/9671/task/9679/stat : 9679 (plingeling) R 9669 9671 9645 0 -1 4202560 1508550 1252 0 0 495908 267 0 0 25 0 9 0 703452403 8452501504 2024800 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9671/tid=9680] ppid=9669 vsize=8254396 CPUtime=4960.41 cores=0-7
/proc/9671/task/9680/stat : 9680 (plingeling) R 9669 9671 9645 0 -1 4202560 1829506 1252 0 0 495743 298 0 0 25 0 9 0 703452403 8452501504 2024800 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9671/tid=9681] ppid=9669 vsize=8254396 CPUtime=4961.08 cores=0-7
/proc/9671/task/9681/stat : 9681 (plingeling) R 9669 9671 9645 0 -1 4202560 1141986 1252 0 0 495832 276 0 0 25 0 9 0 703452403 8452501504 2024800 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9671/tid=9682] ppid=9669 vsize=8254396 CPUtime=4961.86 cores=0-7
/proc/9671/task/9682/stat : 9682 (plingeling) R 9669 9671 9645 0 -1 4202560 1553968 1252 0 0 495926 260 0 0 25 0 9 0 703452403 8452501504 2024800 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9671/tid=9683] ppid=9669 vsize=8254396 CPUtime=4961.87 cores=0-7
/proc/9671/task/9683/stat : 9683 (plingeling) R 9669 9671 9645 0 -1 4202560 1443528 1252 0 0 495952 235 0 0 25 0 9 0 703452404 8452501504 2024800 33554432000 4194304 4417452 140733959500672 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9671/tid=9684] ppid=9669 vsize=8254396 CPUtime=4961.88 cores=0-7
/proc/9671/task/9684/stat : 9684 (plingeling) R 9669 9671 9645 0 -1 4202560 1353868 1252 0 0 495957 231 0 0 25 0 9 0 703452404 8452501504 2024800 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9671/tid=9685] ppid=9669 vsize=8254396 CPUtime=4948.7 cores=0-7
/proc/9671/task/9685/stat : 9685 (plingeling) R 9669 9671 9645 0 -1 4202560 1449219 1252 0 0 494647 223 0 0 25 0 9 0 703452404 8452501504 2024800 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364809 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39603.4
Current children cumulated vsize (KiB) 8254396



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5012 s]
/proc/loadavg: 8.06 8.06 8.02 9/537 9831
/proc/meminfo: memFree=7827456/32951124 swapFree=51873512/67111528
[pid=9671] ppid=9669 vsize=7834560 CPUtime=40000.1 cores=0-7
/proc/9671/stat : 9671 (plingeling) S 9669 9671 9645 0 -1 4202496 11711104 1252 0 0 3997967 2044 0 0 25 0 9 0 703452351 8022589440 1919841 33554432000 4194304 4417452 140733959500672 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/9671/statm: 1958640 1919841 180 55 0 1956526 0
[pid=9671/tid=9678] ppid=9669 vsize=7834560 CPUtime=4934.05 cores=0-7
/proc/9671/task/9678/stat : 9678 (plingeling) R 9669 9671 9645 0 -1 4202560 1326371 1252 0 0 493179 226 0 0 25 0 9 0 703452403 8022589440 1919841 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9671/tid=9679] ppid=9669 vsize=7834560 CPUtime=5011.46 cores=0-7
/proc/9671/task/9679/stat : 9679 (plingeling) R 9669 9671 9645 0 -1 4202560 1508550 1252 0 0 500879 267 0 0 25 0 9 0 703452403 8022589440 1919841 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364812 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9671/tid=9680] ppid=9669 vsize=7834560 CPUtime=5010.11 cores=0-7
/proc/9671/task/9680/stat : 9680 (plingeling) R 9669 9671 9645 0 -1 4202560 1834354 1252 0 0 500711 300 0 0 25 0 9 0 703452403 8022589440 1919841 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9671/tid=9681] ppid=9669 vsize=7834560 CPUtime=5010.81 cores=0-7
/proc/9671/task/9681/stat : 9681 (plingeling) R 9669 9671 9645 0 -1 4202560 1141986 1252 0 0 500805 276 0 0 25 0 9 0 703452403 8022589440 1919841 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9671/tid=9682] ppid=9669 vsize=7834560 CPUtime=5011.58 cores=0-7
/proc/9671/task/9682/stat : 9682 (plingeling) R 9669 9671 9645 0 -1 4202560 1553968 1252 0 0 500898 260 0 0 25 0 9 0 703452403 8022589440 1919841 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364653 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9671/tid=9683] ppid=9669 vsize=7834560 CPUtime=5011.59 cores=0-7
/proc/9671/task/9683/stat : 9683 (plingeling) R 9669 9671 9645 0 -1 4202560 1492866 1252 0 0 500911 248 0 0 25 0 9 0 703452404 8022589440 1919841 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9671/tid=9684] ppid=9669 vsize=7834560 CPUtime=5011.6 cores=0-7
/proc/9671/task/9684/stat : 9684 (plingeling) R 9669 9671 9645 0 -1 4202560 1386638 1252 0 0 500922 238 0 0 25 0 9 0 703452404 8022589440 1919841 33554432000 4194304 4417452 140733959500672 18446744073709551615 4255619 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9671/tid=9685] ppid=9669 vsize=7834560 CPUtime=4998.42 cores=0-7
/proc/9671/task/9685/stat : 9685 (plingeling) R 9669 9671 9645 0 -1 4202560 1449219 1252 0 0 499619 223 0 0 25 0 9 0 703452404 8022589440 1919841 33554432000 4194304 4417452 140733959500672 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 40000.1
Current children cumulated vsize (KiB) 7834560

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5012.2 s]
/proc/loadavg: 8.06 8.06 8.02 9/537 9831
/proc/meminfo: memFree=7827456/32951124 swapFree=51873512/67111528
[pid=9671] ppid=9669 vsize=0 CPUtime=40000.4 cores=0-7
/proc/9671/stat : 9671 (plingeling) Z 9669 9671 9645 0 -1 4203548 11711104 1252 0 0 3997981 2061 0 0 18 0 2 0 703452351 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/9671/statm: 0 0 0 0 0 0 0
[pid=9671/tid=9678] ppid=9669 vsize=0 CPUtime=4934.24 cores=0-7
/proc/9671/task/9678/stat : 9678 (plingeling) R 9669 9671 9645 0 -1 4203588 1326371 1252 0 0 493180 244 0 0 25 0 2 0 703452403 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 0

[startup+5012.3 s]
/proc/loadavg: 8.06 8.06 8.02 9/537 9831
/proc/meminfo: memFree=7827456/32951124 swapFree=51873512/67111528
[pid=9671] ppid=9669 vsize=0 CPUtime=40000.5 cores=0-7
/proc/9671/stat : 9671 (plingeling) Z 9669 9671 9645 0 -1 4203548 11711104 1252 0 0 3997981 2071 0 0 18 0 2 0 703452351 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 5 0 0 0
/proc/9671/statm: 0 0 0 0 0 0 0
[pid=9671/tid=9678] ppid=9669 vsize=0 CPUtime=4934.34 cores=0-7
/proc/9671/task/9678/stat : 9678 (plingeling) S 9669 9671 9645 0 -1 4203588 1326371 1252 0 0 493180 254 0 0 25 0 2 0 703452403 0 0 33554432000 0 0 0 0 0 0 2147483391 4096 0 18446744073709551615 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 40000.5
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5012.3
CPU time (s): 40000.5
CPU user time (s): 39979.8
CPU system time (s): 20.7208
CPU usage (%): 798.047
Max. virtual memory (cumulated for all children) (KiB): 8516548

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39979.8
system time used= 20.7208
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 11712356
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 461
involuntary context switches= 194235

runsolver used 25.2872 second user time and 65.3951 second system time

The end

Launcher Data 

Begin job on node146 at 2011-05-23 02:26:01
IDJOB=3395002
IDBENCH=88209
IDSOLVER=1851
FILE ID=node146/3395002-1306110361
RUNJOBID= node146-1306110360-9659
PBS_JOBID= 13388046
Free space on /tmp= 73028 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/crafted/anton/SRHD-SGI/srhd-sgi-m47-q742.5-n45-p15-s28972035.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3395002-1306110361/watcher-3395002-1306110361 -o /tmp/evaluation-result-3395002-1306110361/solver-3395002-1306110361 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3395002-1306110361.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= bfa662ff57088c54e0febd091ce11e5c
RANDOM SEED=1209688300

node146.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      15521228 kB
Buffers:        312568 kB
Cached:        4771520 kB
SwapCached:      56204 kB
Active:       12512324 kB
Inactive:      4602168 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      15521228 kB
SwapTotal:    67111528 kB
SwapFree:     51873512 kB
Dirty:            5464 kB
Writeback:           0 kB
AnonPages:    11981480 kB
Mapped:          29516 kB
Slab:           144268 kB
PageTables:      96844 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 160666992 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73028 MiB
End job on node146 at 2011-05-23 03:49:35