Trace number 3371274

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.08 1205.03

General information on the benchmark

Nameindustrial/anbulagan/
hard-unsat/dated-10-17-u.cnf
MD5SUM780d946576ec5746b4ea9832bb01dbc5
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark1033.89
Satisfiable
(Un)Satisfiability was proved
Number of variables229544
Number of clauses1070757
Sum of the clauses size2471122
Maximum clause length10
Minimum clause length1
Number of clauses of size 119156
Number of clauses of size 2831957
Number of clauses of size 3176604
Number of clauses of size 44780
Number of clauses of size 519120
Number of clauses of size over 519140

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3371274-1305415399/watcher-3371274-1305415399 -o /tmp/evaluation-result-3371274-1305415399/solver-3371274-1305415399 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3371274-1305415399.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.46 4.06 5.25 2/279 5738
/proc/meminfo: memFree=19020472/32951132 swapFree=67093824/67111528
[pid=5738] ppid=5736 vsize=8728 CPUtime=0 cores=1,3,5,7
/proc/5738/stat : 5738 (plingeling) S 5736 5738 5706 0 -1 4202496 216 0 0 0 0 0 0 0 21 0 1 0 653139612 8937472 147 33554432000 4194304 4417452 140733503590240 18446744073709551615 274664808192 0 0 4096 0 18446744071562362840 0 0 17 1 0 0 0
/proc/5738/statm: 2182 147 122 55 0 68 0

[startup+0.0797831 s]
/proc/loadavg: 3.46 4.06 5.25 2/279 5738
/proc/meminfo: memFree=19020472/32951132 swapFree=67093824/67111528
[pid=5738] ppid=5736 vsize=16940 CPUtime=0 cores=1,3,5,7
/proc/5738/stat : 5738 (plingeling) R 5736 5738 5706 0 -1 4202496 2297 1250 0 2 0 0 0 0 21 0 1 0 653139612 17346560 2227 33554432000 4194304 4417452 140733503590240 18446744073709551615 274664808192 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/5738/statm: 4235 2227 148 55 0 2121 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 16940

[startup+0.100776 s]
/proc/loadavg: 3.46 4.06 5.25 2/279 5738
/proc/meminfo: memFree=19020472/32951132 swapFree=67093824/67111528
[pid=5738] ppid=5736 vsize=23452 CPUtime=0.02 cores=1,3,5,7
/proc/5738/stat : 5738 (plingeling) R 5736 5738 5706 0 -1 4202496 3973 1250 0 2 2 0 0 0 21 0 1 0 653139612 24014848 3887 33554432000 4194304 4417452 140733503590240 18446744073709551615 274664427056 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/5738/statm: 5863 3887 148 55 0 3749 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 23452

[startup+0.300742 s]
/proc/loadavg: 3.46 4.06 5.25 2/279 5738
/proc/meminfo: memFree=19020472/32951132 swapFree=67093824/67111528
[pid=5738] ppid=5736 vsize=67116 CPUtime=0.22 cores=1,3,5,7
/proc/5738/stat : 5738 (plingeling) R 5736 5738 5706 0 -1 4202496 14889 1250 0 2 21 1 0 0 21 0 1 0 653139612 68726784 14803 33554432000 4194304 4417452 140733503590240 18446744073709551615 4264909 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/5738/statm: 16779 14803 148 55 0 14665 0
Current children cumulated CPU time (s) 0.22
Current children cumulated vsize (KiB) 67116

[startup+0.700674 s]
/proc/loadavg: 3.46 4.06 5.25 2/279 5738
/proc/meminfo: memFree=19020472/32951132 swapFree=67093824/67111528
[pid=5738] ppid=5736 vsize=121556 CPUtime=0.62 cores=1,3,5,7
/proc/5738/stat : 5738 (plingeling) R 5736 5738 5706 0 -1 4202496 28523 1250 0 2 59 3 0 0 22 0 1 0 653139612 124473344 28414 33554432000 4194304 4417452 140733503590240 18446744073709551615 274664808192 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/5738/statm: 30389 28414 149 55 0 28275 0
Current children cumulated CPU time (s) 0.62
Current children cumulated vsize (KiB) 121556

[startup+1.50049 s]
/proc/loadavg: 3.35 4.03 5.23 3/283 5755
/proc/meminfo: memFree=18677896/32951132 swapFree=67093824/67111528
[pid=5738] ppid=5736 vsize=272584 CPUtime=1.71 cores=1,3,5,7
/proc/5738/stat : 5738 (plingeling) S 5736 5738 5706 0 -1 4202496 60313 1250 0 2 163 8 0 0 25 0 5 0 653139612 279126016 54563 33554432000 4194304 4417452 140733503590240 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/5738/statm: 68146 54563 159 55 0 66032 0
Current children cumulated CPU time (s) 1.71
Current children cumulated vsize (KiB) 272584

[startup+3.10122 s]
/proc/loadavg: 3.35 4.03 5.23 6/287 5759
/proc/meminfo: memFree=18509100/32951132 swapFree=67093824/67111528
[pid=5738] ppid=5736 vsize=311588 CPUtime=8.1 cores=1,3,5,7
/proc/5738/stat : 5738 (plingeling) S 5736 5738 5706 0 -1 4202496 79882 1250 0 2 799 11 0 0 25 0 5 0 653139612 319066112 52787 33554432000 4194304 4417452 140733503590240 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/5738/statm: 77897 52787 175 55 0 75783 0
[pid=5738/tid=5756] ppid=5736 vsize=311588 CPUtime=1.7 cores=1,3,5,7
/proc/5738/task/5756/stat : 5756 (plingeling) R 5736 5738 5706 0 -1 4202560 6712 1250 0 2 169 1 0 0 25 0 5 0 653139752 319066112 52787 33554432000 4194304 4417452 140733503590240 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=5738/tid=5757] ppid=5736 vsize=311588 CPUtime=1.69 cores=1,3,5,7
/proc/5738/task/5757/stat : 5757 (plingeling) R 5736 5738 5706 0 -1 4202560 7630 1250 0 2 168 1 0 0 20 0 5 0 653139752 319066112 52787 33554432000 4194304 4417452 140733503590240 18446744073709551615 4275944 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=5738/tid=5758] ppid=5736 vsize=311588 CPUtime=1.68 cores=1,3,5,7
/proc/5738/task/5758/stat : 5758 (plingeling) R 5736 5738 5706 0 -1 4202560 6248 1250 0 2 168 0 0 0 25 0 5 0 653139752 319066112 52787 33554432000 4194304 4417452 140733503590240 18446744073709551615 4236835 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=5738/tid=5759] ppid=5736 vsize=311588 CPUtime=1.68 cores=1,3,5,7
/proc/5738/task/5759/stat : 5759 (plingeling) R 5736 5738 5706 0 -1 4202560 4385 1250 0 2 168 0 0 0 25 0 5 0 653139752 319066112 52787 33554432000 4194304 4417452 140733503590240 18446744073709551615 4237290 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 8.1
Current children cumulated vsize (KiB) 311588

[startup+6.30066 s]
/proc/loadavg: 3.35 4.03 5.23 11/291 5764
/proc/meminfo: memFree=18588420/32951132 swapFree=67093824/67111528
[pid=5738] ppid=5736 vsize=293576 CPUtime=20.87 cores=1,3,5,7
/proc/5738/stat : 5738 (plingeling) S 5736 5738 5706 0 -1 4202496 80210 1250 0 2 2076 11 0 0 25 0 5 0 653139612 300621824 48302 33554432000 4194304 4417452 140733503590240 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/5738/statm: 73394 48302 179 55 0 71280 0
[pid=5738/tid=5756] ppid=5736 vsize=293576 CPUtime=4.9 cores=1,3,5,7
/proc/5738/task/5756/stat : 5756 (plingeling) R 5736 5738 5706 0 -1 4202560 6847 1250 0 2 489 1 0 0 25 0 5 0 653139752 300621824 48302 33554432000 4194304 4417452 140733503590240 18446744073709551615 4233477 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=5738/tid=5757] ppid=5736 vsize=293576 CPUtime=4.89 cores=1,3,5,7
/proc/5738/task/5757/stat : 5757 (plingeling) R 5736 5738 5706 0 -1 4202560 7819 1250 0 2 488 1 0 0 25 0 5 0 653139752 300621824 48302 33554432000 4194304 4417452 140733503590240 18446744073709551615 4336667 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=5738/tid=5758] ppid=5736 vsize=293576 CPUtime=4.88 cores=1,3,5,7
/proc/5738/task/5758/stat : 5758 (plingeling) R 5736 5738 5706 0 -1 4202560 6252 1250 0 2 488 0 0 0 25 0 5 0 653139752 300621824 48302 33554432000 4194304 4417452 140733503590240 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=5738/tid=5759] ppid=5736 vsize=293576 CPUtime=4.85 cores=1,3,5,7
/proc/5738/task/5759/stat : 5759 (plingeling) R 5736 5738 5706 0 -1 4202560 4385 1250 0 2 485 0 0 0 25 0 5 0 653139752 300621824 48302 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364466 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 20.87
Current children cumulated vsize (KiB) 293576

[startup+12.7006 s]
/proc/loadavg: 4.53 4.26 5.29 9/291 5764
/proc/meminfo: memFree=18711312/32951132 swapFree=67093824/67111528
[pid=5738] ppid=5736 vsize=219008 CPUtime=44.16 cores=1,3,5,7
/proc/5738/stat : 5738 (plingeling) S 5736 5738 5706 0 -1 4202496 80232 1250 0 2 4373 43 0 0 25 0 5 0 653139612 224264192 29661 33554432000 4194304 4417452 140733503590240 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/5738/statm: 54752 29661 180 55 0 52638 0
[pid=5738/tid=5756] ppid=5736 vsize=219008 CPUtime=10.63 cores=1,3,5,7
/proc/5738/task/5756/stat : 5756 (plingeling) R 5736 5738 5706 0 -1 4202560 6853 1250 0 2 1058 5 0 0 25 0 5 0 653139752 224264192 29661 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364537 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5738/tid=5757] ppid=5736 vsize=219008 CPUtime=11.19 cores=1,3,5,7
/proc/5738/task/5757/stat : 5757 (plingeling) R 5736 5738 5706 0 -1 4202560 7826 1250 0 2 1116 3 0 0 25 0 5 0 653139752 224264192 29661 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=5738/tid=5758] ppid=5736 vsize=219008 CPUtime=11.28 cores=1,3,5,7
/proc/5738/task/5758/stat : 5758 (plingeling) R 5736 5738 5706 0 -1 4202560 6261 1250 0 2 1126 2 0 0 25 0 5 0 653139752 224264192 29661 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=5738/tid=5759] ppid=5736 vsize=219008 CPUtime=9.7 cores=1,3,5,7
/proc/5738/task/5759/stat : 5759 (plingeling) R 5736 5738 5706 0 -1 4202560 4385 1250 0 2 946 24 0 0 25 0 5 0 653139752 224264192 29661 33554432000 4194304 4417452 140733503590240 18446744073709551615 4281668 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 44.16
Current children cumulated vsize (KiB) 219008

[startup+25.5003 s]
/proc/loadavg: 5.21 4.41 5.33 9/291 5764
/proc/meminfo: memFree=18775240/32951132 swapFree=67093824/67111528
[pid=5738] ppid=5736 vsize=204724 CPUtime=92.76 cores=1,3,5,7
/proc/5738/stat : 5738 (plingeling) S 5736 5738 5706 0 -1 4202496 82942 1250 0 2 9182 94 0 0 25 0 5 0 653139612 209637376 26096 33554432000 4194304 4417452 140733503590240 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/5738/statm: 51181 26096 180 55 0 49067 0
[pid=5738/tid=5756] ppid=5736 vsize=204724 CPUtime=23.32 cores=1,3,5,7
/proc/5738/task/5756/stat : 5756 (plingeling) R 5736 5738 5706 0 -1 4202560 7892 1250 0 2 2325 7 0 0 25 0 5 0 653139752 209637376 26096 33554432000 4194304 4417452 140733503590240 18446744073709551615 4281668 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5738/tid=5757] ppid=5736 vsize=204724 CPUtime=23.99 cores=1,3,5,7
/proc/5738/task/5757/stat : 5757 (plingeling) R 5736 5738 5706 0 -1 4202560 9312 1250 0 2 2395 4 0 0 25 0 5 0 653139752 209637376 26096 33554432000 4194304 4417452 140733503590240 18446744073709551615 4277446 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=5738/tid=5758] ppid=5736 vsize=204724 CPUtime=24.08 cores=1,3,5,7
/proc/5738/task/5758/stat : 5758 (plingeling) R 5736 5738 5706 0 -1 4202560 6286 1250 0 2 2405 3 0 0 25 0 5 0 653139752 209637376 26096 33554432000 4194304 4417452 140733503590240 18446744073709551615 4366494 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=5738/tid=5759] ppid=5736 vsize=204724 CPUtime=20.01 cores=1,3,5,7
/proc/5738/task/5759/stat : 5759 (plingeling) R 5736 5738 5706 0 -1 4202560 4545 1250 0 2 1930 71 0 0 25 0 5 0 653139752 209637376 26096 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364466 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 92.76
Current children cumulated vsize (KiB) 204724

[startup+51.1009 s]
/proc/loadavg: 6.16 4.70 5.40 9/291 5765
/proc/meminfo: memFree=18722612/32951132 swapFree=67093824/67111528
[pid=5738] ppid=5736 vsize=234676 CPUtime=194.88 cores=1,3,5,7
/proc/5738/stat : 5738 (plingeling) S 5736 5738 5706 0 -1 4202496 93042 1250 0 2 19389 99 0 0 25 0 5 0 653139612 240308224 33584 33554432000 4194304 4417452 140733503590240 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/5738/statm: 58669 33584 180 55 0 56555 0
[pid=5738/tid=5756] ppid=5736 vsize=234676 CPUtime=48.68 cores=1,3,5,7
/proc/5738/task/5756/stat : 5756 (plingeling) R 5736 5738 5706 0 -1 4202560 11555 1250 0 2 4861 7 0 0 25 0 5 0 653139752 240308224 33584 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364741 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5738/tid=5757] ppid=5736 vsize=234676 CPUtime=49.57 cores=1,3,5,7
/proc/5738/task/5757/stat : 5757 (plingeling) R 5736 5738 5706 0 -1 4202560 13219 1250 0 2 4953 4 0 0 25 0 5 0 653139752 240308224 33584 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364420 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=5738/tid=5758] ppid=5736 vsize=234676 CPUtime=49.68 cores=1,3,5,7
/proc/5738/task/5758/stat : 5758 (plingeling) R 5736 5738 5706 0 -1 4202560 8813 1250 0 2 4964 4 0 0 25 0 5 0 653139752 240308224 33584 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=5738/tid=5759] ppid=5736 vsize=234676 CPUtime=45.59 cores=1,3,5,7
/proc/5738/task/5759/stat : 5759 (plingeling) R 5736 5738 5706 0 -1 4202560 4548 1250 0 2 4484 75 0 0 25 0 5 0 653139752 240308224 33584 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 194.88
Current children cumulated vsize (KiB) 234676

[startup+102.311 s]
/proc/loadavg: 7.31 5.27 5.55 9/291 5766
/proc/meminfo: memFree=18677872/32951132 swapFree=67093824/67111528
[pid=5738] ppid=5736 vsize=249940 CPUtime=399.24 cores=1,3,5,7
/proc/5738/stat : 5738 (plingeling) S 5736 5738 5706 0 -1 4202496 107130 1250 0 2 39822 102 0 0 25 0 5 0 653139612 255938560 37400 33554432000 4194304 4417452 140733503590240 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/5738/statm: 62485 37400 180 55 0 60371 0
[pid=5738/tid=5756] ppid=5736 vsize=249940 CPUtime=99.41 cores=1,3,5,7
/proc/5738/task/5756/stat : 5756 (plingeling) R 5736 5738 5706 0 -1 4202560 16192 1250 0 2 9933 8 0 0 25 0 5 0 653139752 255938560 37400 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5738/tid=5757] ppid=5736 vsize=249940 CPUtime=100.78 cores=1,3,5,7
/proc/5738/task/5757/stat : 5757 (plingeling) R 5736 5738 5706 0 -1 4202560 19238 1250 0 2 10073 5 0 0 25 0 5 0 653139752 255938560 37400 33554432000 4194304 4417452 140733503590240 18446744073709551615 4236928 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=5738/tid=5758] ppid=5736 vsize=249940 CPUtime=100.88 cores=1,3,5,7
/proc/5738/task/5758/stat : 5758 (plingeling) R 5736 5738 5706 0 -1 4202560 12245 1250 0 2 10084 4 0 0 25 0 5 0 653139752 255938560 37400 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=5738/tid=5759] ppid=5736 vsize=249940 CPUtime=96.79 cores=1,3,5,7
/proc/5738/task/5759/stat : 5759 (plingeling) R 5736 5738 5706 0 -1 4202560 4548 1250 0 2 9604 75 0 0 25 0 5 0 653139752 255938560 37400 33554432000 4194304 4417452 140733503590240 18446744073709551615 4365058 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 399.24
Current children cumulated vsize (KiB) 249940

[startup+162.301 s]
/proc/loadavg: 7.83 5.79 5.71 9/291 5768
/proc/meminfo: memFree=18585224/32951132 swapFree=67093824/67111528
[pid=5738] ppid=5736 vsize=293820 CPUtime=638.64 cores=1,3,5,7
/proc/5738/stat : 5738 (plingeling) S 5736 5738 5706 0 -1 4202496 124273 1250 0 2 63760 104 0 0 25 0 5 0 653139612 300871680 48370 33554432000 4194304 4417452 140733503590240 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/5738/statm: 73455 48370 180 55 0 71341 0
[pid=5738/tid=5756] ppid=5736 vsize=293820 CPUtime=158.86 cores=1,3,5,7
/proc/5738/task/5756/stat : 5756 (plingeling) R 5736 5738 5706 0 -1 4202560 24466 1250 0 2 15877 9 0 0 25 0 5 0 653139752 300871680 48370 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5738/tid=5757] ppid=5736 vsize=293820 CPUtime=160.77 cores=1,3,5,7
/proc/5738/task/5757/stat : 5757 (plingeling) R 5736 5738 5706 0 -1 4202560 24007 1250 0 2 16071 6 0 0 25 0 5 0 653139752 300871680 48370 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=5738/tid=5758] ppid=5736 vsize=293820 CPUtime=160.87 cores=1,3,5,7
/proc/5738/task/5758/stat : 5758 (plingeling) R 5736 5738 5706 0 -1 4202560 16345 1250 0 2 16082 5 0 0 25 0 5 0 653139752 300871680 48370 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=5738/tid=5759] ppid=5736 vsize=293820 CPUtime=156.78 cores=1,3,5,7
/proc/5738/task/5759/stat : 5759 (plingeling) R 5736 5738 5706 0 -1 4202560 4548 1250 0 2 15603 75 0 0 25 0 5 0 653139752 300871680 48370 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 638.64
Current children cumulated vsize (KiB) 293820

[startup+222.3 s]
/proc/loadavg: 7.94 6.19 5.85 9/291 5769
/proc/meminfo: memFree=18564884/32951132 swapFree=67093824/67111528
[pid=5738] ppid=5736 vsize=301172 CPUtime=878.09 cores=1,3,5,7
/proc/5738/stat : 5738 (plingeling) S 5736 5738 5706 0 -1 4202496 138411 1250 0 2 87704 105 0 0 25 0 5 0 653139612 308400128 50208 33554432000 4194304 4417452 140733503590240 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/5738/statm: 75293 50208 180 55 0 73179 0
[pid=5738/tid=5756] ppid=5736 vsize=301172 CPUtime=218.32 cores=1,3,5,7
/proc/5738/task/5756/stat : 5756 (plingeling) R 5736 5738 5706 0 -1 4202560 27773 1250 0 2 21822 10 0 0 25 0 5 0 653139752 308400128 50208 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 7 0 0 0

################
# More data... #
################

[pid=5738/tid=5757] ppid=5736 vsize=372556 CPUtime=700.76 cores=1,3,5,7
/proc/5738/task/5757/stat : 5757 (plingeling) R 5736 5738 5706 0 -1 4202560 112572 1250 0 2 70058 18 0 0 25 0 5 0 653139752 381497344 68085 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=5738/tid=5758] ppid=5736 vsize=372556 CPUtime=700.86 cores=1,3,5,7
/proc/5738/task/5758/stat : 5758 (plingeling) R 5736 5738 5706 0 -1 4202560 65941 1250 0 2 70075 11 0 0 25 0 5 0 653139752 381497344 68085 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=5738/tid=5759] ppid=5736 vsize=372556 CPUtime=696.79 cores=1,3,5,7
/proc/5738/task/5759/stat : 5759 (plingeling) R 5736 5738 5706 0 -1 4202560 14258 1250 0 2 69602 77 0 0 25 0 5 0 653139752 381497344 68085 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 2793.75
Current children cumulated vsize (KiB) 372556

[startup+762.311 s]
/proc/loadavg: 8.09 7.77 6.82 9/291 5784
/proc/meminfo: memFree=18400388/32951132 swapFree=67093824/67111528
[pid=5738] ppid=5736 vsize=389484 CPUtime=3033.24 cores=1,3,5,7
/proc/5738/stat : 5738 (plingeling) S 5736 5738 5706 0 -1 4202496 377226 1250 0 2 303184 140 0 0 25 0 5 0 653139612 398831616 72317 33554432000 4194304 4417452 140733503590240 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/5738/statm: 97371 72317 180 55 0 95257 0
[pid=5738/tid=5756] ppid=5736 vsize=389484 CPUtime=753.43 cores=1,3,5,7
/proc/5738/task/5756/stat : 5756 (plingeling) R 5736 5738 5706 0 -1 4202560 97560 1250 0 2 75323 20 0 0 25 0 5 0 653139752 398831616 72317 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5738/tid=5757] ppid=5736 vsize=389484 CPUtime=760.78 cores=1,3,5,7
/proc/5738/task/5757/stat : 5757 (plingeling) R 5736 5738 5706 0 -1 4202560 125461 1250 0 2 76057 21 0 0 25 0 5 0 653139752 398831616 72317 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=5738/tid=5758] ppid=5736 vsize=389484 CPUtime=760.88 cores=1,3,5,7
/proc/5738/task/5758/stat : 5758 (plingeling) R 5736 5738 5706 0 -1 4202560 78085 1250 0 2 76075 13 0 0 25 0 5 0 653139752 398831616 72317 33554432000 4194304 4417452 140733503590240 18446744073709551615 4365198 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=5738/tid=5759] ppid=5736 vsize=389484 CPUtime=756.8 cores=1,3,5,7
/proc/5738/task/5759/stat : 5759 (plingeling) R 5736 5738 5706 0 -1 4202560 21213 1250 0 2 75602 78 0 0 25 0 5 0 653139752 398831616 72317 33554432000 4194304 4417452 140733503590240 18446744073709551615 4366491 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3033.24
Current children cumulated vsize (KiB) 389484

[startup+822.3 s]
/proc/loadavg: 8.03 7.81 6.89 9/291 5785
/proc/meminfo: memFree=18420344/32951132 swapFree=67093824/67111528
[pid=5738] ppid=5736 vsize=389824 CPUtime=3272.66 cores=1,3,5,7
/proc/5738/stat : 5738 (plingeling) S 5736 5738 5706 0 -1 4202496 414621 1250 0 2 327120 146 0 0 25 0 5 0 653139612 399179776 72402 33554432000 4194304 4417452 140733503590240 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/5738/statm: 97456 72402 180 55 0 95342 0
[pid=5738/tid=5756] ppid=5736 vsize=389824 CPUtime=812.88 cores=1,3,5,7
/proc/5738/task/5756/stat : 5756 (plingeling) R 5736 5738 5706 0 -1 4202560 109022 1250 0 2 81267 21 0 0 25 0 5 0 653139752 399179776 72402 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5738/tid=5757] ppid=5736 vsize=389824 CPUtime=820.76 cores=1,3,5,7
/proc/5738/task/5757/stat : 5757 (plingeling) R 5736 5738 5706 0 -1 4202560 135340 1250 0 2 82054 22 0 0 25 0 5 0 653139752 399179776 72402 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=5738/tid=5758] ppid=5736 vsize=389824 CPUtime=820.87 cores=1,3,5,7
/proc/5738/task/5758/stat : 5758 (plingeling) R 5736 5738 5706 0 -1 4202560 89771 1250 0 2 82072 15 0 0 25 0 5 0 653139752 399179776 72402 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=5738/tid=5759] ppid=5736 vsize=389824 CPUtime=816.79 cores=1,3,5,7
/proc/5738/task/5759/stat : 5759 (plingeling) R 5736 5738 5706 0 -1 4202560 25581 1250 0 2 81601 78 0 0 25 0 5 0 653139752 399179776 72402 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3272.66
Current children cumulated vsize (KiB) 389824

[startup+882.301 s]
/proc/loadavg: 8.04 7.86 6.96 9/291 5787
/proc/meminfo: memFree=18420832/32951132 swapFree=67093824/67111528
[pid=5738] ppid=5736 vsize=391676 CPUtime=3512.12 cores=1,3,5,7
/proc/5738/stat : 5738 (plingeling) S 5736 5738 5706 0 -1 4202496 449789 1250 0 2 351061 151 0 0 25 0 5 0 653139612 401076224 72865 33554432000 4194304 4417452 140733503590240 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/5738/statm: 97919 72865 180 55 0 95805 0
[pid=5738/tid=5756] ppid=5736 vsize=391676 CPUtime=872.33 cores=1,3,5,7
/proc/5738/task/5756/stat : 5756 (plingeling) R 5736 5738 5706 0 -1 4202560 114482 1250 0 2 87211 22 0 0 25 0 5 0 653139752 401076224 72865 33554432000 4194304 4417452 140733503590240 18446744073709551615 4355860 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5738/tid=5757] ppid=5736 vsize=391676 CPUtime=880.77 cores=1,3,5,7
/proc/5738/task/5757/stat : 5757 (plingeling) R 5736 5738 5706 0 -1 4202560 146379 1250 0 2 88053 24 0 0 25 0 5 0 653139752 401076224 72865 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=5738/tid=5758] ppid=5736 vsize=391676 CPUtime=880.87 cores=1,3,5,7
/proc/5738/task/5758/stat : 5758 (plingeling) R 5736 5738 5706 0 -1 4202560 96558 1250 0 2 88071 16 0 0 25 0 5 0 653139752 401076224 72865 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=5738/tid=5759] ppid=5736 vsize=391676 CPUtime=876.79 cores=1,3,5,7
/proc/5738/task/5759/stat : 5759 (plingeling) R 5736 5738 5706 0 -1 4202560 37463 1250 0 2 87599 80 0 0 25 0 5 0 653139752 401076224 72865 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3512.12
Current children cumulated vsize (KiB) 391676

[startup+942.301 s]
/proc/loadavg: 8.13 7.91 7.04 9/291 5788
/proc/meminfo: memFree=18372096/32951132 swapFree=67093824/67111528
[pid=5738] ppid=5736 vsize=408280 CPUtime=3751.58 cores=1,3,5,7
/proc/5738/stat : 5738 (plingeling) S 5736 5738 5706 0 -1 4202496 483228 1250 0 2 375002 156 0 0 25 0 5 0 653139612 418078720 77016 33554432000 4194304 4417452 140733503590240 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/5738/statm: 102070 77016 180 55 0 99956 0
[pid=5738/tid=5756] ppid=5736 vsize=408280 CPUtime=931.8 cores=1,3,5,7
/proc/5738/task/5756/stat : 5756 (plingeling) R 5736 5738 5706 0 -1 4202560 124711 1250 0 2 93156 24 0 0 25 0 5 0 653139752 418078720 77016 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5738/tid=5757] ppid=5736 vsize=408280 CPUtime=940.77 cores=1,3,5,7
/proc/5738/task/5757/stat : 5757 (plingeling) R 5736 5738 5706 0 -1 4202560 152441 1250 0 2 94052 25 0 0 25 0 5 0 653139752 418078720 77016 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=5738/tid=5758] ppid=5736 vsize=408280 CPUtime=940.86 cores=1,3,5,7
/proc/5738/task/5758/stat : 5758 (plingeling) R 5736 5738 5706 0 -1 4202560 109916 1250 0 2 94069 17 0 0 25 0 5 0 653139752 418078720 77016 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364772 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=5738/tid=5759] ppid=5736 vsize=408280 CPUtime=936.8 cores=1,3,5,7
/proc/5738/task/5759/stat : 5759 (plingeling) R 5736 5738 5706 0 -1 4202560 41253 1250 0 2 93599 81 0 0 25 0 5 0 653139752 418078720 77016 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3751.58
Current children cumulated vsize (KiB) 408280

[startup+1002.3 s]
/proc/loadavg: 8.12 7.94 7.10 9/291 5790
/proc/meminfo: memFree=18336608/32951132 swapFree=67093824/67111528
[pid=5738] ppid=5736 vsize=427160 CPUtime=3991.03 cores=1,3,5,7
/proc/5738/stat : 5738 (plingeling) S 5736 5738 5706 0 -1 4202496 530061 1250 0 2 398940 163 0 0 25 0 5 0 653139612 437411840 81705 33554432000 4194304 4417452 140733503590240 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/5738/statm: 106790 81705 180 55 0 104676 0
[pid=5738/tid=5756] ppid=5736 vsize=427160 CPUtime=991.25 cores=1,3,5,7
/proc/5738/task/5756/stat : 5756 (plingeling) R 5736 5738 5706 0 -1 4202560 135268 1250 0 2 99099 26 0 0 25 0 5 0 653139752 437411840 81705 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5738/tid=5757] ppid=5736 vsize=427160 CPUtime=1000.77 cores=1,3,5,7
/proc/5738/task/5757/stat : 5757 (plingeling) R 5736 5738 5706 0 -1 4202560 171442 1250 0 2 100049 28 0 0 25 0 5 0 653139752 437411840 81705 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364675 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=5738/tid=5758] ppid=5736 vsize=427160 CPUtime=1000.86 cores=1,3,5,7
/proc/5738/task/5758/stat : 5758 (plingeling) R 5736 5738 5706 0 -1 4202560 122241 1250 0 2 100067 19 0 0 25 0 5 0 653139752 437411840 81705 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=5738/tid=5759] ppid=5736 vsize=427160 CPUtime=996.8 cores=1,3,5,7
/proc/5738/task/5759/stat : 5759 (plingeling) R 5736 5738 5706 0 -1 4202560 46203 1250 0 2 99598 82 0 0 25 0 5 0 653139752 437411840 81705 33554432000 4194304 4417452 140733503590240 18446744073709551615 4365088 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3991.03
Current children cumulated vsize (KiB) 427160

[startup+1062.3 s]
/proc/loadavg: 8.11 7.96 7.16 9/291 5792
/proc/meminfo: memFree=18315776/32951132 swapFree=67093824/67111528
[pid=5738] ppid=5736 vsize=424184 CPUtime=4230.5 cores=1,3,5,7
/proc/5738/stat : 5738 (plingeling) S 5736 5738 5706 0 -1 4202496 572985 1250 0 2 422881 169 0 0 25 0 5 0 653139612 434364416 80961 33554432000 4194304 4417452 140733503590240 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/5738/statm: 106046 80961 180 55 0 103932 0
[pid=5738/tid=5756] ppid=5736 vsize=424184 CPUtime=1050.71 cores=1,3,5,7
/proc/5738/task/5756/stat : 5756 (plingeling) R 5736 5738 5706 0 -1 4202560 151512 1250 0 2 105043 28 0 0 25 0 5 0 653139752 434364416 80961 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5738/tid=5757] ppid=5736 vsize=424184 CPUtime=1060.76 cores=1,3,5,7
/proc/5738/task/5757/stat : 5757 (plingeling) R 5736 5738 5706 0 -1 4202560 184723 1250 0 2 106047 29 0 0 25 0 5 0 653139752 434364416 80961 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=5738/tid=5758] ppid=5736 vsize=424184 CPUtime=1060.87 cores=1,3,5,7
/proc/5738/task/5758/stat : 5758 (plingeling) R 5736 5738 5706 0 -1 4202560 130896 1250 0 2 106066 21 0 0 25 0 5 0 653139752 434364416 80961 33554432000 4194304 4417452 140733503590240 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=5738/tid=5759] ppid=5736 vsize=424184 CPUtime=1056.79 cores=1,3,5,7
/proc/5738/task/5759/stat : 5759 (plingeling) R 5736 5738 5706 0 -1 4202560 50947 1250 0 2 105597 82 0 0 25 0 5 0 653139752 434364416 80961 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4230.5
Current children cumulated vsize (KiB) 424184

[startup+1122.3 s]
/proc/loadavg: 8.04 7.97 7.21 9/291 5793
/proc/meminfo: memFree=18385948/32951132 swapFree=67093824/67111528
[pid=5738] ppid=5736 vsize=411884 CPUtime=4469.94 cores=1,3,5,7
/proc/5738/stat : 5738 (plingeling) S 5736 5738 5706 0 -1 4202496 608093 1250 0 2 446820 174 0 0 25 0 5 0 653139612 421769216 77886 33554432000 4194304 4417452 140733503590240 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/5738/statm: 102971 77886 180 55 0 100857 0
[pid=5738/tid=5756] ppid=5736 vsize=411884 CPUtime=1110.16 cores=1,3,5,7
/proc/5738/task/5756/stat : 5756 (plingeling) R 5736 5738 5706 0 -1 4202560 160238 1250 0 2 110987 29 0 0 25 0 5 0 653139752 421769216 77886 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364807 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5738/tid=5757] ppid=5736 vsize=411884 CPUtime=1120.77 cores=1,3,5,7
/proc/5738/task/5757/stat : 5757 (plingeling) R 5736 5738 5706 0 -1 4202560 197314 1250 0 2 112046 31 0 0 25 0 5 0 653139752 421769216 77886 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=5738/tid=5758] ppid=5736 vsize=411884 CPUtime=1120.87 cores=1,3,5,7
/proc/5738/task/5758/stat : 5758 (plingeling) R 5736 5738 5706 0 -1 4202560 140730 1250 0 2 112064 23 0 0 25 0 5 0 653139752 421769216 77886 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=5738/tid=5759] ppid=5736 vsize=411884 CPUtime=1116.8 cores=1,3,5,7
/proc/5738/task/5759/stat : 5759 (plingeling) R 5736 5738 5706 0 -1 4202560 54904 1250 0 2 111597 83 0 0 25 0 5 0 653139752 421769216 77886 33554432000 4194304 4417452 140733503590240 18446744073709551615 4281633 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4469.94
Current children cumulated vsize (KiB) 411884

[startup+1182.3 s]
/proc/loadavg: 8.05 7.99 7.26 9/291 5795
/proc/meminfo: memFree=18308200/32951132 swapFree=67093824/67111528
[pid=5738] ppid=5736 vsize=428120 CPUtime=4709.41 cores=1,3,5,7
/proc/5738/stat : 5738 (plingeling) S 5736 5738 5706 0 -1 4202496 643749 1250 0 2 470762 179 0 0 25 0 5 0 653139612 438394880 81945 33554432000 4194304 4417452 140733503590240 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/5738/statm: 107030 81945 180 55 0 104916 0
[pid=5738/tid=5756] ppid=5736 vsize=428120 CPUtime=1169.62 cores=1,3,5,7
/proc/5738/task/5756/stat : 5756 (plingeling) R 5736 5738 5706 0 -1 4202560 171959 1250 0 2 116931 31 0 0 25 0 5 0 653139752 438394880 81945 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364765 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5738/tid=5757] ppid=5736 vsize=428120 CPUtime=1180.77 cores=1,3,5,7
/proc/5738/task/5757/stat : 5757 (plingeling) R 5736 5738 5706 0 -1 4202560 207188 1250 0 2 118045 32 0 0 25 0 5 0 653139752 438394880 81945 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=5738/tid=5758] ppid=5736 vsize=428120 CPUtime=1180.86 cores=1,3,5,7
/proc/5738/task/5758/stat : 5758 (plingeling) R 5736 5738 5706 0 -1 4202560 151342 1250 0 2 118062 24 0 0 25 0 5 0 653139752 438394880 81945 33554432000 4194304 4417452 140733503590240 18446744073709551615 4346484 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=5738/tid=5759] ppid=5736 vsize=428120 CPUtime=1176.8 cores=1,3,5,7
/proc/5738/task/5759/stat : 5759 (plingeling) R 5736 5738 5706 0 -1 4202560 58353 1250 0 2 117597 83 0 0 25 0 5 0 653139752 438394880 81945 33554432000 4194304 4417452 140733503590240 18446744073709551615 4236995 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4709.41
Current children cumulated vsize (KiB) 428120



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1205 s]
/proc/loadavg: 8.10 8.00 7.28 9/291 5796
/proc/meminfo: memFree=18317748/32951132 swapFree=67093824/67111528
[pid=5738] ppid=5736 vsize=435544 CPUtime=4800.01 cores=1,3,5,7
/proc/5738/stat : 5738 (plingeling) S 5736 5738 5706 0 -1 4202496 665295 1250 0 2 479818 183 0 0 25 0 5 0 653139612 445997056 83801 33554432000 4194304 4417452 140733503590240 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/5738/statm: 108886 83801 180 55 0 106772 0
[pid=5738/tid=5756] ppid=5736 vsize=435544 CPUtime=1192.11 cores=1,3,5,7
/proc/5738/task/5756/stat : 5756 (plingeling) R 5736 5738 5706 0 -1 4202560 177558 1250 0 2 119179 32 0 0 25 0 5 0 653139752 445997056 83801 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5738/tid=5757] ppid=5736 vsize=435544 CPUtime=1203.47 cores=1,3,5,7
/proc/5738/task/5757/stat : 5757 (plingeling) R 5736 5738 5706 0 -1 4202560 215967 1250 0 2 120314 33 0 0 25 0 5 0 653139752 445997056 83801 33554432000 4194304 4417452 140733503590240 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=5738/tid=5758] ppid=5736 vsize=435544 CPUtime=1203.57 cores=1,3,5,7
/proc/5738/task/5758/stat : 5758 (plingeling) R 5736 5738 5706 0 -1 4202560 156800 1250 0 2 120332 25 0 0 25 0 5 0 653139752 445997056 83801 33554432000 4194304 4417452 140733503590240 18446744073709551615 4364572 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=5738/tid=5759] ppid=5736 vsize=435544 CPUtime=1199.51 cores=1,3,5,7
/proc/5738/task/5759/stat : 5759 (plingeling) R 5736 5738 5706 0 -1 4202560 60063 1250 0 2 119867 84 0 0 25 0 5 0 653139752 445997056 83801 33554432000 4194304 4417452 140733503590240 18446744073709551615 4355819 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4800.01
Current children cumulated vsize (KiB) 435544

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1205.03
CPU time (s): 4800.08
CPU user time (s): 4798.22
CPU system time (s): 1.85072
CPU usage (%): 398.336
Max. virtual memory (cumulated for all children) (KiB): 452424

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4798.22
system time used= 1.85072
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 666545
page faults= 2
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 129
involuntary context switches= 36703

runsolver used 3.06153 second user time and 7.9088 second system time

The end

Launcher Data 

Begin job on node131 at 2011-05-15 01:23:20
IDJOB=3371274
IDBENCH=20384
IDSOLVER=1848
FILE ID=node131/3371274-1305415399
RUNJOBID= node131-1305415399-5725
PBS_JOBID= 13324564
Free space on /tmp= 73652 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT07/industrial/anbulagan/hard-unsat/dated-10-17-u.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3371274-1305415399/watcher-3371274-1305415399 -o /tmp/evaluation-result-3371274-1305415399/solver-3371274-1305415399 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3371274-1305415399.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 780d946576ec5746b4ea9832bb01dbc5
RANDOM SEED=301721715

node131.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951132 kB
MemFree:      19021000 kB
Buffers:        276064 kB
Cached:        3673488 kB
SwapCached:       1504 kB
Active:       10585028 kB
Inactive:      3085484 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951132 kB
LowFree:      19021000 kB
SwapTotal:    67111528 kB
SwapFree:     67093824 kB
Dirty:           39508 kB
Writeback:           0 kB
AnonPages:     9719792 kB
Mapped:          26428 kB
Slab:           165008 kB
PageTables:      30776 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587092 kB
Committed_AS: 27997548 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296000 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73644 MiB
End job on node131 at 2011-05-15 01:43:27