Trace number 3395030

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40001 5013.04

General information on the benchmark

Namecrafted/anton/SRHD-SGI/
srhd-sgi-m42-q663-n40-p15-s72490337.cnf
MD5SUMab8e40d31c79e7a71114e3126a46874f
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark0.165974
Satisfiable
(Un)Satisfiability was proved
Number of variables1649
Number of clauses256038
Sum of the clauses size513645
Maximum clause length42
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2255998
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 540

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3395030-1306111438/watcher-3395030-1306111438 -o /tmp/evaluation-result-3395030-1306111438/solver-3395030-1306111438 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3395030-1306111438.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.47 7.99 8.02 1/551 544
/proc/meminfo: memFree=25538984/32951124 swapFree=39849776/67111528
[pid=544] ppid=542 vsize=8728 CPUtime=0 cores=0-7
/proc/544/stat : 544 (plingeling) S 542 544 32736 0 -1 4202496 215 0 0 0 0 0 0 0 22 0 1 0 801374499 8937472 146 33554432000 4194304 4417452 140736665663632 18446744073709551615 224284925696 0 0 4096 0 18446744071562362840 0 0 17 5 0 0 0
/proc/544/statm: 2182 146 122 55 0 68 0

[startup+0.098443 s]
/proc/loadavg: 7.47 7.99 8.02 1/551 544
/proc/meminfo: memFree=25538984/32951124 swapFree=39849776/67111528
[pid=544] ppid=542 vsize=26536 CPUtime=0.09 cores=0-7
/proc/544/stat : 544 (plingeling) R 542 544 32736 0 -1 4202496 4856 1262 0 0 8 1 0 0 22 0 1 0 801374499 27172864 4657 33554432000 4194304 4417452 140736665663632 18446744073709551615 4262065 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/544/statm: 6634 4657 148 55 0 4520 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 26536

[startup+0.100437 s]
/proc/loadavg: 7.47 7.99 8.02 1/551 544
/proc/meminfo: memFree=25538984/32951124 swapFree=39849776/67111528
[pid=544] ppid=542 vsize=26536 CPUtime=0.09 cores=0-7
/proc/544/stat : 544 (plingeling) R 542 544 32736 0 -1 4202496 4856 1262 0 0 8 1 0 0 22 0 1 0 801374499 27172864 4657 33554432000 4194304 4417452 140736665663632 18446744073709551615 4231619 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/544/statm: 6634 4657 148 55 0 4520 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 26536

[startup+0.30041 s]
/proc/loadavg: 7.47 7.99 8.02 1/551 544
/proc/meminfo: memFree=25538984/32951124 swapFree=39849776/67111528
[pid=544] ppid=542 vsize=75688 CPUtime=0.28 cores=0-7
/proc/544/stat : 544 (plingeling) R 542 544 32736 0 -1 4202496 17144 1262 0 0 26 2 0 0 23 0 1 0 801374499 77504512 16945 33554432000 4194304 4417452 140736665663632 18446744073709551615 4231291 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/544/statm: 18922 16945 148 55 0 16808 0
Current children cumulated CPU time (s) 0.28
Current children cumulated vsize (KiB) 75688

[startup+0.700305 s]
/proc/loadavg: 7.47 7.99 8.02 1/551 544
/proc/meminfo: memFree=25538984/32951124 swapFree=39849776/67111528
[pid=544] ppid=542 vsize=229360 CPUtime=3.29 cores=0-7
/proc/544/stat : 544 (plingeling) S 542 544 32736 0 -1 4202496 34626 1262 0 0 323 6 0 0 18 0 9 0 801374499 234864640 18539 33554432000 4194304 4417452 140736665663632 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/544/statm: 57340 18539 179 55 0 55226 0
Current children cumulated CPU time (s) 3.29
Current children cumulated vsize (KiB) 229360

[startup+1.50117 s]
/proc/loadavg: 7.47 7.99 8.02 9/560 558
/proc/meminfo: memFree=25445920/32951124 swapFree=39849776/67111528
[pid=544] ppid=542 vsize=255596 CPUtime=9.68 cores=0-7
/proc/544/stat : 544 (plingeling) S 542 544 32736 0 -1 4202496 42525 1262 0 0 960 8 0 0 18 0 9 0 801374499 261730304 24872 33554432000 4194304 4417452 140736665663632 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/544/statm: 63899 24872 180 55 0 61785 0
[pid=544/tid=551] ppid=542 vsize=255596 CPUtime=1.16 cores=0-7
/proc/544/task/551/stat : 551 (plingeling) R 542 544 32736 0 -1 4202560 2349 1262 0 0 116 0 0 0 25 0 9 0 801374531 261730304 24872 33554432000 4194304 4417452 140736665663632 18446744073709551615 4249584 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=544/tid=552] ppid=542 vsize=255596 CPUtime=1.17 cores=0-7
/proc/544/task/552/stat : 552 (plingeling) R 542 544 32736 0 -1 4202560 2910 1262 0 0 117 0 0 0 25 0 9 0 801374531 261730304 24872 33554432000 4194304 4417452 140736665663632 18446744073709551615 4249305 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=544/tid=553] ppid=542 vsize=255596 CPUtime=1.16 cores=0-7
/proc/544/task/553/stat : 553 (plingeling) R 542 544 32736 0 -1 4202560 1691 1262 0 0 116 0 0 0 25 0 9 0 801374531 261730304 24872 33554432000 4194304 4417452 140736665663632 18446744073709551615 4249597 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=544/tid=554] ppid=542 vsize=255596 CPUtime=1.16 cores=0-7
/proc/544/task/554/stat : 554 (plingeling) R 542 544 32736 0 -1 4202560 6285 1262 0 0 115 1 0 0 22 0 9 0 801374531 261730304 24872 33554432000 4194304 4417452 140736665663632 18446744073709551615 4356796 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=544/tid=555] ppid=542 vsize=255596 CPUtime=1.17 cores=0-7
/proc/544/task/555/stat : 555 (plingeling) R 542 544 32736 0 -1 4202560 3292 1262 0 0 117 0 0 0 19 0 9 0 801374531 261730304 24872 33554432000 4194304 4417452 140736665663632 18446744073709551615 4249209 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=544/tid=556] ppid=542 vsize=255596 CPUtime=1.16 cores=0-7
/proc/544/task/556/stat : 556 (plingeling) R 542 544 32736 0 -1 4202560 2582 1262 0 0 116 0 0 0 19 0 9 0 801374531 261730304 24872 33554432000 4194304 4417452 140736665663632 18446744073709551615 4249171 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=544/tid=557] ppid=542 vsize=255596 CPUtime=1.14 cores=0-7
/proc/544/task/557/stat : 557 (plingeling) R 542 544 32736 0 -1 4202560 3409 1262 0 0 114 0 0 0 22 0 9 0 801374531 261730304 24872 33554432000 4194304 4417452 140736665663632 18446744073709551615 4376521 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=544/tid=558] ppid=542 vsize=255596 CPUtime=1.16 cores=0-7
/proc/544/task/558/stat : 558 (plingeling) R 542 544 32736 0 -1 4202560 2847 1262 0 0 116 0 0 0 19 0 9 0 801374531 261730304 24872 33554432000 4194304 4417452 140736665663632 18446744073709551615 4249991 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 9.68
Current children cumulated vsize (KiB) 255596

[startup+3.1009 s]
/proc/loadavg: 7.47 7.99 8.02 10/560 558
/proc/meminfo: memFree=25436000/32951124 swapFree=39849776/67111528
[pid=544] ppid=542 vsize=266000 CPUtime=22.46 cores=0-7
/proc/544/stat : 544 (plingeling) S 542 544 32736 0 -1 4202496 50375 1262 0 0 2236 10 0 0 18 0 9 0 801374499 272384000 27700 33554432000 4194304 4417452 140736665663632 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/544/statm: 66500 27700 180 55 0 64386 0
[pid=544/tid=551] ppid=542 vsize=266000 CPUtime=2.76 cores=0-7
/proc/544/task/551/stat : 551 (plingeling) R 542 544 32736 0 -1 4202560 2349 1262 0 0 276 0 0 0 25 0 9 0 801374531 272384000 27700 33554432000 4194304 4417452 140736665663632 18446744073709551615 4236296 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=544/tid=552] ppid=542 vsize=266000 CPUtime=2.77 cores=0-7
/proc/544/task/552/stat : 552 (plingeling) R 542 544 32736 0 -1 4202560 2910 1262 0 0 277 0 0 0 25 0 9 0 801374531 272384000 27700 33554432000 4194304 4417452 140736665663632 18446744073709551615 4249143 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=544/tid=553] ppid=542 vsize=266000 CPUtime=2.76 cores=0-7
/proc/544/task/553/stat : 553 (plingeling) R 542 544 32736 0 -1 4202560 1691 1262 0 0 276 0 0 0 25 0 9 0 801374531 272384000 27700 33554432000 4194304 4417452 140736665663632 18446744073709551615 4376559 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=544/tid=554] ppid=542 vsize=266000 CPUtime=2.76 cores=0-7
/proc/544/task/554/stat : 554 (plingeling) R 542 544 32736 0 -1 4202560 14135 1262 0 0 273 3 0 0 25 0 9 0 801374531 272384000 27700 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=544/tid=555] ppid=542 vsize=266000 CPUtime=2.77 cores=0-7
/proc/544/task/555/stat : 555 (plingeling) R 542 544 32736 0 -1 4202560 3292 1262 0 0 277 0 0 0 23 0 9 0 801374531 272384000 27700 33554432000 4194304 4417452 140736665663632 18446744073709551615 4250080 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=544/tid=556] ppid=542 vsize=266000 CPUtime=2.76 cores=0-7
/proc/544/task/556/stat : 556 (plingeling) R 542 544 32736 0 -1 4202560 2582 1262 0 0 276 0 0 0 22 0 9 0 801374531 272384000 27700 33554432000 4194304 4417452 140736665663632 18446744073709551615 4235378 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=544/tid=557] ppid=542 vsize=266000 CPUtime=2.72 cores=0-7
/proc/544/task/557/stat : 557 (plingeling) R 542 544 32736 0 -1 4202560 3409 1262 0 0 272 0 0 0 25 0 9 0 801374531 272384000 27700 33554432000 4194304 4417452 140736665663632 18446744073709551615 4249209 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=544/tid=558] ppid=542 vsize=266000 CPUtime=2.76 cores=0-7
/proc/544/task/558/stat : 558 (plingeling) R 542 544 32736 0 -1 4202560 2847 1262 0 0 276 0 0 0 22 0 9 0 801374531 272384000 27700 33554432000 4194304 4417452 140736665663632 18446744073709551615 4249584 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 22.46
Current children cumulated vsize (KiB) 266000

[startup+6.30037 s]
/proc/loadavg: 7.51 7.99 8.02 9/560 558
/proc/meminfo: memFree=25408196/32951124 swapFree=39849776/67111528
[pid=544] ppid=542 vsize=298604 CPUtime=48 cores=0-7
/proc/544/stat : 544 (plingeling) S 542 544 32736 0 -1 4202496 64296 1262 0 0 4787 13 0 0 18 0 9 0 801374499 305770496 35851 33554432000 4194304 4417452 140736665663632 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/544/statm: 74651 35851 180 55 0 72537 0
[pid=544/tid=551] ppid=542 vsize=298604 CPUtime=5.96 cores=0-7
/proc/544/task/551/stat : 551 (plingeling) R 542 544 32736 0 -1 4202560 2349 1262 0 0 596 0 0 0 25 0 9 0 801374531 305770496 35851 33554432000 4194304 4417452 140736665663632 18446744073709551615 4250082 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=544/tid=552] ppid=542 vsize=298604 CPUtime=5.97 cores=0-7
/proc/544/task/552/stat : 552 (plingeling) R 542 544 32736 0 -1 4202560 2910 1262 0 0 597 0 0 0 25 0 9 0 801374531 305770496 35851 33554432000 4194304 4417452 140736665663632 18446744073709551615 4249193 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=544/tid=553] ppid=542 vsize=298604 CPUtime=5.96 cores=0-7
/proc/544/task/553/stat : 553 (plingeling) R 542 544 32736 0 -1 4202560 3912 1262 0 0 596 0 0 0 25 0 9 0 801374531 305770496 35851 33554432000 4194304 4417452 140736665663632 18446744073709551615 4236380 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=544/tid=554] ppid=542 vsize=298604 CPUtime=5.96 cores=0-7
/proc/544/task/554/stat : 554 (plingeling) R 542 544 32736 0 -1 4202560 25145 1262 0 0 590 6 0 0 25 0 9 0 801374531 305770496 35851 33554432000 4194304 4417452 140736665663632 18446744073709551615 4251081 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=544/tid=555] ppid=542 vsize=298604 CPUtime=5.97 cores=0-7
/proc/544/task/555/stat : 555 (plingeling) R 542 544 32736 0 -1 4202560 3292 1262 0 0 597 0 0 0 25 0 9 0 801374531 305770496 35851 33554432000 4194304 4417452 140736665663632 18446744073709551615 4249859 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=544/tid=556] ppid=542 vsize=298604 CPUtime=5.96 cores=0-7
/proc/544/task/556/stat : 556 (plingeling) R 542 544 32736 0 -1 4202560 2582 1262 0 0 596 0 0 0 25 0 9 0 801374531 305770496 35851 33554432000 4194304 4417452 140736665663632 18446744073709551615 4236267 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=544/tid=557] ppid=542 vsize=298604 CPUtime=5.86 cores=0-7
/proc/544/task/557/stat : 557 (plingeling) R 542 544 32736 0 -1 4202560 4099 1262 0 0 586 0 0 0 25 0 9 0 801374531 305770496 35851 33554432000 4194304 4417452 140736665663632 18446744073709551615 4254862 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=544/tid=558] ppid=542 vsize=298604 CPUtime=5.96 cores=0-7
/proc/544/task/558/stat : 558 (plingeling) R 542 544 32736 0 -1 4202560 2847 1262 0 0 596 0 0 0 25 0 9 0 801374531 305770496 35851 33554432000 4194304 4417452 140736665663632 18446744073709551615 4250003 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 48
Current children cumulated vsize (KiB) 298604

[startup+12.7013 s]
/proc/loadavg: 7.55 7.99 8.02 9/560 558
/proc/meminfo: memFree=25326324/32951124 swapFree=39849776/67111528
[pid=544] ppid=542 vsize=368384 CPUtime=99.08 cores=0-7
/proc/544/stat : 544 (plingeling) S 542 544 32736 0 -1 4202496 95767 1262 0 0 9888 20 0 0 18 0 9 0 801374499 377225216 53296 33554432000 4194304 4417452 140736665663632 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/544/statm: 92096 53296 180 55 0 89982 0
[pid=544/tid=551] ppid=542 vsize=368384 CPUtime=12.37 cores=0-7
/proc/544/task/551/stat : 551 (plingeling) R 542 544 32736 0 -1 4202560 2349 1262 0 0 1237 0 0 0 25 0 9 0 801374531 377225216 53296 33554432000 4194304 4417452 140736665663632 18446744073709551615 4258575 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=544/tid=552] ppid=542 vsize=368384 CPUtime=12.37 cores=0-7
/proc/544/task/552/stat : 552 (plingeling) R 542 544 32736 0 -1 4202560 2910 1262 0 0 1237 0 0 0 25 0 9 0 801374531 377225216 53296 33554432000 4194304 4417452 140736665663632 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=544/tid=553] ppid=542 vsize=368384 CPUtime=12.36 cores=0-7
/proc/544/task/553/stat : 553 (plingeling) R 542 544 32736 0 -1 4202560 5395 1262 0 0 1236 0 0 0 25 0 9 0 801374531 377225216 53296 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=544/tid=554] ppid=542 vsize=368384 CPUtime=12.36 cores=0-7
/proc/544/task/554/stat : 554 (plingeling) R 542 544 32736 0 -1 4202560 55133 1262 0 0 1223 13 0 0 25 0 9 0 801374531 377225216 53296 33554432000 4194304 4417452 140736665663632 18446744073709551615 4353561 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=544/tid=555] ppid=542 vsize=368384 CPUtime=12.37 cores=0-7
/proc/544/task/555/stat : 555 (plingeling) R 542 544 32736 0 -1 4202560 3292 1262 0 0 1237 0 0 0 25 0 9 0 801374531 377225216 53296 33554432000 4194304 4417452 140736665663632 18446744073709551615 4249601 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=544/tid=556] ppid=542 vsize=368384 CPUtime=12.36 cores=0-7
/proc/544/task/556/stat : 556 (plingeling) R 542 544 32736 0 -1 4202560 2582 1262 0 0 1236 0 0 0 25 0 9 0 801374531 377225216 53296 33554432000 4194304 4417452 140736665663632 18446744073709551615 4258201 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=544/tid=557] ppid=542 vsize=368384 CPUtime=12.14 cores=0-7
/proc/544/task/557/stat : 557 (plingeling) R 542 544 32736 0 -1 4202560 4099 1262 0 0 1214 0 0 0 25 0 9 0 801374531 377225216 53296 33554432000 4194304 4417452 140736665663632 18446744073709551615 4232317 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=544/tid=558] ppid=542 vsize=368384 CPUtime=12.36 cores=0-7
/proc/544/task/558/stat : 558 (plingeling) R 542 544 32736 0 -1 4202560 2847 1262 0 0 1236 0 0 0 25 0 9 0 801374531 377225216 53296 33554432000 4194304 4417452 140736665663632 18446744073709551615 4249110 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 99.08
Current children cumulated vsize (KiB) 368384

[startup+25.5012 s]
/proc/loadavg: 7.65 7.99 8.01 9/560 558
/proc/meminfo: memFree=25245512/32951124 swapFree=39849776/67111528
[pid=544] ppid=542 vsize=451232 CPUtime=201.22 cores=0-7
/proc/544/stat : 544 (plingeling) S 542 544 32736 0 -1 4202496 141324 1262 0 0 20096 26 0 0 18 0 9 0 801374499 462061568 74008 33554432000 4194304 4417452 140736665663632 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/544/statm: 112808 74008 180 55 0 110694 0
[pid=544/tid=551] ppid=542 vsize=451232 CPUtime=25.16 cores=0-7
/proc/544/task/551/stat : 551 (plingeling) R 542 544 32736 0 -1 4202560 4896 1262 0 0 2516 0 0 0 25 0 9 0 801374531 462061568 74008 33554432000 4194304 4417452 140736665663632 18446744073709551615 4355812 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=544/tid=552] ppid=542 vsize=451232 CPUtime=25.16 cores=0-7
/proc/544/task/552/stat : 552 (plingeling) R 542 544 32736 0 -1 4202560 5790 1262 0 0 2516 0 0 0 25 0 9 0 801374531 462061568 74008 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364650 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=544/tid=553] ppid=542 vsize=451232 CPUtime=25.16 cores=0-7
/proc/544/task/553/stat : 553 (plingeling) R 542 544 32736 0 -1 4202560 9294 1262 0 0 2515 1 0 0 25 0 9 0 801374531 462061568 74008 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=544/tid=554] ppid=542 vsize=451232 CPUtime=25.16 cores=0-7
/proc/544/task/554/stat : 554 (plingeling) R 542 544 32736 0 -1 4202560 77208 1262 0 0 2500 16 0 0 25 0 9 0 801374531 462061568 74008 33554432000 4194304 4417452 140736665663632 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=544/tid=555] ppid=542 vsize=451232 CPUtime=25.17 cores=0-7
/proc/544/task/555/stat : 555 (plingeling) R 542 544 32736 0 -1 4202560 5749 1262 0 0 2517 0 0 0 25 0 9 0 801374531 462061568 74008 33554432000 4194304 4417452 140736665663632 18446744073709551615 4246970 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=544/tid=556] ppid=542 vsize=451232 CPUtime=25.16 cores=0-7
/proc/544/task/556/stat : 556 (plingeling) R 542 544 32736 0 -1 4202560 5708 1262 0 0 2516 0 0 0 25 0 9 0 801374531 462061568 74008 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=544/tid=557] ppid=542 vsize=451232 CPUtime=24.7 cores=0-7
/proc/544/task/557/stat : 557 (plingeling) R 542 544 32736 0 -1 4202560 10989 1262 0 0 2469 1 0 0 25 0 9 0 801374531 462061568 74008 33554432000 4194304 4417452 140736665663632 18446744073709551615 4232654 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=544/tid=558] ppid=542 vsize=451232 CPUtime=25.14 cores=0-7
/proc/544/task/558/stat : 558 (plingeling) R 542 544 32736 0 -1 4202560 4530 1262 0 0 2514 0 0 0 25 0 9 0 801374531 462061568 74008 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 201.22
Current children cumulated vsize (KiB) 451232

[startup+51.1009 s]
/proc/loadavg: 7.77 7.99 8.01 9/560 560
/proc/meminfo: memFree=25192964/32951124 swapFree=39849776/67111528
[pid=544] ppid=542 vsize=497380 CPUtime=405.54 cores=0-7
/proc/544/stat : 544 (plingeling) S 542 544 32736 0 -1 4202496 224213 1262 0 0 40515 39 0 0 18 0 9 0 801374499 509317120 85617 33554432000 4194304 4417452 140736665663632 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/544/statm: 124345 85617 180 55 0 122231 0
[pid=544/tid=551] ppid=542 vsize=497380 CPUtime=50.76 cores=0-7
/proc/544/task/551/stat : 551 (plingeling) R 542 544 32736 0 -1 4202560 10927 1262 0 0 5076 0 0 0 25 0 9 0 801374531 509317120 85617 33554432000 4194304 4417452 140736665663632 18446744073709551615 4258263 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=544/tid=552] ppid=542 vsize=497380 CPUtime=50.76 cores=0-7
/proc/544/task/552/stat : 552 (plingeling) R 542 544 32736 0 -1 4202560 13508 1262 0 0 5075 1 0 0 25 0 9 0 801374531 509317120 85617 33554432000 4194304 4417452 140736665663632 18446744073709551615 4236294 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=544/tid=553] ppid=542 vsize=497380 CPUtime=50.76 cores=0-7

################
# More data... #
################


[startup+4722.32 s]
/proc/loadavg: 8.76 8.23 8.06 10/565 1155
/proc/meminfo: memFree=16394088/32951124 swapFree=39849776/67111528
[pid=544] ppid=542 vsize=9263884 CPUtime=37683.6 cores=0-7
/proc/544/stat : 544 (plingeling) S 542 544 32736 0 -1 4202496 15039382 1262 0 0 3765817 2547 0 0 18 0 9 0 801374499 9486217216 2277114 33554432000 4194304 4417452 140736665663632 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/544/statm: 2315971 2277114 180 55 0 2313857 0
[pid=544/tid=551] ppid=542 vsize=9263884 CPUtime=4720.84 cores=0-7
/proc/544/task/551/stat : 551 (plingeling) R 542 544 32736 0 -1 4202560 1805819 1262 0 0 471777 307 0 0 25 0 9 0 801374531 9486217216 2277114 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=544/tid=552] ppid=542 vsize=9263884 CPUtime=4721.95 cores=0-7
/proc/544/task/552/stat : 552 (plingeling) R 542 544 32736 0 -1 4202560 1655918 1262 0 0 471931 264 0 0 25 0 9 0 801374531 9486217216 2277114 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=544/tid=553] ppid=542 vsize=9263884 CPUtime=4722.01 cores=0-7
/proc/544/task/553/stat : 553 (plingeling) R 542 544 32736 0 -1 4202560 2718616 1262 0 0 471769 432 0 0 25 0 9 0 801374531 9486217216 2277114 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=544/tid=554] ppid=542 vsize=9263884 CPUtime=4635.56 cores=0-7
/proc/544/task/554/stat : 554 (plingeling) R 542 544 32736 0 -1 4202560 1213972 1262 0 0 463247 309 0 0 25 0 9 0 801374531 9486217216 2277114 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=544/tid=555] ppid=542 vsize=9263884 CPUtime=4722 cores=0-7
/proc/544/task/555/stat : 555 (plingeling) R 542 544 32736 0 -1 4202560 1620126 1262 0 0 471897 303 0 0 25 0 9 0 801374531 9486217216 2277114 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364885 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=544/tid=556] ppid=542 vsize=9263884 CPUtime=4721.98 cores=0-7
/proc/544/task/556/stat : 556 (plingeling) R 542 544 32736 0 -1 4202560 1590817 1262 0 0 471934 264 0 0 25 0 9 0 801374531 9486217216 2277114 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=544/tid=557] ppid=542 vsize=9263884 CPUtime=4719.27 cores=0-7
/proc/544/task/557/stat : 557 (plingeling) R 542 544 32736 0 -1 4202560 2613063 1262 0 0 471527 400 0 0 25 0 9 0 801374531 9486217216 2277114 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=544/tid=558] ppid=542 vsize=9263884 CPUtime=4719.63 cores=0-7
/proc/544/task/558/stat : 558 (plingeling) R 542 544 32736 0 -1 4202560 1803891 1262 0 0 471702 261 0 0 25 0 9 0 801374531 9486217216 2277114 33554432000 4194304 4417452 140736665663632 18446744073709551615 4365042 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 37683.6
Current children cumulated vsize (KiB) 9263884

[startup+4782.3 s]
/proc/loadavg: 9.03 8.40 8.13 10/565 1157
/proc/meminfo: memFree=15931740/32951124 swapFree=39849776/67111528
[pid=544] ppid=542 vsize=9723808 CPUtime=38162.3 cores=0-7
/proc/544/stat : 544 (plingeling) S 542 544 32736 0 -1 4202496 15219899 1262 0 0 3813641 2592 0 0 18 0 9 0 801374499 9957179392 2392095 33554432000 4194304 4417452 140736665663632 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/544/statm: 2430952 2392095 180 55 0 2428838 0
[pid=544/tid=551] ppid=542 vsize=9723808 CPUtime=4780.82 cores=0-7
/proc/544/task/551/stat : 551 (plingeling) R 542 544 32736 0 -1 4202560 1838587 1262 0 0 477765 317 0 0 25 0 9 0 801374531 9957179392 2392095 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=544/tid=552] ppid=542 vsize=9723808 CPUtime=4781.93 cores=0-7
/proc/544/task/552/stat : 552 (plingeling) R 542 544 32736 0 -1 4202560 1688686 1262 0 0 477925 268 0 0 25 0 9 0 801374531 9957179392 2392095 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=544/tid=553] ppid=542 vsize=9723808 CPUtime=4781.99 cores=0-7
/proc/544/task/553/stat : 553 (plingeling) R 542 544 32736 0 -1 4202560 2751384 1262 0 0 477756 443 0 0 25 0 9 0 801374531 9957179392 2392095 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=544/tid=554] ppid=542 vsize=9723808 CPUtime=4694.41 cores=0-7
/proc/544/task/554/stat : 554 (plingeling) R 542 544 32736 0 -1 4202560 1213972 1262 0 0 469132 309 0 0 25 0 9 0 801374531 9957179392 2392095 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364821 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=544/tid=555] ppid=542 vsize=9723808 CPUtime=4781.98 cores=0-7
/proc/544/task/555/stat : 555 (plingeling) R 542 544 32736 0 -1 4202560 1620126 1262 0 0 477895 303 0 0 25 0 9 0 801374531 9957179392 2392095 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=544/tid=556] ppid=542 vsize=9723808 CPUtime=4781.96 cores=0-7
/proc/544/task/556/stat : 556 (plingeling) R 542 544 32736 0 -1 4202560 1623585 1262 0 0 477928 268 0 0 25 0 9 0 801374531 9957179392 2392095 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=544/tid=557] ppid=542 vsize=9723808 CPUtime=4779.25 cores=0-7
/proc/544/task/557/stat : 557 (plingeling) R 542 544 32736 0 -1 4202560 2657004 1262 0 0 477511 414 0 0 25 0 9 0 801374531 9957179392 2392095 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=544/tid=558] ppid=542 vsize=9723808 CPUtime=4779.59 cores=0-7
/proc/544/task/558/stat : 558 (plingeling) R 542 544 32736 0 -1 4202560 1809395 1262 0 0 477696 263 0 0 25 0 9 0 801374531 9957179392 2392095 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 38162.3
Current children cumulated vsize (KiB) 9723808

[startup+4842.3 s]
/proc/loadavg: 9.01 8.51 8.18 10/565 1159
/proc/meminfo: memFree=16237008/32951124 swapFree=39849776/67111528
[pid=544] ppid=542 vsize=9415692 CPUtime=38641.1 cores=0-7
/proc/544/stat : 544 (plingeling) S 542 544 32736 0 -1 4202496 15295883 1262 0 0 3861511 2604 0 0 18 0 9 0 801374499 9641668608 2315035 33554432000 4194304 4417452 140736665663632 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/544/statm: 2353923 2315035 180 55 0 2351809 0
[pid=544/tid=551] ppid=542 vsize=9415692 CPUtime=4840.82 cores=0-7
/proc/544/task/551/stat : 551 (plingeling) R 542 544 32736 0 -1 4202560 1838587 1262 0 0 483765 317 0 0 25 0 9 0 801374531 9641668608 2315035 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=544/tid=552] ppid=542 vsize=9415692 CPUtime=4841.94 cores=0-7
/proc/544/task/552/stat : 552 (plingeling) R 542 544 32736 0 -1 4202560 1688686 1262 0 0 483926 268 0 0 25 0 9 0 801374531 9641668608 2315035 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=544/tid=553] ppid=542 vsize=9415692 CPUtime=4842 cores=0-7
/proc/544/task/553/stat : 553 (plingeling) R 542 544 32736 0 -1 4202560 2777196 1262 0 0 483752 448 0 0 25 0 9 0 801374531 9641668608 2315035 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=544/tid=554] ppid=542 vsize=9415692 CPUtime=4753.25 cores=0-7
/proc/544/task/554/stat : 554 (plingeling) R 542 544 32736 0 -1 4202560 1213972 1262 0 0 475016 309 0 0 25 0 9 0 801374531 9641668608 2315035 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=544/tid=555] ppid=542 vsize=9415692 CPUtime=4841.98 cores=0-7
/proc/544/task/555/stat : 555 (plingeling) R 542 544 32736 0 -1 4202560 1620126 1262 0 0 483895 303 0 0 25 0 9 0 801374531 9641668608 2315035 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364587 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=544/tid=556] ppid=542 vsize=9415692 CPUtime=4841.96 cores=0-7
/proc/544/task/556/stat : 556 (plingeling) R 542 544 32736 0 -1 4202560 1623585 1262 0 0 483928 268 0 0 25 0 9 0 801374531 9641668608 2315035 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=544/tid=557] ppid=542 vsize=9415692 CPUtime=4839.25 cores=0-7
/proc/544/task/557/stat : 557 (plingeling) R 542 544 32736 0 -1 4202560 2659359 1262 0 0 483508 417 0 0 25 0 9 0 801374531 9641668608 2315035 33554432000 4194304 4417452 140736665663632 18446744073709551615 4253440 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=544/tid=558] ppid=542 vsize=9415692 CPUtime=4839.57 cores=0-7
/proc/544/task/558/stat : 558 (plingeling) R 542 544 32736 0 -1 4202560 1857212 1262 0 0 483688 269 0 0 25 0 9 0 801374531 9641668608 2315035 33554432000 4194304 4417452 140736665663632 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 38641.1
Current children cumulated vsize (KiB) 9415692

[startup+4902.3 s]
/proc/loadavg: 9.06 8.61 8.23 10/565 1161
/proc/meminfo: memFree=16204448/32951124 swapFree=39849776/67111528
[pid=544] ppid=542 vsize=9443372 CPUtime=39120 cores=0-7
/proc/544/stat : 544 (plingeling) S 542 544 32736 0 -1 4202496 15368339 1262 0 0 3909382 2619 0 0 18 0 9 0 801374499 9670012928 2321955 33554432000 4194304 4417452 140736665663632 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/544/statm: 2360843 2321955 180 55 0 2358729 0
[pid=544/tid=551] ppid=542 vsize=9443372 CPUtime=4900.82 cores=0-7
/proc/544/task/551/stat : 551 (plingeling) R 542 544 32736 0 -1 4202560 1838587 1262 0 0 489765 317 0 0 25 0 9 0 801374531 9670012928 2321955 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=544/tid=552] ppid=542 vsize=9443372 CPUtime=4901.94 cores=0-7
/proc/544/task/552/stat : 552 (plingeling) R 542 544 32736 0 -1 4202560 1688686 1262 0 0 489925 269 0 0 25 0 9 0 801374531 9670012928 2321955 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=544/tid=553] ppid=542 vsize=9443372 CPUtime=4902 cores=0-7
/proc/544/task/553/stat : 553 (plingeling) R 542 544 32736 0 -1 4202560 2777196 1262 0 0 489752 448 0 0 25 0 9 0 801374531 9670012928 2321955 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=544/tid=554] ppid=542 vsize=9443372 CPUtime=4812.12 cores=0-7
/proc/544/task/554/stat : 554 (plingeling) R 542 544 32736 0 -1 4202560 1213972 1262 0 0 480903 309 0 0 25 0 9 0 801374531 9670012928 2321955 33554432000 4194304 4417452 140736665663632 18446744073709551615 4365075 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=544/tid=555] ppid=542 vsize=9443372 CPUtime=4901.99 cores=0-7
/proc/544/task/555/stat : 555 (plingeling) R 542 544 32736 0 -1 4202560 1620126 1262 0 0 489896 303 0 0 25 0 9 0 801374531 9670012928 2321955 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364741 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=544/tid=556] ppid=542 vsize=9443372 CPUtime=4901.96 cores=0-7
/proc/544/task/556/stat : 556 (plingeling) R 542 544 32736 0 -1 4202560 1623585 1262 0 0 489928 268 0 0 25 0 9 0 801374531 9670012928 2321955 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=544/tid=557] ppid=542 vsize=9443372 CPUtime=4899.25 cores=0-7
/proc/544/task/557/stat : 557 (plingeling) R 542 544 32736 0 -1 4202560 2731815 1262 0 0 489495 430 0 0 25 0 9 0 801374531 9670012928 2321955 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364741 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=544/tid=558] ppid=542 vsize=9443372 CPUtime=4899.56 cores=0-7
/proc/544/task/558/stat : 558 (plingeling) R 542 544 32736 0 -1 4202560 1857212 1262 0 0 489686 270 0 0 25 0 9 0 801374531 9670012928 2321955 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 39120
Current children cumulated vsize (KiB) 9443372

[startup+4962.3 s]
/proc/loadavg: 9.02 8.68 8.28 10/565 1161
/proc/meminfo: memFree=16338932/32951124 swapFree=39849776/67111528
[pid=544] ppid=542 vsize=9305388 CPUtime=39598.9 cores=0-7
/proc/544/stat : 544 (plingeling) S 542 544 32736 0 -1 4202496 15448531 1262 0 0 3957252 2633 0 0 18 0 9 0 801374499 9528717312 2287459 33554432000 4194304 4417452 140736665663632 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/544/statm: 2326347 2287459 180 55 0 2324233 0
[pid=544/tid=551] ppid=542 vsize=9305388 CPUtime=4960.82 cores=0-7
/proc/544/task/551/stat : 551 (plingeling) R 542 544 32736 0 -1 4202560 1838587 1262 0 0 495765 317 0 0 25 0 9 0 801374531 9528717312 2287459 33554432000 4194304 4417452 140736665663632 18446744073709551615 4365058 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=544/tid=552] ppid=542 vsize=9305388 CPUtime=4961.93 cores=0-7
/proc/544/task/552/stat : 552 (plingeling) R 542 544 32736 0 -1 4202560 1721454 1262 0 0 495921 272 0 0 25 0 9 0 801374531 9528717312 2287459 33554432000 4194304 4417452 140736665663632 18446744073709551615 4365150 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=544/tid=553] ppid=542 vsize=9305388 CPUtime=4961.99 cores=0-7
/proc/544/task/553/stat : 553 (plingeling) R 542 544 32736 0 -1 4202560 2791852 1262 0 0 495747 452 0 0 25 0 9 0 801374531 9528717312 2287459 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=544/tid=554] ppid=542 vsize=9305388 CPUtime=4870.97 cores=0-7
/proc/544/task/554/stat : 554 (plingeling) R 542 544 32736 0 -1 4202560 1213972 1262 0 0 486788 309 0 0 25 0 9 0 801374531 9528717312 2287459 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=544/tid=555] ppid=542 vsize=9305388 CPUtime=4961.99 cores=0-7
/proc/544/task/555/stat : 555 (plingeling) R 542 544 32736 0 -1 4202560 1620126 1262 0 0 495896 303 0 0 25 0 9 0 801374531 9528717312 2287459 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=544/tid=556] ppid=542 vsize=9305388 CPUtime=4961.96 cores=0-7
/proc/544/task/556/stat : 556 (plingeling) R 542 544 32736 0 -1 4202560 1623585 1262 0 0 495928 268 0 0 25 0 9 0 801374531 9528717312 2287459 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=544/tid=557] ppid=542 vsize=9305388 CPUtime=4959.25 cores=0-7
/proc/544/task/557/stat : 557 (plingeling) R 542 544 32736 0 -1 4202560 2764583 1262 0 0 495492 433 0 0 25 0 9 0 801374531 9528717312 2287459 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=544/tid=558] ppid=542 vsize=9305388 CPUtime=4959.54 cores=0-7
/proc/544/task/558/stat : 558 (plingeling) R 542 544 32736 0 -1 4202560 1857212 1262 0 0 495684 270 0 0 25 0 9 0 801374531 9528717312 2287459 33554432000 4194304 4417452 140736665663632 18446744073709551615 4278211 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 39598.8
Current children cumulated vsize (KiB) 9305388



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5012.6 s]
/proc/loadavg: 9.07 8.74 8.32 11/565 1163
/proc/meminfo: memFree=16176744/32951124 swapFree=39849776/67111528
[pid=544] ppid=542 vsize=9464348 CPUtime=40000.4 cores=0-7
/proc/544/stat : 544 (plingeling) S 542 544 32736 0 -1 4202496 15580170 1262 0 0 3997393 2652 0 0 18 0 9 0 801374499 9691492352 2327199 33554432000 4194304 4417452 140736665663632 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/544/statm: 2366087 2327199 180 55 0 2363973 0
[pid=544/tid=551] ppid=542 vsize=9464348 CPUtime=5011.16 cores=0-7
/proc/544/task/551/stat : 551 (plingeling) R 542 544 32736 0 -1 4202560 1838587 1262 0 0 500799 317 0 0 25 0 9 0 801374531 9691492352 2327199 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364984 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=544/tid=552] ppid=542 vsize=9464348 CPUtime=5012.28 cores=0-7
/proc/544/task/552/stat : 552 (plingeling) R 542 544 32736 0 -1 4202560 1721454 1262 0 0 500956 272 0 0 25 0 9 0 801374531 9691492352 2327199 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=544/tid=553] ppid=542 vsize=9464348 CPUtime=5012.34 cores=0-7
/proc/544/task/553/stat : 553 (plingeling) R 542 544 32736 0 -1 4202560 2791852 1262 0 0 500782 452 0 0 25 0 9 0 801374531 9691492352 2327199 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=544/tid=554] ppid=542 vsize=9464348 CPUtime=4920.32 cores=0-7
/proc/544/task/554/stat : 554 (plingeling) R 542 544 32736 0 -1 4202560 1213972 1262 0 0 491723 309 0 0 25 0 9 0 801374531 9691492352 2327199 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=544/tid=555] ppid=542 vsize=9464348 CPUtime=5012.33 cores=0-7
/proc/544/task/555/stat : 555 (plingeling) R 542 544 32736 0 -1 4202560 1620126 1262 0 0 500930 303 0 0 25 0 9 0 801374531 9691492352 2327199 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=544/tid=556] ppid=542 vsize=9464348 CPUtime=5012.3 cores=0-7
/proc/544/task/556/stat : 556 (plingeling) R 542 544 32736 0 -1 4202560 1656353 1262 0 0 500958 272 0 0 25 0 9 0 801374531 9691492352 2327199 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364838 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=544/tid=557] ppid=542 vsize=9464348 CPUtime=5009.59 cores=0-7
/proc/544/task/557/stat : 557 (plingeling) R 542 544 32736 0 -1 4202560 2782080 1262 0 0 500524 435 0 0 25 0 9 0 801374531 9691492352 2327199 33554432000 4194304 4417452 140736665663632 18446744073709551615 4364809 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=544/tid=558] ppid=542 vsize=9464348 CPUtime=5009.86 cores=0-7
/proc/544/task/558/stat : 558 (plingeling) R 542 544 32736 0 -1 4202560 1938586 1262 0 0 500702 284 0 0 25 0 9 0 801374531 9691492352 2327199 33554432000 4194304 4417452 140736665663632 18446744073709551615 4366385 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 9464348

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5013 s]
/proc/loadavg: 9.07 8.74 8.32 11/565 1163
/proc/meminfo: memFree=16176744/32951124 swapFree=39849776/67111528
[pid=544] ppid=542 vsize=0 CPUtime=40001 cores=0-7
/proc/544/stat : 544 (plingeling) Z 542 544 32736 0 -1 4203548 15580171 1262 0 0 3997410 2687 0 0 18 0 2 0 801374499 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/544/statm: 0 0 0 0 0 0 0
[pid=544/tid=554] ppid=542 vsize=0 CPUtime=4920.68 cores=0-7
/proc/544/task/554/stat : 554 (plingeling) R 542 544 32736 0 -1 4203588 1213972 1262 0 0 491724 344 0 0 25 0 2 0 801374531 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40001
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5013.04
CPU time (s): 40001
CPU user time (s): 39974.1
CPU system time (s): 26.9149
CPU usage (%): 797.94
Max. virtual memory (cumulated for all children) (KiB): 9854880

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39974.1
system time used= 26.9149
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 15581433
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 436
involuntary context switches= 200584

runsolver used 26.6579 second user time and 67.6137 second system time

The end

Launcher Data 

Begin job on node118 at 2011-05-23 02:43:58
IDJOB=3395030
IDBENCH=88223
IDSOLVER=1851
FILE ID=node118/3395030-1306111438
RUNJOBID= node118-1306106414-32750
PBS_JOBID= 13388024
Free space on /tmp= 72940 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/crafted/anton/SRHD-SGI/srhd-sgi-m42-q663-n40-p15-s72490337.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3395030-1306111438/watcher-3395030-1306111438 -o /tmp/evaluation-result-3395030-1306111438/solver-3395030-1306111438 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3395030-1306111438.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= ab8e40d31c79e7a71114e3126a46874f
RANDOM SEED=1083979442

node118.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5374.59
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      25539264 kB
Buffers:         18148 kB
Cached:         150828 kB
SwapCached:      52024 kB
Active:         485868 kB
Inactive:      6663768 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      25539264 kB
SwapTotal:    67111528 kB
SwapFree:     39849776 kB
Dirty:            3496 kB
Writeback:           0 kB
AnonPages:     6942028 kB
Mapped:          24960 kB
Slab:            74868 kB
PageTables:     115324 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 183776516 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72940 MiB
End job on node118 at 2011-05-23 04:07:33