Trace number 3371155

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.17 1203.18

General information on the benchmark

NameCRAFTED/rbsat/crafted/unforced/
rbsat-v1820c171155g5.cnf
MD5SUM70a781814cd432fb0f934a7edd92ccff
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables1820
Number of clauses171155
Sum of the clauses size344000
Maximum clause length28
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2171090
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 565

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3371155-1305419616/watcher-3371155-1305419616 -o /tmp/evaluation-result-3371155-1305419616/solver-3371155-1305419616 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3371155-1305419616.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.22 7.88 7.78 2/324 25587
/proc/meminfo: memFree=25270620/32951124 swapFree=44485764/67111528
[pid=25587] ppid=25585 vsize=10312 CPUtime=0 cores=1,3,5,7
/proc/25587/stat : 25587 (plingeling) R 25585 25587 25544 0 -1 4202496 639 1253 0 0 0 0 0 0 22 0 1 0 653562056 10559488 569 33554432000 4194304 4417452 140736924725824 18446744073709551615 244446945024 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/25587/statm: 2578 576 148 55 0 464 0

[startup+0.088871 s]
/proc/loadavg: 7.22 7.88 7.78 2/324 25587
/proc/meminfo: memFree=25270620/32951124 swapFree=44485764/67111528
[pid=25587] ppid=25585 vsize=25832 CPUtime=0.07 cores=1,3,5,7
/proc/25587/stat : 25587 (plingeling) R 25585 25587 25544 0 -1 4202496 4527 1253 0 0 7 0 0 0 22 0 1 0 653562056 26451968 4457 33554432000 4194304 4417452 140736924725824 18446744073709551615 244446945024 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/25587/statm: 6458 4457 148 55 0 4344 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 25832

[startup+0.100864 s]
/proc/loadavg: 7.22 7.88 7.78 2/324 25587
/proc/meminfo: memFree=25270620/32951124 swapFree=44485764/67111528
[pid=25587] ppid=25585 vsize=25832 CPUtime=0.09 cores=1,3,5,7
/proc/25587/stat : 25587 (plingeling) R 25585 25587 25544 0 -1 4202496 4527 1253 0 0 9 0 0 0 22 0 1 0 653562056 26451968 4457 33554432000 4194304 4417452 140736924725824 18446744073709551615 4209589 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/25587/statm: 6458 4457 148 55 0 4344 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 25832

[startup+0.300796 s]
/proc/loadavg: 7.22 7.88 7.78 2/324 25587
/proc/meminfo: memFree=25270620/32951124 swapFree=44485764/67111528
[pid=25587] ppid=25585 vsize=75228 CPUtime=0.84 cores=1,3,5,7
/proc/25587/stat : 25587 (plingeling) S 25585 25587 25544 0 -1 4202496 8650 1253 0 0 83 1 0 0 22 0 5 0 653562056 77033472 6591 33554432000 4194304 4417452 140736924725824 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/25587/statm: 18807 6591 177 55 0 16693 0
Current children cumulated CPU time (s) 0.84
Current children cumulated vsize (KiB) 75228

[startup+0.700719 s]
/proc/loadavg: 7.22 7.88 7.78 2/324 25587
/proc/meminfo: memFree=25270620/32951124 swapFree=44485764/67111528
[pid=25587] ppid=25585 vsize=82824 CPUtime=2.45 cores=1,3,5,7
/proc/25587/stat : 25587 (plingeling) S 25585 25587 25544 0 -1 4202496 13583 1253 0 0 243 2 0 0 22 0 5 0 653562056 84811776 8493 33554432000 4194304 4417452 140736924725824 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/25587/statm: 20706 8493 180 55 0 18592 0
Current children cumulated CPU time (s) 2.45
Current children cumulated vsize (KiB) 82824

[startup+1.50057 s]
/proc/loadavg: 7.22 7.88 7.78 10/333 25601
/proc/meminfo: memFree=25217104/32951124 swapFree=44485764/67111528
[pid=25587] ppid=25585 vsize=89752 CPUtime=5.63 cores=1,3,5,7
/proc/25587/stat : 25587 (plingeling) S 25585 25587 25544 0 -1 4202496 18053 1253 0 0 561 2 0 0 22 0 5 0 653562056 91906048 10225 33554432000 4194304 4417452 140736924725824 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/25587/statm: 22438 10225 180 55 0 20324 0
[pid=25587/tid=25598] ppid=25585 vsize=89752 CPUtime=1.38 cores=1,3,5,7
/proc/25587/task/25598/stat : 25598 (plingeling) R 25585 25587 25544 0 -1 4202560 1308 1253 0 0 138 0 0 0 25 0 5 0 653562067 91906048 10225 33554432000 4194304 4417452 140736924725824 18446744073709551615 4249222 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25587/tid=25599] ppid=25585 vsize=89752 CPUtime=1.37 cores=1,3,5,7
/proc/25587/task/25599/stat : 25599 (plingeling) R 25585 25587 25544 0 -1 4202560 1500 1253 0 0 137 0 0 0 25 0 5 0 653562067 91906048 10225 33554432000 4194304 4417452 140736924725824 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25587/tid=25600] ppid=25585 vsize=89752 CPUtime=1.38 cores=1,3,5,7
/proc/25587/task/25600/stat : 25600 (plingeling) R 25585 25587 25544 0 -1 4202560 1549 1253 0 0 138 0 0 0 25 0 5 0 653562067 91906048 10225 33554432000 4194304 4417452 140736924725824 18446744073709551615 4258445 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25587/tid=25601] ppid=25585 vsize=89752 CPUtime=1.37 cores=1,3,5,7
/proc/25587/task/25601/stat : 25601 (plingeling) R 25585 25587 25544 0 -1 4202560 9157 1253 0 0 136 1 0 0 25 0 5 0 653562067 91906048 10225 33554432000 4194304 4417452 140736924725824 18446744073709551615 4250637 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 5.63
Current children cumulated vsize (KiB) 89752

[startup+3.10027 s]
/proc/loadavg: 7.22 7.88 7.78 10/333 25601
/proc/meminfo: memFree=25206452/32951124 swapFree=44485764/67111528
[pid=25587] ppid=25585 vsize=109816 CPUtime=12.02 cores=1,3,5,7
/proc/25587/stat : 25587 (plingeling) S 25585 25587 25544 0 -1 4202496 26565 1253 0 0 1199 3 0 0 22 0 5 0 653562056 112451584 15241 33554432000 4194304 4417452 140736924725824 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/25587/statm: 27454 15241 180 55 0 25340 0
[pid=25587/tid=25598] ppid=25585 vsize=109816 CPUtime=2.98 cores=1,3,5,7
/proc/25587/task/25598/stat : 25598 (plingeling) R 25585 25587 25544 0 -1 4202560 2251 1253 0 0 298 0 0 0 25 0 5 0 653562067 112451584 15241 33554432000 4194304 4417452 140736924725824 18446744073709551615 4249158 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25587/tid=25599] ppid=25585 vsize=109816 CPUtime=2.97 cores=1,3,5,7
/proc/25587/task/25599/stat : 25599 (plingeling) R 25585 25587 25544 0 -1 4202560 2443 1253 0 0 297 0 0 0 25 0 5 0 653562067 112451584 15241 33554432000 4194304 4417452 140736924725824 18446744073709551615 4249222 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25587/tid=25600] ppid=25585 vsize=109816 CPUtime=2.98 cores=1,3,5,7
/proc/25587/task/25600/stat : 25600 (plingeling) R 25585 25587 25544 0 -1 4202560 1549 1253 0 0 298 0 0 0 25 0 5 0 653562067 112451584 15241 33554432000 4194304 4417452 140736924725824 18446744073709551615 4365779 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25587/tid=25601] ppid=25585 vsize=109816 CPUtime=2.95 cores=1,3,5,7
/proc/25587/task/25601/stat : 25601 (plingeling) R 25585 25587 25544 0 -1 4202560 15783 1253 0 0 294 1 0 0 25 0 5 0 653562067 112451584 15241 33554432000 4194304 4417452 140736924725824 18446744073709551615 4250651 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 12.02
Current children cumulated vsize (KiB) 109816

[startup+6.30067 s]
/proc/loadavg: 7.45 7.92 7.80 9/333 25601
/proc/meminfo: memFree=25186252/32951124 swapFree=44485764/67111528
[pid=25587] ppid=25585 vsize=131732 CPUtime=24.78 cores=1,3,5,7
/proc/25587/stat : 25587 (plingeling) S 25585 25587 25544 0 -1 4202496 41222 1253 0 0 2473 5 0 0 22 0 5 0 653562056 134893568 20720 33554432000 4194304 4417452 140736924725824 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/25587/statm: 32933 20720 180 55 0 30819 0
[pid=25587/tid=25598] ppid=25585 vsize=131732 CPUtime=6.18 cores=1,3,5,7
/proc/25587/task/25598/stat : 25598 (plingeling) R 25585 25587 25544 0 -1 4202560 2251 1253 0 0 618 0 0 0 25 0 5 0 653562067 134893568 20720 33554432000 4194304 4417452 140736924725824 18446744073709551615 4250080 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25587/tid=25599] ppid=25585 vsize=131732 CPUtime=6.17 cores=1,3,5,7
/proc/25587/task/25599/stat : 25599 (plingeling) R 25585 25587 25544 0 -1 4202560 2443 1253 0 0 617 0 0 0 25 0 5 0 653562067 134893568 20720 33554432000 4194304 4417452 140736924725824 18446744073709551615 4249181 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25587/tid=25600] ppid=25585 vsize=131732 CPUtime=6.18 cores=1,3,5,7
/proc/25587/task/25600/stat : 25600 (plingeling) R 25585 25587 25544 0 -1 4202560 1549 1253 0 0 618 0 0 0 25 0 5 0 653562067 134893568 20720 33554432000 4194304 4417452 140736924725824 18446744073709551615 4277897 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25587/tid=25601] ppid=25585 vsize=131732 CPUtime=6.13 cores=1,3,5,7
/proc/25587/task/25601/stat : 25601 (plingeling) R 25585 25587 25544 0 -1 4202560 30440 1253 0 0 609 4 0 0 25 0 5 0 653562067 134893568 20720 33554432000 4194304 4417452 140736924725824 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 24.78
Current children cumulated vsize (KiB) 131732

[startup+12.7005 s]
/proc/loadavg: 7.49 7.92 7.80 10/333 25601
/proc/meminfo: memFree=25128612/32951124 swapFree=44485764/67111528
[pid=25587] ppid=25585 vsize=136896 CPUtime=50.32 cores=1,3,5,7
/proc/25587/stat : 25587 (plingeling) S 25585 25587 25544 0 -1 4202496 67512 1253 0 0 5023 9 0 0 22 0 5 0 653562056 140181504 22011 33554432000 4194304 4417452 140736924725824 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/25587/statm: 34224 22011 180 55 0 32110 0
[pid=25587/tid=25598] ppid=25585 vsize=136896 CPUtime=12.58 cores=1,3,5,7
/proc/25587/task/25598/stat : 25598 (plingeling) R 25585 25587 25544 0 -1 4202560 3194 1253 0 0 1258 0 0 0 25 0 5 0 653562067 140181504 22011 33554432000 4194304 4417452 140736924725824 18446744073709551615 4235359 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25587/tid=25599] ppid=25585 vsize=136896 CPUtime=12.57 cores=1,3,5,7
/proc/25587/task/25599/stat : 25599 (plingeling) R 25585 25587 25544 0 -1 4202560 3386 1253 0 0 1257 0 0 0 25 0 5 0 653562067 140181504 22011 33554432000 4194304 4417452 140736924725824 18446744073709551615 4335867 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25587/tid=25600] ppid=25585 vsize=136896 CPUtime=12.57 cores=1,3,5,7
/proc/25587/task/25600/stat : 25600 (plingeling) R 25585 25587 25544 0 -1 4202560 3137 1253 0 0 1257 0 0 0 25 0 5 0 653562067 140181504 22011 33554432000 4194304 4417452 140736924725824 18446744073709551615 4309183 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25587/tid=25601] ppid=25585 vsize=136896 CPUtime=12.46 cores=1,3,5,7
/proc/25587/task/25601/stat : 25601 (plingeling) R 25585 25587 25544 0 -1 4202560 53256 1253 0 0 1239 7 0 0 25 0 5 0 653562067 140181504 22011 33554432000 4194304 4417452 140736924725824 18446744073709551615 4250848 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 50.32
Current children cumulated vsize (KiB) 136896

[startup+25.5011 s]
/proc/loadavg: 7.60 7.92 7.80 9/333 25602
/proc/meminfo: memFree=25051032/32951124 swapFree=44485764/67111528
[pid=25587] ppid=25585 vsize=265388 CPUtime=101.39 cores=1,3,5,7
/proc/25587/stat : 25587 (plingeling) S 25585 25587 25544 0 -1 4202496 108269 1253 0 0 10123 16 0 0 22 0 5 0 653562056 271757312 37758 33554432000 4194304 4417452 140736924725824 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/25587/statm: 66347 37758 180 55 0 64233 0
[pid=25587/tid=25598] ppid=25585 vsize=265388 CPUtime=25.38 cores=1,3,5,7
/proc/25587/task/25598/stat : 25598 (plingeling) R 25585 25587 25544 0 -1 4202560 5144 1253 0 0 2538 0 0 0 25 0 5 0 653562067 271757312 37758 33554432000 4194304 4417452 140736924725824 18446744073709551615 4250633 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25587/tid=25599] ppid=25585 vsize=265388 CPUtime=25.37 cores=1,3,5,7
/proc/25587/task/25599/stat : 25599 (plingeling) R 25585 25587 25544 0 -1 4202560 3890 1253 0 0 2537 0 0 0 25 0 5 0 653562067 271757312 37758 33554432000 4194304 4417452 140736924725824 18446744073709551615 4365008 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25587/tid=25600] ppid=25585 vsize=265388 CPUtime=25.38 cores=1,3,5,7
/proc/25587/task/25600/stat : 25600 (plingeling) R 25585 25587 25544 0 -1 4202560 8407 1253 0 0 2537 1 0 0 25 0 5 0 653562067 271757312 37758 33554432000 4194304 4417452 140736924725824 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25587/tid=25601] ppid=25585 vsize=265388 CPUtime=25.12 cores=1,3,5,7
/proc/25587/task/25601/stat : 25601 (plingeling) R 25585 25587 25544 0 -1 4202560 86289 1253 0 0 2500 12 0 0 25 0 5 0 653562067 271757312 37758 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 101.39
Current children cumulated vsize (KiB) 265388

[startup+51.1003 s]
/proc/loadavg: 7.74 7.93 7.80 10/333 25602
/proc/meminfo: memFree=24983512/32951124 swapFree=44485764/67111528
[pid=25587] ppid=25585 vsize=291988 CPUtime=203.54 cores=1,3,5,7
/proc/25587/stat : 25587 (plingeling) S 25585 25587 25544 0 -1 4202496 173171 1253 0 0 20329 25 0 0 22 0 5 0 653562056 298995712 44408 33554432000 4194304 4417452 140736924725824 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/25587/statm: 72997 44408 180 55 0 70883 0
[pid=25587/tid=25598] ppid=25585 vsize=291988 CPUtime=50.98 cores=1,3,5,7
/proc/25587/task/25598/stat : 25598 (plingeling) R 25585 25587 25544 0 -1 4202560 9235 1253 0 0 5097 1 0 0 25 0 5 0 653562067 298995712 44408 33554432000 4194304 4417452 140736924725824 18446744073709551615 4249110 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25587/tid=25599] ppid=25585 vsize=291988 CPUtime=50.97 cores=1,3,5,7
/proc/25587/task/25599/stat : 25599 (plingeling) R 25585 25587 25544 0 -1 4202560 7650 1253 0 0 5096 1 0 0 25 0 5 0 653562067 298995712 44408 33554432000 4194304 4417452 140736924725824 18446744073709551615 4249151 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25587/tid=25600] ppid=25585 vsize=291988 CPUtime=50.98 cores=1,3,5,7
/proc/25587/task/25600/stat : 25600 (plingeling) R 25585 25587 25544 0 -1 4202560 10941 1253 0 0 5097 1 0 0 25 0 5 0 653562067 298995712 44408 33554432000 4194304 4417452 140736924725824 18446744073709551615 4302940 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25587/tid=25601] ppid=25585 vsize=291988 CPUtime=50.47 cores=1,3,5,7
/proc/25587/task/25601/stat : 25601 (plingeling) R 25585 25587 25544 0 -1 4202560 140806 1253 0 0 5027 20 0 0 25 0 5 0 653562067 298995712 44408 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 203.54
Current children cumulated vsize (KiB) 291988

[startup+102.312 s]
/proc/loadavg: 7.93 7.95 7.82 10/333 25604
/proc/meminfo: memFree=24816952/32951124 swapFree=44485764/67111528
[pid=25587] ppid=25585 vsize=386632 CPUtime=407.87 cores=1,3,5,7
/proc/25587/stat : 25587 (plingeling) S 25585 25587 25544 0 -1 4202496 261643 1253 0 0 40750 37 0 0 22 0 5 0 653562056 395911168 68069 33554432000 4194304 4417452 140736924725824 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/25587/statm: 96658 68069 180 55 0 94544 0
[pid=25587/tid=25598] ppid=25585 vsize=386632 CPUtime=102.19 cores=1,3,5,7
/proc/25587/task/25598/stat : 25598 (plingeling) R 25585 25587 25544 0 -1 4202560 11905 1253 0 0 10218 1 0 0 25 0 5 0 653562067 395911168 68069 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364650 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25587/tid=25599] ppid=25585 vsize=386632 CPUtime=102.18 cores=1,3,5,7
/proc/25587/task/25599/stat : 25599 (plingeling) R 25585 25587 25544 0 -1 4202560 11089 1253 0 0 10217 1 0 0 25 0 5 0 653562067 395911168 68069 33554432000 4194304 4417452 140736924725824 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25587/tid=25600] ppid=25585 vsize=386632 CPUtime=102.19 cores=1,3,5,7
/proc/25587/task/25600/stat : 25600 (plingeling) R 25585 25587 25544 0 -1 4202560 25912 1253 0 0 10216 3 0 0 25 0 5 0 653562067 395911168 68069 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25587/tid=25601] ppid=25585 vsize=386632 CPUtime=101.17 cores=1,3,5,7
/proc/25587/task/25601/stat : 25601 (plingeling) R 25585 25587 25544 0 -1 4202560 208198 1253 0 0 10087 30 0 0 25 0 5 0 653562067 395911168 68069 33554432000 4194304 4417452 140736924725824 18446744073709551615 4366385 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 407.87
Current children cumulated vsize (KiB) 386632

[startup+162.301 s]
/proc/loadavg: 8.05 7.98 7.84 10/333 25605
/proc/meminfo: memFree=24529848/32951124 swapFree=44485764/67111528
[pid=25587] ppid=25585 vsize=558696 CPUtime=647.25 cores=1,3,5,7
/proc/25587/stat : 25587 (plingeling) S 25585 25587 25544 0 -1 4202496 321461 1253 0 0 64679 46 0 0 22 0 5 0 653562056 572104704 111110 33554432000 4194304 4417452 140736924725824 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/25587/statm: 139674 111110 180 55 0 137560 0
[pid=25587/tid=25598] ppid=25585 vsize=558696 CPUtime=162.18 cores=1,3,5,7
/proc/25587/task/25598/stat : 25598 (plingeling) R 25585 25587 25544 0 -1 4202560 12310 1253 0 0 16217 1 0 0 25 0 5 0 653562067 572104704 111110 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25587/tid=25599] ppid=25585 vsize=558696 CPUtime=162.17 cores=1,3,5,7
/proc/25587/task/25599/stat : 25599 (plingeling) R 25585 25587 25544 0 -1 4202560 13879 1253 0 0 16216 1 0 0 25 0 5 0 653562067 572104704 111110 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25587/tid=25600] ppid=25585 vsize=558696 CPUtime=162.18 cores=1,3,5,7
/proc/25587/task/25600/stat : 25600 (plingeling) R 25585 25587 25544 0 -1 4202560 44409 1253 0 0 16213 5 0 0 25 0 5 0 653562067 572104704 111110 33554432000 4194304 4417452 140736924725824 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25587/tid=25601] ppid=25585 vsize=558696 CPUtime=160.57 cores=1,3,5,7
/proc/25587/task/25601/stat : 25601 (plingeling) R 25585 25587 25544 0 -1 4202560 246324 1253 0 0 16021 36 0 0 25 0 5 0 653562067 572104704 111110 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 647.25
Current children cumulated vsize (KiB) 558696


################
# More data... #
################

[pid=25587/tid=25599] ppid=25585 vsize=897652 CPUtime=762.21 cores=1,3,5,7
/proc/25587/task/25599/stat : 25599 (plingeling) R 25585 25587 25544 0 -1 4202560 33413 1253 0 0 76217 4 0 0 25 0 5 0 653562067 919195648 195849 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25587/tid=25600] ppid=25585 vsize=897652 CPUtime=762.21 cores=1,3,5,7
/proc/25587/task/25600/stat : 25600 (plingeling) R 25585 25587 25544 0 -1 4202560 174237 1253 0 0 76200 21 0 0 25 0 5 0 653562067 919195648 195849 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25587/tid=25601] ppid=25585 vsize=897652 CPUtime=754.55 cores=1,3,5,7
/proc/25587/task/25601/stat : 25601 (plingeling) R 25585 25587 25544 0 -1 4202560 611630 1253 0 0 75358 97 0 0 25 0 5 0 653562067 919195648 195849 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3041.31
Current children cumulated vsize (KiB) 897652

[startup+822.3 s]
/proc/loadavg: 8.08 8.09 7.96 10/333 25623
/proc/meminfo: memFree=23613336/32951124 swapFree=44485764/67111528
[pid=25587] ppid=25585 vsize=1027752 CPUtime=3280.66 cores=1,3,5,7
/proc/25587/stat : 25587 (plingeling) S 25585 25587 25544 0 -1 4202496 897611 1253 0 0 327935 131 0 0 22 0 5 0 653562056 1052418048 228374 33554432000 4194304 4417452 140736924725824 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/25587/statm: 256938 228374 180 55 0 254824 0
[pid=25587/tid=25598] ppid=25585 vsize=1027752 CPUtime=822.2 cores=1,3,5,7
/proc/25587/task/25598/stat : 25598 (plingeling) R 25585 25587 25544 0 -1 4202560 35467 1253 0 0 82217 3 0 0 25 0 5 0 653562067 1052418048 228374 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25587/tid=25599] ppid=25585 vsize=1027752 CPUtime=822.2 cores=1,3,5,7
/proc/25587/task/25599/stat : 25599 (plingeling) R 25585 25587 25544 0 -1 4202560 33413 1253 0 0 82216 4 0 0 25 0 5 0 653562067 1052418048 228374 33554432000 4194304 4417452 140736924725824 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25587/tid=25600] ppid=25585 vsize=1027752 CPUtime=822.2 cores=1,3,5,7
/proc/25587/task/25600/stat : 25600 (plingeling) R 25585 25587 25544 0 -1 4202560 182106 1253 0 0 82198 22 0 0 25 0 5 0 653562067 1052418048 228374 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364629 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25587/tid=25601] ppid=25585 vsize=1027752 CPUtime=813.93 cores=1,3,5,7
/proc/25587/task/25601/stat : 25601 (plingeling) R 25585 25587 25544 0 -1 4202560 642086 1253 0 0 81293 100 0 0 25 0 5 0 653562067 1052418048 228374 33554432000 4194304 4417452 140736924725824 18446744073709551615 4366655 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3280.66
Current children cumulated vsize (KiB) 1027752

[startup+882.3 s]
/proc/loadavg: 8.08 8.09 7.96 10/333 25624
/proc/meminfo: memFree=23381528/32951124 swapFree=44485764/67111528
[pid=25587] ppid=25585 vsize=1121812 CPUtime=3520.05 cores=1,3,5,7
/proc/25587/stat : 25587 (plingeling) S 25585 25587 25544 0 -1 4202496 933315 1253 0 0 351868 137 0 0 22 0 5 0 653562056 1148735488 251889 33554432000 4194304 4417452 140736924725824 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/25587/statm: 280453 251889 180 55 0 278339 0
[pid=25587/tid=25598] ppid=25585 vsize=1121812 CPUtime=882.2 cores=1,3,5,7
/proc/25587/task/25598/stat : 25598 (plingeling) R 25585 25587 25544 0 -1 4202560 35467 1253 0 0 88217 3 0 0 25 0 5 0 653562067 1148735488 251889 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364821 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25587/tid=25599] ppid=25585 vsize=1121812 CPUtime=882.2 cores=1,3,5,7
/proc/25587/task/25599/stat : 25599 (plingeling) R 25585 25587 25544 0 -1 4202560 33413 1253 0 0 88216 4 0 0 25 0 5 0 653562067 1148735488 251889 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364508 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25587/tid=25600] ppid=25585 vsize=1121812 CPUtime=882.19 cores=1,3,5,7
/proc/25587/task/25600/stat : 25600 (plingeling) R 25585 25587 25544 0 -1 4202560 193645 1253 0 0 88196 23 0 0 25 0 5 0 653562067 1148735488 251889 33554432000 4194304 4417452 140736924725824 18446744073709551615 4352536 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25587/tid=25601] ppid=25585 vsize=1121812 CPUtime=873.32 cores=1,3,5,7
/proc/25587/task/25601/stat : 25601 (plingeling) R 25585 25587 25544 0 -1 4202560 666251 1253 0 0 87227 105 0 0 25 0 5 0 653562067 1148735488 251889 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3520.05
Current children cumulated vsize (KiB) 1121812

[startup+942.301 s]
/proc/loadavg: 8.06 8.08 7.97 9/333 25626
/proc/meminfo: memFree=23471504/32951124 swapFree=44485764/67111528
[pid=25587] ppid=25585 vsize=1237864 CPUtime=3759.47 cores=1,3,5,7
/proc/25587/stat : 25587 (plingeling) S 25585 25587 25544 0 -1 4202496 970795 1253 0 0 375804 143 0 0 22 0 5 0 653562056 1267572736 280902 33554432000 4194304 4417452 140736924725824 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/25587/statm: 309466 280902 180 55 0 307352 0
[pid=25587/tid=25598] ppid=25585 vsize=1237864 CPUtime=942.2 cores=1,3,5,7
/proc/25587/task/25598/stat : 25598 (plingeling) R 25585 25587 25544 0 -1 4202560 35467 1253 0 0 94217 3 0 0 25 0 5 0 653562067 1267572736 280902 33554432000 4194304 4417452 140736924725824 18446744073709551615 4263213 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25587/tid=25599] ppid=25585 vsize=1237864 CPUtime=942.2 cores=1,3,5,7
/proc/25587/task/25599/stat : 25599 (plingeling) R 25585 25587 25544 0 -1 4202560 33413 1253 0 0 94216 4 0 0 25 0 5 0 653562067 1267572736 280902 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25587/tid=25600] ppid=25585 vsize=1237864 CPUtime=942.19 cores=1,3,5,7
/proc/25587/task/25600/stat : 25600 (plingeling) R 25585 25587 25544 0 -1 4202560 200962 1253 0 0 94195 24 0 0 25 0 5 0 653562067 1267572736 280902 33554432000 4194304 4417452 140736924725824 18446744073709551615 4236630 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25587/tid=25601] ppid=25585 vsize=1237864 CPUtime=932.72 cores=1,3,5,7
/proc/25587/task/25601/stat : 25601 (plingeling) R 25585 25587 25544 0 -1 4202560 696414 1253 0 0 93163 109 0 0 25 0 5 0 653562067 1267572736 280902 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3759.47
Current children cumulated vsize (KiB) 1237864

[startup+1002.3 s]
/proc/loadavg: 8.10 8.09 7.98 9/333 25628
/proc/meminfo: memFree=23429920/32951124 swapFree=44485764/67111528
[pid=25587] ppid=25585 vsize=1291968 CPUtime=3998.86 cores=1,3,5,7
/proc/25587/stat : 25587 (plingeling) S 25585 25587 25544 0 -1 4202496 995894 1253 0 0 399739 147 0 0 22 0 5 0 653562056 1322975232 294428 33554432000 4194304 4417452 140736924725824 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/25587/statm: 322992 294428 180 55 0 320878 0
[pid=25587/tid=25598] ppid=25585 vsize=1291968 CPUtime=1002.21 cores=1,3,5,7
/proc/25587/task/25598/stat : 25598 (plingeling) R 25585 25587 25544 0 -1 4202560 38805 1253 0 0 100217 4 0 0 25 0 5 0 653562067 1322975232 294428 33554432000 4194304 4417452 140736924725824 18446744073709551615 4353557 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25587/tid=25599] ppid=25585 vsize=1291968 CPUtime=1002.2 cores=1,3,5,7
/proc/25587/task/25599/stat : 25599 (plingeling) R 25585 25587 25544 0 -1 4202560 33413 1253 0 0 100216 4 0 0 25 0 5 0 653562067 1322975232 294428 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364829 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25587/tid=25600] ppid=25585 vsize=1291968 CPUtime=1002.2 cores=1,3,5,7
/proc/25587/task/25600/stat : 25600 (plingeling) R 25585 25587 25544 0 -1 4202560 212641 1253 0 0 100194 26 0 0 25 0 5 0 653562067 1322975232 294428 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25587/tid=25601] ppid=25585 vsize=1291968 CPUtime=992.11 cores=1,3,5,7
/proc/25587/task/25601/stat : 25601 (plingeling) R 25585 25587 25544 0 -1 4202560 706496 1253 0 0 99100 111 0 0 25 0 5 0 653562067 1322975232 294428 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3998.86
Current children cumulated vsize (KiB) 1291968

[startup+1062.3 s]
/proc/loadavg: 8.07 8.09 7.98 9/333 25629
/proc/meminfo: memFree=23359680/32951124 swapFree=44485764/67111528
[pid=25587] ppid=25585 vsize=1299340 CPUtime=4238.26 cores=1,3,5,7
/proc/25587/stat : 25587 (plingeling) S 25585 25587 25544 0 -1 4202496 1006671 1253 0 0 423678 148 0 0 22 0 5 0 653562056 1330524160 296271 33554432000 4194304 4417452 140736924725824 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/25587/statm: 324835 296271 180 55 0 322721 0
[pid=25587/tid=25598] ppid=25585 vsize=1299340 CPUtime=1062.21 cores=1,3,5,7
/proc/25587/task/25598/stat : 25598 (plingeling) R 25585 25587 25544 0 -1 4202560 38805 1253 0 0 106217 4 0 0 25 0 5 0 653562067 1330524160 296271 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25587/tid=25599] ppid=25585 vsize=1299340 CPUtime=1062.21 cores=1,3,5,7
/proc/25587/task/25599/stat : 25599 (plingeling) R 25585 25587 25544 0 -1 4202560 33413 1253 0 0 106217 4 0 0 25 0 5 0 653562067 1330524160 296271 33554432000 4194304 4417452 140736924725824 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25587/tid=25600] ppid=25585 vsize=1299340 CPUtime=1062.21 cores=1,3,5,7
/proc/25587/task/25600/stat : 25600 (plingeling) R 25585 25587 25544 0 -1 4202560 223418 1253 0 0 106194 27 0 0 25 0 5 0 653562067 1330524160 296271 33554432000 4194304 4417452 140736924725824 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25587/tid=25601] ppid=25585 vsize=1299340 CPUtime=1051.5 cores=1,3,5,7
/proc/25587/task/25601/stat : 25601 (plingeling) R 25585 25587 25544 0 -1 4202560 706496 1253 0 0 105039 111 0 0 25 0 5 0 653562067 1330524160 296271 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4238.26
Current children cumulated vsize (KiB) 1299340

[startup+1122.3 s]
/proc/loadavg: 8.09 8.10 7.99 9/333 25631
/proc/meminfo: memFree=22879876/32951124 swapFree=44485764/67111528
[pid=25587] ppid=25585 vsize=1636740 CPUtime=4477.66 cores=1,3,5,7
/proc/25587/stat : 25587 (plingeling) S 25585 25587 25544 0 -1 4202496 1110797 1253 0 0 447601 165 0 0 22 0 5 0 653562056 1676021760 380621 33554432000 4194304 4417452 140736924725824 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/25587/statm: 409185 380621 180 55 0 407071 0
[pid=25587/tid=25598] ppid=25585 vsize=1636740 CPUtime=1122.22 cores=1,3,5,7
/proc/25587/task/25598/stat : 25598 (plingeling) R 25585 25587 25544 0 -1 4202560 46902 1253 0 0 112217 5 0 0 25 0 5 0 653562067 1676021760 380621 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364542 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25587/tid=25599] ppid=25585 vsize=1636740 CPUtime=1122.21 cores=1,3,5,7
/proc/25587/task/25599/stat : 25599 (plingeling) R 25585 25587 25544 0 -1 4202560 56144 1253 0 0 112215 6 0 0 25 0 5 0 653562067 1676021760 380621 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25587/tid=25600] ppid=25585 vsize=1636740 CPUtime=1122.2 cores=1,3,5,7
/proc/25587/task/25600/stat : 25600 (plingeling) R 25585 25587 25544 0 -1 4202560 235804 1253 0 0 112192 28 0 0 25 0 5 0 653562067 1676021760 380621 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25587/tid=25601] ppid=25585 vsize=1636740 CPUtime=1110.89 cores=1,3,5,7
/proc/25587/task/25601/stat : 25601 (plingeling) R 25585 25587 25544 0 -1 4202560 767408 1253 0 0 110965 124 0 0 25 0 5 0 653562067 1676021760 380621 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4477.66
Current children cumulated vsize (KiB) 1636740

[startup+1182.3 s]
/proc/loadavg: 8.22 8.12 8.01 9/333 25632
/proc/meminfo: memFree=22827356/32951124 swapFree=44485764/67111528
[pid=25587] ppid=25585 vsize=1662560 CPUtime=4717.06 cores=1,3,5,7
/proc/25587/stat : 25587 (plingeling) S 25585 25587 25544 0 -1 4202496 1138847 1253 0 0 471536 170 0 0 22 0 5 0 653562056 1702461440 387076 33554432000 4194304 4417452 140736924725824 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/25587/statm: 415640 387076 180 55 0 413526 0
[pid=25587/tid=25598] ppid=25585 vsize=1662560 CPUtime=1182.22 cores=1,3,5,7
/proc/25587/task/25598/stat : 25598 (plingeling) R 25585 25587 25544 0 -1 4202560 46902 1253 0 0 118217 5 0 0 25 0 5 0 653562067 1702461440 387076 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25587/tid=25599] ppid=25585 vsize=1662560 CPUtime=1182.21 cores=1,3,5,7
/proc/25587/task/25599/stat : 25599 (plingeling) R 25585 25587 25544 0 -1 4202560 59087 1253 0 0 118215 6 0 0 25 0 5 0 653562067 1702461440 387076 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364861 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25587/tid=25600] ppid=25585 vsize=1662560 CPUtime=1182.21 cores=1,3,5,7
/proc/25587/task/25600/stat : 25600 (plingeling) R 25585 25587 25544 0 -1 4202560 247823 1253 0 0 118191 30 0 0 25 0 5 0 653562067 1702461440 387076 33554432000 4194304 4417452 140736924725824 18446744073709551615 4366659 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25587/tid=25601] ppid=25585 vsize=1662560 CPUtime=1170.3 cores=1,3,5,7
/proc/25587/task/25601/stat : 25601 (plingeling) R 25585 25587 25544 0 -1 4202560 780496 1253 0 0 116903 127 0 0 25 0 5 0 653562067 1702461440 387076 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4717.06
Current children cumulated vsize (KiB) 1662560



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.1 s]
/proc/loadavg: 8.15 8.11 8.01 5/330 25634
/proc/meminfo: memFree=23669688/32951124 swapFree=44485764/67111528
[pid=25587] ppid=25585 vsize=1719816 CPUtime=4800.08 cores=1,3,5,7
/proc/25587/stat : 25587 (plingeling) S 25585 25587 25544 0 -1 4202496 1157676 1253 0 0 479837 171 0 0 22 0 5 0 653562056 1761091584 401390 33554432000 4194304 4417452 140736924725824 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/25587/statm: 429954 401390 180 55 0 427840 0
[pid=25587/tid=25598] ppid=25585 vsize=1719816 CPUtime=1203.03 cores=1,3,5,7
/proc/25587/task/25598/stat : 25598 (plingeling) R 25585 25587 25544 0 -1 4202560 46902 1253 0 0 120298 5 0 0 25 0 5 0 653562067 1761091584 401390 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25587/tid=25599] ppid=25585 vsize=1719816 CPUtime=1203.02 cores=1,3,5,7
/proc/25587/task/25599/stat : 25599 (plingeling) R 25585 25587 25544 0 -1 4202560 74484 1253 0 0 120294 8 0 0 25 0 5 0 653562067 1761091584 401390 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25587/tid=25600] ppid=25585 vsize=1719816 CPUtime=1203.02 cores=1,3,5,7
/proc/25587/task/25600/stat : 25600 (plingeling) R 25585 25587 25544 0 -1 4202560 251255 1253 0 0 120272 30 0 0 25 0 5 0 653562067 1761091584 401390 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364780 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25587/tid=25601] ppid=25585 vsize=1719816 CPUtime=1190.88 cores=1,3,5,7
/proc/25587/task/25601/stat : 25601 (plingeling) R 25585 25587 25544 0 -1 4202560 780496 1253 0 0 118961 127 0 0 25 0 5 0 653562067 1761091584 401390 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4800.08
Current children cumulated vsize (KiB) 1719816

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1203.1 s]
/proc/loadavg: 8.15 8.11 8.01 5/330 25634
/proc/meminfo: memFree=23669688/32951124 swapFree=44485764/67111528
[pid=25587] ppid=25585 vsize=1719816 CPUtime=4800.08 cores=1,3,5,7
/proc/25587/stat : 25587 (plingeling) S 25585 25587 25544 0 -1 4202496 1157676 1253 0 0 479837 171 0 0 22 0 5 0 653562056 1761091584 401390 33554432000 4194304 4417452 140736924725824 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/25587/statm: 429954 401390 180 55 0 427840 0
[pid=25587/tid=25598] ppid=25585 vsize=1719816 CPUtime=1203.03 cores=1,3,5,7
/proc/25587/task/25598/stat : 25598 (plingeling) R 25585 25587 25544 0 -1 4202560 46902 1253 0 0 120298 5 0 0 25 0 5 0 653562067 1761091584 401390 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25587/tid=25599] ppid=25585 vsize=1719816 CPUtime=1203.02 cores=1,3,5,7
/proc/25587/task/25599/stat : 25599 (plingeling) R 25585 25587 25544 0 -1 4202560 74484 1253 0 0 120294 8 0 0 25 0 5 0 653562067 1761091584 401390 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25587/tid=25600] ppid=25585 vsize=1719816 CPUtime=1203.02 cores=1,3,5,7
/proc/25587/task/25600/stat : 25600 (plingeling) R 25585 25587 25544 0 -1 4202560 251255 1253 0 0 120272 30 0 0 25 0 5 0 653562067 1761091584 401390 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364780 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25587/tid=25601] ppid=25585 vsize=1719816 CPUtime=1190.88 cores=1,3,5,7
/proc/25587/task/25601/stat : 25601 (plingeling) R 25585 25587 25544 0 -1 4202560 780496 1253 0 0 118961 127 0 0 25 0 5 0 653562067 1761091584 401390 33554432000 4194304 4417452 140736924725824 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4800.08
Current children cumulated vsize (KiB) 1719816

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1203.18
CPU time (s): 4800.17
CPU user time (s): 4798.4
CPU system time (s): 1.77873
CPU usage (%): 398.959
Max. virtual memory (cumulated for all children) (KiB): 1732632

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4798.4
system time used= 1.77873
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1158929
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 121
involuntary context switches= 21787

runsolver used 3.48047 second user time and 8.68668 second system time

The end

Launcher Data 

Begin job on node105 at 2011-05-15 02:33:36
IDJOB=3371155
IDBENCH=71443
IDSOLVER=1848
FILE ID=node105/3371155-1305419616
RUNJOBID= node105-1305419616-25563
PBS_JOBID= 13324579
Free space on /tmp= 73588 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT09/CRAFTED/rbsat/crafted/unforced/rbsat-v1820c171155g5.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3371155-1305419616/watcher-3371155-1305419616 -o /tmp/evaluation-result-3371155-1305419616/solver-3371155-1305419616 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3371155-1305419616.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 70a781814cd432fb0f934a7edd92ccff
RANDOM SEED=920412543

node105.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5410.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      25270900 kB
Buffers:        281764 kB
Cached:        5918748 kB
SwapCached:      39656 kB
Active:         580308 kB
Inactive:      6838564 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      25270900 kB
SwapTotal:    67111528 kB
SwapFree:     44485764 kB
Dirty:            3240 kB
Writeback:           0 kB
AnonPages:     1187392 kB
Mapped:           9928 kB
Slab:           128608 kB
PageTables:      66172 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 65347692 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73588 MiB
End job on node105 at 2011-05-15 02:53:41