Trace number 3370938

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.4 1205.19

General information on the benchmark

Namecrafted/mosoi/289/
289-unsat-13x32.cnf
MD5SUM73c8ae5bbdfe41463e8e40ea211c5e14
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables1664
Number of clauses157664
Sum of the clauses size625664
Maximum clause length4
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 22496
Number of clauses of size 30
Number of clauses of size 4155168
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3370938-1305425102/watcher-3370938-1305425102 -o /tmp/evaluation-result-3370938-1305425102/solver-3370938-1305425102 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3370938-1305425102.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.37 7.73 7.92 2/408 10687
/proc/meminfo: memFree=21399228/32950940 swapFree=67111356/67111528
[pid=10687] ppid=10685 vsize=9624 CPUtime=0 cores=0,2,4,6
/proc/10687/stat : 10687 (plingeling) R 10685 10687 9516 0 -1 4202496 482 1252 0 0 0 0 0 0 22 0 1 0 732740991 9854976 412 33554432000 4194304 4417452 140733258398480 18446744073709551615 215988592384 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/10687/statm: 2406 412 147 55 0 292 0

[startup+0.016495 s]
/proc/loadavg: 6.37 7.73 7.92 2/408 10687
/proc/meminfo: memFree=21399228/32950940 swapFree=67111356/67111528
[pid=10687] ppid=10685 vsize=12480 CPUtime=0.01 cores=0,2,4,6
/proc/10687/stat : 10687 (plingeling) R 10685 10687 9516 0 -1 4202496 1241 1252 0 0 1 0 0 0 22 0 1 0 732740991 12779520 1143 33554432000 4194304 4417452 140733258398480 18446744073709551615 4235997 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/10687/statm: 3120 1143 148 55 0 1006 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 12480

[startup+0.100521 s]
/proc/loadavg: 6.37 7.73 7.92 2/408 10687
/proc/meminfo: memFree=21399228/32950940 swapFree=67111356/67111528
[pid=10687] ppid=10685 vsize=25928 CPUtime=0.08 cores=0,2,4,6
/proc/10687/stat : 10687 (plingeling) R 10685 10687 9516 0 -1 4202496 4699 1252 0 0 8 0 0 0 22 0 1 0 732740991 26550272 4505 33554432000 4194304 4417452 140733258398480 18446744073709551615 4323376 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/10687/statm: 6482 4505 148 55 0 4368 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 25928

[startup+0.300372 s]
/proc/loadavg: 6.37 7.73 7.92 2/408 10687
/proc/meminfo: memFree=21399228/32950940 swapFree=67111356/67111528
[pid=10687] ppid=10685 vsize=80024 CPUtime=0.63 cores=0,2,4,6
/proc/10687/stat : 10687 (plingeling) S 10685 10687 9516 0 -1 4202496 9304 1252 0 0 62 1 0 0 22 0 5 0 732740991 81944576 7797 33554432000 4194304 4417452 140733258398480 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10687/statm: 20006 7797 173 55 0 17892 0
Current children cumulated CPU time (s) 0.63
Current children cumulated vsize (KiB) 80024

[startup+0.700308 s]
/proc/loadavg: 6.37 7.73 7.92 2/408 10687
/proc/meminfo: memFree=21399228/32950940 swapFree=67111356/67111528
[pid=10687] ppid=10685 vsize=109684 CPUtime=2.22 cores=0,2,4,6
/proc/10687/stat : 10687 (plingeling) S 10685 10687 9516 0 -1 4202496 17614 1252 0 0 220 2 0 0 22 0 5 0 732740991 112316416 15239 33554432000 4194304 4417452 140733258398480 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10687/statm: 27421 15239 179 55 0 25307 0
Current children cumulated CPU time (s) 2.22
Current children cumulated vsize (KiB) 109684

[startup+1.50116 s]
/proc/loadavg: 6.37 7.73 7.92 9/422 10716
/proc/meminfo: memFree=21300668/32950940 swapFree=67111356/67111528
[pid=10687] ppid=10685 vsize=143436 CPUtime=5.41 cores=0,2,4,6
/proc/10687/stat : 10687 (plingeling) S 10685 10687 9516 0 -1 4202496 28267 1252 0 0 537 4 0 0 22 0 5 0 732740991 146878464 23646 33554432000 4194304 4417452 140733258398480 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10687/statm: 35859 23646 180 55 0 33745 0
[pid=10687/tid=10698] ppid=10685 vsize=143436 CPUtime=1.3 cores=0,2,4,6
/proc/10687/task/10698/stat : 10698 (plingeling) R 10685 10687 9516 0 -1 4202560 5513 1252 0 0 130 0 0 0 25 0 5 0 732741010 146878464 23646 33554432000 4194304 4417452 140733258398480 18446744073709551615 4239395 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10687/tid=10699] ppid=10685 vsize=143436 CPUtime=1.29 cores=0,2,4,6
/proc/10687/task/10699/stat : 10699 (plingeling) R 10685 10687 9516 0 -1 4202560 5514 1252 0 0 128 1 0 0 25 0 5 0 732741010 146878464 23646 33554432000 4194304 4417452 140733258398480 18446744073709551615 4320940 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10687/tid=10700] ppid=10685 vsize=143436 CPUtime=1.3 cores=0,2,4,6
/proc/10687/task/10700/stat : 10700 (plingeling) R 10685 10687 9516 0 -1 4202560 3336 1252 0 0 130 0 0 0 25 0 5 0 732741010 146878464 23646 33554432000 4194304 4417452 140733258398480 18446744073709551615 4249209 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10687/tid=10701] ppid=10685 vsize=143436 CPUtime=1.3 cores=0,2,4,6
/proc/10687/task/10701/stat : 10701 (plingeling) R 10685 10687 9516 0 -1 4202560 5097 1252 0 0 130 0 0 0 25 0 5 0 732741010 146878464 23646 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 5.41
Current children cumulated vsize (KiB) 143436

[startup+3.10088 s]
/proc/loadavg: 6.50 7.74 7.92 9/422 10716
/proc/meminfo: memFree=21259680/32950940 swapFree=67111356/67111528
[pid=10687] ppid=10685 vsize=148340 CPUtime=11.8 cores=0,2,4,6
/proc/10687/stat : 10687 (plingeling) S 10685 10687 9516 0 -1 4202496 30918 1252 0 0 1175 5 0 0 22 0 5 0 732740991 151900160 24872 33554432000 4194304 4417452 140733258398480 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10687/statm: 37085 24872 180 55 0 34971 0
[pid=10687/tid=10698] ppid=10685 vsize=148340 CPUtime=2.9 cores=0,2,4,6
/proc/10687/task/10698/stat : 10698 (plingeling) R 10685 10687 9516 0 -1 4202560 5513 1252 0 0 290 0 0 0 25 0 5 0 732741010 151900160 24872 33554432000 4194304 4417452 140733258398480 18446744073709551615 4249110 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10687/tid=10699] ppid=10685 vsize=148340 CPUtime=2.88 cores=0,2,4,6
/proc/10687/task/10699/stat : 10699 (plingeling) R 10685 10687 9516 0 -1 4202560 5514 1252 0 0 287 1 0 0 25 0 5 0 732741010 151900160 24872 33554432000 4194304 4417452 140733258398480 18446744073709551615 4249209 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10687/tid=10700] ppid=10685 vsize=148340 CPUtime=2.9 cores=0,2,4,6
/proc/10687/task/10700/stat : 10700 (plingeling) R 10685 10687 9516 0 -1 4202560 3336 1252 0 0 290 0 0 0 25 0 5 0 732741010 151900160 24872 33554432000 4194304 4417452 140733258398480 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10687/tid=10701] ppid=10685 vsize=148340 CPUtime=2.91 cores=0,2,4,6
/proc/10687/task/10701/stat : 10701 (plingeling) R 10685 10687 9516 0 -1 4202560 7748 1252 0 0 290 1 0 0 25 0 5 0 732741010 151900160 24872 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 11.8
Current children cumulated vsize (KiB) 148340

[startup+6.30032 s]
/proc/loadavg: 6.50 7.74 7.92 9/422 10824
/proc/meminfo: memFree=21294356/32950940 swapFree=67111356/67111528
[pid=10687] ppid=10685 vsize=148804 CPUtime=24.55 cores=0,2,4,6
/proc/10687/stat : 10687 (plingeling) S 10685 10687 9516 0 -1 4202496 48397 1252 0 0 2447 8 0 0 22 0 5 0 732740991 152375296 25020 33554432000 4194304 4417452 140733258398480 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10687/statm: 37201 25020 180 55 0 35087 0
[pid=10687/tid=10698] ppid=10685 vsize=148804 CPUtime=6.1 cores=0,2,4,6
/proc/10687/task/10698/stat : 10698 (plingeling) R 10685 10687 9516 0 -1 4202560 10482 1252 0 0 608 2 0 0 25 0 5 0 732741010 152375296 25020 33554432000 4194304 4417452 140733258398480 18446744073709551615 4249547 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10687/tid=10699] ppid=10685 vsize=148804 CPUtime=6.03 cores=0,2,4,6
/proc/10687/task/10699/stat : 10699 (plingeling) R 10685 10687 9516 0 -1 4202560 9228 1252 0 0 602 1 0 0 25 0 5 0 732741010 152375296 25020 33554432000 4194304 4417452 140733258398480 18446744073709551615 4254868 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10687/tid=10700] ppid=10685 vsize=148804 CPUtime=6.09 cores=0,2,4,6
/proc/10687/task/10700/stat : 10700 (plingeling) R 10685 10687 9516 0 -1 4202560 8225 1252 0 0 609 0 0 0 25 0 5 0 732741010 152375296 25020 33554432000 4194304 4417452 140733258398480 18446744073709551615 4233804 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10687/tid=10701] ppid=10685 vsize=148804 CPUtime=6.1 cores=0,2,4,6
/proc/10687/task/10701/stat : 10701 (plingeling) R 10685 10687 9516 0 -1 4202560 11655 1252 0 0 609 1 0 0 25 0 5 0 732741010 152375296 25020 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 24.55
Current children cumulated vsize (KiB) 148804

[startup+12.7012 s]
/proc/loadavg: 6.81 7.76 7.93 9/422 10824
/proc/meminfo: memFree=21160876/32950940 swapFree=67111356/67111528
[pid=10687] ppid=10685 vsize=175364 CPUtime=50.06 cores=0,2,4,6
/proc/10687/stat : 10687 (plingeling) S 10685 10687 9516 0 -1 4202496 87116 1252 0 0 4992 14 0 0 22 0 5 0 732740991 179572736 31628 33554432000 4194304 4417452 140733258398480 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10687/statm: 43841 31628 180 55 0 41727 0
[pid=10687/tid=10698] ppid=10685 vsize=175364 CPUtime=12.5 cores=0,2,4,6
/proc/10687/task/10698/stat : 10698 (plingeling) R 10685 10687 9516 0 -1 4202560 21472 1252 0 0 1246 4 0 0 25 0 5 0 732741010 179572736 31628 33554432000 4194304 4417452 140733258398480 18446744073709551615 4249978 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10687/tid=10699] ppid=10685 vsize=175364 CPUtime=12.34 cores=0,2,4,6
/proc/10687/task/10699/stat : 10699 (plingeling) R 10685 10687 9516 0 -1 4202560 18583 1252 0 0 1232 2 0 0 25 0 5 0 732741010 179572736 31628 33554432000 4194304 4417452 140733258398480 18446744073709551615 4260546 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10687/tid=10700] ppid=10685 vsize=175364 CPUtime=12.49 cores=0,2,4,6
/proc/10687/task/10700/stat : 10700 (plingeling) R 10685 10687 9516 0 -1 4202560 17294 1252 0 0 1247 2 0 0 25 0 5 0 732741010 179572736 31628 33554432000 4194304 4417452 140733258398480 18446744073709551615 4253513 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10687/tid=10701] ppid=10685 vsize=175364 CPUtime=12.51 cores=0,2,4,6
/proc/10687/task/10701/stat : 10701 (plingeling) R 10685 10687 9516 0 -1 4202560 20960 1252 0 0 1248 3 0 0 25 0 5 0 732741010 179572736 31628 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 50.06
Current children cumulated vsize (KiB) 175364

[startup+25.501 s]
/proc/loadavg: 6.99 7.77 7.93 9/422 10825
/proc/meminfo: memFree=20952692/32950940 swapFree=67111356/67111528
[pid=10687] ppid=10685 vsize=298828 CPUtime=101.08 cores=0,2,4,6
/proc/10687/stat : 10687 (plingeling) S 10685 10687 9516 0 -1 4202496 128771 1252 0 0 10088 20 0 0 22 0 5 0 732740991 305999872 46143 33554432000 4194304 4417452 140733258398480 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10687/statm: 74707 46143 180 55 0 72593 0
[pid=10687/tid=10698] ppid=10685 vsize=298828 CPUtime=25.31 cores=0,2,4,6
/proc/10687/task/10698/stat : 10698 (plingeling) R 10685 10687 9516 0 -1 4202560 33088 1252 0 0 2525 6 0 0 25 0 5 0 732741010 305999872 46143 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10687/tid=10699] ppid=10685 vsize=298828 CPUtime=24.96 cores=0,2,4,6
/proc/10687/task/10699/stat : 10699 (plingeling) R 10685 10687 9516 0 -1 4202560 26966 1252 0 0 2493 3 0 0 25 0 5 0 732741010 305999872 46143 33554432000 4194304 4417452 140733258398480 18446744073709551615 4354506 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10687/tid=10700] ppid=10685 vsize=298828 CPUtime=25.29 cores=0,2,4,6
/proc/10687/task/10700/stat : 10700 (plingeling) R 10685 10687 9516 0 -1 4202560 30956 1252 0 0 2525 4 0 0 25 0 5 0 732741010 305999872 46143 33554432000 4194304 4417452 140733258398480 18446744073709551615 4357204 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10687/tid=10701] ppid=10685 vsize=298828 CPUtime=25.3 cores=0,2,4,6
/proc/10687/task/10701/stat : 10701 (plingeling) R 10685 10687 9516 0 -1 4202560 28954 1252 0 0 2526 4 0 0 25 0 5 0 732741010 305999872 46143 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 101.08
Current children cumulated vsize (KiB) 298828

[startup+51.1005 s]
/proc/loadavg: 7.33 7.78 7.93 9/422 10825
/proc/meminfo: memFree=20639368/32950940 swapFree=67111356/67111528
[pid=10687] ppid=10685 vsize=374392 CPUtime=203.13 cores=0,2,4,6
/proc/10687/stat : 10687 (plingeling) S 10685 10687 9516 0 -1 4202496 220659 1252 0 0 20280 33 0 0 22 0 5 0 732740991 383377408 65034 33554432000 4194304 4417452 140733258398480 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10687/statm: 93598 65034 180 55 0 91484 0
[pid=10687/tid=10698] ppid=10685 vsize=374392 CPUtime=50.9 cores=0,2,4,6
/proc/10687/task/10698/stat : 10698 (plingeling) R 10685 10687 9516 0 -1 4202560 60687 1252 0 0 5081 9 0 0 25 0 5 0 732741010 383377408 65034 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10687/tid=10699] ppid=10685 vsize=374392 CPUtime=50.23 cores=0,2,4,6
/proc/10687/task/10699/stat : 10699 (plingeling) R 10685 10687 9516 0 -1 4202560 45494 1252 0 0 5017 6 0 0 25 0 5 0 732741010 383377408 65034 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10687/tid=10700] ppid=10685 vsize=374392 CPUtime=50.89 cores=0,2,4,6
/proc/10687/task/10700/stat : 10700 (plingeling) R 10685 10687 9516 0 -1 4202560 56756 1252 0 0 5081 8 0 0 25 0 5 0 732741010 383377408 65034 33554432000 4194304 4417452 140733258398480 18446744073709551615 4249222 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10687/tid=10701] ppid=10685 vsize=374392 CPUtime=50.91 cores=0,2,4,6
/proc/10687/task/10701/stat : 10701 (plingeling) R 10685 10687 9516 0 -1 4202560 48915 1252 0 0 5083 8 0 0 25 0 5 0 732741010 383377408 65034 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 203.13
Current children cumulated vsize (KiB) 374392

[startup+102.316 s]
/proc/loadavg: 8.00 7.88 7.95 9/422 10827
/proc/meminfo: memFree=20237332/32950940 swapFree=67111356/67111528
[pid=10687] ppid=10685 vsize=557924 CPUtime=407.28 cores=0,2,4,6
/proc/10687/stat : 10687 (plingeling) S 10685 10687 9516 0 -1 4202496 435501 1252 0 0 40662 66 0 0 22 0 5 0 732740991 571314176 110917 33554432000 4194304 4417452 140733258398480 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10687/statm: 139481 110917 180 55 0 137367 0
[pid=10687/tid=10698] ppid=10685 vsize=557924 CPUtime=102.12 cores=0,2,4,6
/proc/10687/task/10698/stat : 10698 (plingeling) R 10685 10687 9516 0 -1 4202560 129076 1252 0 0 10192 20 0 0 25 0 5 0 732741010 571314176 110917 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10687/tid=10699] ppid=10685 vsize=557924 CPUtime=100.73 cores=0,2,4,6
/proc/10687/task/10699/stat : 10699 (plingeling) R 10685 10687 9516 0 -1 4202560 91824 1252 0 0 10061 12 0 0 25 0 5 0 732741010 571314176 110917 33554432000 4194304 4417452 140733258398480 18446744073709551615 4249110 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10687/tid=10700] ppid=10685 vsize=557924 CPUtime=102.08 cores=0,2,4,6
/proc/10687/task/10700/stat : 10700 (plingeling) R 10685 10687 9516 0 -1 4202560 139874 1252 0 0 10188 20 0 0 25 0 5 0 732741010 571314176 110917 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364812 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10687/tid=10701] ppid=10685 vsize=557924 CPUtime=102.12 cores=0,2,4,6
/proc/10687/task/10701/stat : 10701 (plingeling) R 10685 10687 9516 0 -1 4202560 65920 1252 0 0 10202 10 0 0 25 0 5 0 732741010 571314176 110917 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 407.28
Current children cumulated vsize (KiB) 557924

[startup+162.301 s]
/proc/loadavg: 8.06 7.92 7.96 9/422 10828
/proc/meminfo: memFree=19726792/32950940 swapFree=67111356/67111528
[pid=10687] ppid=10685 vsize=665708 CPUtime=646.4 cores=0,2,4,6
/proc/10687/stat : 10687 (plingeling) S 10685 10687 9516 0 -1 4202496 650337 1252 0 0 64540 100 0 0 22 0 5 0 732740991 681684992 137863 33554432000 4194304 4417452 140733258398480 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10687/statm: 166427 137863 180 55 0 164313 0
[pid=10687/tid=10698] ppid=10685 vsize=665708 CPUtime=162.11 cores=0,2,4,6
/proc/10687/task/10698/stat : 10698 (plingeling) R 10685 10687 9516 0 -1 4202560 202552 1252 0 0 16178 33 0 0 25 0 5 0 732741010 681684992 137863 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364984 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10687/tid=10699] ppid=10685 vsize=665708 CPUtime=159.91 cores=0,2,4,6
/proc/10687/task/10699/stat : 10699 (plingeling) R 10685 10687 9516 0 -1 4202560 148494 1252 0 0 15970 21 0 0 25 0 5 0 732741010 681684992 137863 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10687/tid=10700] ppid=10685 vsize=665708 CPUtime=162.06 cores=0,2,4,6
/proc/10687/task/10700/stat : 10700 (plingeling) R 10685 10687 9516 0 -1 4202560 203720 1252 0 0 16176 30 0 0 25 0 5 0 732741010 681684992 137863 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10687/tid=10701] ppid=10685 vsize=665708 CPUtime=162.11 cores=0,2,4,6
/proc/10687/task/10701/stat : 10701 (plingeling) R 10685 10687 9516 0 -1 4202560 86764 1252 0 0 16197 14 0 0 25 0 5 0 732741010 681684992 137863 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 646.4
Current children cumulated vsize (KiB) 665708


################
# More data... #
################

[pid=10687/tid=10699] ppid=10685 vsize=1038352 CPUtime=751.76 cores=0,2,4,6
/proc/10687/task/10699/stat : 10699 (plingeling) R 10685 10687 9516 0 -1 4202560 497058 1252 0 0 75100 76 0 0 25 0 5 0 732741010 1063272448 231056 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10687/tid=10700] ppid=10685 vsize=1038352 CPUtime=761.94 cores=0,2,4,6
/proc/10687/task/10700/stat : 10700 (plingeling) R 10685 10687 9516 0 -1 4202560 546073 1252 0 0 76112 82 0 0 25 0 5 0 732741010 1063272448 231056 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364576 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10687/tid=10701] ppid=10685 vsize=1038352 CPUtime=762.14 cores=0,2,4,6
/proc/10687/task/10701/stat : 10701 (plingeling) R 10685 10687 9516 0 -1 4202560 263466 1252 0 0 76174 40 0 0 25 0 5 0 732741010 1063272448 231056 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3038.19
Current children cumulated vsize (KiB) 1038352

[startup+822.301 s]
/proc/loadavg: 8.13 8.09 8.02 9/422 10846
/proc/meminfo: memFree=17406100/32950940 swapFree=67111356/67111528
[pid=10687] ppid=10685 vsize=1063204 CPUtime=3277.29 cores=0,2,4,6
/proc/10687/stat : 10687 (plingeling) S 10685 10687 9516 0 -1 4202496 2142552 1252 0 0 327401 328 0 0 22 0 5 0 732740991 1088720896 237269 33554432000 4194304 4417452 140733258398480 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10687/statm: 265801 237269 180 55 0 263687 0
[pid=10687/tid=10698] ppid=10685 vsize=1063204 CPUtime=822.12 cores=0,2,4,6
/proc/10687/task/10698/stat : 10698 (plingeling) R 10685 10687 9516 0 -1 4202560 770768 1252 0 0 82093 119 0 0 25 0 5 0 732741010 1088720896 237269 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364724 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10687/tid=10699] ppid=10685 vsize=1063204 CPUtime=810.93 cores=0,2,4,6
/proc/10687/task/10699/stat : 10699 (plingeling) R 10685 10687 9516 0 -1 4202560 540862 1252 0 0 81012 81 0 0 25 0 5 0 732741010 1088720896 237269 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364765 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10687/tid=10700] ppid=10685 vsize=1063204 CPUtime=821.91 cores=0,2,4,6
/proc/10687/task/10700/stat : 10700 (plingeling) R 10685 10687 9516 0 -1 4202560 558649 1252 0 0 82106 85 0 0 25 0 5 0 732741010 1088720896 237269 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10687/tid=10701] ppid=10685 vsize=1063204 CPUtime=822.12 cores=0,2,4,6
/proc/10687/task/10701/stat : 10701 (plingeling) R 10685 10687 9516 0 -1 4202560 263466 1252 0 0 82172 40 0 0 25 0 5 0 732741010 1088720896 237269 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3277.29
Current children cumulated vsize (KiB) 1063204

[startup+882.301 s]
/proc/loadavg: 8.19 8.10 8.02 9/422 10847
/proc/meminfo: memFree=16857140/32950940 swapFree=67111356/67111528
[pid=10687] ppid=10685 vsize=1304640 CPUtime=3516.46 cores=0,2,4,6
/proc/10687/stat : 10687 (plingeling) S 10685 10687 9516 0 -1 4202496 2266036 1252 0 0 351301 345 0 0 22 0 5 0 732740991 1335951360 297596 33554432000 4194304 4417452 140733258398480 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10687/statm: 326160 297596 180 55 0 324046 0
[pid=10687/tid=10698] ppid=10685 vsize=1304640 CPUtime=882.12 cores=0,2,4,6
/proc/10687/task/10698/stat : 10698 (plingeling) R 10685 10687 9516 0 -1 4202560 808709 1252 0 0 88088 124 0 0 25 0 5 0 732741010 1335951360 297596 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10687/tid=10699] ppid=10685 vsize=1304640 CPUtime=870.11 cores=0,2,4,6
/proc/10687/task/10699/stat : 10699 (plingeling) R 10685 10687 9516 0 -1 4202560 567071 1252 0 0 86925 86 0 0 25 0 5 0 732741010 1335951360 297596 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364604 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10687/tid=10700] ppid=10685 vsize=1304640 CPUtime=881.9 cores=0,2,4,6
/proc/10687/task/10700/stat : 10700 (plingeling) R 10685 10687 9516 0 -1 4202560 593605 1252 0 0 88100 90 0 0 25 0 5 0 732741010 1335951360 297596 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10687/tid=10701] ppid=10685 vsize=1304640 CPUtime=882.12 cores=0,2,4,6
/proc/10687/task/10701/stat : 10701 (plingeling) R 10685 10687 9516 0 -1 4202560 287844 1252 0 0 88169 43 0 0 25 0 5 0 732741010 1335951360 297596 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3516.46
Current children cumulated vsize (KiB) 1304640

[startup+942.301 s]
/proc/loadavg: 8.13 8.10 8.02 9/422 10849
/proc/meminfo: memFree=16832440/32950940 swapFree=67111356/67111528
[pid=10687] ppid=10685 vsize=1170132 CPUtime=3755.65 cores=0,2,4,6
/proc/10687/stat : 10687 (plingeling) S 10685 10687 9516 0 -1 4202496 2344366 1252 0 0 375206 359 0 0 22 0 5 0 732740991 1198215168 263969 33554432000 4194304 4417452 140733258398480 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10687/statm: 292533 263969 180 55 0 290419 0
[pid=10687/tid=10698] ppid=10685 vsize=1170132 CPUtime=942.12 cores=0,2,4,6
/proc/10687/task/10698/stat : 10698 (plingeling) R 10685 10687 9516 0 -1 4202560 837519 1252 0 0 94084 128 0 0 25 0 5 0 732741010 1198215168 263969 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10687/tid=10699] ppid=10685 vsize=1170132 CPUtime=929.31 cores=0,2,4,6
/proc/10687/task/10699/stat : 10699 (plingeling) R 10685 10687 9516 0 -1 4202560 596551 1252 0 0 92840 91 0 0 25 0 5 0 732741010 1198215168 263969 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10687/tid=10700] ppid=10685 vsize=1170132 CPUtime=941.88 cores=0,2,4,6
/proc/10687/task/10700/stat : 10700 (plingeling) R 10685 10687 9516 0 -1 4202560 610003 1252 0 0 94096 92 0 0 25 0 5 0 732741010 1198215168 263969 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10687/tid=10701] ppid=10685 vsize=1170132 CPUtime=942.12 cores=0,2,4,6
/proc/10687/task/10701/stat : 10701 (plingeling) R 10685 10687 9516 0 -1 4202560 291486 1252 0 0 94168 44 0 0 25 0 5 0 732741010 1198215168 263969 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3755.65
Current children cumulated vsize (KiB) 1170132

[startup+1002.3 s]
/proc/loadavg: 8.29 8.14 8.04 9/422 10851
/proc/meminfo: memFree=16174720/32950940 swapFree=67111356/67111528
[pid=10687] ppid=10685 vsize=1292188 CPUtime=3994.8 cores=0,2,4,6
/proc/10687/stat : 10687 (plingeling) S 10685 10687 9516 0 -1 4202496 2385200 1252 0 0 399115 365 0 0 22 0 5 0 732740991 1323200512 294483 33554432000 4194304 4417452 140733258398480 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10687/statm: 323047 294483 180 55 0 320933 0
[pid=10687/tid=10698] ppid=10685 vsize=1292188 CPUtime=1002.12 cores=0,2,4,6
/proc/10687/task/10698/stat : 10698 (plingeling) R 10685 10687 9516 0 -1 4202560 849440 1252 0 0 100082 130 0 0 25 0 5 0 732741010 1323200512 294483 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10687/tid=10699] ppid=10685 vsize=1292188 CPUtime=988.48 cores=0,2,4,6
/proc/10687/task/10699/stat : 10699 (plingeling) R 10685 10687 9516 0 -1 4202560 618270 1252 0 0 98754 94 0 0 25 0 5 0 732741010 1323200512 294483 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10687/tid=10700] ppid=10685 vsize=1292188 CPUtime=1001.86 cores=0,2,4,6
/proc/10687/task/10700/stat : 10700 (plingeling) R 10685 10687 9516 0 -1 4202560 617197 1252 0 0 100092 94 0 0 25 0 5 0 732741010 1323200512 294483 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10687/tid=10701] ppid=10685 vsize=1292188 CPUtime=1002.13 cores=0,2,4,6
/proc/10687/task/10701/stat : 10701 (plingeling) R 10685 10687 9516 0 -1 4202560 291486 1252 0 0 100169 44 0 0 25 0 5 0 732741010 1323200512 294483 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3994.8
Current children cumulated vsize (KiB) 1292188

[startup+1062.3 s]
/proc/loadavg: 8.80 8.30 8.09 10/430 12068
/proc/meminfo: memFree=16592184/32950940 swapFree=67111356/67111528
[pid=10687] ppid=10685 vsize=1341280 CPUtime=4233.19 cores=0,2,4,6
/proc/10687/stat : 10687 (plingeling) S 10685 10687 9516 0 -1 4202496 2431462 1252 0 0 422944 375 0 0 22 0 5 0 732740991 1373470720 306756 33554432000 4194304 4417452 140733258398480 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10687/statm: 335320 306756 180 55 0 333206 0
[pid=10687/tid=10698] ppid=10685 vsize=1341280 CPUtime=1061.94 cores=0,2,4,6
/proc/10687/task/10698/stat : 10698 (plingeling) R 10685 10687 9516 0 -1 4202560 870770 1252 0 0 106060 134 0 0 25 0 5 0 732741010 1373470720 306756 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10687/tid=10699] ppid=10685 vsize=1341280 CPUtime=1047.54 cores=0,2,4,6
/proc/10687/task/10699/stat : 10699 (plingeling) R 10685 10687 9516 0 -1 4202560 620622 1252 0 0 104659 95 0 0 25 0 5 0 732741010 1373470720 306756 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10687/tid=10700] ppid=10685 vsize=1341280 CPUtime=1061.74 cores=0,2,4,6
/proc/10687/task/10700/stat : 10700 (plingeling) R 10685 10687 9516 0 -1 4202560 639777 1252 0 0 106075 99 0 0 25 0 5 0 732741010 1373470720 306756 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10687/tid=10701] ppid=10685 vsize=1341280 CPUtime=1061.75 cores=0,2,4,6
/proc/10687/task/10701/stat : 10701 (plingeling) R 10685 10687 9516 0 -1 4202560 291486 1252 0 0 106131 44 0 0 25 0 5 0 732741010 1373470720 306756 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4233.19
Current children cumulated vsize (KiB) 1341280

[startup+1122.3 s]
/proc/loadavg: 9.03 8.45 8.16 10/430 13816
/proc/meminfo: memFree=16569696/32950940 swapFree=67111356/67111528
[pid=10687] ppid=10685 vsize=1453964 CPUtime=4471.47 cores=0,2,4,6
/proc/10687/stat : 10687 (plingeling) S 10685 10687 9516 0 -1 4202496 2519572 1252 0 0 446759 388 0 0 22 0 5 0 732740991 1488859136 334927 33554432000 4194304 4417452 140733258398480 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10687/statm: 363491 334927 180 55 0 361377 0
[pid=10687/tid=10698] ppid=10685 vsize=1453964 CPUtime=1121.8 cores=0,2,4,6
/proc/10687/task/10698/stat : 10698 (plingeling) R 10685 10687 9516 0 -1 4202560 885745 1252 0 0 112044 136 0 0 25 0 5 0 732741010 1488859136 334927 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10687/tid=10699] ppid=10685 vsize=1453964 CPUtime=1106.54 cores=0,2,4,6
/proc/10687/task/10699/stat : 10699 (plingeling) R 10685 10687 9516 0 -1 4202560 639948 1252 0 0 110554 100 0 0 25 0 5 0 732741010 1488859136 334927 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10687/tid=10700] ppid=10685 vsize=1453964 CPUtime=1121.53 cores=0,2,4,6
/proc/10687/task/10700/stat : 10700 (plingeling) R 10685 10687 9516 0 -1 4202560 674226 1252 0 0 112050 103 0 0 25 0 5 0 732741010 1488859136 334927 33554432000 4194304 4417452 140733258398480 18446744073709551615 4343986 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10687/tid=10701] ppid=10685 vsize=1453964 CPUtime=1121.39 cores=0,2,4,6
/proc/10687/task/10701/stat : 10701 (plingeling) R 10685 10687 9516 0 -1 4202560 310846 1252 0 0 112092 47 0 0 25 0 5 0 732741010 1488859136 334927 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4471.47
Current children cumulated vsize (KiB) 1453964

[startup+1182.3 s]
/proc/loadavg: 9.19 8.60 8.22 10/430 15603
/proc/meminfo: memFree=16170140/32950940 swapFree=67111356/67111528
[pid=10687] ppid=10685 vsize=1492472 CPUtime=4709.74 cores=0,2,4,6
/proc/10687/stat : 10687 (plingeling) S 10685 10687 9516 0 -1 4202496 2701254 1252 0 0 470555 419 0 0 22 0 5 0 732740991 1528291328 344554 33554432000 4194304 4417452 140733258398480 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10687/statm: 373118 344554 180 55 0 371004 0
[pid=10687/tid=10698] ppid=10685 vsize=1492472 CPUtime=1181.6 cores=0,2,4,6
/proc/10687/task/10698/stat : 10698 (plingeling) R 10685 10687 9516 0 -1 4202560 918613 1252 0 0 118019 141 0 0 25 0 5 0 732741010 1528291328 344554 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10687/tid=10699] ppid=10685 vsize=1492472 CPUtime=1165.5 cores=0,2,4,6
/proc/10687/task/10699/stat : 10699 (plingeling) R 10685 10687 9516 0 -1 4202560 685259 1252 0 0 116443 107 0 0 25 0 5 0 732741010 1528291328 344554 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10687/tid=10700] ppid=10685 vsize=1492472 CPUtime=1181.36 cores=0,2,4,6
/proc/10687/task/10700/stat : 10700 (plingeling) R 10685 10687 9516 0 -1 4202560 749646 1252 0 0 118021 115 0 0 25 0 5 0 732741010 1528291328 344554 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364537 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10687/tid=10701] ppid=10685 vsize=1492472 CPUtime=1181.05 cores=0,2,4,6
/proc/10687/task/10701/stat : 10701 (plingeling) R 10685 10687 9516 0 -1 4202560 338929 1252 0 0 118053 52 0 0 25 0 5 0 732741010 1528291328 344554 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4709.74
Current children cumulated vsize (KiB) 1492472



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1205.1 s]
/proc/loadavg: 9.20 8.64 8.24 10/428 16344
/proc/meminfo: memFree=16040044/32950940 swapFree=67111356/67111528
[pid=10687] ppid=10685 vsize=1593620 CPUtime=4800.27 cores=0,2,4,6
/proc/10687/stat : 10687 (plingeling) S 10685 10687 9516 0 -1 4202496 2733428 1252 0 0 479603 424 0 0 22 0 5 0 732740991 1631866880 369841 33554432000 4194304 4417452 140733258398480 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10687/statm: 398405 369841 180 55 0 396291 0
[pid=10687/tid=10698] ppid=10685 vsize=1593620 CPUtime=1204.34 cores=0,2,4,6
/proc/10687/task/10698/stat : 10698 (plingeling) R 10685 10687 9516 0 -1 4202560 925045 1252 0 0 120292 142 0 0 25 0 5 0 732741010 1631866880 369841 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10687/tid=10699] ppid=10685 vsize=1593620 CPUtime=1187.95 cores=0,2,4,6
/proc/10687/task/10699/stat : 10699 (plingeling) R 10685 10687 9516 0 -1 4202560 709243 1252 0 0 118684 111 0 0 25 0 5 0 732741010 1631866880 369841 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10687/tid=10700] ppid=10685 vsize=1593620 CPUtime=1204.16 cores=0,2,4,6
/proc/10687/task/10700/stat : 10700 (plingeling) R 10685 10687 9516 0 -1 4202560 749646 1252 0 0 120300 116 0 0 25 0 5 0 732741010 1631866880 369841 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10687/tid=10701] ppid=10685 vsize=1593620 CPUtime=1203.64 cores=0,2,4,6
/proc/10687/task/10701/stat : 10701 (plingeling) R 10685 10687 9516 0 -1 4202560 340687 1252 0 0 120311 53 0 0 25 0 5 0 732741010 1631866880 369841 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.27
Current children cumulated vsize (KiB) 1593620

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1205.1 s]
/proc/loadavg: 9.20 8.64 8.24 10/428 16344
/proc/meminfo: memFree=16040044/32950940 swapFree=67111356/67111528
[pid=10687] ppid=10685 vsize=1593620 CPUtime=4800.27 cores=0,2,4,6
/proc/10687/stat : 10687 (plingeling) S 10685 10687 9516 0 -1 4202496 2733428 1252 0 0 479603 424 0 0 22 0 5 0 732740991 1631866880 369841 33554432000 4194304 4417452 140733258398480 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10687/statm: 398405 369841 180 55 0 396291 0
[pid=10687/tid=10698] ppid=10685 vsize=1593620 CPUtime=1204.34 cores=0,2,4,6
/proc/10687/task/10698/stat : 10698 (plingeling) R 10685 10687 9516 0 -1 4202560 925045 1252 0 0 120292 142 0 0 25 0 5 0 732741010 1631866880 369841 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10687/tid=10699] ppid=10685 vsize=1593620 CPUtime=1187.95 cores=0,2,4,6
/proc/10687/task/10699/stat : 10699 (plingeling) R 10685 10687 9516 0 -1 4202560 709243 1252 0 0 118684 111 0 0 25 0 5 0 732741010 1631866880 369841 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10687/tid=10700] ppid=10685 vsize=1593620 CPUtime=1204.16 cores=0,2,4,6
/proc/10687/task/10700/stat : 10700 (plingeling) R 10685 10687 9516 0 -1 4202560 749646 1252 0 0 120300 116 0 0 25 0 5 0 732741010 1631866880 369841 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10687/tid=10701] ppid=10685 vsize=1593620 CPUtime=1203.64 cores=0,2,4,6
/proc/10687/task/10701/stat : 10701 (plingeling) R 10685 10687 9516 0 -1 4202560 340687 1252 0 0 120311 53 0 0 25 0 5 0 732741010 1631866880 369841 33554432000 4194304 4417452 140733258398480 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.27
Current children cumulated vsize (KiB) 1593620

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1205.19
CPU time (s): 4800.4
CPU user time (s): 4796.08
CPU system time (s): 4.32134
CPU usage (%): 398.311
Max. virtual memory (cumulated for all children) (KiB): 1640056

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4796.08
system time used= 4.32134
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2734680
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 164
involuntary context switches= 30108

runsolver used 4.85926 second user time and 11.5702 second system time

The end

Launcher Data 

Begin job on node117 at 2011-05-15 04:05:02
IDJOB=3370938
IDBENCH=83159
IDSOLVER=1848
FILE ID=node117/3370938-1305425102
RUNJOBID= node117-1305425102-10641
PBS_JOBID= 13324567
Free space on /tmp= 73508 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/crafted/mosoi/289/289-unsat-13x32.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3370938-1305425102/watcher-3370938-1305425102 -o /tmp/evaluation-result-3370938-1305425102/solver-3370938-1305425102 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3370938-1305425102.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 73c8ae5bbdfe41463e8e40ea211c5e14
RANDOM SEED=2145994035

node117.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5267.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950940 kB
MemFree:      21399384 kB
Buffers:        302820 kB
Cached:        3022840 kB
SwapCached:          0 kB
Active:        9351444 kB
Inactive:      1852604 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950940 kB
LowFree:      21399384 kB
SwapTotal:    67111528 kB
SwapFree:     67111356 kB
Dirty:            3576 kB
Writeback:           0 kB
AnonPages:     7878328 kB
Mapped:          31948 kB
Slab:           237212 kB
PageTables:      44712 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586996 kB
Committed_AS: 100409072 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73508 MiB
End job on node117 at 2011-05-15 04:25:09