Trace number 3259130

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 4800.29 1201.87

General information on the benchmark

NameCRAFTED/rbsat/crafted/unforced/
rbsat-v2400c266431g9.cnf
MD5SUM35278b3ce28129761cf8ae7994f37d3f
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2400
Number of clauses266431
Sum of the clauses size535112
Maximum clause length32
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2266356
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 575

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3259130-1304054041/watcher-3259130-1304054041 -o /tmp/evaluation-result-3259130-1304054041/solver-3259130-1304054041 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3259130-1304054041.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.74 5.67 7.04 5/181 21996
/proc/meminfo: memFree=13865852/32951124 swapFree=67111528/67111528
[pid=21996] ppid=21994 vsize=9164 CPUtime=0 cores=1,3,5,7
/proc/21996/stat : 21996 (plingeling) R 21994 21996 19631 0 -1 4202496 383 1251 0 0 0 0 0 0 22 0 1 0 517005038 9383936 312 33554432000 4194304 4417332 140737265252672 18446744073709551615 270613110528 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/21996/statm: 2291 312 146 55 0 177 0

[startup+0.010698 s]
/proc/loadavg: 3.74 5.67 7.04 5/181 21996
/proc/meminfo: memFree=13865852/32951124 swapFree=67111528/67111528
[pid=21996] ppid=21994 vsize=9720 CPUtime=0 cores=1,3,5,7
/proc/21996/stat : 21996 (plingeling) R 21994 21996 19631 0 -1 4202496 506 1251 0 0 0 0 0 0 22 0 1 0 517005038 9953280 435 33554432000 4194304 4417332 140737265252672 18446744073709551615 270613110528 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/21996/statm: 2430 435 146 55 0 316 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9720

[startup+0.100686 s]
/proc/loadavg: 3.74 5.67 7.04 5/181 21996
/proc/meminfo: memFree=13865852/32951124 swapFree=67111528/67111528
[pid=21996] ppid=21994 vsize=26604 CPUtime=0.09 cores=1,3,5,7
/proc/21996/stat : 21996 (plingeling) R 21994 21996 19631 0 -1 4202496 4712 1251 0 0 9 0 0 0 22 0 1 0 517005038 27242496 4641 33554432000 4194304 4417332 140737265252672 18446744073709551615 270613110528 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/21996/statm: 6651 4641 147 55 0 4537 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 26604

[startup+0.300614 s]
/proc/loadavg: 3.74 5.67 7.04 5/181 21996
/proc/meminfo: memFree=13865852/32951124 swapFree=67111528/67111528
[pid=21996] ppid=21994 vsize=85996 CPUtime=0.65 cores=1,3,5,7
/proc/21996/stat : 21996 (plingeling) S 21994 21996 19631 0 -1 4202496 9833 1251 0 0 63 2 0 0 23 0 5 0 517005038 88059904 9312 33554432000 4194304 4417332 140737265252672 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21996/statm: 21499 9312 174 55 0 19385 0
Current children cumulated CPU time (s) 0.65
Current children cumulated vsize (KiB) 85996

[startup+0.700542 s]
/proc/loadavg: 3.74 5.67 7.04 5/181 21996
/proc/meminfo: memFree=13865852/32951124 swapFree=67111528/67111528
[pid=21996] ppid=21994 vsize=167576 CPUtime=2.24 cores=1,3,5,7
/proc/21996/stat : 21996 (plingeling) S 21994 21996 19631 0 -1 4202496 21538 1251 0 0 221 3 0 0 23 0 5 0 517005038 171597824 13299 33554432000 4194304 4417332 140737265252672 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21996/statm: 41894 13299 180 55 0 39780 0
Current children cumulated CPU time (s) 2.24
Current children cumulated vsize (KiB) 167576

[startup+1.5004 s]
/proc/loadavg: 3.74 5.67 7.04 10/186 22006
/proc/meminfo: memFree=13798888/32951124 swapFree=67111528/67111528
[pid=21996] ppid=21994 vsize=169912 CPUtime=5.43 cores=1,3,5,7
/proc/21996/stat : 21996 (plingeling) S 21994 21996 19631 0 -1 4202496 28020 1251 0 0 539 4 0 0 23 0 5 0 517005038 173989888 13883 33554432000 4194304 4417332 140737265252672 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21996/statm: 42478 13883 180 55 0 40364 0
[pid=21996/tid=22003] ppid=21994 vsize=169912 CPUtime=1.31 cores=1,3,5,7
/proc/21996/task/22003/stat : 22003 (plingeling) R 21994 21996 19631 0 -1 4202560 3360 1251 0 0 131 0 0 0 25 0 5 0 517005056 173989888 13883 33554432000 4194304 4417332 140737265252672 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=21996/tid=22004] ppid=21994 vsize=169912 CPUtime=1.31 cores=1,3,5,7
/proc/21996/task/22004/stat : 22004 (plingeling) R 21994 21996 19631 0 -1 4202560 2780 1251 0 0 131 0 0 0 25 0 5 0 517005056 173989888 13883 33554432000 4194304 4417332 140737265252672 18446744073709551615 4249186 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21996/tid=22005] ppid=21994 vsize=169912 CPUtime=1.31 cores=1,3,5,7
/proc/21996/task/22005/stat : 22005 (plingeling) R 21994 21996 19631 0 -1 4202560 3371 1251 0 0 131 0 0 0 25 0 5 0 517005056 173989888 13883 33554432000 4194304 4417332 140737265252672 18446744073709551615 4249597 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21996/tid=22006] ppid=21994 vsize=169912 CPUtime=1.3 cores=1,3,5,7
/proc/21996/task/22006/stat : 22006 (plingeling) R 21994 21996 19631 0 -1 4202560 9689 1251 0 0 129 1 0 0 25 0 5 0 517005056 173989888 13883 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364713 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 5.43
Current children cumulated vsize (KiB) 169912

[startup+3.10112 s]
/proc/loadavg: 3.74 5.67 7.04 10/186 22006
/proc/meminfo: memFree=13785496/32951124 swapFree=67111528/67111528
[pid=21996] ppid=21994 vsize=196112 CPUtime=11.83 cores=1,3,5,7
/proc/21996/stat : 21996 (plingeling) S 21994 21996 19631 0 -1 4202496 38992 1251 0 0 1177 6 0 0 23 0 5 0 517005038 200818688 20434 33554432000 4194304 4417332 140737265252672 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21996/statm: 49028 20434 181 55 0 46914 0
[pid=21996/tid=22003] ppid=21994 vsize=196112 CPUtime=2.91 cores=1,3,5,7
/proc/21996/task/22003/stat : 22003 (plingeling) R 21994 21996 19631 0 -1 4202560 4598 1251 0 0 291 0 0 0 25 0 5 0 517005056 200818688 20434 33554432000 4194304 4417332 140737265252672 18446744073709551615 4250254 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=21996/tid=22004] ppid=21994 vsize=196112 CPUtime=2.91 cores=1,3,5,7
/proc/21996/task/22004/stat : 22004 (plingeling) R 21994 21996 19631 0 -1 4202560 4018 1251 0 0 291 0 0 0 25 0 5 0 517005056 200818688 20434 33554432000 4194304 4417332 140737265252672 18446744073709551615 4249110 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21996/tid=22005] ppid=21994 vsize=196112 CPUtime=2.91 cores=1,3,5,7
/proc/21996/task/22005/stat : 22005 (plingeling) R 21994 21996 19631 0 -1 4202560 3372 1251 0 0 291 0 0 0 25 0 5 0 517005056 200818688 20434 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364568 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21996/tid=22006] ppid=21994 vsize=196112 CPUtime=2.9 cores=1,3,5,7
/proc/21996/task/22006/stat : 22006 (plingeling) R 21994 21996 19631 0 -1 4202560 18184 1251 0 0 287 3 0 0 25 0 5 0 517005056 200818688 20434 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 11.83
Current children cumulated vsize (KiB) 196112

[startup+6.30055 s]
/proc/loadavg: 4.08 5.70 7.04 10/186 22006
/proc/meminfo: memFree=13759772/32951124 swapFree=67111528/67111528
[pid=21996] ppid=21994 vsize=219316 CPUtime=24.62 cores=1,3,5,7
/proc/21996/stat : 21996 (plingeling) S 21994 21996 19631 0 -1 4202496 56747 1251 0 0 2453 9 0 0 23 0 5 0 517005038 224579584 25724 33554432000 4194304 4417332 140737265252672 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21996/statm: 54829 25724 181 55 0 52715 0
[pid=21996/tid=22003] ppid=21994 vsize=219316 CPUtime=6.11 cores=1,3,5,7
/proc/21996/task/22003/stat : 22003 (plingeling) R 21994 21996 19631 0 -1 4202560 4598 1251 0 0 611 0 0 0 25 0 5 0 517005056 224579584 25724 33554432000 4194304 4417332 140737265252672 18446744073709551615 4258394 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=21996/tid=22004] ppid=21994 vsize=219316 CPUtime=6.11 cores=1,3,5,7
/proc/21996/task/22004/stat : 22004 (plingeling) R 21994 21996 19631 0 -1 4202560 4018 1251 0 0 611 0 0 0 25 0 5 0 517005056 224579584 25724 33554432000 4194304 4417332 140737265252672 18446744073709551615 4249601 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21996/tid=22005] ppid=21994 vsize=219316 CPUtime=6.11 cores=1,3,5,7
/proc/21996/task/22005/stat : 22005 (plingeling) R 21994 21996 19631 0 -1 4202560 3445 1251 0 0 611 0 0 0 25 0 5 0 517005056 224579584 25724 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364919 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21996/tid=22006] ppid=21994 vsize=219316 CPUtime=6.08 cores=1,3,5,7
/proc/21996/task/22006/stat : 22006 (plingeling) R 21994 21996 19631 0 -1 4202560 35866 1251 0 0 603 5 0 0 25 0 5 0 517005056 224579584 25724 33554432000 4194304 4417332 140737265252672 18446744073709551615 4236564 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 24.62
Current children cumulated vsize (KiB) 219316

[startup+12.7004 s]
/proc/loadavg: 4.47 5.76 7.05 10/186 22006
/proc/meminfo: memFree=13718576/32951124 swapFree=67111528/67111528
[pid=21996] ppid=21994 vsize=238804 CPUtime=50.18 cores=1,3,5,7
/proc/21996/stat : 21996 (plingeling) S 21994 21996 19631 0 -1 4202496 89023 1251 0 0 5005 13 0 0 23 0 5 0 517005038 244535296 31107 33554432000 4194304 4417332 140737265252672 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21996/statm: 59701 31107 181 55 0 57587 0
[pid=21996/tid=22003] ppid=21994 vsize=238804 CPUtime=12.51 cores=1,3,5,7
/proc/21996/task/22003/stat : 22003 (plingeling) R 21994 21996 19631 0 -1 4202560 5836 1251 0 0 1251 0 0 0 25 0 5 0 517005056 244535296 31107 33554432000 4194304 4417332 140737265252672 18446744073709551615 4249222 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=21996/tid=22004] ppid=21994 vsize=238804 CPUtime=12.51 cores=1,3,5,7
/proc/21996/task/22004/stat : 22004 (plingeling) R 21994 21996 19631 0 -1 4202560 5256 1251 0 0 1251 0 0 0 25 0 5 0 517005056 244535296 31107 33554432000 4194304 4417332 140737265252672 18446744073709551615 4236017 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21996/tid=22005] ppid=21994 vsize=238804 CPUtime=12.51 cores=1,3,5,7
/proc/21996/task/22005/stat : 22005 (plingeling) R 21994 21996 19631 0 -1 4202560 3445 1251 0 0 1251 0 0 0 25 0 5 0 517005056 244535296 31107 33554432000 4194304 4417332 140737265252672 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21996/tid=22006] ppid=21994 vsize=238804 CPUtime=12.44 cores=1,3,5,7
/proc/21996/task/22006/stat : 22006 (plingeling) R 21994 21996 19631 0 -1 4202560 65666 1251 0 0 1235 9 0 0 25 0 5 0 517005056 244535296 31107 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364796 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 50.18
Current children cumulated vsize (KiB) 238804

[startup+25.5012 s]
/proc/loadavg: 5.25 5.87 7.07 10/186 22006
/proc/meminfo: memFree=13630528/32951124 swapFree=67111528/67111528
[pid=21996] ppid=21994 vsize=291836 CPUtime=101.32 cores=1,3,5,7
/proc/21996/stat : 21996 (plingeling) S 21994 21996 19631 0 -1 4202496 140435 1251 0 0 10112 20 0 0 23 0 5 0 517005038 298840064 44365 33554432000 4194304 4417332 140737265252672 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21996/statm: 72959 44365 181 55 0 70845 0
[pid=21996/tid=22003] ppid=21994 vsize=291836 CPUtime=25.31 cores=1,3,5,7
/proc/21996/task/22003/stat : 22003 (plingeling) R 21994 21996 19631 0 -1 4202560 7442 1251 0 0 2531 0 0 0 25 0 5 0 517005056 298840064 44365 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=21996/tid=22004] ppid=21994 vsize=291836 CPUtime=25.31 cores=1,3,5,7
/proc/21996/task/22004/stat : 22004 (plingeling) R 21994 21996 19631 0 -1 4202560 5976 1251 0 0 2531 0 0 0 25 0 5 0 517005056 298840064 44365 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21996/tid=22005] ppid=21994 vsize=291836 CPUtime=25.31 cores=1,3,5,7
/proc/21996/task/22005/stat : 22005 (plingeling) R 21994 21996 19631 0 -1 4202560 5123 1251 0 0 2531 0 0 0 25 0 5 0 517005056 298840064 44365 33554432000 4194304 4417332 140737265252672 18446744073709551615 4235359 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21996/tid=22006] ppid=21994 vsize=291836 CPUtime=25.19 cores=1,3,5,7
/proc/21996/task/22006/stat : 22006 (plingeling) R 21994 21996 19631 0 -1 4202560 113074 1251 0 0 2503 16 0 0 25 0 5 0 517005056 298840064 44365 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364608 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 101.32
Current children cumulated vsize (KiB) 291836

[startup+51.1006 s]
/proc/loadavg: 6.19 6.04 7.09 10/186 22007
/proc/meminfo: memFree=13438916/32951124 swapFree=67111528/67111528
[pid=21996] ppid=21994 vsize=433224 CPUtime=203.58 cores=1,3,5,7
/proc/21996/stat : 21996 (plingeling) S 21994 21996 19631 0 -1 4202496 212279 1251 0 0 20328 30 0 0 23 0 5 0 517005038 443621376 79720 33554432000 4194304 4417332 140737265252672 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21996/statm: 108306 79720 181 55 0 106192 0
[pid=21996/tid=22003] ppid=21994 vsize=433224 CPUtime=50.91 cores=1,3,5,7
/proc/21996/task/22003/stat : 22003 (plingeling) R 21994 21996 19631 0 -1 4202560 8816 1251 0 0 5091 0 0 0 25 0 5 0 517005056 443621376 79720 33554432000 4194304 4417332 140737265252672 18446744073709551615 4353484 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=21996/tid=22004] ppid=21994 vsize=433224 CPUtime=50.91 cores=1,3,5,7
/proc/21996/task/22004/stat : 22004 (plingeling) R 21994 21996 19631 0 -1 4202560 6333 1251 0 0 5091 0 0 0 25 0 5 0 517005056 443621376 79720 33554432000 4194304 4417332 140737265252672 18446744073709551615 4237211 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21996/tid=22005] ppid=21994 vsize=433224 CPUtime=50.9 cores=1,3,5,7
/proc/21996/task/22005/stat : 22005 (plingeling) R 21994 21996 19631 0 -1 4202560 7928 1251 0 0 5090 0 0 0 25 0 5 0 517005056 443621376 79720 33554432000 4194304 4417332 140737265252672 18446744073709551615 4248888 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21996/tid=22006] ppid=21994 vsize=433224 CPUtime=50.65 cores=1,3,5,7
/proc/21996/task/22006/stat : 22006 (plingeling) R 21994 21996 19631 0 -1 4202560 180382 1251 0 0 5039 26 0 0 25 0 5 0 517005056 443621376 79720 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364565 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 203.58
Current children cumulated vsize (KiB) 433224

[startup+102.307 s]
/proc/loadavg: 7.30 6.37 7.15 10/186 22009
/proc/meminfo: memFree=13379384/32951124 swapFree=67111528/67111528
[pid=21996] ppid=21994 vsize=459128 CPUtime=408.14 cores=1,3,5,7
/proc/21996/stat : 21996 (plingeling) S 21994 21996 19631 0 -1 4202496 327740 1251 0 0 40767 47 0 0 23 0 5 0 517005038 470147072 86196 33554432000 4194304 4417332 140737265252672 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21996/statm: 114782 86196 181 55 0 112668 0
[pid=21996/tid=22003] ppid=21994 vsize=459128 CPUtime=102.12 cores=1,3,5,7
/proc/21996/task/22003/stat : 22003 (plingeling) R 21994 21996 19631 0 -1 4202560 12609 1251 0 0 10211 1 0 0 25 0 5 0 517005056 470147072 86196 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=21996/tid=22004] ppid=21994 vsize=459128 CPUtime=102.12 cores=1,3,5,7
/proc/21996/task/22004/stat : 22004 (plingeling) R 21994 21996 19631 0 -1 4202560 8642 1251 0 0 10211 1 0 0 25 0 5 0 517005056 470147072 86196 33554432000 4194304 4417332 140737265252672 18446744073709551615 4366359 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21996/tid=22005] ppid=21994 vsize=459128 CPUtime=102.12 cores=1,3,5,7
/proc/21996/task/22005/stat : 22005 (plingeling) R 21994 21996 19631 0 -1 4202560 8975 1251 0 0 10211 1 0 0 25 0 5 0 517005056 470147072 86196 33554432000 4194304 4417332 140737265252672 18446744073709551615 4249110 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21996/tid=22006] ppid=21994 vsize=459128 CPUtime=101.59 cores=1,3,5,7
/proc/21996/task/22006/stat : 22006 (plingeling) R 21994 21996 19631 0 -1 4202560 288694 1251 0 0 10118 41 0 0 25 0 5 0 517005056 470147072 86196 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 408.14
Current children cumulated vsize (KiB) 459128

[startup+162.301 s]
/proc/loadavg: 7.82 6.69 7.21 10/186 22010
/proc/meminfo: memFree=12975508/32951124 swapFree=67111528/67111528
[pid=21996] ppid=21994 vsize=792864 CPUtime=647.8 cores=1,3,5,7
/proc/21996/stat : 21996 (plingeling) S 21994 21996 19631 0 -1 4202496 412146 1251 0 0 64721 59 0 0 23 0 5 0 517005038 811892736 169630 33554432000 4194304 4417332 140737265252672 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21996/statm: 198216 169630 181 55 0 196102 0
[pid=21996/tid=22003] ppid=21994 vsize=792864 CPUtime=162.11 cores=1,3,5,7
/proc/21996/task/22003/stat : 22003 (plingeling) R 21994 21996 19631 0 -1 4202560 15529 1251 0 0 16210 1 0 0 25 0 5 0 517005056 811892736 169630 33554432000 4194304 4417332 140737265252672 18446744073709551615 4356700 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=21996/tid=22004] ppid=21994 vsize=792864 CPUtime=162.11 cores=1,3,5,7
/proc/21996/task/22004/stat : 22004 (plingeling) R 21994 21996 19631 0 -1 4202560 8959 1251 0 0 16210 1 0 0 25 0 5 0 517005056 811892736 169630 33554432000 4194304 4417332 140737265252672 18446744073709551615 4277291 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21996/tid=22005] ppid=21994 vsize=792864 CPUtime=162.11 cores=1,3,5,7
/proc/21996/task/22005/stat : 22005 (plingeling) R 21994 21996 19631 0 -1 4202560 13472 1251 0 0 16210 1 0 0 25 0 5 0 517005056 811892736 169630 33554432000 4194304 4417332 140737265252672 18446744073709551615 4353463 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21996/tid=22006] ppid=21994 vsize=792864 CPUtime=161.26 cores=1,3,5,7
/proc/21996/task/22006/stat : 22006 (plingeling) R 21994 21996 19631 0 -1 4202560 365366 1251 0 0 16074 52 0 0 25 0 5 0 517005056 811892736 169630 33554432000 4194304 4417332 140737265252672 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 647.8
Current children cumulated vsize (KiB) 792864


################
# More data... #
################

[pid=21996/tid=22004] ppid=21994 vsize=1727540 CPUtime=702.11 cores=1,3,5,7
/proc/21996/task/22004/stat : 22004 (plingeling) R 21994 21996 19631 0 -1 4202560 31602 1251 0 0 70208 3 0 0 25 0 5 0 517005056 1769000960 403322 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364750 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21996/tid=22005] ppid=21994 vsize=1727540 CPUtime=702.1 cores=1,3,5,7
/proc/21996/task/22005/stat : 22005 (plingeling) R 21994 21996 19631 0 -1 4202560 35407 1251 0 0 70206 4 0 0 25 0 5 0 517005056 1769000960 403322 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364695 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21996/tid=22006] ppid=21994 vsize=1727540 CPUtime=698.32 cores=1,3,5,7
/proc/21996/task/22006/stat : 22006 (plingeling) R 21994 21996 19631 0 -1 4202560 847547 1251 0 0 69710 122 0 0 25 0 5 0 517005056 1769000960 403322 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 2804.87
Current children cumulated vsize (KiB) 1727540

[startup+762.307 s]
/proc/loadavg: 8.08 7.90 7.62 10/186 22026
/proc/meminfo: memFree=11679920/32951124 swapFree=67111528/67111528
[pid=21996] ppid=21994 vsize=1729136 CPUtime=3044.57 cores=1,3,5,7
/proc/21996/stat : 21996 (plingeling) S 21994 21996 19631 0 -1 4202496 978196 1251 0 0 304318 139 0 0 23 0 5 0 517005038 1770635264 403721 33554432000 4194304 4417332 140737265252672 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21996/statm: 432284 403721 181 55 0 430170 0
[pid=21996/tid=22003] ppid=21994 vsize=1729136 CPUtime=762.13 cores=1,3,5,7
/proc/21996/task/22003/stat : 22003 (plingeling) R 21994 21996 19631 0 -1 4202560 54421 1251 0 0 76207 6 0 0 25 0 5 0 517005056 1770635264 403721 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364526 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=21996/tid=22004] ppid=21994 vsize=1729136 CPUtime=762.12 cores=1,3,5,7
/proc/21996/task/22004/stat : 22004 (plingeling) R 21994 21996 19631 0 -1 4202560 32001 1251 0 0 76209 3 0 0 25 0 5 0 517005056 1770635264 403721 33554432000 4194304 4417332 140737265252672 18446744073709551615 4343913 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21996/tid=22005] ppid=21994 vsize=1729136 CPUtime=762.11 cores=1,3,5,7
/proc/21996/task/22005/stat : 22005 (plingeling) R 21994 21996 19631 0 -1 4202560 35407 1251 0 0 76207 4 0 0 25 0 5 0 517005056 1770635264 403721 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364776 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21996/tid=22006] ppid=21994 vsize=1729136 CPUtime=758 cores=1,3,5,7
/proc/21996/task/22006/stat : 22006 (plingeling) R 21994 21996 19631 0 -1 4202560 847547 1251 0 0 75678 122 0 0 25 0 5 0 517005056 1770635264 403721 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364487 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3044.57
Current children cumulated vsize (KiB) 1729136

[startup+822.301 s]
/proc/loadavg: 8.08 7.93 7.64 10/186 22028
/proc/meminfo: memFree=11529256/32951124 swapFree=67111528/67111528
[pid=21996] ppid=21994 vsize=1918776 CPUtime=3284.23 cores=1,3,5,7
/proc/21996/stat : 21996 (plingeling) S 21994 21996 19631 0 -1 4202496 1025606 1251 0 0 328279 144 0 0 23 0 5 0 517005038 1964826624 451131 33554432000 4194304 4417332 140737265252672 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21996/statm: 479694 451131 181 55 0 477580 0
[pid=21996/tid=22003] ppid=21994 vsize=1918776 CPUtime=822.13 cores=1,3,5,7
/proc/21996/task/22003/stat : 22003 (plingeling) R 21994 21996 19631 0 -1 4202560 54421 1251 0 0 82207 6 0 0 25 0 5 0 517005056 1964826624 451131 33554432000 4194304 4417332 140737265252672 18446744073709551615 4251028 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=21996/tid=22004] ppid=21994 vsize=1918776 CPUtime=822.11 cores=1,3,5,7
/proc/21996/task/22004/stat : 22004 (plingeling) R 21994 21996 19631 0 -1 4202560 32001 1251 0 0 82208 3 0 0 25 0 5 0 517005056 1964826624 451131 33554432000 4194304 4417332 140737265252672 18446744073709551615 4365963 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21996/tid=22005] ppid=21994 vsize=1918776 CPUtime=822.1 cores=1,3,5,7
/proc/21996/task/22005/stat : 22005 (plingeling) R 21994 21996 19631 0 -1 4202560 35407 1251 0 0 82206 4 0 0 25 0 5 0 517005056 1964826624 451131 33554432000 4194304 4417332 140737265252672 18446744073709551615 4309662 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21996/tid=22006] ppid=21994 vsize=1918776 CPUtime=817.67 cores=1,3,5,7
/proc/21996/task/22006/stat : 22006 (plingeling) R 21994 21996 19631 0 -1 4202560 894957 1251 0 0 81641 126 0 0 25 0 5 0 517005056 1964826624 451131 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3284.23
Current children cumulated vsize (KiB) 1918776

[startup+882.3 s]
/proc/loadavg: 8.12 7.97 7.67 10/186 22029
/proc/meminfo: memFree=11271824/32951124 swapFree=67111528/67111528
[pid=21996] ppid=21994 vsize=2097016 CPUtime=3523.9 cores=1,3,5,7
/proc/21996/stat : 21996 (plingeling) S 21994 21996 19631 0 -1 4202496 1070288 1251 0 0 352236 154 0 0 23 0 5 0 517005038 2147344384 495691 33554432000 4194304 4417332 140737265252672 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21996/statm: 524254 495691 181 55 0 522140 0
[pid=21996/tid=22003] ppid=21994 vsize=2097016 CPUtime=882.13 cores=1,3,5,7
/proc/21996/task/22003/stat : 22003 (plingeling) R 21994 21996 19631 0 -1 4202560 54421 1251 0 0 88207 6 0 0 25 0 5 0 517005056 2147344384 495691 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=21996/tid=22004] ppid=21994 vsize=2097016 CPUtime=882.11 cores=1,3,5,7
/proc/21996/task/22004/stat : 22004 (plingeling) R 21994 21996 19631 0 -1 4202560 32001 1251 0 0 88208 3 0 0 25 0 5 0 517005056 2147344384 495691 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21996/tid=22005] ppid=21994 vsize=2097016 CPUtime=882.09 cores=1,3,5,7
/proc/21996/task/22005/stat : 22005 (plingeling) R 21994 21996 19631 0 -1 4202560 35529 1251 0 0 88205 4 0 0 25 0 5 0 517005056 2147344384 495691 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364502 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21996/tid=22006] ppid=21994 vsize=2097016 CPUtime=877.34 cores=1,3,5,7
/proc/21996/task/22006/stat : 22006 (plingeling) R 21994 21996 19631 0 -1 4202560 939517 1251 0 0 87598 136 0 0 25 0 5 0 517005056 2147344384 495691 33554432000 4194304 4417332 140737265252672 18446744073709551615 270613168554 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3523.9
Current children cumulated vsize (KiB) 2097016

[startup+942.301 s]
/proc/loadavg: 8.12 8.00 7.70 10/186 22031
/proc/meminfo: memFree=11035348/32951124 swapFree=67111528/67111528
[pid=21996] ppid=21994 vsize=2193848 CPUtime=3763.57 cores=1,3,5,7
/proc/21996/stat : 21996 (plingeling) S 21994 21996 19631 0 -1 4202496 1094496 1251 0 0 376200 157 0 0 23 0 5 0 517005038 2246500352 519899 33554432000 4194304 4417332 140737265252672 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21996/statm: 548462 519899 181 55 0 546348 0
[pid=21996/tid=22003] ppid=21994 vsize=2193848 CPUtime=942.13 cores=1,3,5,7
/proc/21996/task/22003/stat : 22003 (plingeling) R 21994 21996 19631 0 -1 4202560 54421 1251 0 0 94207 6 0 0 25 0 5 0 517005056 2246500352 519899 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364565 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=21996/tid=22004] ppid=21994 vsize=2193848 CPUtime=942.12 cores=1,3,5,7
/proc/21996/task/22004/stat : 22004 (plingeling) R 21994 21996 19631 0 -1 4202560 32001 1251 0 0 94209 3 0 0 25 0 5 0 517005056 2246500352 519899 33554432000 4194304 4417332 140737265252672 18446744073709551615 4343913 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21996/tid=22005] ppid=21994 vsize=2193848 CPUtime=942.1 cores=1,3,5,7
/proc/21996/task/22005/stat : 22005 (plingeling) R 21994 21996 19631 0 -1 4202560 37652 1251 0 0 94205 5 0 0 25 0 5 0 517005056 2246500352 519899 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21996/tid=22006] ppid=21994 vsize=2193848 CPUtime=937.03 cores=1,3,5,7
/proc/21996/task/22006/stat : 22006 (plingeling) R 21994 21996 19631 0 -1 4202560 961602 1251 0 0 93563 140 0 0 25 0 5 0 517005056 2246500352 519899 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3763.57
Current children cumulated vsize (KiB) 2193848

[startup+1002.3 s]
/proc/loadavg: 8.12 8.03 7.73 10/186 22033
/proc/meminfo: memFree=11998580/32951124 swapFree=67111528/67111528
[pid=21996] ppid=21994 vsize=1199844 CPUtime=4003.26 cores=1,3,5,7
/proc/21996/stat : 21996 (plingeling) S 21994 21996 19631 0 -1 4202496 1168465 1251 0 0 400154 172 0 0 23 0 5 0 517005038 1228640256 271398 33554432000 4194304 4417332 140737265252672 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21996/statm: 299961 271398 181 55 0 297847 0
[pid=21996/tid=22003] ppid=21994 vsize=1199844 CPUtime=1002.13 cores=1,3,5,7
/proc/21996/task/22003/stat : 22003 (plingeling) R 21994 21996 19631 0 -1 4202560 63226 1251 0 0 100206 7 0 0 25 0 5 0 517005056 1228640256 271398 33554432000 4194304 4417332 140737265252672 18446744073709551615 4365003 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=21996/tid=22004] ppid=21994 vsize=1199844 CPUtime=1002.12 cores=1,3,5,7
/proc/21996/task/22004/stat : 22004 (plingeling) R 21994 21996 19631 0 -1 4202560 32001 1251 0 0 100209 3 0 0 25 0 5 0 517005056 1228640256 271398 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364899 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21996/tid=22005] ppid=21994 vsize=1199844 CPUtime=1002.1 cores=1,3,5,7
/proc/21996/task/22005/stat : 22005 (plingeling) R 21994 21996 19631 0 -1 4202560 37652 1251 0 0 100205 5 0 0 25 0 5 0 517005056 1228640256 271398 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21996/tid=22006] ppid=21994 vsize=1199844 CPUtime=996.7 cores=1,3,5,7
/proc/21996/task/22006/stat : 22006 (plingeling) R 21994 21996 19631 0 -1 4202560 1026766 1251 0 0 99517 153 0 0 25 0 5 0 517005056 1228640256 271398 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4003.26
Current children cumulated vsize (KiB) 1199844

[startup+1062.3 s]
/proc/loadavg: 8.11 8.05 7.75 10/186 22034
/proc/meminfo: memFree=11855228/32951124 swapFree=67111528/67111528
[pid=21996] ppid=21994 vsize=1335588 CPUtime=4242.93 cores=1,3,5,7
/proc/21996/stat : 21996 (plingeling) S 21994 21996 19631 0 -1 4202496 1202401 1251 0 0 424116 177 0 0 23 0 5 0 517005038 1367642112 305334 33554432000 4194304 4417332 140737265252672 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21996/statm: 333897 305334 181 55 0 331783 0
[pid=21996/tid=22003] ppid=21994 vsize=1335588 CPUtime=1062.03 cores=1,3,5,7
/proc/21996/task/22003/stat : 22003 (plingeling) R 21994 21996 19631 0 -1 4202560 63226 1251 0 0 106196 7 0 0 25 0 5 0 517005056 1367642112 305334 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364536 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21996/tid=22004] ppid=21994 vsize=1335588 CPUtime=1062.12 cores=1,3,5,7
/proc/21996/task/22004/stat : 22004 (plingeling) R 21994 21996 19631 0 -1 4202560 32001 1251 0 0 106209 3 0 0 25 0 5 0 517005056 1367642112 305334 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364990 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21996/tid=22005] ppid=21994 vsize=1335588 CPUtime=1062.1 cores=1,3,5,7
/proc/21996/task/22005/stat : 22005 (plingeling) R 21994 21996 19631 0 -1 4202560 37652 1251 0 0 106205 5 0 0 25 0 5 0 517005056 1367642112 305334 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21996/tid=22006] ppid=21994 vsize=1335588 CPUtime=1056.48 cores=1,3,5,7
/proc/21996/task/22006/stat : 22006 (plingeling) R 21994 21996 19631 0 -1 4202560 1060702 1251 0 0 105489 159 0 0 25 0 5 0 517005056 1367642112 305334 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4242.93
Current children cumulated vsize (KiB) 1335588

[startup+1122.3 s]
/proc/loadavg: 8.04 8.04 7.76 10/186 22036
/proc/meminfo: memFree=11565564/32951124 swapFree=67111528/67111528
[pid=21996] ppid=21994 vsize=1540216 CPUtime=4482.6 cores=1,3,5,7
/proc/21996/stat : 21996 (plingeling) S 21994 21996 19631 0 -1 4202496 1253558 1251 0 0 448077 183 0 0 23 0 5 0 517005038 1577181184 356491 33554432000 4194304 4417332 140737265252672 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21996/statm: 385054 356491 181 55 0 382940 0
[pid=21996/tid=22003] ppid=21994 vsize=1540216 CPUtime=1121.7 cores=1,3,5,7
/proc/21996/task/22003/stat : 22003 (plingeling) R 21994 21996 19631 0 -1 4202560 87711 1251 0 0 112160 10 0 0 25 0 5 0 517005056 1577181184 356491 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364732 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21996/tid=22004] ppid=21994 vsize=1540216 CPUtime=1122.12 cores=1,3,5,7
/proc/21996/task/22004/stat : 22004 (plingeling) R 21994 21996 19631 0 -1 4202560 32001 1251 0 0 112209 3 0 0 25 0 5 0 517005056 1577181184 356491 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21996/tid=22005] ppid=21994 vsize=1540216 CPUtime=1122.1 cores=1,3,5,7
/proc/21996/task/22005/stat : 22005 (plingeling) R 21994 21996 19631 0 -1 4202560 37652 1251 0 0 112205 5 0 0 25 0 5 0 517005056 1577181184 356491 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21996/tid=22006] ppid=21994 vsize=1540216 CPUtime=1116.47 cores=1,3,5,7
/proc/21996/task/22006/stat : 22006 (plingeling) R 21994 21996 19631 0 -1 4202560 1087374 1251 0 0 111485 162 0 0 25 0 5 0 517005056 1577181184 356491 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4482.6
Current children cumulated vsize (KiB) 1540216

[startup+1182.3 s]
/proc/loadavg: 8.05 8.04 7.78 10/186 22037
/proc/meminfo: memFree=11453332/32951124 swapFree=67111528/67111528
[pid=21996] ppid=21994 vsize=1573944 CPUtime=4722.28 cores=1,3,5,7
/proc/21996/stat : 21996 (plingeling) S 21994 21996 19631 0 -1 4202496 1261990 1251 0 0 472044 184 0 0 23 0 5 0 517005038 1611718656 364923 33554432000 4194304 4417332 140737265252672 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21996/statm: 393486 364923 181 55 0 391372 0
[pid=21996/tid=22003] ppid=21994 vsize=1573944 CPUtime=1181.38 cores=1,3,5,7
/proc/21996/task/22003/stat : 22003 (plingeling) R 21994 21996 19631 0 -1 4202560 87711 1251 0 0 118128 10 0 0 25 0 5 0 517005056 1611718656 364923 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21996/tid=22004] ppid=21994 vsize=1573944 CPUtime=1182.12 cores=1,3,5,7
/proc/21996/task/22004/stat : 22004 (plingeling) R 21994 21996 19631 0 -1 4202560 32001 1251 0 0 118209 3 0 0 25 0 5 0 517005056 1611718656 364923 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364536 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21996/tid=22005] ppid=21994 vsize=1573944 CPUtime=1182.1 cores=1,3,5,7
/proc/21996/task/22005/stat : 22005 (plingeling) R 21994 21996 19631 0 -1 4202560 37652 1251 0 0 118205 5 0 0 25 0 5 0 517005056 1611718656 364923 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21996/tid=22006] ppid=21994 vsize=1573944 CPUtime=1176.48 cores=1,3,5,7
/proc/21996/task/22006/stat : 22006 (plingeling) R 21994 21996 19631 0 -1 4202560 1095806 1251 0 0 117485 163 0 0 25 0 5 0 517005056 1611718656 364923 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4722.28
Current children cumulated vsize (KiB) 1573944



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.8 s]
/proc/loadavg: 8.10 8.05 7.79 5/180 22039
/proc/meminfo: memFree=12408552/32951124 swapFree=67111528/67111528
[pid=21996] ppid=21994 vsize=1573944 CPUtime=4800.19 cores=1,3,5,7
/proc/21996/stat : 21996 (plingeling) S 21994 21996 19631 0 -1 4202496 1261990 1251 0 0 479835 184 0 0 23 0 5 0 517005038 1611718656 364923 33554432000 4194304 4417332 140737265252672 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/21996/statm: 393486 364923 181 55 0 391372 0
[pid=21996/tid=22003] ppid=21994 vsize=1573944 CPUtime=1200.77 cores=1,3,5,7
/proc/21996/task/22003/stat : 22003 (plingeling) R 21994 21996 19631 0 -1 4202560 87711 1251 0 0 120067 10 0 0 25 0 5 0 517005056 1611718656 364923 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21996/tid=22004] ppid=21994 vsize=1573944 CPUtime=1201.63 cores=1,3,5,7
/proc/21996/task/22004/stat : 22004 (plingeling) R 21994 21996 19631 0 -1 4202560 32001 1251 0 0 120160 3 0 0 25 0 5 0 517005056 1611718656 364923 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364536 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21996/tid=22005] ppid=21994 vsize=1573944 CPUtime=1201.61 cores=1,3,5,7
/proc/21996/task/22005/stat : 22005 (plingeling) R 21994 21996 19631 0 -1 4202560 37652 1251 0 0 120156 5 0 0 25 0 5 0 517005056 1611718656 364923 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21996/tid=22006] ppid=21994 vsize=1573944 CPUtime=1195.98 cores=1,3,5,7
/proc/21996/task/22006/stat : 22006 (plingeling) R 21994 21996 19631 0 -1 4202560 1095806 1251 0 0 119435 163 0 0 25 0 5 0 517005056 1611718656 364923 33554432000 4194304 4417332 140737265252672 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4800.19
Current children cumulated vsize (KiB) 1573944

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1201.87
CPU time (s): 4800.29
CPU user time (s): 4798.38
CPU system time (s): 1.90871
CPU usage (%): 399.401
Max. virtual memory (cumulated for all children) (KiB): 2193848

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4798.38
system time used= 1.90871
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1263241
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 161
involuntary context switches= 34577

runsolver used 1.89571 second user time and 4.63929 second system time

The end

Launcher Data 

Begin job on node108 at 2011-04-29 07:14:01
IDJOB=3259130
IDBENCH=71492
IDSOLVER=1605
FILE ID=node108/3259130-1304054041
RUNJOBID= node108-1304054041-21968
PBS_JOBID= 13170368
Free space on /tmp= 72116 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT09/CRAFTED/rbsat/crafted/unforced/rbsat-v2400c266431g9.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3259130-1304054041/watcher-3259130-1304054041 -o /tmp/evaluation-result-3259130-1304054041/solver-3259130-1304054041 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3259130-1304054041.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 35278b3ce28129761cf8ae7994f37d3f
RANDOM SEED=112602428

node108.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      13865884 kB
Buffers:       2296632 kB
Cached:       16136228 kB
SwapCached:          0 kB
Active:       10387228 kB
Inactive:      8115748 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      13865884 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:            4300 kB
Writeback:           0 kB
AnonPages:       69120 kB
Mapped:          15272 kB
Slab:           518372 kB
PageTables:       5132 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   251980 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72116 MiB
End job on node108 at 2011-04-29 07:34:05