Trace number 3372827

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40001.1 5006.44

General information on the benchmark

NameSAT03/industrial/goldberg/hard_eq_check/
c6288mul.miter.shuffled-as.sat03-346.cnf
MD5SUM9799c8db3562549113e4a12f98721530
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark195.02
Satisfiable
(Un)Satisfiability was proved
Number of variables9540
Number of clauses61421
Sum of the clauses size207840
Maximum clause length64
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 214116
Number of clauses of size 328488
Number of clauses of size 40
Number of clauses of size 518816
Number of clauses of size over 51

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3372827-1305442536/watcher-3372827-1305442536 -o /tmp/evaluation-result-3372827-1305442536/solver-3372827-1305442536 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3372827-1305442536.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.96 8.02 8.04 1/277 15207
/proc/meminfo: memFree=21466064/32951124 swapFree=66457188/67111528
[pid=15207] ppid=15205 vsize=12068 CPUtime=0 cores=0-7
/proc/15207/stat : 15207 (plingeling) R 15205 15207 14543 0 -1 4202496 1100 1252 0 0 0 0 0 0 25 0 1 0 655851755 12357632 1029 33554432000 4194304 4417452 140737057235648 18446744073709551615 232335586251 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/15207/statm: 3017 1029 149 55 0 903 0

[startup+0.0679589 s]
/proc/loadavg: 7.96 8.02 8.04 1/277 15207
/proc/meminfo: memFree=21466064/32951124 swapFree=66457188/67111528
[pid=15207] ppid=15205 vsize=23856 CPUtime=0.06 cores=0-7
/proc/15207/stat : 15207 (plingeling) R 15205 15207 14543 0 -1 4202496 4059 1252 0 0 6 0 0 0 25 0 1 0 655851755 24428544 3988 33554432000 4194304 4417452 140737057235648 18446744073709551615 4227779 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/15207/statm: 5964 3988 149 55 0 3850 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 23856

[startup+0.100955 s]
/proc/loadavg: 7.96 8.02 8.04 1/277 15207
/proc/meminfo: memFree=21466064/32951124 swapFree=66457188/67111528
[pid=15207] ppid=15205 vsize=29376 CPUtime=0.09 cores=0-7
/proc/15207/stat : 15207 (plingeling) R 15205 15207 14543 0 -1 4202496 5439 1252 0 0 9 0 0 0 25 0 1 0 655851755 30081024 5368 33554432000 4194304 4417452 140737057235648 18446744073709551615 4231321 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/15207/statm: 7344 5368 149 55 0 5230 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 29376

[startup+0.605259 s]
/proc/loadavg: 7.96 8.02 8.04 1/277 15207
/proc/meminfo: memFree=21466064/32951124 swapFree=66457188/67111528
[pid=15207] ppid=15205 vsize=187036 CPUtime=7.43 cores=0-7
/proc/15207/stat : 15207 (plingeling) S 15205 15207 14543 0 -1 4202496 13384 1252 0 0 742 1 0 0 18 0 9 0 655851755 191524864 8036 33554432000 4194304 4417452 140737057235648 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/15207/statm: 46759 8036 180 55 0 44645 0
Current children cumulated CPU time (s) 7.43
Current children cumulated vsize (KiB) 187036

[startup+1.2487 s]
/proc/loadavg: 7.96 8.02 8.04 1/277 15207
/proc/meminfo: memFree=21466064/32951124 swapFree=66457188/67111528
[pid=15207] ppid=15205 vsize=186252 CPUtime=8.23 cores=0-7
/proc/15207/stat : 15207 (plingeling) S 15205 15207 14543 0 -1 4202496 13510 1252 0 0 822 1 0 0 18 0 9 0 655851755 190722048 7870 33554432000 4194304 4417452 140737057235648 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/15207/statm: 46563 7870 180 55 0 44449 0
Current children cumulated CPU time (s) 8.23
Current children cumulated vsize (KiB) 186252

[startup+1.50065 s]
/proc/loadavg: 7.96 8.02 8.04 1/277 15207
/proc/meminfo: memFree=21466064/32951124 swapFree=66457188/67111528
[pid=15207] ppid=15205 vsize=188256 CPUtime=10.24 cores=0-7
/proc/15207/stat : 15207 (plingeling) S 15205 15207 14543 0 -1 4202496 14041 1252 0 0 1023 1 0 0 18 0 9 0 655851755 192774144 8401 33554432000 4194304 4417452 140737057235648 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/15207/statm: 47064 8401 180 55 0 44950 0
Current children cumulated CPU time (s) 10.24
Current children cumulated vsize (KiB) 188256

[startup+3.10037 s]
/proc/loadavg: 8.04 8.04 8.05 9/286 15221
/proc/meminfo: memFree=21424560/32951124 swapFree=66457188/67111528
[pid=15207] ppid=15205 vsize=198392 CPUtime=23.03 cores=0-7
/proc/15207/stat : 15207 (plingeling) S 15205 15207 14543 0 -1 4202496 18296 1252 0 0 2301 2 0 0 18 0 9 0 655851755 203153408 10935 33554432000 4194304 4417452 140737057235648 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/15207/statm: 49598 10935 180 55 0 47484 0
[pid=15207/tid=15214] ppid=15205 vsize=198392 CPUtime=2.85 cores=0-7
/proc/15207/task/15214/stat : 15214 (plingeling) R 15205 15207 14543 0 -1 4202560 1407 1252 0 0 285 0 0 0 24 0 9 0 655851779 203153408 10935 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=15207/tid=15215] ppid=15205 vsize=198392 CPUtime=2.83 cores=0-7
/proc/15207/task/15215/stat : 15215 (plingeling) R 15205 15207 14543 0 -1 4202560 1233 1252 0 0 283 0 0 0 24 0 9 0 655851779 203153408 10935 33554432000 4194304 4417452 140737057235648 18446744073709551615 4356960 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=15207/tid=15216] ppid=15205 vsize=198392 CPUtime=2.85 cores=0-7
/proc/15207/task/15216/stat : 15216 (plingeling) R 15205 15207 14543 0 -1 4202560 599 1252 0 0 285 0 0 0 23 0 9 0 655851779 203153408 10935 33554432000 4194304 4417452 140737057235648 18446744073709551615 4250977 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=15207/tid=15217] ppid=15205 vsize=198392 CPUtime=2.85 cores=0-7
/proc/15207/task/15217/stat : 15217 (plingeling) R 15205 15207 14543 0 -1 4202560 1467 1252 0 0 285 0 0 0 25 0 9 0 655851779 203153408 10935 33554432000 4194304 4417452 140737057235648 18446744073709551615 4366102 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=15207/tid=15218] ppid=15205 vsize=198392 CPUtime=2.85 cores=0-7
/proc/15207/task/15218/stat : 15218 (plingeling) R 15205 15207 14543 0 -1 4202560 1797 1252 0 0 285 0 0 0 25 0 9 0 655851779 203153408 10935 33554432000 4194304 4417452 140737057235648 18446744073709551615 4321570 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=15207/tid=15219] ppid=15205 vsize=198392 CPUtime=2.84 cores=0-7
/proc/15207/task/15219/stat : 15219 (plingeling) R 15205 15207 14543 0 -1 4202560 1870 1252 0 0 284 0 0 0 25 0 9 0 655851779 203153408 10935 33554432000 4194304 4417452 140737057235648 18446744073709551615 4353761 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=15207/tid=15220] ppid=15205 vsize=198392 CPUtime=2.84 cores=0-7
/proc/15207/task/15220/stat : 15220 (plingeling) R 15205 15207 14543 0 -1 4202560 508 1252 0 0 284 0 0 0 24 0 9 0 655851779 203153408 10935 33554432000 4194304 4417452 140737057235648 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=15207/tid=15221] ppid=15205 vsize=198392 CPUtime=2.84 cores=0-7
/proc/15207/task/15221/stat : 15221 (plingeling) R 15205 15207 14543 0 -1 4202560 1335 1252 0 0 284 0 0 0 25 0 9 0 655851780 203153408 10935 33554432000 4194304 4417452 140737057235648 18446744073709551615 4236816 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 23.03
Current children cumulated vsize (KiB) 198392

[startup+6.3108 s]
/proc/loadavg: 8.04 8.04 8.05 9/286 15221
/proc/meminfo: memFree=21409060/32951124 swapFree=66457188/67111528
[pid=15207] ppid=15205 vsize=208520 CPUtime=48.68 cores=0-7
/proc/15207/stat : 15207 (plingeling) S 15205 15207 14543 0 -1 4202496 29182 1252 0 0 4864 4 0 0 18 0 9 0 655851755 213524480 13467 33554432000 4194304 4417452 140737057235648 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/15207/statm: 52130 13467 180 55 0 50016 0
[pid=15207/tid=15214] ppid=15205 vsize=208520 CPUtime=6.06 cores=0-7
/proc/15207/task/15214/stat : 15214 (plingeling) R 15205 15207 14543 0 -1 4202560 2310 1252 0 0 606 0 0 0 25 0 9 0 655851779 213524480 13467 33554432000 4194304 4417452 140737057235648 18446744073709551615 4358080 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=15207/tid=15215] ppid=15205 vsize=208520 CPUtime=6 cores=0-7
/proc/15207/task/15215/stat : 15215 (plingeling) R 15205 15207 14543 0 -1 4202560 2856 1252 0 0 600 0 0 0 25 0 9 0 655851779 213524480 13467 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=15207/tid=15216] ppid=15205 vsize=208520 CPUtime=6.06 cores=0-7
/proc/15207/task/15216/stat : 15216 (plingeling) R 15205 15207 14543 0 -1 4202560 636 1252 0 0 606 0 0 0 25 0 9 0 655851779 213524480 13467 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=15207/tid=15217] ppid=15205 vsize=208520 CPUtime=6.06 cores=0-7
/proc/15207/task/15217/stat : 15217 (plingeling) R 15205 15207 14543 0 -1 4202560 2503 1252 0 0 606 0 0 0 25 0 9 0 655851779 213524480 13467 33554432000 4194304 4417452 140737057235648 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=15207/tid=15218] ppid=15205 vsize=208520 CPUtime=6.05 cores=0-7
/proc/15207/task/15218/stat : 15218 (plingeling) R 15205 15207 14543 0 -1 4202560 4170 1252 0 0 605 0 0 0 25 0 9 0 655851779 213524480 13467 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=15207/tid=15219] ppid=15205 vsize=208520 CPUtime=6.06 cores=0-7
/proc/15207/task/15219/stat : 15219 (plingeling) R 15205 15207 14543 0 -1 4202560 4643 1252 0 0 605 1 0 0 25 0 9 0 655851779 213524480 13467 33554432000 4194304 4417452 140737057235648 18446744073709551615 4366450 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=15207/tid=15220] ppid=15205 vsize=208520 CPUtime=6.04 cores=0-7
/proc/15207/task/15220/stat : 15220 (plingeling) R 15205 15207 14543 0 -1 4202560 1481 1252 0 0 604 0 0 0 25 0 9 0 655851779 213524480 13467 33554432000 4194304 4417452 140737057235648 18446744073709551615 4357902 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=15207/tid=15221] ppid=15205 vsize=208520 CPUtime=6.05 cores=0-7
/proc/15207/task/15221/stat : 15221 (plingeling) R 15205 15207 14543 0 -1 4202560 2503 1252 0 0 605 0 0 0 25 0 9 0 655851780 213524480 13467 33554432000 4194304 4417452 140737057235648 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 48.68
Current children cumulated vsize (KiB) 208520

[startup+12.7007 s]
/proc/loadavg: 8.04 8.04 8.05 9/286 15221
/proc/meminfo: memFree=21394956/32951124 swapFree=66457188/67111528
[pid=15207] ppid=15205 vsize=223112 CPUtime=99.75 cores=0-7
/proc/15207/stat : 15207 (plingeling) S 15205 15207 14543 0 -1 4202496 62417 1252 0 0 9963 12 0 0 18 0 9 0 655851755 228466688 17115 33554432000 4194304 4417452 140737057235648 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/15207/statm: 55778 17115 180 55 0 53664 0
[pid=15207/tid=15214] ppid=15205 vsize=223112 CPUtime=12.44 cores=0-7
/proc/15207/task/15214/stat : 15214 (plingeling) R 15205 15207 14543 0 -1 4202560 8189 1252 0 0 1243 1 0 0 25 0 9 0 655851779 228466688 17115 33554432000 4194304 4417452 140737057235648 18446744073709551615 4251148 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=15207/tid=15215] ppid=15205 vsize=223112 CPUtime=12.35 cores=0-7
/proc/15207/task/15215/stat : 15215 (plingeling) R 15205 15207 14543 0 -1 4202560 6476 1252 0 0 1234 1 0 0 25 0 9 0 655851779 228466688 17115 33554432000 4194304 4417452 140737057235648 18446744073709551615 18446744073699066034 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=15207/tid=15216] ppid=15205 vsize=223112 CPUtime=12.45 cores=0-7
/proc/15207/task/15216/stat : 15216 (plingeling) R 15205 15207 14543 0 -1 4202560 3611 1252 0 0 1245 0 0 0 25 0 9 0 655851779 228466688 17115 33554432000 4194304 4417452 140737057235648 18446744073709551615 4366090 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=15207/tid=15217] ppid=15205 vsize=223112 CPUtime=12.45 cores=0-7
/proc/15207/task/15217/stat : 15217 (plingeling) R 15205 15207 14543 0 -1 4202560 5653 1252 0 0 1244 1 0 0 25 0 9 0 655851779 228466688 17115 33554432000 4194304 4417452 140737057235648 18446744073709551615 4250786 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=15207/tid=15218] ppid=15205 vsize=223112 CPUtime=12.44 cores=0-7
/proc/15207/task/15218/stat : 15218 (plingeling) R 15205 15207 14543 0 -1 4202560 9420 1252 0 0 1242 2 0 0 25 0 9 0 655851779 228466688 17115 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=15207/tid=15219] ppid=15205 vsize=223112 CPUtime=12.44 cores=0-7
/proc/15207/task/15219/stat : 15219 (plingeling) R 15205 15207 14543 0 -1 4202560 10829 1252 0 0 1242 2 0 0 25 0 9 0 655851779 228466688 17115 33554432000 4194304 4417452 140737057235648 18446744073709551615 4358082 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=15207/tid=15220] ppid=15205 vsize=223112 CPUtime=12.43 cores=0-7
/proc/15207/task/15220/stat : 15220 (plingeling) R 15205 15207 14543 0 -1 4202560 3166 1252 0 0 1243 0 0 0 25 0 9 0 655851779 228466688 17115 33554432000 4194304 4417452 140737057235648 18446744073709551615 4365195 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=15207/tid=15221] ppid=15205 vsize=223112 CPUtime=12.44 cores=0-7
/proc/15207/task/15221/stat : 15221 (plingeling) R 15205 15207 14543 0 -1 4202560 6993 1252 0 0 1243 1 0 0 25 0 9 0 655851780 228466688 17115 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 99.75
Current children cumulated vsize (KiB) 223112

[startup+25.5004 s]
/proc/loadavg: 8.10 8.05 8.05 9/286 15222
/proc/meminfo: memFree=21348104/32951124 swapFree=66457188/67111528
[pid=15207] ppid=15205 vsize=273984 CPUtime=202.02 cores=0-7
/proc/15207/stat : 15207 (plingeling) S 15205 15207 14543 0 -1 4202496 128738 1252 0 0 20178 24 0 0 18 0 9 0 655851755 280559616 29845 33554432000 4194304 4417452 140737057235648 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/15207/statm: 68496 29845 180 55 0 66382 0
[pid=15207/tid=15214] ppid=15205 vsize=273984 CPUtime=25.25 cores=0-7
/proc/15207/task/15214/stat : 15214 (plingeling) R 15205 15207 14543 0 -1 4202560 17410 1252 0 0 2522 3 0 0 25 0 9 0 655851779 280559616 29845 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=15207/tid=15215] ppid=15205 vsize=273984 CPUtime=25.02 cores=0-7
/proc/15207/task/15215/stat : 15215 (plingeling) R 15205 15207 14543 0 -1 4202560 13220 1252 0 0 2500 2 0 0 25 0 9 0 655851779 280559616 29845 33554432000 4194304 4417452 140737057235648 18446744073709551615 4365936 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=15207/tid=15216] ppid=15205 vsize=273984 CPUtime=25.24 cores=0-7
/proc/15207/task/15216/stat : 15216 (plingeling) R 15205 15207 14543 0 -1 4202560 9898 1252 0 0 2523 1 0 0 25 0 9 0 655851779 280559616 29845 33554432000 4194304 4417452 140737057235648 18446744073709551615 4357879 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=15207/tid=15217] ppid=15205 vsize=273984 CPUtime=25.24 cores=0-7
/proc/15207/task/15217/stat : 15217 (plingeling) R 15205 15207 14543 0 -1 4202560 13490 1252 0 0 2522 2 0 0 25 0 9 0 655851779 280559616 29845 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=15207/tid=15218] ppid=15205 vsize=273984 CPUtime=25.25 cores=0-7
/proc/15207/task/15218/stat : 15218 (plingeling) R 15205 15207 14543 0 -1 4202560 21108 1252 0 0 2520 5 0 0 25 0 9 0 655851779 280559616 29845 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364629 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=15207/tid=15219] ppid=15205 vsize=273984 CPUtime=25.24 cores=0-7
/proc/15207/task/15219/stat : 15219 (plingeling) R 15205 15207 14543 0 -1 4202560 19679 1252 0 0 2521 3 0 0 25 0 9 0 655851779 280559616 29845 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364567 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=15207/tid=15220] ppid=15205 vsize=273984 CPUtime=25.23 cores=0-7
/proc/15207/task/15220/stat : 15220 (plingeling) R 15205 15207 14543 0 -1 4202560 8024 1252 0 0 2522 1 0 0 25 0 9 0 655851779 280559616 29845 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364807 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=15207/tid=15221] ppid=15205 vsize=273984 CPUtime=25.24 cores=0-7
/proc/15207/task/15221/stat : 15221 (plingeling) R 15205 15207 14543 0 -1 4202560 17829 1252 0 0 2522 2 0 0 25 0 9 0 655851780 280559616 29845 33554432000 4194304 4417452 140737057235648 18446744073709551615 4357265 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 202.02
Current children cumulated vsize (KiB) 273984

[startup+51.1009 s]
/proc/loadavg: 8.14 8.06 8.06 9/286 15222
/proc/meminfo: memFree=21323536/32951124 swapFree=66457188/67111528
[pid=15207] ppid=15205 vsize=310676 CPUtime=406.59 cores=0-7
/proc/15207/stat : 15207 (plingeling) S 15205 15207 14543 0 -1 4202496 235930 1252 0 0 40612 47 0 0 18 0 9 0 655851755 318132224 39018 33554432000 4194304 4417452 140737057235648 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/15207/statm: 77669 39018 180 55 0 75555 0
[pid=15207/tid=15214] ppid=15205 vsize=310676 CPUtime=50.84 cores=0-7
/proc/15207/task/15214/stat : 15214 (plingeling) R 15205 15207 14543 0 -1 4202560 30340 1252 0 0 5078 6 0 0 25 0 9 0 655851779 318132224 39018 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364958 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=15207/tid=15215] ppid=15205 vsize=310676 CPUtime=50.39 cores=0-7
/proc/15207/task/15215/stat : 15215 (plingeling) R 15205 15207 14543 0 -1 4202560 26616 1252 0 0 5034 5 0 0 25 0 9 0 655851779 318132224 39018 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=15207/tid=15216] ppid=15205 vsize=310676 CPUtime=50.85 cores=0-7
/proc/15207/task/15216/stat : 15216 (plingeling) R 15205 15207 14543 0 -1 4202560 22741 1252 0 0 5081 4 0 0 25 0 9 0 655851779 318132224 39018 33554432000 4194304 4417452 140737057235648 18446744073709551615 4366253 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=15207/tid=15217] ppid=15205 vsize=310676 CPUtime=50.84 cores=0-7
/proc/15207/task/15217/stat : 15217 (plingeling) R 15205 15207 14543 0 -1 4202560 22716 1252 0 0 5080 4 0 0 25 0 9 0 655851779 318132224 39018 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=15207/tid=15218] ppid=15205 vsize=310676 CPUtime=50.84 cores=0-7
/proc/15207/task/15218/stat : 15218 (plingeling) R 15205 15207 14543 0 -1 4202560 38906 1252 0 0 5075 9 0 0 25 0 9 0 655851779 318132224 39018 33554432000 4194304 4417452 140737057235648 18446744073709551615 4365946 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=15207/tid=15219] ppid=15205 vsize=310676 CPUtime=50.84 cores=0-7
/proc/15207/task/15219/stat : 15219 (plingeling) R 15205 15207 14543 0 -1 4202560 34168 1252 0 0 5078 6 0 0 25 0 9 0 655851779 318132224 39018 33554432000 4194304 4417452 140737057235648 18446744073709551615 4251164 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=15207/tid=15220] ppid=15205 vsize=310676 CPUtime=50.83 cores=0-7
/proc/15207/task/15220/stat : 15220 (plingeling) R 15205 15207 14543 0 -1 4202560 16411 1252 0 0 5080 3 0 0 25 0 9 0 655851779 318132224 39018 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364537 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=15207/tid=15221] ppid=15205 vsize=310676 CPUtime=50.84 cores=0-7
/proc/15207/task/15221/stat : 15221 (plingeling) R 15205 15207 14543 0 -1 4202560 35952 1252 0 0 5078 6 0 0 25 0 9 0 655851780 318132224 39018 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 406.59
Current children cumulated vsize (KiB) 310676

[startup+102.301 s]
/proc/loadavg: 8.23 8.10 8.07 9/286 15224

################
# More data... #
################


[startup+4722.3 s]
/proc/loadavg: 8.12 8.18 8.17 9/286 15371
/proc/meminfo: memFree=16685008/32951124 swapFree=66457188/67111528
[pid=15207] ppid=15205 vsize=4926960 CPUtime=37732.1 cores=0-7
/proc/15207/stat : 15207 (plingeling) S 15205 15207 14543 0 -1 4202496 5336963 1252 0 0 3772012 1196 0 0 18 0 9 0 655851755 5045207040 1193539 33554432000 4194304 4417452 140737057235648 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/15207/statm: 1231740 1193539 180 55 0 1229626 0
[pid=15207/tid=15214] ppid=15205 vsize=4926960 CPUtime=4722.15 cores=0-7
/proc/15207/task/15214/stat : 15214 (plingeling) R 15205 15207 14543 0 -1 4202560 579155 1252 0 0 472077 138 0 0 25 0 9 0 655851779 5045207040 1193539 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=15207/tid=15215] ppid=15205 vsize=4926960 CPUtime=4679.24 cores=0-7
/proc/15207/task/15215/stat : 15215 (plingeling) R 15205 15207 14543 0 -1 4202560 701077 1252 0 0 467759 165 0 0 25 0 9 0 655851779 5045207040 1193539 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=15207/tid=15216] ppid=15205 vsize=4926960 CPUtime=4721.8 cores=0-7
/proc/15207/task/15216/stat : 15216 (plingeling) R 15205 15207 14543 0 -1 4202560 847023 1252 0 0 472010 170 0 0 25 0 9 0 655851779 5045207040 1193539 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=15207/tid=15217] ppid=15205 vsize=4926960 CPUtime=4721.78 cores=0-7
/proc/15207/task/15217/stat : 15217 (plingeling) R 15205 15207 14543 0 -1 4202560 631834 1252 0 0 472036 142 0 0 25 0 9 0 655851779 5045207040 1193539 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=15207/tid=15218] ppid=15205 vsize=4926960 CPUtime=4721.47 cores=0-7
/proc/15207/task/15218/stat : 15218 (plingeling) R 15205 15207 14543 0 -1 4202560 598998 1252 0 0 472008 139 0 0 25 0 9 0 655851779 5045207040 1193539 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=15207/tid=15219] ppid=15205 vsize=4926960 CPUtime=4722.14 cores=0-7
/proc/15207/task/15219/stat : 15219 (plingeling) R 15205 15207 14543 0 -1 4202560 651591 1252 0 0 472071 143 0 0 25 0 9 0 655851779 5045207040 1193539 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=15207/tid=15220] ppid=15205 vsize=4926960 CPUtime=4721.35 cores=0-7
/proc/15207/task/15220/stat : 15220 (plingeling) R 15205 15207 14543 0 -1 4202560 655187 1252 0 0 471983 152 0 0 25 0 9 0 655851779 5045207040 1193539 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=15207/tid=15221] ppid=15205 vsize=4926960 CPUtime=4721.84 cores=0-7
/proc/15207/task/15221/stat : 15221 (plingeling) R 15205 15207 14543 0 -1 4202560 664018 1252 0 0 472041 143 0 0 25 0 9 0 655851780 5045207040 1193539 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 37732.1
Current children cumulated vsize (KiB) 4926960

[startup+4782.3 s]
/proc/loadavg: 8.16 8.17 8.17 9/286 15372
/proc/meminfo: memFree=16475064/32951124 swapFree=66457188/67111528
[pid=15207] ppid=15205 vsize=5136364 CPUtime=38211.5 cores=0-7
/proc/15207/stat : 15207 (plingeling) S 15205 15207 14543 0 -1 4202496 5389314 1252 0 0 3819941 1213 0 0 18 0 9 0 655851755 5259636736 1245890 33554432000 4194304 4417452 140737057235648 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/15207/statm: 1284091 1245890 180 55 0 1281977 0
[pid=15207/tid=15214] ppid=15205 vsize=5136364 CPUtime=4782.15 cores=0-7
/proc/15207/task/15214/stat : 15214 (plingeling) R 15205 15207 14543 0 -1 4202560 579155 1252 0 0 478077 138 0 0 25 0 9 0 655851779 5259636736 1245890 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=15207/tid=15215] ppid=15205 vsize=5136364 CPUtime=4738.7 cores=0-7
/proc/15207/task/15215/stat : 15215 (plingeling) R 15205 15207 14543 0 -1 4202560 701077 1252 0 0 473705 165 0 0 25 0 9 0 655851779 5259636736 1245890 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=15207/tid=15216] ppid=15205 vsize=5136364 CPUtime=4781.81 cores=0-7
/proc/15207/task/15216/stat : 15216 (plingeling) R 15205 15207 14543 0 -1 4202560 863407 1252 0 0 478005 176 0 0 25 0 9 0 655851779 5259636736 1245890 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=15207/tid=15217] ppid=15205 vsize=5136364 CPUtime=4781.78 cores=0-7
/proc/15207/task/15217/stat : 15217 (plingeling) R 15205 15207 14543 0 -1 4202560 631834 1252 0 0 478036 142 0 0 25 0 9 0 655851779 5259636736 1245890 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=15207/tid=15218] ppid=15205 vsize=5136364 CPUtime=4781.47 cores=0-7
/proc/15207/task/15218/stat : 15218 (plingeling) R 15205 15207 14543 0 -1 4202560 598998 1252 0 0 478008 139 0 0 25 0 9 0 655851779 5259636736 1245890 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=15207/tid=15219] ppid=15205 vsize=5136364 CPUtime=4782.14 cores=0-7
/proc/15207/task/15219/stat : 15219 (plingeling) R 15205 15207 14543 0 -1 4202560 651591 1252 0 0 478071 143 0 0 25 0 9 0 655851779 5259636736 1245890 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364669 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=15207/tid=15220] ppid=15205 vsize=5136364 CPUtime=4781.35 cores=0-7
/proc/15207/task/15220/stat : 15220 (plingeling) R 15205 15207 14543 0 -1 4202560 655187 1252 0 0 477983 152 0 0 25 0 9 0 655851779 5259636736 1245890 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364765 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=15207/tid=15221] ppid=15205 vsize=5136364 CPUtime=4781.84 cores=0-7
/proc/15207/task/15221/stat : 15221 (plingeling) R 15205 15207 14543 0 -1 4202560 699985 1252 0 0 478029 155 0 0 25 0 9 0 655851780 5259636736 1245890 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38211.5
Current children cumulated vsize (KiB) 5136364

[startup+4842.3 s]
/proc/loadavg: 8.10 8.16 8.16 9/286 15374
/proc/meminfo: memFree=16416772/32951124 swapFree=66457188/67111528
[pid=15207] ppid=15205 vsize=5194492 CPUtime=38691 cores=0-7
/proc/15207/stat : 15207 (plingeling) S 15205 15207 14543 0 -1 4202496 5403846 1252 0 0 3867882 1218 0 0 18 0 9 0 655851755 5319159808 1260422 33554432000 4194304 4417452 140737057235648 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/15207/statm: 1298623 1260422 180 55 0 1296509 0
[pid=15207/tid=15214] ppid=15205 vsize=5194492 CPUtime=4842.15 cores=0-7
/proc/15207/task/15214/stat : 15214 (plingeling) R 15205 15207 14543 0 -1 4202560 579155 1252 0 0 484077 138 0 0 25 0 9 0 655851779 5319159808 1260422 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=15207/tid=15215] ppid=15205 vsize=5194492 CPUtime=4798.16 cores=0-7
/proc/15207/task/15215/stat : 15215 (plingeling) R 15205 15207 14543 0 -1 4202560 701077 1252 0 0 479651 165 0 0 25 0 9 0 655851779 5319159808 1260422 33554432000 4194304 4417452 140737057235648 18446744073709551615 4365027 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=15207/tid=15216] ppid=15205 vsize=5194492 CPUtime=4841.8 cores=0-7
/proc/15207/task/15216/stat : 15216 (plingeling) R 15205 15207 14543 0 -1 4202560 863407 1252 0 0 484004 176 0 0 25 0 9 0 655851779 5319159808 1260422 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=15207/tid=15217] ppid=15205 vsize=5194492 CPUtime=4841.78 cores=0-7
/proc/15207/task/15217/stat : 15217 (plingeling) R 15205 15207 14543 0 -1 4202560 646366 1252 0 0 484032 146 0 0 25 0 9 0 655851779 5319159808 1260422 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=15207/tid=15218] ppid=15205 vsize=5194492 CPUtime=4841.47 cores=0-7
/proc/15207/task/15218/stat : 15218 (plingeling) R 15205 15207 14543 0 -1 4202560 598998 1252 0 0 484008 139 0 0 25 0 9 0 655851779 5319159808 1260422 33554432000 4194304 4417452 140737057235648 18446744073709551615 4365027 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=15207/tid=15219] ppid=15205 vsize=5194492 CPUtime=4842.15 cores=0-7
/proc/15207/task/15219/stat : 15219 (plingeling) R 15205 15207 14543 0 -1 4202560 651591 1252 0 0 484072 143 0 0 25 0 9 0 655851779 5319159808 1260422 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=15207/tid=15220] ppid=15205 vsize=5194492 CPUtime=4841.35 cores=0-7
/proc/15207/task/15220/stat : 15220 (plingeling) R 15205 15207 14543 0 -1 4202560 655187 1252 0 0 483983 152 0 0 25 0 9 0 655851779 5319159808 1260422 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=15207/tid=15221] ppid=15205 vsize=5194492 CPUtime=4841.84 cores=0-7
/proc/15207/task/15221/stat : 15221 (plingeling) R 15205 15207 14543 0 -1 4202560 699985 1252 0 0 484029 155 0 0 25 0 9 0 655851780 5319159808 1260422 33554432000 4194304 4417452 140737057235648 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38691
Current children cumulated vsize (KiB) 5194492

[startup+4902.3 s]
/proc/loadavg: 8.19 8.17 8.17 9/286 15376
/proc/meminfo: memFree=16416380/32951124 swapFree=66457188/67111528
[pid=15207] ppid=15205 vsize=5194492 CPUtime=39170.5 cores=0-7
/proc/15207/stat : 15207 (plingeling) S 15205 15207 14543 0 -1 4202496 5403846 1252 0 0 3915833 1218 0 0 18 0 9 0 655851755 5319159808 1260422 33554432000 4194304 4417452 140737057235648 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/15207/statm: 1298623 1260422 180 55 0 1296509 0
[pid=15207/tid=15214] ppid=15205 vsize=5194492 CPUtime=4902.16 cores=0-7
/proc/15207/task/15214/stat : 15214 (plingeling) R 15205 15207 14543 0 -1 4202560 579155 1252 0 0 490078 138 0 0 25 0 9 0 655851779 5319159808 1260422 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=15207/tid=15215] ppid=15205 vsize=5194492 CPUtime=4857.62 cores=0-7
/proc/15207/task/15215/stat : 15215 (plingeling) R 15205 15207 14543 0 -1 4202560 701077 1252 0 0 485597 165 0 0 25 0 9 0 655851779 5319159808 1260422 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364690 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=15207/tid=15216] ppid=15205 vsize=5194492 CPUtime=4901.8 cores=0-7
/proc/15207/task/15216/stat : 15216 (plingeling) R 15205 15207 14543 0 -1 4202560 863407 1252 0 0 490004 176 0 0 25 0 9 0 655851779 5319159808 1260422 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=15207/tid=15217] ppid=15205 vsize=5194492 CPUtime=4901.79 cores=0-7
/proc/15207/task/15217/stat : 15217 (plingeling) R 15205 15207 14543 0 -1 4202560 646366 1252 0 0 490033 146 0 0 25 0 9 0 655851779 5319159808 1260422 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=15207/tid=15218] ppid=15205 vsize=5194492 CPUtime=4901.49 cores=0-7
/proc/15207/task/15218/stat : 15218 (plingeling) R 15205 15207 14543 0 -1 4202560 598998 1252 0 0 490010 139 0 0 25 0 9 0 655851779 5319159808 1260422 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=15207/tid=15219] ppid=15205 vsize=5194492 CPUtime=4902.16 cores=0-7
/proc/15207/task/15219/stat : 15219 (plingeling) R 15205 15207 14543 0 -1 4202560 651591 1252 0 0 490073 143 0 0 25 0 9 0 655851779 5319159808 1260422 33554432000 4194304 4417452 140737057235648 18446744073709551615 4357984 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=15207/tid=15220] ppid=15205 vsize=5194492 CPUtime=4901.36 cores=0-7
/proc/15207/task/15220/stat : 15220 (plingeling) R 15205 15207 14543 0 -1 4202560 655187 1252 0 0 489984 152 0 0 25 0 9 0 655851779 5319159808 1260422 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=15207/tid=15221] ppid=15205 vsize=5194492 CPUtime=4901.85 cores=0-7
/proc/15207/task/15221/stat : 15221 (plingeling) R 15205 15207 14543 0 -1 4202560 699985 1252 0 0 490030 155 0 0 25 0 9 0 655851780 5319159808 1260422 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39170.5
Current children cumulated vsize (KiB) 5194492

[startup+4962.3 s]
/proc/loadavg: 8.11 8.15 8.16 9/286 15377
/proc/meminfo: memFree=16181148/32951124 swapFree=66457188/67111528
[pid=15207] ppid=15205 vsize=5429828 CPUtime=39649.9 cores=0-7
/proc/15207/stat : 15207 (plingeling) S 15205 15207 14543 0 -1 4202496 5462680 1252 0 0 3963757 1234 0 0 18 0 9 0 655851755 5560143872 1319256 33554432000 4194304 4417452 140737057235648 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/15207/statm: 1357457 1319256 180 55 0 1355343 0
[pid=15207/tid=15214] ppid=15205 vsize=5429828 CPUtime=4962.15 cores=0-7
/proc/15207/task/15214/stat : 15214 (plingeling) R 15205 15207 14543 0 -1 4202560 595539 1252 0 0 496075 140 0 0 25 0 9 0 655851779 5560143872 1319256 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=15207/tid=15215] ppid=15205 vsize=5429828 CPUtime=4917.07 cores=0-7
/proc/15207/task/15215/stat : 15215 (plingeling) R 15205 15207 14543 0 -1 4202560 701077 1252 0 0 491542 165 0 0 25 0 9 0 655851779 5560143872 1319256 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=15207/tid=15216] ppid=15205 vsize=5429828 CPUtime=4961.78 cores=0-7
/proc/15207/task/15216/stat : 15216 (plingeling) R 15205 15207 14543 0 -1 4202560 863407 1252 0 0 496002 176 0 0 25 0 9 0 655851779 5560143872 1319256 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=15207/tid=15217] ppid=15205 vsize=5429828 CPUtime=4961.78 cores=0-7
/proc/15207/task/15217/stat : 15217 (plingeling) R 15205 15207 14543 0 -1 4202560 656157 1252 0 0 496029 149 0 0 25 0 9 0 655851779 5560143872 1319256 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=15207/tid=15218] ppid=15205 vsize=5429828 CPUtime=4961.48 cores=0-7
/proc/15207/task/15218/stat : 15218 (plingeling) R 15205 15207 14543 0 -1 4202560 631657 1252 0 0 495999 149 0 0 25 0 9 0 655851779 5560143872 1319256 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=15207/tid=15219] ppid=15205 vsize=5429828 CPUtime=4962.15 cores=0-7
/proc/15207/task/15219/stat : 15219 (plingeling) R 15205 15207 14543 0 -1 4202560 651591 1252 0 0 496072 143 0 0 25 0 9 0 655851779 5560143872 1319256 33554432000 4194304 4417452 140737057235648 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=15207/tid=15220] ppid=15205 vsize=5429828 CPUtime=4961.35 cores=0-7
/proc/15207/task/15220/stat : 15220 (plingeling) R 15205 15207 14543 0 -1 4202560 655187 1252 0 0 495983 152 0 0 25 0 9 0 655851779 5560143872 1319256 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364587 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=15207/tid=15221] ppid=15205 vsize=5429828 CPUtime=4961.84 cores=0-7
/proc/15207/task/15221/stat : 15221 (plingeling) R 15205 15207 14543 0 -1 4202560 699985 1252 0 0 496029 155 0 0 25 0 9 0 655851780 5560143872 1319256 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39649.9
Current children cumulated vsize (KiB) 5429828



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5006.2 s]
/proc/loadavg: 8.25 8.17 8.17 9/286 15378
/proc/meminfo: memFree=16403104/32951124 swapFree=66457188/67111528
[pid=15207] ppid=15205 vsize=5209788 CPUtime=40000.8 cores=0-7
/proc/15207/stat : 15207 (plingeling) S 15205 15207 14543 0 -1 4202496 5543709 1252 0 0 3998822 1254 0 0 18 0 9 0 655851755 5334822912 1263877 33554432000 4194304 4417452 140737057235648 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/15207/statm: 1302447 1263877 180 55 0 1300333 0
[pid=15207/tid=15214] ppid=15205 vsize=5209788 CPUtime=5006.06 cores=0-7
/proc/15207/task/15214/stat : 15214 (plingeling) R 15205 15207 14543 0 -1 4202560 595539 1252 0 0 500466 140 0 0 25 0 9 0 655851779 5334822912 1263877 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=15207/tid=15215] ppid=15205 vsize=5209788 CPUtime=4960.57 cores=0-7
/proc/15207/task/15215/stat : 15215 (plingeling) R 15205 15207 14543 0 -1 4202560 701077 1252 0 0 495892 165 0 0 25 0 9 0 655851779 5334822912 1263877 33554432000 4194304 4417452 140737057235648 18446744073709551615 4357207 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=15207/tid=15216] ppid=15205 vsize=5209788 CPUtime=5005.68 cores=0-7
/proc/15207/task/15216/stat : 15216 (plingeling) R 15205 15207 14543 0 -1 4202560 929835 1252 0 0 500377 191 0 0 25 0 9 0 655851779 5334822912 1263877 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364583 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=15207/tid=15217] ppid=15205 vsize=5209788 CPUtime=5005.69 cores=0-7
/proc/15207/task/15217/stat : 15217 (plingeling) R 15205 15207 14543 0 -1 4202560 670758 1252 0 0 500415 154 0 0 25 0 9 0 655851779 5334822912 1263877 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=15207/tid=15218] ppid=15205 vsize=5209788 CPUtime=5005.39 cores=0-7
/proc/15207/task/15218/stat : 15218 (plingeling) R 15205 15207 14543 0 -1 4202560 631657 1252 0 0 500390 149 0 0 25 0 9 0 655851779 5334822912 1263877 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=15207/tid=15219] ppid=15205 vsize=5209788 CPUtime=5006.06 cores=0-7
/proc/15207/task/15219/stat : 15219 (plingeling) R 15205 15207 14543 0 -1 4202560 651591 1252 0 0 500463 143 0 0 25 0 9 0 655851779 5334822912 1263877 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364616 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=15207/tid=15220] ppid=15205 vsize=5209788 CPUtime=5005.25 cores=0-7
/proc/15207/task/15220/stat : 15220 (plingeling) R 15205 15207 14543 0 -1 4202560 655187 1252 0 0 500373 152 0 0 25 0 9 0 655851779 5334822912 1263877 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=15207/tid=15221] ppid=15205 vsize=5209788 CPUtime=5005.75 cores=0-7
/proc/15207/task/15221/stat : 15221 (plingeling) R 15205 15207 14543 0 -1 4202560 699985 1252 0 0 500420 155 0 0 25 0 9 0 655851780 5334822912 1263877 33554432000 4194304 4417452 140737057235648 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 40000.8
Current children cumulated vsize (KiB) 5209788

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5006.4 s]
/proc/loadavg: 8.25 8.17 8.17 9/286 15378
/proc/meminfo: memFree=16403104/32951124 swapFree=66457188/67111528
[pid=15207] ppid=15205 vsize=0 CPUtime=40001 cores=0-7
/proc/15207/stat : 15207 (plingeling) Z 15205 15207 14543 0 -1 4203548 5543709 1252 0 0 3998829 1272 0 0 18 0 2 0 655851755 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/15207/statm: 0 0 0 0 0 0 0
[pid=15207/tid=15215] ppid=15205 vsize=0 CPUtime=4960.77 cores=0-7
/proc/15207/task/15215/stat : 15215 (plingeling) R 15205 15207 14543 0 -1 4203588 701077 1252 0 0 495893 184 0 0 25 0 2 0 655851779 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40001
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5006.44
CPU time (s): 40001.1
CPU user time (s): 39988.3
CPU system time (s): 12.7671
CPU usage (%): 798.992
Max. virtual memory (cumulated for all children) (KiB): 5589160

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39988.3
system time used= 12.7671
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 5544961
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 274
involuntary context switches= 348737

runsolver used 12.5071 second user time and 33.012 second system time

The end

Launcher Data 

Begin job on node132 at 2011-05-15 08:55:36
IDJOB=3372827
IDBENCH=84205
IDSOLVER=1851
FILE ID=node132/3372827-1305442536
RUNJOBID= node132-1305437109-14557
PBS_JOBID= 13324765
Free space on /tmp= 73368 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT03/industrial/goldberg/hard_eq_check/c6288mul.miter.shuffled-as.sat03-346.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3372827-1305442536/watcher-3372827-1305442536 -o /tmp/evaluation-result-3372827-1305442536/solver-3372827-1305442536 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3372827-1305442536.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 9799c8db3562549113e4a12f98721530
RANDOM SEED=1089437249

node132.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      21466220 kB
Buffers:        291368 kB
Cached:        5642660 kB
SwapCached:       6904 kB
Active:        4811092 kB
Inactive:      6366696 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      21466220 kB
SwapTotal:    67111528 kB
SwapFree:     66457188 kB
Dirty:            1868 kB
Writeback:           0 kB
AnonPages:     5241936 kB
Mapped:          18472 kB
Slab:           217712 kB
PageTables:      23532 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 34736424 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73368 MiB
End job on node132 at 2011-05-15 10:19:04