Trace number 3395036

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40000.6 5013.22

General information on the benchmark

Namecrafted/anton/SRHD-SGI/
srhd-sgi-m52-q1041.25-n50-p15-s99099953.cnf
MD5SUMdda220d08783aabf4e7a9e99eeeccae6
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2444
Number of clauses498568
Sum of the clauses size999480
Maximum clause length52
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2498518
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 550

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3395036-1306111661/watcher-3395036-1306111661 -o /tmp/evaluation-result-3395036-1306111661/solver-3395036-1306111661 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3395036-1306111661.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.04 8.08 8.06 2/543 23076
/proc/meminfo: memFree=16251080/32951124 swapFree=54323904/67111528
[pid=23076] ppid=23074 vsize=12580 CPUtime=0.01 cores=0-7
/proc/23076/stat : 23076 (plingeling) R 23074 23076 22863 0 -1 4202496 1237 1252 0 0 1 0 0 0 23 0 1 0 722766515 12881920 1167 33554432000 4194304 4417452 140734418781376 18446744073709551615 253640859392 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/23076/statm: 3145 1167 147 55 0 1031 0

[startup+0.0656141 s]
/proc/loadavg: 8.04 8.08 8.06 2/543 23076
/proc/meminfo: memFree=16251080/32951124 swapFree=54323904/67111528
[pid=23076] ppid=23074 vsize=19668 CPUtime=0.05 cores=0-7
/proc/23076/stat : 23076 (plingeling) R 23074 23076 22863 0 -1 4202496 3162 1252 0 0 5 0 0 0 23 0 1 0 722766515 20140032 2940 33554432000 4194304 4417452 140734418781376 18446744073709551615 4231393 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/23076/statm: 4917 2940 148 55 0 2803 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 19668

[startup+0.100609 s]
/proc/loadavg: 8.04 8.08 8.06 2/543 23076
/proc/meminfo: memFree=16251080/32951124 swapFree=54323904/67111528
[pid=23076] ppid=23074 vsize=27860 CPUtime=0.09 cores=0-7
/proc/23076/stat : 23076 (plingeling) R 23074 23076 22863 0 -1 4202496 5210 1252 0 0 9 0 0 0 23 0 1 0 722766515 28528640 4988 33554432000 4194304 4417452 140734418781376 18446744073709551615 4320665 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/23076/statm: 6965 4988 148 55 0 4851 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 27860

[startup+0.300569 s]
/proc/loadavg: 8.04 8.08 8.06 2/543 23076
/proc/meminfo: memFree=16251080/32951124 swapFree=54323904/67111528
[pid=23076] ppid=23074 vsize=77012 CPUtime=0.29 cores=0-7
/proc/23076/stat : 23076 (plingeling) R 23074 23076 22863 0 -1 4202496 17498 1252 0 0 27 2 0 0 24 0 1 0 722766515 78860288 17276 33554432000 4194304 4417452 140734418781376 18446744073709551615 4231569 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/23076/statm: 19253 17276 148 55 0 17139 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 77012

[startup+0.700463 s]
/proc/loadavg: 8.04 8.08 8.06 2/543 23076
/proc/meminfo: memFree=16251080/32951124 swapFree=54323904/67111528
[pid=23076] ppid=23074 vsize=289848 CPUtime=0.75 cores=0-7
/proc/23076/stat : 23076 (plingeling) S 23074 23076 22863 0 -1 4202496 33931 1252 0 0 72 3 0 0 18 0 9 0 722766515 296804352 33617 33554432000 4194304 4417452 140734418781376 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/23076/statm: 72462 33617 166 55 0 70348 0
Current children cumulated CPU time (s) 0.75
Current children cumulated vsize (KiB) 289848

[startup+1.50132 s]
/proc/loadavg: 8.04 8.08 8.06 9/552 23090
/proc/meminfo: memFree=16112680/32951124 swapFree=54323904/67111528
[pid=23076] ppid=23074 vsize=304376 CPUtime=7.11 cores=0-7
/proc/23076/stat : 23076 (plingeling) S 23074 23076 22863 0 -1 4202496 66126 1252 0 0 700 11 0 0 18 0 9 0 722766515 311681024 37332 33554432000 4194304 4417452 140734418781376 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/23076/statm: 76094 37332 179 55 0 73980 0
[pid=23076/tid=23083] ppid=23074 vsize=304376 CPUtime=0.79 cores=0-7
/proc/23076/task/23083/stat : 23083 (plingeling) R 23074 23076 22863 0 -1 4202560 4147 1252 0 0 78 1 0 0 18 0 9 0 722766584 311681024 37332 33554432000 4194304 4417452 140734418781376 18446744073709551615 4240013 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23076/tid=23084] ppid=23074 vsize=304376 CPUtime=0.81 cores=0-7
/proc/23076/task/23084/stat : 23084 (plingeling) R 23074 23076 22863 0 -1 4202560 4134 1252 0 0 80 1 0 0 25 0 9 0 722766584 311681024 37332 33554432000 4194304 4417452 140734418781376 18446744073709551615 4254977 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23076/tid=23085] ppid=23074 vsize=304376 CPUtime=0.8 cores=0-7
/proc/23076/task/23085/stat : 23085 (plingeling) R 23074 23076 22863 0 -1 4202560 4373 1252 0 0 80 0 0 0 25 0 9 0 722766584 311681024 37332 33554432000 4194304 4417452 140734418781376 18446744073709551615 4255010 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=23076/tid=23086] ppid=23074 vsize=304376 CPUtime=0.8 cores=0-7
/proc/23076/task/23086/stat : 23086 (plingeling) R 23074 23076 22863 0 -1 4202560 6154 1252 0 0 79 1 0 0 25 0 9 0 722766584 311681024 37332 33554432000 4194304 4417452 140734418781376 18446744073709551615 4357562 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23076/tid=23087] ppid=23074 vsize=304376 CPUtime=0.79 cores=0-7
/proc/23076/task/23087/stat : 23087 (plingeling) R 23074 23076 22863 0 -1 4202560 4132 1252 0 0 79 0 0 0 25 0 9 0 722766584 311681024 37332 33554432000 4194304 4417452 140734418781376 18446744073709551615 4254990 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23076/tid=23088] ppid=23074 vsize=304376 CPUtime=0.79 cores=0-7
/proc/23076/task/23088/stat : 23088 (plingeling) R 23074 23076 22863 0 -1 4202560 2083 1252 0 0 79 0 0 0 25 0 9 0 722766585 311681024 37332 33554432000 4194304 4417452 140734418781376 18446744073709551615 4240106 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23076/tid=23089] ppid=23074 vsize=304376 CPUtime=0.8 cores=0-7
/proc/23076/task/23089/stat : 23089 (plingeling) R 23074 23076 22863 0 -1 4202560 4132 1252 0 0 79 1 0 0 18 0 9 0 722766585 311681024 37332 33554432000 4194304 4417452 140734418781376 18446744073709551615 4254990 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23076/tid=23090] ppid=23074 vsize=304376 CPUtime=0.78 cores=0-7
/proc/23076/task/23090/stat : 23090 (plingeling) R 23074 23076 22863 0 -1 4202560 3074 1252 0 0 78 0 0 0 21 0 9 0 722766585 311681024 37332 33554432000 4194304 4417452 140734418781376 18446744073709551615 4239777 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 7.11
Current children cumulated vsize (KiB) 304376

[startup+3.10102 s]
/proc/loadavg: 8.03 8.08 8.06 9/552 23090
/proc/meminfo: memFree=16077960/32951124 swapFree=54323904/67111528
[pid=23076] ppid=23074 vsize=362812 CPUtime=19.89 cores=0-7
/proc/23076/stat : 23076 (plingeling) S 23074 23076 22863 0 -1 4202496 84659 1252 0 0 1975 14 0 0 18 0 9 0 722766515 371519488 51999 33554432000 4194304 4417452 140734418781376 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/23076/statm: 90703 51999 180 55 0 88589 0
[pid=23076/tid=23083] ppid=23074 vsize=362812 CPUtime=2.39 cores=0-7
/proc/23076/task/23083/stat : 23083 (plingeling) R 23074 23076 22863 0 -1 4202560 6116 1252 0 0 238 1 0 0 20 0 9 0 722766584 371519488 51999 33554432000 4194304 4417452 140734418781376 18446744073709551615 4249584 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23076/tid=23084] ppid=23074 vsize=362812 CPUtime=2.4 cores=0-7
/proc/23076/task/23084/stat : 23084 (plingeling) R 23074 23076 22863 0 -1 4202560 5243 1252 0 0 239 1 0 0 25 0 9 0 722766584 371519488 51999 33554432000 4194304 4417452 140734418781376 18446744073709551615 4249151 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23076/tid=23085] ppid=23074 vsize=362812 CPUtime=2.4 cores=0-7
/proc/23076/task/23085/stat : 23085 (plingeling) R 23074 23076 22863 0 -1 4202560 5528 1252 0 0 240 0 0 0 25 0 9 0 722766584 371519488 51999 33554432000 4194304 4417452 140734418781376 18446744073709551615 4249193 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=23076/tid=23086] ppid=23074 vsize=362812 CPUtime=2.4 cores=0-7
/proc/23076/task/23086/stat : 23086 (plingeling) R 23074 23076 22863 0 -1 4202560 15266 1252 0 0 238 2 0 0 25 0 9 0 722766584 371519488 51999 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23076/tid=23087] ppid=23074 vsize=362812 CPUtime=2.39 cores=0-7
/proc/23076/task/23087/stat : 23087 (plingeling) R 23074 23076 22863 0 -1 4202560 6485 1252 0 0 238 1 0 0 25 0 9 0 722766584 371519488 51999 33554432000 4194304 4417452 140734418781376 18446744073709551615 4249890 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23076/tid=23088] ppid=23074 vsize=362812 CPUtime=2.39 cores=0-7
/proc/23076/task/23088/stat : 23088 (plingeling) R 23074 23076 22863 0 -1 4202560 4158 1252 0 0 239 0 0 0 25 0 9 0 722766585 371519488 51999 33554432000 4194304 4417452 140734418781376 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23076/tid=23089] ppid=23074 vsize=362812 CPUtime=2.4 cores=0-7
/proc/23076/task/23089/stat : 23089 (plingeling) R 23074 23076 22863 0 -1 4202560 4490 1252 0 0 239 1 0 0 20 0 9 0 722766585 371519488 51999 33554432000 4194304 4417452 140734418781376 18446744073709551615 4249890 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23076/tid=23090] ppid=23074 vsize=362812 CPUtime=2.36 cores=0-7
/proc/23076/task/23090/stat : 23090 (plingeling) R 23074 23076 22863 0 -1 4202560 3476 1252 0 0 236 0 0 0 25 0 9 0 722766585 371519488 51999 33554432000 4194304 4417452 140734418781376 18446744073709551615 4249209 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 19.89
Current children cumulated vsize (KiB) 362812

[startup+6.30043 s]
/proc/loadavg: 8.03 8.08 8.06 9/552 23090
/proc/meminfo: memFree=16018024/32951124 swapFree=54323904/67111528
[pid=23076] ppid=23074 vsize=374396 CPUtime=45.42 cores=0-7
/proc/23076/stat : 23076 (plingeling) S 23074 23076 22863 0 -1 4202496 97076 1252 0 0 4527 15 0 0 18 0 9 0 722766515 383381504 53872 33554432000 4194304 4417452 140734418781376 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/23076/statm: 93599 53872 180 55 0 91485 0
[pid=23076/tid=23083] ppid=23074 vsize=374396 CPUtime=5.59 cores=0-7
/proc/23076/task/23083/stat : 23083 (plingeling) R 23074 23076 22863 0 -1 4202560 6116 1252 0 0 558 1 0 0 25 0 9 0 722766584 383381504 53872 33554432000 4194304 4417452 140734418781376 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23076/tid=23084] ppid=23074 vsize=374396 CPUtime=5.6 cores=0-7
/proc/23076/task/23084/stat : 23084 (plingeling) R 23074 23076 22863 0 -1 4202560 5243 1252 0 0 559 1 0 0 25 0 9 0 722766584 383381504 53872 33554432000 4194304 4417452 140734418781376 18446744073709551615 4236296 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23076/tid=23085] ppid=23074 vsize=374396 CPUtime=5.6 cores=0-7
/proc/23076/task/23085/stat : 23085 (plingeling) R 23074 23076 22863 0 -1 4202560 5528 1252 0 0 560 0 0 0 25 0 9 0 722766584 383381504 53872 33554432000 4194304 4417452 140734418781376 18446744073709551615 4250072 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=23076/tid=23086] ppid=23074 vsize=374396 CPUtime=5.6 cores=0-7
/proc/23076/task/23086/stat : 23086 (plingeling) R 23074 23076 22863 0 -1 4202560 27683 1252 0 0 556 4 0 0 25 0 9 0 722766584 383381504 53872 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23076/tid=23087] ppid=23074 vsize=374396 CPUtime=5.59 cores=0-7
/proc/23076/task/23087/stat : 23087 (plingeling) R 23074 23076 22863 0 -1 4202560 6485 1252 0 0 558 1 0 0 25 0 9 0 722766584 383381504 53872 33554432000 4194304 4417452 140734418781376 18446744073709551615 4249222 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23076/tid=23088] ppid=23074 vsize=374396 CPUtime=5.59 cores=0-7
/proc/23076/task/23088/stat : 23088 (plingeling) R 23074 23076 22863 0 -1 4202560 4158 1252 0 0 559 0 0 0 25 0 9 0 722766585 383381504 53872 33554432000 4194304 4417452 140734418781376 18446744073709551615 4258392 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23076/tid=23089] ppid=23074 vsize=374396 CPUtime=5.6 cores=0-7
/proc/23076/task/23089/stat : 23089 (plingeling) R 23074 23076 22863 0 -1 4202560 4490 1252 0 0 559 1 0 0 25 0 9 0 722766585 383381504 53872 33554432000 4194304 4417452 140734418781376 18446744073709551615 4249222 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23076/tid=23090] ppid=23074 vsize=374396 CPUtime=5.5 cores=0-7
/proc/23076/task/23090/stat : 23090 (plingeling) R 23074 23076 22863 0 -1 4202560 3476 1252 0 0 550 0 0 0 25 0 9 0 722766585 383381504 53872 33554432000 4194304 4417452 140734418781376 18446744073709551615 4249143 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 45.42
Current children cumulated vsize (KiB) 374396

[startup+12.7013 s]
/proc/loadavg: 8.03 8.08 8.06 9/552 23090
/proc/meminfo: memFree=15989784/32951124 swapFree=54323904/67111528
[pid=23076] ppid=23074 vsize=420076 CPUtime=96.49 cores=0-7
/proc/23076/stat : 23076 (plingeling) S 23074 23076 22863 0 -1 4202496 119327 1252 0 0 9630 19 0 0 18 0 9 0 722766515 430157824 66315 33554432000 4194304 4417452 140734418781376 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/23076/statm: 105019 66315 180 55 0 102905 0
[pid=23076/tid=23083] ppid=23074 vsize=420076 CPUtime=11.99 cores=0-7
/proc/23076/task/23083/stat : 23083 (plingeling) R 23074 23076 22863 0 -1 4202560 6116 1252 0 0 1198 1 0 0 25 0 9 0 722766584 430157824 66315 33554432000 4194304 4417452 140734418781376 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23076/tid=23084] ppid=23074 vsize=420076 CPUtime=12.01 cores=0-7
/proc/23076/task/23084/stat : 23084 (plingeling) R 23074 23076 22863 0 -1 4202560 5243 1252 0 0 1200 1 0 0 25 0 9 0 722766584 430157824 66315 33554432000 4194304 4417452 140734418781376 18446744073709551615 4258244 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23076/tid=23085] ppid=23074 vsize=420076 CPUtime=12.01 cores=0-7
/proc/23076/task/23085/stat : 23085 (plingeling) R 23074 23076 22863 0 -1 4202560 5528 1252 0 0 1200 1 0 0 25 0 9 0 722766584 430157824 66315 33554432000 4194304 4417452 140734418781376 18446744073709551615 4258525 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=23076/tid=23086] ppid=23074 vsize=420076 CPUtime=12.01 cores=0-7
/proc/23076/task/23086/stat : 23086 (plingeling) R 23074 23076 22863 0 -1 4202560 49934 1252 0 0 1193 8 0 0 25 0 9 0 722766584 430157824 66315 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23076/tid=23087] ppid=23074 vsize=420076 CPUtime=11.98 cores=0-7
/proc/23076/task/23087/stat : 23087 (plingeling) R 23074 23076 22863 0 -1 4202560 6485 1252 0 0 1197 1 0 0 25 0 9 0 722766584 430157824 66315 33554432000 4194304 4417452 140734418781376 18446744073709551615 4248920 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23076/tid=23088] ppid=23074 vsize=420076 CPUtime=11.99 cores=0-7
/proc/23076/task/23088/stat : 23088 (plingeling) R 23074 23076 22863 0 -1 4202560 4158 1252 0 0 1199 0 0 0 25 0 9 0 722766585 430157824 66315 33554432000 4194304 4417452 140734418781376 18446744073709551615 4258244 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23076/tid=23089] ppid=23074 vsize=420076 CPUtime=11.99 cores=0-7
/proc/23076/task/23089/stat : 23089 (plingeling) R 23074 23076 22863 0 -1 4202560 4490 1252 0 0 1198 1 0 0 25 0 9 0 722766585 430157824 66315 33554432000 4194304 4417452 140734418781376 18446744073709551615 4258382 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23076/tid=23090] ppid=23074 vsize=420076 CPUtime=11.78 cores=0-7
/proc/23076/task/23090/stat : 23090 (plingeling) R 23074 23076 22863 0 -1 4202560 3476 1252 0 0 1178 0 0 0 25 0 9 0 722766585 430157824 66315 33554432000 4194304 4417452 140734418781376 18446744073709551615 4235225 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 96.49
Current children cumulated vsize (KiB) 420076

[startup+25.5009 s]
/proc/loadavg: 8.02 8.07 8.06 9/552 23091
/proc/meminfo: memFree=15871996/32951124 swapFree=54323904/67111528
[pid=23076] ppid=23074 vsize=534424 CPUtime=198.62 cores=0-7
/proc/23076/stat : 23076 (plingeling) S 23074 23076 22863 0 -1 4202496 168201 1252 0 0 19834 28 0 0 18 0 9 0 722766515 547250176 94957 33554432000 4194304 4417452 140734418781376 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/23076/statm: 133606 94957 180 55 0 131492 0
[pid=23076/tid=23083] ppid=23074 vsize=534424 CPUtime=24.79 cores=0-7
/proc/23076/task/23083/stat : 23083 (plingeling) R 23074 23076 22863 0 -1 4202560 8452 1252 0 0 2478 1 0 0 25 0 9 0 722766584 547250176 94957 33554432000 4194304 4417452 140734418781376 18446744073709551615 4248999 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23076/tid=23084] ppid=23074 vsize=534424 CPUtime=24.8 cores=0-7
/proc/23076/task/23084/stat : 23084 (plingeling) R 23074 23076 22863 0 -1 4202560 6389 1252 0 0 2479 1 0 0 25 0 9 0 722766584 547250176 94957 33554432000 4194304 4417452 140734418781376 18446744073709551615 4270927 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23076/tid=23085] ppid=23074 vsize=534424 CPUtime=24.79 cores=0-7
/proc/23076/task/23085/stat : 23085 (plingeling) R 23074 23076 22863 0 -1 4202560 8422 1252 0 0 2478 1 0 0 25 0 9 0 722766584 547250176 94957 33554432000 4194304 4417452 140734418781376 18446744073709551615 4249512 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=23076/tid=23086] ppid=23074 vsize=534424 CPUtime=24.8 cores=0-7
/proc/23076/task/23086/stat : 23086 (plingeling) R 23074 23076 22863 0 -1 4202560 86053 1252 0 0 2465 15 0 0 25 0 9 0 722766584 547250176 94957 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23076/tid=23087] ppid=23074 vsize=534424 CPUtime=24.77 cores=0-7
/proc/23076/task/23087/stat : 23087 (plingeling) R 23074 23076 22863 0 -1 4202560 7675 1252 0 0 2476 1 0 0 25 0 9 0 722766584 547250176 94957 33554432000 4194304 4417452 140734418781376 18446744073709551615 4232693 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23076/tid=23088] ppid=23074 vsize=534424 CPUtime=24.79 cores=0-7
/proc/23076/task/23088/stat : 23088 (plingeling) R 23074 23076 22863 0 -1 4202560 8713 1252 0 0 2478 1 0 0 25 0 9 0 722766585 547250176 94957 33554432000 4194304 4417452 140734418781376 18446744073709551615 4236008 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23076/tid=23089] ppid=23074 vsize=534424 CPUtime=24.79 cores=0-7
/proc/23076/task/23089/stat : 23089 (plingeling) R 23074 23076 22863 0 -1 4202560 5124 1252 0 0 2478 1 0 0 25 0 9 0 722766585 547250176 94957 33554432000 4194304 4417452 140734418781376 18446744073709551615 4249584 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23076/tid=23090] ppid=23074 vsize=534424 CPUtime=24.34 cores=0-7
/proc/23076/task/23090/stat : 23090 (plingeling) R 23074 23076 22863 0 -1 4202560 3476 1252 0 0 2434 0 0 0 25 0 9 0 722766585 547250176 94957 33554432000 4194304 4417452 140734418781376 18446744073709551615 4249209 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 198.62
Current children cumulated vsize (KiB) 534424

[startup+51.1013 s]
/proc/loadavg: 8.01 8.07 8.06 9/552 23091
/proc/meminfo: memFree=15724268/32951124 swapFree=54323904/67111528
[pid=23076] ppid=23074 vsize=681884 CPUtime=402.96 cores=0-7
/proc/23076/stat : 23076 (plingeling) S 23074 23076 22863 0 -1 4202496 258510 1252 0 0 40251 45 0 0 18 0 9 0 722766515 698249216 131997 33554432000 4194304 4417452 140734418781376 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/23076/statm: 170471 131997 180 55 0 168357 0
[pid=23076/tid=23083] ppid=23074 vsize=681884 CPUtime=50.38 cores=0-7
/proc/23076/task/23083/stat : 23083 (plingeling) R 23074 23076 22863 0 -1 4202560 10500 1252 0 0 5037 1 0 0 25 0 9 0 722766584 698249216 131997 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364687 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23076/tid=23084] ppid=23074 vsize=681884 CPUtime=50.4 cores=0-7
/proc/23076/task/23084/stat : 23084 (plingeling) R 23074 23076 22863 0 -1 4202560 8413 1252 0 0 5039 1 0 0 25 0 9 0 722766584 698249216 131997 33554432000 4194304 4417452 140734418781376 18446744073709551615 4358947 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23076/tid=23085] ppid=23074 vsize=681884 CPUtime=50.39 cores=0-7

################
# More data... #
################

[pid=23076/tid=23085] ppid=23074 vsize=10362756 CPUtime=4641.19 cores=0-7
/proc/23076/task/23085/stat : 23085 (plingeling) R 23074 23076 22863 0 -1 4202560 2000810 1252 0 0 463801 318 0 0 25 0 9 0 722766584 10611462144 2551677 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23076/tid=23086] ppid=23074 vsize=10362756 CPUtime=4721.76 cores=0-7
/proc/23076/task/23086/stat : 23086 (plingeling) R 23074 23076 22863 0 -1 4202560 1949518 1252 0 0 471696 480 0 0 25 0 9 0 722766584 10611462144 2551677 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23076/tid=23087] ppid=23074 vsize=10362756 CPUtime=4720.17 cores=0-7
/proc/23076/task/23087/stat : 23087 (plingeling) R 23074 23076 22863 0 -1 4202560 1611330 1252 0 0 471712 305 0 0 25 0 9 0 722766584 10611462144 2551677 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23076/tid=23088] ppid=23074 vsize=10362756 CPUtime=4716.11 cores=0-7
/proc/23076/task/23088/stat : 23088 (plingeling) R 23074 23076 22863 0 -1 4202560 2033988 1252 0 0 471304 307 0 0 25 0 9 0 722766585 10611462144 2551677 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364850 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23076/tid=23089] ppid=23074 vsize=10362756 CPUtime=4721.71 cores=0-7
/proc/23076/task/23089/stat : 23089 (plingeling) R 23074 23076 22863 0 -1 4202560 1904056 1252 0 0 471863 308 0 0 25 0 9 0 722766585 10611462144 2551677 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23076/tid=23090] ppid=23074 vsize=10362756 CPUtime=4717.67 cores=0-7
/proc/23076/task/23090/stat : 23090 (plingeling) R 23074 23076 22863 0 -1 4202560 1881322 1252 0 0 471486 281 0 0 25 0 9 0 722766585 10611462144 2551677 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 37681.5
Current children cumulated vsize (KiB) 10362756

[startup+4782.3 s]
/proc/loadavg: 8.07 8.13 8.08 9/552 23738
/proc/meminfo: memFree=6263704/32951124 swapFree=54323904/67111528
[pid=23076] ppid=23074 vsize=10395528 CPUtime=38160.3 cores=0-7
/proc/23076/stat : 23076 (plingeling) S 23074 23076 22863 0 -1 4202496 16740993 1252 0 0 3813251 2775 0 0 18 0 9 0 722766515 10645020672 2559870 33554432000 4194304 4417452 140734418781376 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/23076/statm: 2598882 2559870 180 55 0 2596768 0
[pid=23076/tid=23083] ppid=23074 vsize=10395528 CPUtime=4781.46 cores=0-7
/proc/23076/task/23083/stat : 23083 (plingeling) R 23074 23076 22863 0 -1 4202560 2512872 1252 0 0 477790 356 0 0 25 0 9 0 722766584 10645020672 2559870 33554432000 4194304 4417452 140734418781376 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23076/tid=23084] ppid=23074 vsize=10395528 CPUtime=4780.69 cores=0-7
/proc/23076/task/23084/stat : 23084 (plingeling) R 23074 23076 22863 0 -1 4202560 2764048 1252 0 0 477662 407 0 0 25 0 9 0 722766584 10645020672 2559870 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23076/tid=23085] ppid=23074 vsize=10395528 CPUtime=4701.17 cores=0-7
/proc/23076/task/23085/stat : 23085 (plingeling) R 23074 23076 22863 0 -1 4202560 2017194 1252 0 0 469797 320 0 0 25 0 9 0 722766584 10645020672 2559870 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23076/tid=23086] ppid=23074 vsize=10395528 CPUtime=4781.74 cores=0-7
/proc/23076/task/23086/stat : 23086 (plingeling) R 23074 23076 22863 0 -1 4202560 1949518 1252 0 0 477694 480 0 0 25 0 9 0 722766584 10645020672 2559870 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23076/tid=23087] ppid=23074 vsize=10395528 CPUtime=4780.14 cores=0-7
/proc/23076/task/23087/stat : 23087 (plingeling) R 23074 23076 22863 0 -1 4202560 1611330 1252 0 0 477709 305 0 0 25 0 9 0 722766584 10645020672 2559870 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364548 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23076/tid=23088] ppid=23074 vsize=10395528 CPUtime=4774.97 cores=0-7
/proc/23076/task/23088/stat : 23088 (plingeling) R 23074 23076 22863 0 -1 4202560 2066756 1252 0 0 477186 311 0 0 25 0 9 0 722766585 10645020672 2559870 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23076/tid=23089] ppid=23074 vsize=10395528 CPUtime=4781.69 cores=0-7
/proc/23076/task/23089/stat : 23089 (plingeling) R 23074 23076 22863 0 -1 4202560 1904056 1252 0 0 477861 308 0 0 25 0 9 0 722766585 10645020672 2559870 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23076/tid=23090] ppid=23074 vsize=10395528 CPUtime=4777.65 cores=0-7
/proc/23076/task/23090/stat : 23090 (plingeling) R 23074 23076 22863 0 -1 4202560 1881322 1252 0 0 477484 281 0 0 25 0 9 0 722766585 10645020672 2559870 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 38160.3
Current children cumulated vsize (KiB) 10395528

[startup+4842.32 s]
/proc/loadavg: 8.02 8.11 8.07 9/552 23739
/proc/meminfo: memFree=6447840/32951124 swapFree=54323904/67111528
[pid=23076] ppid=23074 vsize=10079868 CPUtime=38639.3 cores=0-7
/proc/23076/stat : 23076 (plingeling) S 23074 23076 22863 0 -1 4202496 16839328 1252 0 0 3861132 2794 0 0 18 0 9 0 722766515 10321784832 2480986 33554432000 4194304 4417452 140734418781376 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/23076/statm: 2519967 2480986 180 55 0 2517853 0
[pid=23076/tid=23083] ppid=23074 vsize=10079868 CPUtime=4841.49 cores=0-7
/proc/23076/task/23083/stat : 23083 (plingeling) R 23074 23076 22863 0 -1 4202560 2529256 1252 0 0 483790 359 0 0 25 0 9 0 722766584 10321784832 2480986 33554432000 4194304 4417452 140734418781376 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23076/tid=23084] ppid=23074 vsize=10079868 CPUtime=4840.71 cores=0-7
/proc/23076/task/23084/stat : 23084 (plingeling) R 23074 23076 22863 0 -1 4202560 2796816 1252 0 0 483661 410 0 0 25 0 9 0 722766584 10321784832 2480986 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23076/tid=23085] ppid=23074 vsize=10079868 CPUtime=4761.19 cores=0-7
/proc/23076/task/23085/stat : 23085 (plingeling) R 23074 23076 22863 0 -1 4202560 2049962 1252 0 0 475790 329 0 0 25 0 9 0 722766584 10321784832 2480986 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23076/tid=23086] ppid=23074 vsize=10079868 CPUtime=4841.76 cores=0-7
/proc/23076/task/23086/stat : 23086 (plingeling) R 23074 23076 22863 0 -1 4202560 1949518 1252 0 0 483696 480 0 0 25 0 9 0 722766584 10321784832 2480986 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23076/tid=23087] ppid=23074 vsize=10079868 CPUtime=4840.14 cores=0-7
/proc/23076/task/23087/stat : 23087 (plingeling) R 23074 23076 22863 0 -1 4202560 1627714 1252 0 0 483707 307 0 0 25 0 9 0 722766584 10321784832 2480986 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23076/tid=23088] ppid=23074 vsize=10079868 CPUtime=4833.84 cores=0-7
/proc/23076/task/23088/stat : 23088 (plingeling) R 23074 23076 22863 0 -1 4202560 2066787 1252 0 0 483072 312 0 0 25 0 9 0 722766585 10321784832 2480986 33554432000 4194304 4417452 140734418781376 18446744073709551615 4346490 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23076/tid=23089] ppid=23074 vsize=10079868 CPUtime=4841.71 cores=0-7
/proc/23076/task/23089/stat : 23089 (plingeling) R 23074 23076 22863 0 -1 4202560 1904056 1252 0 0 483863 308 0 0 25 0 9 0 722766585 10321784832 2480986 33554432000 4194304 4417452 140734418781376 18446744073709551615 4262948 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23076/tid=23090] ppid=23074 vsize=10079868 CPUtime=4837.67 cores=0-7
/proc/23076/task/23090/stat : 23090 (plingeling) R 23074 23076 22863 0 -1 4202560 1881322 1252 0 0 483485 282 0 0 25 0 9 0 722766585 10321784832 2480986 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 38639.3
Current children cumulated vsize (KiB) 10079868

[startup+4902.3 s]
/proc/loadavg: 8.08 8.10 8.07 9/552 23741
/proc/meminfo: memFree=6082420/32951124 swapFree=54323904/67111528
[pid=23076] ppid=23074 vsize=10444924 CPUtime=39118 cores=0-7
/proc/23076/stat : 23076 (plingeling) S 23074 23076 22863 0 -1 4202496 17028927 1252 0 0 3908965 2834 0 0 18 0 9 0 722766515 10695602176 2572281 33554432000 4194304 4417452 140734418781376 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/23076/statm: 2611231 2572281 180 55 0 2609117 0
[pid=23076/tid=23083] ppid=23074 vsize=10444924 CPUtime=4901.47 cores=0-7
/proc/23076/task/23083/stat : 23083 (plingeling) R 23074 23076 22863 0 -1 4202560 2562024 1252 0 0 489785 362 0 0 25 0 9 0 722766584 10695602176 2572281 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23076/tid=23084] ppid=23074 vsize=10444924 CPUtime=4900.69 cores=0-7
/proc/23076/task/23084/stat : 23084 (plingeling) R 23074 23076 22863 0 -1 4202560 2813200 1252 0 0 489657 412 0 0 25 0 9 0 722766584 10695602176 2572281 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23076/tid=23085] ppid=23074 vsize=10444924 CPUtime=4821.17 cores=0-7
/proc/23076/task/23085/stat : 23085 (plingeling) R 23074 23076 22863 0 -1 4202560 2049962 1252 0 0 481787 330 0 0 25 0 9 0 722766584 10695602176 2572281 33554432000 4194304 4417452 140734418781376 18446744073709551615 4354159 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23076/tid=23086] ppid=23074 vsize=10444924 CPUtime=4901.74 cores=0-7
/proc/23076/task/23086/stat : 23086 (plingeling) R 23074 23076 22863 0 -1 4202560 1949518 1252 0 0 489694 480 0 0 25 0 9 0 722766584 10695602176 2572281 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364711 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23076/tid=23087] ppid=23074 vsize=10444924 CPUtime=4900.11 cores=0-7
/proc/23076/task/23087/stat : 23087 (plingeling) R 23074 23076 22863 0 -1 4202560 1627714 1252 0 0 489704 307 0 0 25 0 9 0 722766584 10695602176 2572281 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23076/tid=23088] ppid=23074 vsize=10444924 CPUtime=4892.71 cores=0-7
/proc/23076/task/23088/stat : 23088 (plingeling) R 23074 23076 22863 0 -1 4202560 2207234 1252 0 0 488925 346 0 0 25 0 9 0 722766585 10695602176 2572281 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23076/tid=23089] ppid=23074 vsize=10444924 CPUtime=4901.69 cores=0-7
/proc/23076/task/23089/stat : 23089 (plingeling) R 23074 23076 22863 0 -1 4202560 1904056 1252 0 0 489861 308 0 0 25 0 9 0 722766585 10695602176 2572281 33554432000 4194304 4417452 140734418781376 18446744073709551615 4366094 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23076/tid=23090] ppid=23074 vsize=10444924 CPUtime=4897.65 cores=0-7
/proc/23076/task/23090/stat : 23090 (plingeling) R 23074 23076 22863 0 -1 4202560 1881322 1252 0 0 489483 282 0 0 25 0 9 0 722766585 10695602176 2572281 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 39118
Current children cumulated vsize (KiB) 10444924

[startup+4962.3 s]
/proc/loadavg: 8.10 8.09 8.07 9/552 23742
/proc/meminfo: memFree=5865448/32951124 swapFree=54323904/67111528
[pid=23076] ppid=23074 vsize=10662076 CPUtime=39596.9 cores=0-7
/proc/23076/stat : 23076 (plingeling) S 23074 23076 22863 0 -1 4202496 17148751 1252 0 0 3956838 2848 0 0 18 0 9 0 722766515 10917965824 2626569 33554432000 4194304 4417452 140734418781376 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/23076/statm: 2665519 2626569 180 55 0 2663405 0
[pid=23076/tid=23083] ppid=23074 vsize=10662076 CPUtime=4961.47 cores=0-7
/proc/23076/task/23083/stat : 23083 (plingeling) R 23074 23076 22863 0 -1 4202560 2578408 1252 0 0 495783 364 0 0 25 0 9 0 722766584 10917965824 2626569 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23076/tid=23084] ppid=23074 vsize=10662076 CPUtime=4960.69 cores=0-7
/proc/23076/task/23084/stat : 23084 (plingeling) R 23074 23076 22863 0 -1 4202560 2834720 1252 0 0 495653 416 0 0 25 0 9 0 722766584 10917965824 2626569 33554432000 4194304 4417452 140734418781376 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23076/tid=23085] ppid=23074 vsize=10662076 CPUtime=4881.17 cores=0-7
/proc/23076/task/23085/stat : 23085 (plingeling) R 23074 23076 22863 0 -1 4202560 2082730 1252 0 0 487784 333 0 0 25 0 9 0 722766584 10917965824 2626569 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23076/tid=23086] ppid=23074 vsize=10662076 CPUtime=4961.75 cores=0-7
/proc/23076/task/23086/stat : 23086 (plingeling) R 23074 23076 22863 0 -1 4202560 1949518 1252 0 0 495695 480 0 0 25 0 9 0 722766584 10917965824 2626569 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23076/tid=23087] ppid=23074 vsize=10662076 CPUtime=4960.09 cores=0-7
/proc/23076/task/23087/stat : 23087 (plingeling) R 23074 23076 22863 0 -1 4202560 1627714 1252 0 0 495702 307 0 0 25 0 9 0 722766584 10917965824 2626569 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23076/tid=23088] ppid=23074 vsize=10662076 CPUtime=4951.59 cores=0-7
/proc/23076/task/23088/stat : 23088 (plingeling) R 23074 23076 22863 0 -1 4202560 2223618 1252 0 0 494811 348 0 0 25 0 9 0 722766585 10917965824 2626569 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23076/tid=23089] ppid=23074 vsize=10662076 CPUtime=4961.7 cores=0-7
/proc/23076/task/23089/stat : 23089 (plingeling) R 23074 23076 22863 0 -1 4202560 1904056 1252 0 0 495862 308 0 0 25 0 9 0 722766585 10917965824 2626569 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23076/tid=23090] ppid=23074 vsize=10662076 CPUtime=4957.66 cores=0-7
/proc/23076/task/23090/stat : 23090 (plingeling) R 23074 23076 22863 0 -1 4202560 1914090 1252 0 0 495480 286 0 0 25 0 9 0 722766585 10917965824 2626569 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 39596.9
Current children cumulated vsize (KiB) 10662076



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5012.82 s]
/proc/loadavg: 8.11 8.09 8.07 9/552 23744
/proc/meminfo: memFree=5859488/32951124 swapFree=54323904/67111528
[pid=23076] ppid=23074 vsize=10666340 CPUtime=40000.2 cores=0-7
/proc/23076/stat : 23076 (plingeling) S 23074 23076 22863 0 -1 4202496 17248121 1252 0 0 3997162 2859 0 0 18 0 9 0 722766515 10922332160 2627635 33554432000 4194304 4417452 140734418781376 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/23076/statm: 2666585 2627635 180 55 0 2664471 0
[pid=23076/tid=23083] ppid=23074 vsize=10666340 CPUtime=5012.01 cores=0-7
/proc/23076/task/23083/stat : 23083 (plingeling) R 23074 23076 22863 0 -1 4202560 2594792 1252 0 0 500834 367 0 0 25 0 9 0 722766584 10922332160 2627635 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23076/tid=23084] ppid=23074 vsize=10666340 CPUtime=5011.23 cores=0-7
/proc/23076/task/23084/stat : 23084 (plingeling) R 23074 23076 22863 0 -1 4202560 2851104 1252 0 0 500705 418 0 0 25 0 9 0 722766584 10922332160 2627635 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23076/tid=23085] ppid=23074 vsize=10666340 CPUtime=4931.71 cores=0-7
/proc/23076/task/23085/stat : 23085 (plingeling) R 23074 23076 22863 0 -1 4202560 2082730 1252 0 0 492838 333 0 0 25 0 9 0 722766584 10922332160 2627635 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23076/tid=23086] ppid=23074 vsize=10666340 CPUtime=5012.29 cores=0-7
/proc/23076/task/23086/stat : 23086 (plingeling) R 23074 23076 22863 0 -1 4202560 1949518 1252 0 0 500749 480 0 0 25 0 9 0 722766584 10922332160 2627635 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23076/tid=23087] ppid=23074 vsize=10666340 CPUtime=5010.61 cores=0-7
/proc/23076/task/23087/stat : 23087 (plingeling) R 23074 23076 22863 0 -1 4202560 1661548 1252 0 0 500751 310 0 0 25 0 9 0 722766584 10922332160 2627635 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23076/tid=23088] ppid=23074 vsize=10666340 CPUtime=5001.17 cores=0-7
/proc/23076/task/23088/stat : 23088 (plingeling) R 23074 23076 22863 0 -1 4202560 2256386 1252 0 0 499766 351 0 0 25 0 9 0 722766585 10922332160 2627635 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23076/tid=23089] ppid=23074 vsize=10666340 CPUtime=5012.24 cores=0-7
/proc/23076/task/23089/stat : 23089 (plingeling) R 23074 23076 22863 0 -1 4202560 1904056 1252 0 0 500916 308 0 0 25 0 9 0 722766585 10922332160 2627635 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23076/tid=23090] ppid=23074 vsize=10666340 CPUtime=5008.2 cores=0-7
/proc/23076/task/23090/stat : 23090 (plingeling) R 23074 23076 22863 0 -1 4202560 1914090 1252 0 0 500534 286 0 0 25 0 9 0 722766585 10922332160 2627635 33554432000 4194304 4417452 140734418781376 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 40000.2
Current children cumulated vsize (KiB) 10666340

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5013.1 s]
/proc/loadavg: 8.11 8.09 8.07 9/552 23744
/proc/meminfo: memFree=5859488/32951124 swapFree=54323904/67111528
[pid=23076] ppid=23074 vsize=0 CPUtime=40000.5 cores=0-7
/proc/23076/stat : 23076 (plingeling) Z 23074 23076 22863 0 -1 4203548 17248121 1252 0 0 3997166 2885 0 0 18 0 2 0 722766515 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/23076/statm: 0 0 0 0 0 0 0
[pid=23076/tid=23088] ppid=23074 vsize=0 CPUtime=5001.43 cores=0-7
/proc/23076/task/23088/stat : 23088 (plingeling) R 23074 23076 22863 0 -1 4203588 2256386 1252 0 0 499767 376 0 0 25 0 2 0 722766585 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40000.5
Current children cumulated vsize (KiB) 0

[startup+5013.2 s]
/proc/loadavg: 8.11 8.09 8.07 9/552 23744
/proc/meminfo: memFree=5859488/32951124 swapFree=54323904/67111528
[pid=23076] ppid=23074 vsize=0 CPUtime=40000.6 cores=0-7
/proc/23076/stat : 23076 (plingeling) Z 23074 23076 22863 0 -1 4203548 17248121 1252 0 0 3997166 2895 0 0 18 0 2 0 722766515 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/23076/statm: 0 0 0 0 0 0 0
[pid=23076/tid=23088] ppid=23074 vsize=0 CPUtime=5001.53 cores=0-7
/proc/23076/task/23088/stat : 23088 (plingeling) R 23074 23076 22863 0 -1 4203588 2256386 1252 0 0 499767 386 0 0 25 0 2 0 722766585 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40000.6
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5013.22
CPU time (s): 40000.6
CPU user time (s): 39971.7
CPU system time (s): 28.9696
CPU usage (%): 797.903
Max. virtual memory (cumulated for all children) (KiB): 10928484

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39971.7
system time used= 28.9696
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 17249373
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 435
involuntary context switches= 147641

runsolver used 25.7981 second user time and 67.3098 second system time

The end

Launcher Data 

Begin job on node106 at 2011-05-23 02:47:41
IDJOB=3395036
IDBENCH=88226
IDSOLVER=1851
FILE ID=node106/3395036-1306111661
RUNJOBID= node106-1306106647-22877
PBS_JOBID= 13388028
Free space on /tmp= 73256 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/crafted/anton/SRHD-SGI/srhd-sgi-m52-q1041.25-n50-p15-s99099953.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3395036-1306111661/watcher-3395036-1306111661 -o /tmp/evaluation-result-3395036-1306111661/solver-3395036-1306111661 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3395036-1306111661.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= dda220d08783aabf4e7a9e99eeeccae6
RANDOM SEED=41876365

node106.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.20
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      16251112 kB
Buffers:        343616 kB
Cached:       10396576 kB
SwapCached:      24016 kB
Active:        6124812 kB
Inactive:     10265320 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      16251112 kB
SwapTotal:    67111528 kB
SwapFree:     54323904 kB
Dirty:            6632 kB
Writeback:           0 kB
AnonPages:     5634748 kB
Mapped:          25804 kB
Slab:           158192 kB
PageTables:      77604 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 167518344 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73392 MiB
End job on node106 at 2011-05-23 04:11:16