Trace number 3371171

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.47 1203.59

General information on the benchmark

NameCRAFTED/rbsat/random/unforced/
rbsat-v2100c215164g9.cnf
MD5SUMe033e9c916da557514299cce9ca2468f
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2100
Number of clauses215164
Sum of the clauses size432288
Maximum clause length30
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2215094
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 570

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3371171-1305417954/watcher-3371171-1305417954 -o /tmp/evaluation-result-3371171-1305417954/solver-3371171-1305417954 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3371171-1305417954.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.69 8.01 7.99 5/354 22690
/proc/meminfo: memFree=24007312/32950928 swapFree=45937192/67111528
[pid=22690] ppid=22688 vsize=9668 CPUtime=0 cores=0,2,4,6
/proc/22690/stat : 22690 (plingeling) R 22688 22690 22319 0 -1 4202496 478 1253 0 0 0 0 0 0 25 0 1 0 653392068 9900032 408 33554432000 4194304 4417452 140735837163776 18446744073709551615 254500691712 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/22690/statm: 2417 408 147 55 0 303 0

[startup+0.0635099 s]
/proc/loadavg: 7.69 8.01 7.99 5/354 22690
/proc/meminfo: memFree=24007312/32950928 swapFree=45937192/67111528
[pid=22690] ppid=22688 vsize=18384 CPUtime=0.05 cores=0,2,4,6
/proc/22690/stat : 22690 (plingeling) R 22688 22690 22319 0 -1 4202496 2658 1253 0 0 5 0 0 0 25 0 1 0 653392068 18825216 2588 33554432000 4194304 4417452 140735837163776 18446744073709551615 4231636 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/22690/statm: 4596 2588 148 55 0 2482 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 18384

[startup+0.100506 s]
/proc/loadavg: 7.69 8.01 7.99 5/354 22690
/proc/meminfo: memFree=24007312/32950928 swapFree=45937192/67111528
[pid=22690] ppid=22688 vsize=26576 CPUtime=0.09 cores=0,2,4,6
/proc/22690/stat : 22690 (plingeling) R 22688 22690 22319 0 -1 4202496 4706 1253 0 0 9 0 0 0 25 0 1 0 653392068 27213824 4636 33554432000 4194304 4417452 140735837163776 18446744073709551615 4231283 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/22690/statm: 6644 4636 148 55 0 4530 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 26576

[startup+0.300395 s]
/proc/loadavg: 7.69 8.01 7.99 5/354 22690
/proc/meminfo: memFree=24007312/32950928 swapFree=45937192/67111528
[pid=22690] ppid=22688 vsize=135272 CPUtime=0.76 cores=0,2,4,6
/proc/22690/stat : 22690 (plingeling) S 22688 22690 22319 0 -1 4202496 5709 1253 0 0 76 0 0 0 25 0 5 0 653392068 138518528 5249 33554432000 4194304 4417452 140735837163776 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/22690/statm: 33818 5249 176 55 0 31704 0
Current children cumulated CPU time (s) 0.76
Current children cumulated vsize (KiB) 135272

[startup+0.700315 s]
/proc/loadavg: 7.69 8.01 7.99 5/354 22690
/proc/meminfo: memFree=24007312/32950928 swapFree=45937192/67111528
[pid=22690] ppid=22688 vsize=159500 CPUtime=2.35 cores=0,2,4,6
/proc/22690/stat : 22690 (plingeling) S 22688 22690 22319 0 -1 4202496 19561 1253 0 0 233 2 0 0 25 0 5 0 653392068 163328000 11310 33554432000 4194304 4417452 140735837163776 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/22690/statm: 39875 11310 180 55 0 37761 0
Current children cumulated CPU time (s) 2.35
Current children cumulated vsize (KiB) 159500

[startup+1.50115 s]
/proc/loadavg: 7.69 8.01 7.99 9/359 22700
/proc/meminfo: memFree=23960160/32950928 swapFree=45937192/67111528
[pid=22690] ppid=22688 vsize=163244 CPUtime=5.54 cores=0,2,4,6
/proc/22690/stat : 22690 (plingeling) S 22688 22690 22319 0 -1 4202496 21512 1253 0 0 551 3 0 0 25 0 5 0 653392068 167161856 12246 33554432000 4194304 4417452 140735837163776 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/22690/statm: 40811 12246 180 55 0 38697 0
[pid=22690/tid=22697] ppid=22688 vsize=163244 CPUtime=1.34 cores=0,2,4,6
/proc/22690/task/22697/stat : 22697 (plingeling) R 22688 22690 22319 0 -1 4202560 3301 1253 0 0 134 0 0 0 25 0 5 0 653392082 167161856 12246 33554432000 4194304 4417452 140735837163776 18446744073709551615 4236280 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22690/tid=22698] ppid=22688 vsize=164796 CPUtime=1.33 cores=0,2,4,6
/proc/22690/task/22698/stat : 22698 (plingeling) R 22688 22690 22319 0 -1 4202560 4210 1253 0 0 133 0 0 0 25 0 5 0 653392082 168751104 12246 33554432000 4194304 4417452 140735837163776 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22690/tid=22699] ppid=22688 vsize=164796 CPUtime=1.34 cores=0,2,4,6
/proc/22690/task/22699/stat : 22699 (plingeling) R 22688 22690 22319 0 -1 4202560 4740 1253 0 0 134 0 0 0 19 0 5 0 653392082 168751104 12260 33554432000 4194304 4417452 140735837163776 18446744073709551615 4258575 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22690/tid=22700] ppid=22688 vsize=164796 CPUtime=1.35 cores=0,2,4,6
/proc/22690/task/22700/stat : 22700 (plingeling) R 22688 22690 22319 0 -1 4202560 4570 1253 0 0 135 0 0 0 25 0 5 0 653392082 168751104 12273 33554432000 4194304 4417452 140735837163776 18446744073709551615 254500385885 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 5.54
Current children cumulated vsize (KiB) 163244

[startup+3.10084 s]
/proc/loadavg: 7.69 8.01 7.99 9/359 22700
/proc/meminfo: memFree=23952716/32950928 swapFree=45937192/67111528
[pid=22690] ppid=22688 vsize=177676 CPUtime=11.93 cores=0,2,4,6
/proc/22690/stat : 22690 (plingeling) S 22688 22690 22319 0 -1 4202496 30226 1253 0 0 1189 4 0 0 25 0 5 0 653392068 181940224 15860 33554432000 4194304 4417452 140735837163776 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/22690/statm: 44419 15860 180 55 0 42305 0
[pid=22690/tid=22697] ppid=22688 vsize=177676 CPUtime=2.94 cores=0,2,4,6
/proc/22690/task/22697/stat : 22697 (plingeling) R 22688 22690 22319 0 -1 4202560 3699 1253 0 0 294 0 0 0 25 0 5 0 653392082 181940224 15860 33554432000 4194304 4417452 140735837163776 18446744073709551615 4249976 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22690/tid=22698] ppid=22688 vsize=177676 CPUtime=2.92 cores=0,2,4,6
/proc/22690/task/22698/stat : 22698 (plingeling) R 22688 22690 22319 0 -1 4202560 5296 1253 0 0 292 0 0 0 25 0 5 0 653392082 181940224 15860 33554432000 4194304 4417452 140735837163776 18446744073709551615 4258604 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22690/tid=22699] ppid=22688 vsize=177676 CPUtime=2.94 cores=0,2,4,6
/proc/22690/task/22699/stat : 22699 (plingeling) R 22688 22690 22319 0 -1 4202560 4740 1253 0 0 294 0 0 0 24 0 5 0 653392082 181940224 15860 33554432000 4194304 4417452 140735837163776 18446744073709551615 4366447 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22690/tid=22700] ppid=22688 vsize=177676 CPUtime=2.95 cores=0,2,4,6
/proc/22690/task/22700/stat : 22700 (plingeling) R 22688 22690 22319 0 -1 4202560 11773 1253 0 0 294 1 0 0 25 0 5 0 653392082 181940224 15860 33554432000 4194304 4417452 140735837163776 18446744073709551615 4251096 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 11.93
Current children cumulated vsize (KiB) 177676

[startup+6.3002 s]
/proc/loadavg: 7.71 8.01 7.99 9/359 22700
/proc/meminfo: memFree=23950188/32950928 swapFree=45937192/67111528
[pid=22690] ppid=22688 vsize=184884 CPUtime=24.69 cores=0,2,4,6
/proc/22690/stat : 22690 (plingeling) S 22688 22690 22319 0 -1 4202496 43106 1253 0 0 2463 6 0 0 25 0 5 0 653392068 189321216 17193 33554432000 4194304 4417452 140735837163776 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/22690/statm: 46221 17193 180 55 0 44107 0
[pid=22690/tid=22697] ppid=22688 vsize=184884 CPUtime=6.14 cores=0,2,4,6
/proc/22690/task/22697/stat : 22697 (plingeling) R 22688 22690 22319 0 -1 4202560 3699 1253 0 0 614 0 0 0 25 0 5 0 653392082 189321216 17193 33554432000 4194304 4417452 140735837163776 18446744073709551615 4249943 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22690/tid=22698] ppid=22688 vsize=184884 CPUtime=6.08 cores=0,2,4,6
/proc/22690/task/22698/stat : 22698 (plingeling) R 22688 22690 22319 0 -1 4202560 5296 1253 0 0 608 0 0 0 25 0 5 0 653392082 189321216 17193 33554432000 4194304 4417452 140735837163776 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22690/tid=22699] ppid=22688 vsize=184884 CPUtime=6.14 cores=0,2,4,6
/proc/22690/task/22699/stat : 22699 (plingeling) R 22688 22690 22319 0 -1 4202560 4740 1253 0 0 614 0 0 0 25 0 5 0 653392082 189321216 17193 33554432000 4194304 4417452 140735837163776 18446744073709551615 4249222 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22690/tid=22700] ppid=22688 vsize=184884 CPUtime=6.15 cores=0,2,4,6
/proc/22690/task/22700/stat : 22700 (plingeling) R 22688 22690 22319 0 -1 4202560 24653 1253 0 0 612 3 0 0 25 0 5 0 653392082 189321216 17193 33554432000 4194304 4417452 140735837163776 18446744073709551615 4357229 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 24.69
Current children cumulated vsize (KiB) 184884

[startup+12.7009 s]
/proc/loadavg: 7.73 8.01 7.99 9/359 22700
/proc/meminfo: memFree=23900108/32950928 swapFree=45937192/67111528
[pid=22690] ppid=22688 vsize=232612 CPUtime=50.22 cores=0,2,4,6
/proc/22690/stat : 22690 (plingeling) S 22688 22690 22319 0 -1 4202496 64832 1253 0 0 5013 9 0 0 25 0 5 0 653392068 238194688 29588 33554432000 4194304 4417452 140735837163776 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/22690/statm: 58153 29588 180 55 0 56039 0
[pid=22690/tid=22697] ppid=22688 vsize=232612 CPUtime=12.54 cores=0,2,4,6
/proc/22690/task/22697/stat : 22697 (plingeling) R 22688 22690 22319 0 -1 4202560 5870 1253 0 0 1254 0 0 0 25 0 5 0 653392082 238194688 29588 33554432000 4194304 4417452 140735837163776 18446744073709551615 4236017 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22690/tid=22698] ppid=22688 vsize=232612 CPUtime=12.42 cores=0,2,4,6
/proc/22690/task/22698/stat : 22698 (plingeling) R 22688 22690 22319 0 -1 4202560 6381 1253 0 0 1241 1 0 0 25 0 5 0 653392082 238194688 29588 33554432000 4194304 4417452 140735837163776 18446744073709551615 4249584 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22690/tid=22699] ppid=22688 vsize=232612 CPUtime=12.54 cores=0,2,4,6
/proc/22690/task/22699/stat : 22699 (plingeling) R 22688 22690 22319 0 -1 4202560 4740 1253 0 0 1254 0 0 0 25 0 5 0 653392082 238194688 29588 33554432000 4194304 4417452 140735837163776 18446744073709551615 4251028 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22690/tid=22700] ppid=22688 vsize=232612 CPUtime=12.54 cores=0,2,4,6
/proc/22690/task/22700/stat : 22700 (plingeling) R 22688 22690 22319 0 -1 4202560 43123 1253 0 0 1249 5 0 0 25 0 5 0 653392082 238194688 29588 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 50.22
Current children cumulated vsize (KiB) 232612

[startup+25.5004 s]
/proc/loadavg: 7.79 8.01 7.99 10/359 22700
/proc/meminfo: memFree=23795952/32950928 swapFree=45937192/67111528
[pid=22690] ppid=22688 vsize=275408 CPUtime=101.26 cores=0,2,4,6
/proc/22690/stat : 22690 (plingeling) S 22688 22690 22319 0 -1 4202496 100078 1253 0 0 10112 14 0 0 25 0 5 0 653392068 282017792 40287 33554432000 4194304 4417452 140735837163776 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/22690/statm: 68852 40287 180 55 0 66738 0
[pid=22690/tid=22697] ppid=22688 vsize=275408 CPUtime=25.34 cores=0,2,4,6
/proc/22690/task/22697/stat : 22697 (plingeling) R 22688 22690 22319 0 -1 4202560 5870 1253 0 0 2534 0 0 0 25 0 5 0 653392082 282017792 40287 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22690/tid=22698] ppid=22688 vsize=275408 CPUtime=25.06 cores=0,2,4,6
/proc/22690/task/22698/stat : 22698 (plingeling) R 22688 22690 22319 0 -1 4202560 6381 1253 0 0 2505 1 0 0 25 0 5 0 653392082 282017792 40287 33554432000 4194304 4417452 140735837163776 18446744073709551615 4236554 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22690/tid=22699] ppid=22688 vsize=275408 CPUtime=25.34 cores=0,2,4,6
/proc/22690/task/22699/stat : 22699 (plingeling) R 22688 22690 22319 0 -1 4202560 4740 1253 0 0 2534 0 0 0 25 0 5 0 653392082 282017792 40287 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364873 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22690/tid=22700] ppid=22688 vsize=275408 CPUtime=25.34 cores=0,2,4,6
/proc/22690/task/22700/stat : 22700 (plingeling) R 22688 22690 22319 0 -1 4202560 78369 1253 0 0 2524 10 0 0 25 0 5 0 653392082 282017792 40287 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 101.26
Current children cumulated vsize (KiB) 275408

[startup+51.1003 s]
/proc/loadavg: 8.00 8.04 8.00 9/359 22701
/proc/meminfo: memFree=23732688/32950928 swapFree=45937192/67111528
[pid=22690] ppid=22688 vsize=338136 CPUtime=203.38 cores=0,2,4,6
/proc/22690/stat : 22690 (plingeling) S 22688 22690 22319 0 -1 4202496 150736 1253 0 0 20318 20 0 0 25 0 5 0 653392068 346251264 55969 33554432000 4194304 4417452 140735837163776 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/22690/statm: 84534 55969 180 55 0 82420 0
[pid=22690/tid=22697] ppid=22688 vsize=338136 CPUtime=50.93 cores=0,2,4,6
/proc/22690/task/22697/stat : 22697 (plingeling) R 22688 22690 22319 0 -1 4202560 6656 1253 0 0 5093 0 0 0 25 0 5 0 653392082 346251264 55969 33554432000 4194304 4417452 140735837163776 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22690/tid=22698] ppid=22688 vsize=338136 CPUtime=50.39 cores=0,2,4,6
/proc/22690/task/22698/stat : 22698 (plingeling) R 22688 22690 22319 0 -1 4202560 7168 1253 0 0 5038 1 0 0 25 0 5 0 653392082 346251264 55969 33554432000 4194304 4417452 140735837163776 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22690/tid=22699] ppid=22688 vsize=338136 CPUtime=50.94 cores=0,2,4,6
/proc/22690/task/22699/stat : 22699 (plingeling) R 22688 22690 22319 0 -1 4202560 4740 1253 0 0 5094 0 0 0 25 0 5 0 653392082 346251264 55969 33554432000 4194304 4417452 140735837163776 18446744073709551615 4250291 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22690/tid=22700] ppid=22688 vsize=338136 CPUtime=50.94 cores=0,2,4,6
/proc/22690/task/22700/stat : 22700 (plingeling) R 22688 22690 22319 0 -1 4202560 127454 1253 0 0 5078 16 0 0 25 0 5 0 653392082 346251264 55969 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364807 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 203.38
Current children cumulated vsize (KiB) 338136

[startup+102.313 s]
/proc/loadavg: 8.04 8.05 8.00 9/359 22702
/proc/meminfo: memFree=23592308/32950928 swapFree=45937192/67111528
[pid=22690] ppid=22688 vsize=436596 CPUtime=407.66 cores=0,2,4,6
/proc/22690/stat : 22690 (plingeling) S 22688 22690 22319 0 -1 4202496 217080 1253 0 0 40736 30 0 0 25 0 5 0 653392068 447074304 80586 33554432000 4194304 4417452 140735837163776 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/22690/statm: 109149 80586 180 55 0 107035 0
[pid=22690/tid=22697] ppid=22688 vsize=436596 CPUtime=102.14 cores=0,2,4,6
/proc/22690/task/22697/stat : 22697 (plingeling) R 22688 22690 22319 0 -1 4202560 7127 1253 0 0 10214 0 0 0 25 0 5 0 653392082 447074304 80586 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364572 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22690/tid=22698] ppid=22688 vsize=436596 CPUtime=101.04 cores=0,2,4,6
/proc/22690/task/22698/stat : 22698 (plingeling) R 22688 22690 22319 0 -1 4202560 7411 1253 0 0 10103 1 0 0 25 0 5 0 653392082 447074304 80586 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22690/tid=22699] ppid=22688 vsize=436596 CPUtime=102.16 cores=0,2,4,6
/proc/22690/task/22699/stat : 22699 (plingeling) R 22688 22690 22319 0 -1 4202560 4740 1253 0 0 10216 0 0 0 25 0 5 0 653392082 447074304 80586 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22690/tid=22700] ppid=22688 vsize=436596 CPUtime=102.16 cores=0,2,4,6
/proc/22690/task/22700/stat : 22700 (plingeling) R 22688 22690 22319 0 -1 4202560 193084 1253 0 0 10189 27 0 0 25 0 5 0 653392082 447074304 80586 33554432000 4194304 4417452 140735837163776 18446744073709551615 4366113 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 407.66
Current children cumulated vsize (KiB) 436596

[startup+162.301 s]
/proc/loadavg: 8.17 8.08 8.01 9/359 22704
/proc/meminfo: memFree=23476224/32950928 swapFree=45937192/67111528
[pid=22690] ppid=22688 vsize=426464 CPUtime=646.96 cores=0,2,4,6
/proc/22690/stat : 22690 (plingeling) S 22688 22690 22319 0 -1 4202496 278502 1253 0 0 64655 41 0 0 25 0 5 0 653392068 436699136 78053 33554432000 4194304 4417452 140735837163776 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/22690/statm: 106616 78053 180 55 0 104502 0
[pid=22690/tid=22697] ppid=22688 vsize=426464 CPUtime=162.12 cores=0,2,4,6
/proc/22690/task/22697/stat : 22697 (plingeling) R 22688 22690 22319 0 -1 4202560 7127 1253 0 0 16212 0 0 0 25 0 5 0 653392082 436699136 78053 33554432000 4194304 4417452 140735837163776 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22690/tid=22698] ppid=22688 vsize=426464 CPUtime=160.37 cores=0,2,4,6
/proc/22690/task/22698/stat : 22698 (plingeling) R 22688 22690 22319 0 -1 4202560 7411 1253 0 0 16036 1 0 0 25 0 5 0 653392082 436699136 78053 33554432000 4194304 4417452 140735837163776 18446744073709551615 4264930 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22690/tid=22699] ppid=22688 vsize=426464 CPUtime=162.15 cores=0,2,4,6
/proc/22690/task/22699/stat : 22699 (plingeling) R 22688 22690 22319 0 -1 4202560 4740 1253 0 0 16215 0 0 0 25 0 5 0 653392082 436699136 78053 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364495 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22690/tid=22700] ppid=22688 vsize=426464 CPUtime=162.15 cores=0,2,4,6
/proc/22690/task/22700/stat : 22700 (plingeling) R 22688 22690 22319 0 -1 4202560 254506 1253 0 0 16177 38 0 0 25 0 5 0 653392082 436699136 78053 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 646.96
Current children cumulated vsize (KiB) 426464


################
# More data... #
################

[pid=22690/tid=22698] ppid=22688 vsize=895228 CPUtime=753.84 cores=0,2,4,6
/proc/22690/task/22698/stat : 22698 (plingeling) R 22688 22690 22319 0 -1 4202560 28605 1253 0 0 75381 3 0 0 25 0 5 0 653392082 916713472 195244 33554432000 4194304 4417452 140735837163776 18446744073709551615 4320938 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22690/tid=22699] ppid=22688 vsize=895228 CPUtime=762.19 cores=0,2,4,6
/proc/22690/task/22699/stat : 22699 (plingeling) R 22688 22690 22319 0 -1 4202560 26167 1253 0 0 76216 3 0 0 25 0 5 0 653392082 916713472 195244 33554432000 4194304 4417452 140735837163776 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22690/tid=22700] ppid=22688 vsize=895228 CPUtime=762.19 cores=0,2,4,6
/proc/22690/task/22700/stat : 22700 (plingeling) R 22688 22690 22319 0 -1 4202560 602987 1253 0 0 76128 91 0 0 25 0 5 0 653392082 916713472 195244 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364984 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3040.4
Current children cumulated vsize (KiB) 895228

[startup+822.301 s]
/proc/loadavg: 8.12 8.10 8.02 9/359 22722
/proc/meminfo: memFree=22353208/32950928 swapFree=45937192/67111528
[pid=22690] ppid=22688 vsize=865684 CPUtime=3279.7 cores=0,2,4,6
/proc/22690/stat : 22690 (plingeling) S 22688 22690 22319 0 -1 4202496 691821 1253 0 0 327866 104 0 0 25 0 5 0 653392068 886460416 187890 33554432000 4194304 4417452 140735837163776 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/22690/statm: 216421 187890 180 55 0 214307 0
[pid=22690/tid=22697] ppid=22688 vsize=865684 CPUtime=821.98 cores=0,2,4,6
/proc/22690/task/22697/stat : 22697 (plingeling) R 22688 22690 22319 0 -1 4202560 16352 1253 0 0 82196 2 0 0 25 0 5 0 653392082 886460416 187890 33554432000 4194304 4417452 140735837163776 18446744073709551615 4346484 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22690/tid=22698] ppid=22688 vsize=865684 CPUtime=813.19 cores=0,2,4,6
/proc/22690/task/22698/stat : 22698 (plingeling) R 22688 22690 22319 0 -1 4202560 28605 1253 0 0 81315 4 0 0 25 0 5 0 653392082 886460416 187890 33554432000 4194304 4417452 140735837163776 18446744073709551615 4366447 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22690/tid=22699] ppid=22688 vsize=865684 CPUtime=822.18 cores=0,2,4,6
/proc/22690/task/22699/stat : 22699 (plingeling) R 22688 22690 22319 0 -1 4202560 26167 1253 0 0 82215 3 0 0 25 0 5 0 653392082 886460416 187890 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22690/tid=22700] ppid=22688 vsize=865684 CPUtime=822.18 cores=0,2,4,6
/proc/22690/task/22700/stat : 22700 (plingeling) R 22688 22690 22319 0 -1 4202560 615979 1253 0 0 82125 93 0 0 25 0 5 0 653392082 886460416 187890 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3279.7
Current children cumulated vsize (KiB) 865684

[startup+882.301 s]
/proc/loadavg: 8.08 8.09 8.02 9/359 22723
/proc/meminfo: memFree=22121436/32950928 swapFree=45937192/67111528
[pid=22690] ppid=22688 vsize=1070164 CPUtime=3519.04 cores=0,2,4,6
/proc/22690/stat : 22690 (plingeling) S 22688 22690 22319 0 -1 4202496 742909 1253 0 0 351794 110 0 0 25 0 5 0 653392068 1095847936 238978 33554432000 4194304 4417452 140735837163776 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/22690/statm: 267541 238978 180 55 0 265427 0
[pid=22690/tid=22697] ppid=22688 vsize=1070164 CPUtime=881.98 cores=0,2,4,6
/proc/22690/task/22697/stat : 22697 (plingeling) R 22688 22690 22319 0 -1 4202560 18735 1253 0 0 88195 3 0 0 25 0 5 0 653392082 1095847936 238978 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22690/tid=22698] ppid=22688 vsize=1070164 CPUtime=872.53 cores=0,2,4,6
/proc/22690/task/22698/stat : 22698 (plingeling) R 22688 22690 22319 0 -1 4202560 31952 1253 0 0 87249 4 0 0 25 0 5 0 653392082 1095847936 238978 33554432000 4194304 4417452 140735837163776 18446744073709551615 4357805 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22690/tid=22699] ppid=22688 vsize=1070164 CPUtime=882.18 cores=0,2,4,6
/proc/22690/task/22699/stat : 22699 (plingeling) R 22688 22690 22319 0 -1 4202560 26167 1253 0 0 88215 3 0 0 25 0 5 0 653392082 1095847936 238978 33554432000 4194304 4417452 140735837163776 18446744073709551615 4365118 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22690/tid=22700] ppid=22688 vsize=1070164 CPUtime=882.18 cores=0,2,4,6
/proc/22690/task/22700/stat : 22700 (plingeling) R 22688 22690 22319 0 -1 4202560 661337 1253 0 0 88120 98 0 0 25 0 5 0 653392082 1095847936 238978 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3519.04
Current children cumulated vsize (KiB) 1070164

[startup+942.301 s]
/proc/loadavg: 8.03 8.07 8.02 9/359 22725
/proc/meminfo: memFree=22011800/32950928 swapFree=45937192/67111528
[pid=22690] ppid=22688 vsize=1190868 CPUtime=3758.39 cores=0,2,4,6
/proc/22690/stat : 22690 (plingeling) S 22688 22690 22319 0 -1 4202496 773085 1253 0 0 375723 116 0 0 25 0 5 0 653392068 1219448832 269154 33554432000 4194304 4417452 140735837163776 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/22690/statm: 297717 269154 180 55 0 295603 0
[pid=22690/tid=22697] ppid=22688 vsize=1190868 CPUtime=941.97 cores=0,2,4,6
/proc/22690/task/22697/stat : 22697 (plingeling) R 22688 22690 22319 0 -1 4202560 18735 1253 0 0 94194 3 0 0 25 0 5 0 653392082 1219448832 269154 33554432000 4194304 4417452 140735837163776 18446744073709551615 4237371 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22690/tid=22698] ppid=22688 vsize=1190868 CPUtime=931.88 cores=0,2,4,6
/proc/22690/task/22698/stat : 22698 (plingeling) R 22688 22690 22319 0 -1 4202560 31952 1253 0 0 93184 4 0 0 25 0 5 0 653392082 1219448832 269154 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22690/tid=22699] ppid=22688 vsize=1190868 CPUtime=942.18 cores=0,2,4,6
/proc/22690/task/22699/stat : 22699 (plingeling) R 22688 22690 22319 0 -1 4202560 26167 1253 0 0 94215 3 0 0 25 0 5 0 653392082 1219448832 269154 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364807 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22690/tid=22700] ppid=22688 vsize=1190868 CPUtime=942.19 cores=0,2,4,6
/proc/22690/task/22700/stat : 22700 (plingeling) R 22688 22690 22319 0 -1 4202560 691513 1253 0 0 94114 105 0 0 25 0 5 0 653392082 1219448832 269154 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3758.39
Current children cumulated vsize (KiB) 1190868

[startup+1002.3 s]
/proc/loadavg: 8.13 8.09 8.02 9/359 22726
/proc/meminfo: memFree=21941720/32950928 swapFree=45937192/67111528
[pid=22690] ppid=22688 vsize=1269172 CPUtime=3997.72 cores=0,2,4,6
/proc/22690/stat : 22690 (plingeling) S 22688 22690 22319 0 -1 4202496 792661 1253 0 0 399654 118 0 0 25 0 5 0 653392068 1299632128 288730 33554432000 4194304 4417452 140735837163776 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/22690/statm: 317293 288730 180 55 0 315179 0
[pid=22690/tid=22697] ppid=22688 vsize=1269172 CPUtime=1001.96 cores=0,2,4,6
/proc/22690/task/22697/stat : 22697 (plingeling) R 22688 22690 22319 0 -1 4202560 18735 1253 0 0 100193 3 0 0 25 0 5 0 653392082 1299632128 288730 33554432000 4194304 4417452 140735837163776 18446744073709551615 4321404 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22690/tid=22698] ppid=22688 vsize=1269172 CPUtime=991.23 cores=0,2,4,6
/proc/22690/task/22698/stat : 22698 (plingeling) R 22688 22690 22319 0 -1 4202560 38216 1253 0 0 99118 5 0 0 25 0 5 0 653392082 1299632128 288730 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22690/tid=22699] ppid=22688 vsize=1269172 CPUtime=1002.18 cores=0,2,4,6
/proc/22690/task/22699/stat : 22699 (plingeling) R 22688 22690 22319 0 -1 4202560 26167 1253 0 0 100215 3 0 0 25 0 5 0 653392082 1299632128 288730 33554432000 4194304 4417452 140735837163776 18446744073709551615 4249209 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22690/tid=22700] ppid=22688 vsize=1269172 CPUtime=1002.19 cores=0,2,4,6
/proc/22690/task/22700/stat : 22700 (plingeling) R 22688 22690 22319 0 -1 4202560 704825 1253 0 0 100113 106 0 0 25 0 5 0 653392082 1299632128 288730 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3997.72
Current children cumulated vsize (KiB) 1269172

[startup+1062.3 s]
/proc/loadavg: 8.09 8.08 8.02 9/359 22728
/proc/meminfo: memFree=21701880/32950928 swapFree=45937192/67111528
[pid=22690] ppid=22688 vsize=1489336 CPUtime=4237.07 cores=0,2,4,6
/proc/22690/stat : 22690 (plingeling) S 22688 22690 22319 0 -1 4202496 847702 1253 0 0 423580 127 0 0 25 0 5 0 653392068 1525080064 343771 33554432000 4194304 4417452 140735837163776 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/22690/statm: 372334 343771 180 55 0 370220 0
[pid=22690/tid=22697] ppid=22688 vsize=1489336 CPUtime=1061.95 cores=0,2,4,6
/proc/22690/task/22697/stat : 22697 (plingeling) R 22688 22690 22319 0 -1 4202560 18735 1253 0 0 106192 3 0 0 25 0 5 0 653392082 1525080064 343771 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22690/tid=22698] ppid=22688 vsize=1489336 CPUtime=1050.58 cores=0,2,4,6
/proc/22690/task/22698/stat : 22698 (plingeling) R 22688 22690 22319 0 -1 4202560 38216 1253 0 0 105053 5 0 0 25 0 5 0 653392082 1525080064 343771 33554432000 4194304 4417452 140735837163776 18446744073709551615 4365004 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22690/tid=22699] ppid=22688 vsize=1489336 CPUtime=1062.19 cores=0,2,4,6
/proc/22690/task/22699/stat : 22699 (plingeling) R 22688 22690 22319 0 -1 4202560 26167 1253 0 0 106216 3 0 0 25 0 5 0 653392082 1525080064 343771 33554432000 4194304 4417452 140735837163776 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22690/tid=22700] ppid=22688 vsize=1489336 CPUtime=1062.2 cores=0,2,4,6
/proc/22690/task/22700/stat : 22700 (plingeling) R 22688 22690 22319 0 -1 4202560 759866 1253 0 0 106105 115 0 0 25 0 5 0 653392082 1525080064 343771 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4237.07
Current children cumulated vsize (KiB) 1489336

[startup+1122.3 s]
/proc/loadavg: 8.07 8.08 8.02 9/359 22730
/proc/meminfo: memFree=21759520/32950928 swapFree=45937192/67111528
[pid=22690] ppid=22688 vsize=1489336 CPUtime=4476.41 cores=0,2,4,6
/proc/22690/stat : 22690 (plingeling) S 22688 22690 22319 0 -1 4202496 847702 1253 0 0 447514 127 0 0 25 0 5 0 653392068 1525080064 343771 33554432000 4194304 4417452 140735837163776 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/22690/statm: 372334 343771 180 55 0 370220 0
[pid=22690/tid=22697] ppid=22688 vsize=1489336 CPUtime=1121.94 cores=0,2,4,6
/proc/22690/task/22697/stat : 22697 (plingeling) R 22688 22690 22319 0 -1 4202560 18735 1253 0 0 112191 3 0 0 25 0 5 0 653392082 1525080064 343771 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22690/tid=22698] ppid=22688 vsize=1489336 CPUtime=1109.92 cores=0,2,4,6
/proc/22690/task/22698/stat : 22698 (plingeling) R 22688 22690 22319 0 -1 4202560 38216 1253 0 0 110987 5 0 0 25 0 5 0 653392082 1525080064 343771 33554432000 4194304 4417452 140735837163776 18446744073709551615 4346508 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22690/tid=22699] ppid=22688 vsize=1489336 CPUtime=1122.19 cores=0,2,4,6
/proc/22690/task/22699/stat : 22699 (plingeling) R 22688 22690 22319 0 -1 4202560 26167 1253 0 0 112216 3 0 0 25 0 5 0 653392082 1525080064 343771 33554432000 4194304 4417452 140735837163776 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22690/tid=22700] ppid=22688 vsize=1489336 CPUtime=1122.2 cores=0,2,4,6
/proc/22690/task/22700/stat : 22700 (plingeling) R 22688 22690 22319 0 -1 4202560 759866 1253 0 0 112105 115 0 0 25 0 5 0 653392082 1525080064 343771 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4476.41
Current children cumulated vsize (KiB) 1489336

[startup+1182.3 s]
/proc/loadavg: 8.17 8.11 8.03 5/353 22732
/proc/meminfo: memFree=22665236/32950928 swapFree=45937192/67111528
[pid=22690] ppid=22688 vsize=1571836 CPUtime=4715.75 cores=0,2,4,6
/proc/22690/stat : 22690 (plingeling) S 22688 22690 22319 0 -1 4202496 868327 1253 0 0 471446 129 0 0 25 0 5 0 653392068 1609560064 364396 33554432000 4194304 4417452 140735837163776 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/22690/statm: 392959 364396 180 55 0 390845 0
[pid=22690/tid=22697] ppid=22688 vsize=1571836 CPUtime=1181.93 cores=0,2,4,6
/proc/22690/task/22697/stat : 22697 (plingeling) R 22688 22690 22319 0 -1 4202560 18735 1253 0 0 118190 3 0 0 25 0 5 0 653392082 1609560064 364396 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364466 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22690/tid=22698] ppid=22688 vsize=1571836 CPUtime=1169.27 cores=0,2,4,6
/proc/22690/task/22698/stat : 22698 (plingeling) R 22688 22690 22319 0 -1 4202560 42457 1253 0 0 116922 5 0 0 25 0 5 0 653392082 1609560064 364396 33554432000 4194304 4417452 140735837163776 18446744073709551615 4250651 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22690/tid=22699] ppid=22688 vsize=1571836 CPUtime=1182.19 cores=0,2,4,6
/proc/22690/task/22699/stat : 22699 (plingeling) R 22688 22690 22319 0 -1 4202560 26167 1253 0 0 118216 3 0 0 25 0 5 0 653392082 1609560064 364396 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22690/tid=22700] ppid=22688 vsize=1571836 CPUtime=1182.19 cores=0,2,4,6
/proc/22690/task/22700/stat : 22700 (plingeling) R 22688 22690 22319 0 -1 4202560 776250 1253 0 0 118103 116 0 0 25 0 5 0 653392082 1609560064 364396 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4715.75
Current children cumulated vsize (KiB) 1571836



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.5 s]
/proc/loadavg: 6.98 7.84 7.94 5/349 22750
/proc/meminfo: memFree=22604552/32950928 swapFree=45937192/67111528
[pid=22690] ppid=22688 vsize=1637376 CPUtime=4800.36 cores=0,2,4,6
/proc/22690/stat : 22690 (plingeling) S 22688 22690 22319 0 -1 4202496 884712 1253 0 0 479905 131 0 0 25 0 5 0 653392068 1676673024 380781 33554432000 4194304 4417452 140735837163776 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/22690/statm: 409344 380781 180 55 0 407230 0
[pid=22690/tid=22697] ppid=22688 vsize=1637376 CPUtime=1203.14 cores=0,2,4,6
/proc/22690/task/22697/stat : 22697 (plingeling) R 22688 22690 22319 0 -1 4202560 18735 1253 0 0 120311 3 0 0 25 0 5 0 653392082 1676673024 380781 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364752 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22690/tid=22698] ppid=22688 vsize=1637376 CPUtime=1190.23 cores=0,2,4,6
/proc/22690/task/22698/stat : 22698 (plingeling) R 22688 22690 22319 0 -1 4202560 42457 1253 0 0 119018 5 0 0 25 0 5 0 653392082 1676673024 380781 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364629 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22690/tid=22699] ppid=22688 vsize=1637376 CPUtime=1203.4 cores=0,2,4,6
/proc/22690/task/22699/stat : 22699 (plingeling) R 22688 22690 22319 0 -1 4202560 26167 1253 0 0 120337 3 0 0 25 0 5 0 653392082 1676673024 380781 33554432000 4194304 4417452 140735837163776 18446744073709551615 4343974 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22690/tid=22700] ppid=22688 vsize=1637376 CPUtime=1203.41 cores=0,2,4,6
/proc/22690/task/22700/stat : 22700 (plingeling) R 22688 22690 22319 0 -1 4202560 792635 1253 0 0 120224 117 0 0 25 0 5 0 653392082 1676673024 380781 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.36
Current children cumulated vsize (KiB) 1637376

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1203.5 s]
/proc/loadavg: 6.98 7.84 7.94 5/349 22750
/proc/meminfo: memFree=22604552/32950928 swapFree=45937192/67111528
[pid=22690] ppid=22688 vsize=1637376 CPUtime=4800.36 cores=0,2,4,6
/proc/22690/stat : 22690 (plingeling) S 22688 22690 22319 0 -1 4202496 884712 1253 0 0 479905 131 0 0 25 0 5 0 653392068 1676673024 380781 33554432000 4194304 4417452 140735837163776 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/22690/statm: 409344 380781 180 55 0 407230 0
[pid=22690/tid=22697] ppid=22688 vsize=1637376 CPUtime=1203.14 cores=0,2,4,6
/proc/22690/task/22697/stat : 22697 (plingeling) R 22688 22690 22319 0 -1 4202560 18735 1253 0 0 120311 3 0 0 25 0 5 0 653392082 1676673024 380781 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364752 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22690/tid=22698] ppid=22688 vsize=1637376 CPUtime=1190.23 cores=0,2,4,6
/proc/22690/task/22698/stat : 22698 (plingeling) R 22688 22690 22319 0 -1 4202560 42457 1253 0 0 119018 5 0 0 25 0 5 0 653392082 1676673024 380781 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364629 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22690/tid=22699] ppid=22688 vsize=1637376 CPUtime=1203.4 cores=0,2,4,6
/proc/22690/task/22699/stat : 22699 (plingeling) R 22688 22690 22319 0 -1 4202560 26167 1253 0 0 120337 3 0 0 25 0 5 0 653392082 1676673024 380781 33554432000 4194304 4417452 140735837163776 18446744073709551615 4343974 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22690/tid=22700] ppid=22688 vsize=1637376 CPUtime=1203.41 cores=0,2,4,6
/proc/22690/task/22700/stat : 22700 (plingeling) R 22688 22690 22319 0 -1 4202560 792635 1253 0 0 120224 117 0 0 25 0 5 0 653392082 1676673024 380781 33554432000 4194304 4417452 140735837163776 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.36
Current children cumulated vsize (KiB) 1637376

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1203.59
CPU time (s): 4800.47
CPU user time (s): 4799.08
CPU system time (s): 1.38079
CPU usage (%): 398.847
Max. virtual memory (cumulated for all children) (KiB): 1637376

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4799.08
system time used= 1.38079
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 885965
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 170
involuntary context switches= 35214

runsolver used 3.83342 second user time and 9.32258 second system time

The end

Launcher Data 

Begin job on node149 at 2011-05-15 02:05:54
IDJOB=3371171
IDBENCH=71294
IDSOLVER=1848
FILE ID=node149/3371171-1305417954
RUNJOBID= node149-1305415515-22337
PBS_JOBID= 13324565
Free space on /tmp= 73568 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT09/CRAFTED/rbsat/random/unforced/rbsat-v2100c215164g9.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3371171-1305417954/watcher-3371171-1305417954 -o /tmp/evaluation-result-3371171-1305417954/solver-3371171-1305417954 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3371171-1305417954.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= e033e9c916da557514299cce9ca2468f
RANDOM SEED=1580264226

node149.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.73
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      24007592 kB
Buffers:        312844 kB
Cached:        3876144 kB
SwapCached:      19984 kB
Active:        6442992 kB
Inactive:      2206328 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      24007592 kB
SwapTotal:    67111528 kB
SwapFree:     45937192 kB
Dirty:            4356 kB
Writeback:           0 kB
AnonPages:     4452756 kB
Mapped:          14604 kB
Slab:           157788 kB
PageTables:      71956 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 80372560 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73568 MiB
End job on node149 at 2011-05-15 02:25:59