Trace number 3394622

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40000.3 5010.65

General information on the benchmark

Namecrafted/kullmann/VanDerWaerden/
VanderWaerden3k/VanDerWaerden_2-3-15_218.cnf
MD5SUMafc70649cebcc24cd0a7b28def593295
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables218
Number of clauses13362
Sum of the clauses size59166
Maximum clause length15
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 311772
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 51590

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3394622-1306100471/watcher-3394622-1306100471 -o /tmp/evaluation-result-3394622-1306100471/solver-3394622-1306100471 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3394622-1306100471.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 5.98 7.58 7.84 2/491 27327
/proc/meminfo: memFree=28070500/32951124 swapFree=53876544/67111528
[pid=27327] ppid=27325 vsize=13296 CPUtime=0.01 cores=0-7
/proc/27327/stat : 27327 (plingeling) R 27325 27327 26995 0 -1 4202496 1479 1253 0 0 1 0 0 0 25 0 1 0 721648026 13615104 1331 33554432000 4194304 4417452 140733995714768 18446744073709551615 270613110528 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/27327/statm: 3324 1331 149 55 0 1210 0

[startup+0.162193 s]
/proc/loadavg: 5.98 7.58 7.84 2/491 27327
/proc/meminfo: memFree=28070500/32951124 swapFree=53876544/67111528
[pid=27327] ppid=27325 vsize=233516 CPUtime=1.82 cores=0-7
/proc/27327/stat : 27327 (plingeling) S 27325 27327 26995 0 -1 4202496 4302 1253 0 0 182 0 0 0 18 0 9 0 721648026 239120384 3053 33554432000 4194304 4417452 140733995714768 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/27327/statm: 60313 5303 179 55 0 58199 0
Current children cumulated CPU time (s) 1.82
Current children cumulated vsize (KiB) 233516

[startup+0.993589 s]
/proc/loadavg: 5.98 7.58 7.84 2/491 27327
/proc/meminfo: memFree=28070500/32951124 swapFree=53876544/67111528
[pid=27327] ppid=27325 vsize=241340 CPUtime=7.59 cores=0-7
/proc/27327/stat : 27327 (plingeling) S 27325 27327 26995 0 -1 4202496 7059 1253 0 0 759 0 0 0 18 0 9 0 721648026 247132160 5325 33554432000 4194304 4417452 140733995714768 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/27327/statm: 60335 5325 179 55 0 58221 0
Current children cumulated CPU time (s) 7.59
Current children cumulated vsize (KiB) 241340

[startup+1.09357 s]
/proc/loadavg: 5.98 7.58 7.84 2/491 27327
/proc/meminfo: memFree=28070500/32951124 swapFree=53876544/67111528
[pid=27327] ppid=27325 vsize=241340 CPUtime=8.39 cores=0-7
/proc/27327/stat : 27327 (plingeling) S 27325 27327 26995 0 -1 4202496 7112 1253 0 0 839 0 0 0 18 0 9 0 721648026 247132160 5363 33554432000 4194304 4417452 140733995714768 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/27327/statm: 60335 5363 179 55 0 58221 0
Current children cumulated CPU time (s) 8.39
Current children cumulated vsize (KiB) 241340

[startup+1.19356 s]
/proc/loadavg: 5.98 7.58 7.84 2/491 27327
/proc/meminfo: memFree=28070500/32951124 swapFree=53876544/67111528
[pid=27327] ppid=27325 vsize=241608 CPUtime=9.19 cores=0-7
/proc/27327/stat : 27327 (plingeling) S 27325 27327 26995 0 -1 4202496 7464 1253 0 0 919 0 0 0 18 0 9 0 721648026 247406592 5473 33554432000 4194304 4417452 140733995714768 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/27327/statm: 60402 5473 179 55 0 58288 0
Current children cumulated CPU time (s) 9.19
Current children cumulated vsize (KiB) 241608

[startup+1.5005 s]
/proc/loadavg: 5.98 7.58 7.84 2/491 27327
/proc/meminfo: memFree=28070500/32951124 swapFree=53876544/67111528
[pid=27327] ppid=27325 vsize=242744 CPUtime=11.64 cores=0-7
/proc/27327/stat : 27327 (plingeling) S 27325 27327 26995 0 -1 4202496 8235 1253 0 0 1164 0 0 0 18 0 9 0 721648026 248569856 5937 33554432000 4194304 4417452 140733995714768 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/27327/statm: 60686 5937 179 55 0 58572 0
Current children cumulated CPU time (s) 11.64
Current children cumulated vsize (KiB) 242744

[startup+3.10122 s]
/proc/loadavg: 6.22 7.60 7.85 9/500 27341
/proc/meminfo: memFree=28035676/32951124 swapFree=53876544/67111528
[pid=27327] ppid=27325 vsize=255924 CPUtime=24.42 cores=0-7
/proc/27327/stat : 27327 (plingeling) S 27325 27327 26995 0 -1 4202496 15945 1253 0 0 2440 2 0 0 18 0 9 0 721648026 262066176 9261 33554432000 4194304 4417452 140733995714768 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/27327/statm: 63981 9261 179 55 0 61867 0
[pid=27327/tid=27334] ppid=27325 vsize=255924 CPUtime=3.05 cores=0-7
/proc/27327/task/27334/stat : 27334 (plingeling) R 27325 27327 26995 0 -1 4202560 1707 1253 0 0 305 0 0 0 25 0 9 0 721648030 262066176 9261 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=27327/tid=27335] ppid=27325 vsize=255924 CPUtime=3.05 cores=0-7
/proc/27327/task/27335/stat : 27335 (plingeling) R 27325 27327 26995 0 -1 4202560 1324 1253 0 0 305 0 0 0 25 0 9 0 721648030 262066176 9261 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27327/tid=27336] ppid=27325 vsize=255924 CPUtime=3.05 cores=0-7
/proc/27327/task/27336/stat : 27336 (plingeling) R 27325 27327 26995 0 -1 4202560 737 1253 0 0 305 0 0 0 25 0 9 0 721648030 262066176 9261 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27327/tid=27337] ppid=27325 vsize=255924 CPUtime=3.05 cores=0-7
/proc/27327/task/27337/stat : 27337 (plingeling) R 27325 27327 26995 0 -1 4202560 3450 1253 0 0 305 0 0 0 25 0 9 0 721648030 262066176 9261 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27327/tid=27338] ppid=27325 vsize=255924 CPUtime=3.04 cores=0-7
/proc/27327/task/27338/stat : 27338 (plingeling) R 27325 27327 26995 0 -1 4202560 2051 1253 0 0 304 0 0 0 25 0 9 0 721648030 262066176 9261 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=27327/tid=27339] ppid=27325 vsize=255924 CPUtime=3.05 cores=0-7
/proc/27327/task/27339/stat : 27339 (plingeling) R 27325 27327 26995 0 -1 4202560 490 1253 0 0 305 0 0 0 25 0 9 0 721648030 262066176 9261 33554432000 4194304 4417452 140733995714768 18446744073709551615 18446744073699066094 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27327/tid=27340] ppid=27325 vsize=255924 CPUtime=3.02 cores=0-7
/proc/27327/task/27340/stat : 27340 (plingeling) R 27325 27327 26995 0 -1 4202560 2171 1253 0 0 302 0 0 0 25 0 9 0 721648030 262066176 9261 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364420 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=27327/tid=27341] ppid=27325 vsize=255924 CPUtime=3.01 cores=0-7
/proc/27327/task/27341/stat : 27341 (plingeling) R 27325 27327 26995 0 -1 4202560 1251 1253 0 0 301 0 0 0 25 0 9 0 721648030 262066176 9261 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 24.42
Current children cumulated vsize (KiB) 255924

[startup+6.31865 s]
/proc/loadavg: 6.22 7.60 7.85 9/500 27341
/proc/meminfo: memFree=28027364/32951124 swapFree=53876544/67111528
[pid=27327] ppid=27325 vsize=263668 CPUtime=50.1 cores=0-7
/proc/27327/stat : 27327 (plingeling) S 27325 27327 26995 0 -1 4202496 36941 1253 0 0 5004 6 0 0 18 0 9 0 721648026 269996032 10905 33554432000 4194304 4417452 140733995714768 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/27327/statm: 65917 10905 179 55 0 63803 0
[pid=27327/tid=27334] ppid=27325 vsize=263668 CPUtime=6.26 cores=0-7
/proc/27327/task/27334/stat : 27334 (plingeling) R 27325 27327 26995 0 -1 4202560 4732 1253 0 0 626 0 0 0 25 0 9 0 721648030 269996032 10905 33554432000 4194304 4417452 140733995714768 18446744073709551615 18446744073699065939 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=27327/tid=27335] ppid=27325 vsize=263668 CPUtime=6.27 cores=0-7
/proc/27327/task/27335/stat : 27335 (plingeling) R 27325 27327 26995 0 -1 4202560 3155 1253 0 0 627 0 0 0 25 0 9 0 721648030 269996032 10905 33554432000 4194304 4417452 140733995714768 18446744073709551615 4365999 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27327/tid=27336] ppid=27325 vsize=263668 CPUtime=6.27 cores=0-7
/proc/27327/task/27336/stat : 27336 (plingeling) R 27325 27327 26995 0 -1 4202560 3234 1253 0 0 627 0 0 0 25 0 9 0 721648030 269996032 10905 33554432000 4194304 4417452 140733995714768 18446744073709551615 18446744073699065894 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27327/tid=27337] ppid=27325 vsize=263668 CPUtime=6.27 cores=0-7
/proc/27327/task/27337/stat : 27337 (plingeling) R 27325 27327 26995 0 -1 4202560 6534 1253 0 0 626 1 0 0 25 0 9 0 721648030 269996032 10905 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27327/tid=27338] ppid=27325 vsize=263668 CPUtime=6.25 cores=0-7
/proc/27327/task/27338/stat : 27338 (plingeling) R 27325 27327 26995 0 -1 4202560 4750 1253 0 0 625 0 0 0 25 0 9 0 721648030 269996032 10905 33554432000 4194304 4417452 140733995714768 18446744073709551615 4357851 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=27327/tid=27339] ppid=27325 vsize=263668 CPUtime=6.27 cores=0-7
/proc/27327/task/27339/stat : 27339 (plingeling) R 27325 27327 26995 0 -1 4202560 2269 1253 0 0 627 0 0 0 25 0 9 0 721648030 269996032 10905 33554432000 4194304 4417452 140733995714768 18446744073709551615 4365959 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27327/tid=27340] ppid=27325 vsize=263668 CPUtime=6.24 cores=0-7
/proc/27327/task/27340/stat : 27340 (plingeling) R 27325 27327 26995 0 -1 4202560 5292 1253 0 0 623 1 0 0 25 0 9 0 721648030 269996032 10905 33554432000 4194304 4417452 140733995714768 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=27327/tid=27341] ppid=27325 vsize=263668 CPUtime=6.17 cores=0-7
/proc/27327/task/27341/stat : 27341 (plingeling) R 27325 27327 26995 0 -1 4202560 4211 1253 0 0 617 0 0 0 25 0 9 0 721648030 269996032 10905 33554432000 4194304 4417452 140733995714768 18446744073709551615 4366079 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 50.1
Current children cumulated vsize (KiB) 263668

[startup+12.7005 s]
/proc/loadavg: 6.50 7.62 7.85 9/500 27341
/proc/meminfo: memFree=28024508/32951124 swapFree=53876544/67111528
[pid=27327] ppid=27325 vsize=272196 CPUtime=101.07 cores=0-7
/proc/27327/stat : 27327 (plingeling) S 27325 27327 26995 0 -1 4202496 78974 1253 0 0 10093 14 0 0 18 0 9 0 721648026 278728704 12952 33554432000 4194304 4417452 140733995714768 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/27327/statm: 68049 12952 179 55 0 65935 0
[pid=27327/tid=27334] ppid=27325 vsize=272196 CPUtime=12.65 cores=0-7
/proc/27327/task/27334/stat : 27334 (plingeling) R 27325 27327 26995 0 -1 4202560 9569 1253 0 0 1263 2 0 0 25 0 9 0 721648030 278728704 12952 33554432000 4194304 4417452 140733995714768 18446744073709551615 4365973 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=27327/tid=27335] ppid=27325 vsize=272196 CPUtime=12.65 cores=0-7
/proc/27327/task/27335/stat : 27335 (plingeling) R 27325 27327 26995 0 -1 4202560 9375 1253 0 0 1264 1 0 0 25 0 9 0 721648030 278728704 12952 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27327/tid=27336] ppid=27325 vsize=272196 CPUtime=12.65 cores=0-7
/proc/27327/task/27336/stat : 27336 (plingeling) R 27325 27327 26995 0 -1 4202560 8561 1253 0 0 1264 1 0 0 25 0 9 0 721648030 278728704 12952 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27327/tid=27337] ppid=27325 vsize=272196 CPUtime=12.65 cores=0-7
/proc/27327/task/27337/stat : 27337 (plingeling) R 27325 27327 26995 0 -1 4202560 12270 1253 0 0 1263 2 0 0 25 0 9 0 721648030 278728704 12952 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27327/tid=27338] ppid=27325 vsize=272196 CPUtime=12.64 cores=0-7
/proc/27327/task/27338/stat : 27338 (plingeling) R 27325 27327 26995 0 -1 4202560 9235 1253 0 0 1263 1 0 0 25 0 9 0 721648030 278728704 12952 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=27327/tid=27339] ppid=27325 vsize=272196 CPUtime=12.65 cores=0-7
/proc/27327/task/27339/stat : 27339 (plingeling) R 27325 27327 26995 0 -1 4202560 6861 1253 0 0 1264 1 0 0 25 0 9 0 721648030 278728704 12952 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27327/tid=27340] ppid=27325 vsize=272196 CPUtime=12.62 cores=0-7
/proc/27327/task/27340/stat : 27340 (plingeling) R 27325 27327 26995 0 -1 4202560 10925 1253 0 0 1260 2 0 0 25 0 9 0 721648030 278728704 12952 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364675 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=27327/tid=27341] ppid=27325 vsize=272196 CPUtime=12.47 cores=0-7
/proc/27327/task/27341/stat : 27341 (plingeling) R 27325 27327 26995 0 -1 4202560 9414 1253 0 0 1246 1 0 0 25 0 9 0 721648030 278728704 12952 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 101.07
Current children cumulated vsize (KiB) 272196

[startup+25.5013 s]
/proc/loadavg: 6.73 7.63 7.85 9/500 27342
/proc/meminfo: memFree=28003984/32951124 swapFree=53876544/67111528
[pid=27327] ppid=27325 vsize=286944 CPUtime=203.26 cores=0-7
/proc/27327/stat : 27327 (plingeling) S 27325 27327 26995 0 -1 4202496 157434 1253 0 0 20297 29 0 0 18 0 9 0 721648026 293830656 16637 33554432000 4194304 4417452 140733995714768 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/27327/statm: 71736 16637 179 55 0 69622 0
[pid=27327/tid=27334] ppid=27325 vsize=286944 CPUtime=25.45 cores=0-7
/proc/27327/task/27334/stat : 27334 (plingeling) R 27325 27327 26995 0 -1 4202560 18435 1253 0 0 2542 3 0 0 25 0 9 0 721648030 293830656 16637 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=27327/tid=27335] ppid=27325 vsize=286944 CPUtime=25.45 cores=0-7
/proc/27327/task/27335/stat : 27335 (plingeling) R 27325 27327 26995 0 -1 4202560 20365 1253 0 0 2542 3 0 0 25 0 9 0 721648030 293830656 16637 33554432000 4194304 4417452 140733995714768 18446744073709551615 4354706 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27327/tid=27336] ppid=27325 vsize=286944 CPUtime=25.45 cores=0-7
/proc/27327/task/27336/stat : 27336 (plingeling) R 27325 27327 26995 0 -1 4202560 18061 1253 0 0 2542 3 0 0 25 0 9 0 721648030 293830656 16637 33554432000 4194304 4417452 140733995714768 18446744073709551615 4365023 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27327/tid=27337] ppid=27325 vsize=286944 CPUtime=25.45 cores=0-7
/proc/27327/task/27337/stat : 27337 (plingeling) R 27325 27327 26995 0 -1 4202560 23661 1253 0 0 2540 5 0 0 25 0 9 0 721648030 293830656 16637 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27327/tid=27338] ppid=27325 vsize=286944 CPUtime=25.43 cores=0-7
/proc/27327/task/27338/stat : 27338 (plingeling) R 27325 27327 26995 0 -1 4202560 20415 1253 0 0 2541 2 0 0 25 0 9 0 721648030 293830656 16637 33554432000 4194304 4417452 140733995714768 18446744073709551615 4365996 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=27327/tid=27339] ppid=27325 vsize=286944 CPUtime=25.45 cores=0-7
/proc/27327/task/27339/stat : 27339 (plingeling) R 27325 27327 26995 0 -1 4202560 15472 1253 0 0 2543 2 0 0 25 0 9 0 721648030 293830656 16637 33554432000 4194304 4417452 140733995714768 18446744073709551615 4366804 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27327/tid=27340] ppid=27325 vsize=286944 CPUtime=25.42 cores=0-7
/proc/27327/task/27340/stat : 27340 (plingeling) R 27325 27327 26995 0 -1 4202560 19809 1253 0 0 2538 4 0 0 25 0 9 0 721648030 293830656 16637 33554432000 4194304 4417452 140733995714768 18446744073709551615 4236549 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=27327/tid=27341] ppid=27325 vsize=286944 CPUtime=25.05 cores=0-7
/proc/27327/task/27341/stat : 27341 (plingeling) R 27325 27327 26995 0 -1 4202560 18452 1253 0 0 2502 3 0 0 25 0 9 0 721648030 293830656 16637 33554432000 4194304 4417452 140733995714768 18446744073709551615 18446744073699065949 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 203.26
Current children cumulated vsize (KiB) 286944

[startup+51.1007 s]
/proc/loadavg: 7.16 7.66 7.85 9/500 27342
/proc/meminfo: memFree=27970496/32951124 swapFree=53876544/67111528
[pid=27327] ppid=27325 vsize=306748 CPUtime=407.62 cores=0-7
/proc/27327/stat : 27327 (plingeling) S 27325 27327 26995 0 -1 4202496 306053 1253 0 0 40704 58 0 0 18 0 9 0 721648026 314109952 21578 33554432000 4194304 4417452 140733995714768 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/27327/statm: 76687 21578 179 55 0 74573 0
[pid=27327/tid=27334] ppid=27325 vsize=306748 CPUtime=51.05 cores=0-7
/proc/27327/task/27334/stat : 27334 (plingeling) R 27325 27327 26995 0 -1 4202560 38592 1253 0 0 5099 6 0 0 25 0 9 0 721648030 314109952 21578 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=27327/tid=27335] ppid=27325 vsize=306748 CPUtime=51.04 cores=0-7
/proc/27327/task/27335/stat : 27335 (plingeling) R 27325 27327 26995 0 -1 4202560 38645 1253 0 0 5097 7 0 0 25 0 9 0 721648030 314109952 21578 33554432000 4194304 4417452 140733995714768 18446744073709551615 4366099 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27327/tid=27336] ppid=27325 vsize=306748 CPUtime=51.04 cores=0-7
/proc/27327/task/27336/stat : 27336 (plingeling) R 27325 27327 26995 0 -1 4202560 36544 1253 0 0 5098 6 0 0 25 0 9 0 721648030 314109952 21578 33554432000 4194304 4417452 140733995714768 18446744073709551615 4232644 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27327/tid=27337] ppid=27325 vsize=306748 CPUtime=51.04 cores=0-7
/proc/27327/task/27337/stat : 27337 (plingeling) R 27325 27327 26995 0 -1 4202560 39501 1253 0 0 5096 8 0 0 25 0 9 0 721648030 314109952 21578 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364724 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27327/tid=27338] ppid=27325 vsize=306748 CPUtime=51.04 cores=0-7
/proc/27327/task/27338/stat : 27338 (plingeling) R 27325 27327 26995 0 -1 4202560 41140 1253 0 0 5097 7 0 0 25 0 9 0 721648030 314109952 21578 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=27327/tid=27339] ppid=27325 vsize=306748 CPUtime=51.05 cores=0-7
/proc/27327/task/27339/stat : 27339 (plingeling) R 27325 27327 26995 0 -1 4202560 34588 1253 0 0 5099 6 0 0 25 0 9 0 721648030 314109952 21578 33554432000 4194304 4417452 140733995714768 18446744073709551615 4365984 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27327/tid=27340] ppid=27325 vsize=306748 CPUtime=51.02 cores=0-7
/proc/27327/task/27340/stat : 27340 (plingeling) R 27325 27327 26995 0 -1 4202560 38401 1253 0 0 5094 8 0 0 25 0 9 0 721648030 314109952 21578 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=27327/tid=27341] ppid=27325 vsize=306748 CPUtime=50.24 cores=0-7
/proc/27327/task/27341/stat : 27341 (plingeling) R 27325 27327 26995 0 -1 4202560 35878 1253 0 0 5018 6 0 0 25 0 9 0 721648030 314109952 21578 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 407.62
Current children cumulated vsize (KiB) 306748

[startup+102.301 s]
/proc/loadavg: 7.74 7.73 7.86 9/500 27344

################
# More data... #
################

/proc/27327/task/27337/stat : 27337 (plingeling) R 27325 27327 26995 0 -1 4202560 1453426 1253 0 0 465825 271 0 0 25 0 9 0 721648030 696852480 114950 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27327/tid=27338] ppid=27325 vsize=680520 CPUtime=4662.34 cores=0-7
/proc/27327/task/27338/stat : 27338 (plingeling) R 27325 27327 26995 0 -1 4202560 1303695 1253 0 0 466023 211 0 0 25 0 9 0 721648030 696852480 114950 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=27327/tid=27339] ppid=27325 vsize=680520 CPUtime=4661.7 cores=0-7
/proc/27327/task/27339/stat : 27339 (plingeling) R 27325 27327 26995 0 -1 4202560 1184969 1253 0 0 465959 211 0 0 25 0 9 0 721648030 696852480 114950 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27327/tid=27340] ppid=27325 vsize=680520 CPUtime=4662.33 cores=0-7
/proc/27327/task/27340/stat : 27340 (plingeling) R 27325 27327 26995 0 -1 4202560 1211368 1253 0 0 466031 202 0 0 25 0 9 0 721648030 696852480 114950 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=27327/tid=27341] ppid=27325 vsize=680520 CPUtime=4630.14 cores=0-7
/proc/27327/task/27341/stat : 27341 (plingeling) R 27325 27327 26995 0 -1 4202560 1142179 1253 0 0 462823 191 0 0 25 0 9 0 721648030 696852480 114950 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 37220.3
Current children cumulated vsize (KiB) 680520

[startup+4722.3 s]
/proc/loadavg: 8.07 8.08 8.02 9/500 27479
/proc/meminfo: memFree=27568724/32951124 swapFree=53876544/67111528
[pid=27327] ppid=27325 vsize=720604 CPUtime=37699.3 cores=0-7
/proc/27327/stat : 27327 (plingeling) S 27325 27327 26995 0 -1 4202496 9889142 1253 0 0 3768221 1712 0 0 18 0 9 0 721648026 737898496 124971 33554432000 4194304 4417452 140733995714768 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/27327/statm: 180151 124971 179 55 0 178037 0
[pid=27327/tid=27334] ppid=27325 vsize=720604 CPUtime=4722.32 cores=0-7
/proc/27327/task/27334/stat : 27334 (plingeling) R 27325 27327 26995 0 -1 4202560 1210602 1253 0 0 472020 212 0 0 25 0 9 0 721648030 737898496 124971 33554432000 4194304 4417452 140733995714768 18446744073709551615 4321266 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=27327/tid=27335] ppid=27325 vsize=720604 CPUtime=4677.27 cores=0-7
/proc/27327/task/27335/stat : 27335 (plingeling) R 27325 27327 26995 0 -1 4202560 1163074 1253 0 0 467531 196 0 0 25 0 9 0 721648030 737898496 124971 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=27327/tid=27336] ppid=27325 vsize=720604 CPUtime=4722.19 cores=0-7
/proc/27327/task/27336/stat : 27336 (plingeling) R 27325 27327 26995 0 -1 4202560 1178364 1253 0 0 472011 208 0 0 25 0 9 0 721648030 737898496 124971 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27327/tid=27337] ppid=27325 vsize=720604 CPUtime=4720.93 cores=0-7
/proc/27327/task/27337/stat : 27337 (plingeling) R 27325 27327 26995 0 -1 4202560 1460388 1253 0 0 471821 272 0 0 25 0 9 0 721648030 737898496 124971 33554432000 4194304 4417452 140733995714768 18446744073709551615 4237250 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27327/tid=27338] ppid=27325 vsize=720604 CPUtime=4722.35 cores=0-7
/proc/27327/task/27338/stat : 27338 (plingeling) R 27325 27327 26995 0 -1 4202560 1312212 1253 0 0 472022 213 0 0 25 0 9 0 721648030 737898496 124971 33554432000 4194304 4417452 140733995714768 18446744073709551615 4365058 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=27327/tid=27339] ppid=27325 vsize=720604 CPUtime=4721.7 cores=0-7
/proc/27327/task/27339/stat : 27339 (plingeling) R 27325 27327 26995 0 -1 4202560 1193711 1253 0 0 471957 213 0 0 25 0 9 0 721648030 737898496 124971 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27327/tid=27340] ppid=27325 vsize=720604 CPUtime=4722.32 cores=0-7
/proc/27327/task/27340/stat : 27340 (plingeling) R 27325 27327 26995 0 -1 4202560 1218667 1253 0 0 472030 202 0 0 25 0 9 0 721648030 737898496 124971 33554432000 4194304 4417452 140733995714768 18446744073709551615 4343764 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=27327/tid=27341] ppid=27325 vsize=720604 CPUtime=4690.14 cores=0-7
/proc/27327/task/27341/stat : 27341 (plingeling) R 27325 27327 26995 0 -1 4202560 1149360 1253 0 0 468822 192 0 0 25 0 9 0 721648030 737898496 124971 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 37699.3
Current children cumulated vsize (KiB) 720604

[startup+4782.3 s]
/proc/loadavg: 8.07 8.08 8.02 9/500 27481
/proc/meminfo: memFree=27545420/32951124 swapFree=53876544/67111528
[pid=27327] ppid=27325 vsize=743800 CPUtime=38178.4 cores=0-7
/proc/27327/stat : 27327 (plingeling) S 27325 27327 26995 0 -1 4202496 9984940 1253 0 0 3816107 1729 0 0 18 0 9 0 721648026 761651200 130769 33554432000 4194304 4417452 140733995714768 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/27327/statm: 185950 130769 179 55 0 183836 0
[pid=27327/tid=27334] ppid=27325 vsize=743800 CPUtime=4782.32 cores=0-7
/proc/27327/task/27334/stat : 27334 (plingeling) R 27325 27327 26995 0 -1 4202560 1220175 1253 0 0 478019 213 0 0 25 0 9 0 721648030 761651200 130769 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=27327/tid=27335] ppid=27325 vsize=743800 CPUtime=4736.3 cores=0-7
/proc/27327/task/27335/stat : 27335 (plingeling) R 27325 27327 26995 0 -1 4202560 1175807 1253 0 0 473432 198 0 0 25 0 9 0 721648030 761651200 130769 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=27327/tid=27336] ppid=27325 vsize=743800 CPUtime=4782.19 cores=0-7
/proc/27327/task/27336/stat : 27336 (plingeling) R 27325 27327 26995 0 -1 4202560 1188953 1253 0 0 478009 210 0 0 25 0 9 0 721648030 761651200 130769 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364629 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27327/tid=27337] ppid=27325 vsize=743800 CPUtime=4780.92 cores=0-7
/proc/27327/task/27337/stat : 27337 (plingeling) R 27325 27327 26995 0 -1 4202560 1477706 1253 0 0 477817 275 0 0 25 0 9 0 721648030 761651200 130769 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27327/tid=27338] ppid=27325 vsize=743800 CPUtime=4782.35 cores=0-7
/proc/27327/task/27338/stat : 27338 (plingeling) R 27325 27327 26995 0 -1 4202560 1317910 1253 0 0 478022 213 0 0 25 0 9 0 721648030 761651200 130769 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=27327/tid=27339] ppid=27325 vsize=743800 CPUtime=4781.7 cores=0-7
/proc/27327/task/27339/stat : 27339 (plingeling) R 27325 27327 26995 0 -1 4202560 1203151 1253 0 0 477955 215 0 0 25 0 9 0 721648030 761651200 130769 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27327/tid=27340] ppid=27325 vsize=743800 CPUtime=4782.33 cores=0-7
/proc/27327/task/27340/stat : 27340 (plingeling) R 27325 27327 26995 0 -1 4202560 1231821 1253 0 0 478028 205 0 0 25 0 9 0 721648030 761651200 130769 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=27327/tid=27341] ppid=27325 vsize=743800 CPUtime=4750.14 cores=0-7
/proc/27327/task/27341/stat : 27341 (plingeling) R 27325 27327 26995 0 -1 4202560 1166653 1253 0 0 474818 196 0 0 25 0 9 0 721648030 761651200 130769 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38178.4
Current children cumulated vsize (KiB) 743800

[startup+4842.3 s]
/proc/loadavg: 8.10 8.09 8.02 9/500 27491
/proc/meminfo: memFree=27565872/32951124 swapFree=53876544/67111528
[pid=27327] ppid=27325 vsize=723232 CPUtime=38657.3 cores=0-7
/proc/27327/stat : 27327 (plingeling) S 27325 27327 26995 0 -1 4202496 10064317 1253 0 0 3863992 1741 0 0 18 0 9 0 721648026 740589568 125627 33554432000 4194304 4417452 140733995714768 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/27327/statm: 180808 125627 179 55 0 178694 0
[pid=27327/tid=27334] ppid=27325 vsize=723232 CPUtime=4842.33 cores=0-7
/proc/27327/task/27334/stat : 27334 (plingeling) R 27325 27327 26995 0 -1 4202560 1230183 1253 0 0 484017 216 0 0 25 0 9 0 721648030 740589568 125627 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364984 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=27327/tid=27335] ppid=27325 vsize=723232 CPUtime=4795.3 cores=0-7
/proc/27327/task/27335/stat : 27335 (plingeling) R 27325 27327 26995 0 -1 4202560 1182717 1253 0 0 479331 199 0 0 25 0 9 0 721648030 740589568 125627 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=27327/tid=27336] ppid=27325 vsize=723232 CPUtime=4842.19 cores=0-7
/proc/27327/task/27336/stat : 27336 (plingeling) R 27325 27327 26995 0 -1 4202560 1194533 1253 0 0 484009 210 0 0 25 0 9 0 721648030 740589568 125627 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27327/tid=27337] ppid=27325 vsize=723232 CPUtime=4840.88 cores=0-7
/proc/27327/task/27337/stat : 27337 (plingeling) R 27325 27327 26995 0 -1 4202560 1498830 1253 0 0 483810 278 0 0 25 0 9 0 721648030 740589568 125627 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27327/tid=27338] ppid=27325 vsize=723232 CPUtime=4842.35 cores=0-7
/proc/27327/task/27338/stat : 27338 (plingeling) R 27325 27327 26995 0 -1 4202560 1332605 1253 0 0 484020 215 0 0 25 0 9 0 721648030 740589568 125627 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=27327/tid=27339] ppid=27325 vsize=723232 CPUtime=4841.69 cores=0-7
/proc/27327/task/27339/stat : 27339 (plingeling) R 27325 27327 26995 0 -1 4202560 1210885 1253 0 0 483953 216 0 0 25 0 9 0 721648030 740589568 125627 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27327/tid=27340] ppid=27325 vsize=723232 CPUtime=4842.33 cores=0-7
/proc/27327/task/27340/stat : 27340 (plingeling) R 27325 27327 26995 0 -1 4202560 1238589 1253 0 0 484027 206 0 0 25 0 9 0 721648030 740589568 125627 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364829 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=27327/tid=27341] ppid=27325 vsize=723232 CPUtime=4810.14 cores=0-7
/proc/27327/task/27341/stat : 27341 (plingeling) R 27325 27327 26995 0 -1 4202560 1173211 1253 0 0 480817 197 0 0 25 0 9 0 721648030 740589568 125627 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364765 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38657.3
Current children cumulated vsize (KiB) 723232

[startup+4902.3 s]
/proc/loadavg: 8.07 8.08 8.02 9/500 27493
/proc/meminfo: memFree=27555688/32951124 swapFree=53876544/67111528
[pid=27327] ppid=27325 vsize=733072 CPUtime=39136.5 cores=0-7
/proc/27327/stat : 27327 (plingeling) S 27325 27327 26995 0 -1 4202496 10140029 1253 0 0 3911895 1752 0 0 18 0 9 0 721648026 750665728 128087 33554432000 4194304 4417452 140733995714768 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/27327/statm: 183268 128087 179 55 0 181154 0
[pid=27327/tid=27334] ppid=27325 vsize=733072 CPUtime=4902.34 cores=0-7
/proc/27327/task/27334/stat : 27334 (plingeling) R 27325 27327 26995 0 -1 4202560 1238332 1253 0 0 490017 217 0 0 25 0 9 0 721648030 750665728 128087 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=27327/tid=27335] ppid=27325 vsize=733072 CPUtime=4854.33 cores=0-7
/proc/27327/task/27335/stat : 27335 (plingeling) R 27325 27327 26995 0 -1 4202560 1191322 1253 0 0 485233 200 0 0 25 0 9 0 721648030 750665728 128087 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=27327/tid=27336] ppid=27325 vsize=733072 CPUtime=4902.21 cores=0-7
/proc/27327/task/27336/stat : 27336 (plingeling) R 27325 27327 26995 0 -1 4202560 1196914 1253 0 0 490010 211 0 0 25 0 9 0 721648030 750665728 128087 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27327/tid=27337] ppid=27325 vsize=733072 CPUtime=4900.88 cores=0-7
/proc/27327/task/27337/stat : 27337 (plingeling) R 27325 27327 26995 0 -1 4202560 1516953 1253 0 0 489806 282 0 0 25 0 9 0 721648030 750665728 128087 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364548 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27327/tid=27338] ppid=27325 vsize=733072 CPUtime=4902.37 cores=0-7
/proc/27327/task/27338/stat : 27338 (plingeling) R 27325 27327 26995 0 -1 4202560 1347803 1253 0 0 490020 217 0 0 25 0 9 0 721648030 750665728 128087 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364780 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=27327/tid=27339] ppid=27325 vsize=733072 CPUtime=4901.71 cores=0-7
/proc/27327/task/27339/stat : 27339 (plingeling) R 27325 27327 26995 0 -1 4202560 1219887 1253 0 0 489954 217 0 0 25 0 9 0 721648030 750665728 128087 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364675 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27327/tid=27340] ppid=27325 vsize=733072 CPUtime=4902.35 cores=0-7
/proc/27327/task/27340/stat : 27340 (plingeling) R 27325 27327 26995 0 -1 4202560 1244450 1253 0 0 490028 207 0 0 25 0 9 0 721648030 750665728 128087 33554432000 4194304 4417452 140733995714768 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=27327/tid=27341] ppid=27325 vsize=733072 CPUtime=4870.16 cores=0-7
/proc/27327/task/27341/stat : 27341 (plingeling) R 27325 27327 26995 0 -1 4202560 1181604 1253 0 0 486818 198 0 0 25 0 9 0 721648030 750665728 128087 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39136.5
Current children cumulated vsize (KiB) 733072

[startup+4962.3 s]
/proc/loadavg: 8.02 8.07 8.02 9/500 27496
/proc/meminfo: memFree=27517112/32951124 swapFree=53876544/67111528
[pid=27327] ppid=27325 vsize=772140 CPUtime=39615.4 cores=0-7
/proc/27327/stat : 27327 (plingeling) S 27325 27327 26995 0 -1 4202496 10204722 1253 0 0 3959773 1762 0 0 18 0 9 0 721648026 790671360 137854 33554432000 4194304 4417452 140733995714768 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/27327/statm: 193035 137854 179 55 0 190921 0
[pid=27327/tid=27334] ppid=27325 vsize=772140 CPUtime=4962.33 cores=0-7
/proc/27327/task/27334/stat : 27334 (plingeling) R 27325 27327 26995 0 -1 4202560 1248133 1253 0 0 496014 219 0 0 25 0 9 0 721648030 790671360 137854 33554432000 4194304 4417452 140733995714768 18446744073709551615 4366469 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=27327/tid=27335] ppid=27325 vsize=772140 CPUtime=4913.34 cores=0-7
/proc/27327/task/27335/stat : 27335 (plingeling) R 27325 27327 26995 0 -1 4202560 1196008 1253 0 0 491134 200 0 0 25 0 9 0 721648030 790671360 137854 33554432000 4194304 4417452 140733995714768 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=27327/tid=27336] ppid=27325 vsize=772140 CPUtime=4962.19 cores=0-7
/proc/27327/task/27336/stat : 27336 (plingeling) R 27325 27327 26995 0 -1 4202560 1204153 1253 0 0 496007 212 0 0 25 0 9 0 721648030 790671360 137854 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27327/tid=27337] ppid=27325 vsize=772140 CPUtime=4960.84 cores=0-7
/proc/27327/task/27337/stat : 27337 (plingeling) R 27325 27327 26995 0 -1 4202560 1528148 1253 0 0 495801 283 0 0 25 0 9 0 721648030 790671360 137854 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27327/tid=27338] ppid=27325 vsize=772140 CPUtime=4962.35 cores=0-7
/proc/27327/task/27338/stat : 27338 (plingeling) R 27325 27327 26995 0 -1 4202560 1354332 1253 0 0 496018 217 0 0 25 0 9 0 721648030 790671360 137854 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=27327/tid=27339] ppid=27325 vsize=772140 CPUtime=4961.7 cores=0-7
/proc/27327/task/27339/stat : 27339 (plingeling) R 27325 27327 26995 0 -1 4202560 1231040 1253 0 0 495951 219 0 0 25 0 9 0 721648030 790671360 137854 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27327/tid=27340] ppid=27325 vsize=772140 CPUtime=4962.33 cores=0-7
/proc/27327/task/27340/stat : 27340 (plingeling) R 27325 27327 26995 0 -1 4202560 1252039 1253 0 0 496025 208 0 0 25 0 9 0 721648030 790671360 137854 33554432000 4194304 4417452 140733995714768 18446744073709551615 4343748 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=27327/tid=27341] ppid=27325 vsize=772140 CPUtime=4930.15 cores=0-7
/proc/27327/task/27341/stat : 27341 (plingeling) R 27325 27327 26995 0 -1 4202560 1188105 1253 0 0 492815 200 0 0 25 0 9 0 721648030 790671360 137854 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39615.4
Current children cumulated vsize (KiB) 772140



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5010.6 s]
/proc/loadavg: 8.14 8.10 8.03 9/500 27499
/proc/meminfo: memFree=27555020/32951124 swapFree=53876544/67111528
[pid=27327] ppid=27325 vsize=733944 CPUtime=40000.3 cores=0-7
/proc/27327/stat : 27327 (plingeling) S 27325 27327 26995 0 -1 4202496 10286427 1253 0 0 3998248 1778 0 0 18 0 9 0 721648026 751558656 128331 33554432000 4194304 4417452 140733995714768 18446744073709551615 270624913381 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/27327/statm: 183486 128331 179 55 0 181372 0
[pid=27327/tid=27334] ppid=27325 vsize=733944 CPUtime=5010.64 cores=0-7
/proc/27327/task/27334/stat : 27334 (plingeling) R 27325 27327 26995 0 -1 4202560 1250902 1253 0 0 500845 219 0 0 25 0 9 0 721648030 751558656 128331 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=27327/tid=27335] ppid=27325 vsize=733944 CPUtime=4960.85 cores=0-7
/proc/27327/task/27335/stat : 27335 (plingeling) R 27325 27327 26995 0 -1 4202560 1202584 1253 0 0 495884 201 0 0 25 0 9 0 721648030 751558656 128331 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=27327/tid=27336] ppid=27325 vsize=733944 CPUtime=5010.36 cores=0-7
/proc/27327/task/27336/stat : 27336 (plingeling) R 27325 27327 26995 0 -1 4202560 1225416 1253 0 0 500820 216 0 0 25 0 9 0 721648030 751558656 128331 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27327/tid=27337] ppid=27325 vsize=733944 CPUtime=5009.15 cores=0-7
/proc/27327/task/27337/stat : 27337 (plingeling) R 27325 27327 26995 0 -1 4202560 1542651 1253 0 0 500628 287 0 0 25 0 9 0 721648030 751558656 128331 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27327/tid=27338] ppid=27325 vsize=733944 CPUtime=5010.68 cores=0-7
/proc/27327/task/27338/stat : 27338 (plingeling) R 27325 27327 26995 0 -1 4202560 1360961 1253 0 0 500849 219 0 0 25 0 9 0 721648030 751558656 128331 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=27327/tid=27339] ppid=27325 vsize=733944 CPUtime=5009.35 cores=0-7
/proc/27327/task/27339/stat : 27339 (plingeling) R 27325 27327 26995 0 -1 4202560 1239778 1253 0 0 500714 221 0 0 25 0 9 0 721648030 751558656 128331 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27327/tid=27340] ppid=27325 vsize=733944 CPUtime=5010.65 cores=0-7
/proc/27327/task/27340/stat : 27340 (plingeling) R 27325 27327 26995 0 -1 4202560 1268586 1253 0 0 500854 211 0 0 25 0 9 0 721648030 751558656 128331 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=27327/tid=27341] ppid=27325 vsize=733944 CPUtime=4978.46 cores=0-7
/proc/27327/task/27341/stat : 27341 (plingeling) R 27325 27327 26995 0 -1 4202560 1192785 1253 0 0 497646 200 0 0 25 0 9 0 721648030 751558656 128331 33554432000 4194304 4417452 140733995714768 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 40000.3
Current children cumulated vsize (KiB) 733944

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5010.65
CPU time (s): 40000.3
CPU user time (s): 39982.5
CPU system time (s): 17.8153
CPU usage (%): 798.306
Max. virtual memory (cumulated for all children) (KiB): 823292

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39982.5
system time used= 17.8153
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 10287681
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 549
involuntary context switches= 189818

runsolver used 23.1795 second user time and 58.8521 second system time

The end

Launcher Data 

Begin job on node108 at 2011-05-22 23:41:11
IDJOB=3394622
IDBENCH=83353
IDSOLVER=1851
FILE ID=node108/3394622-1306100471
RUNJOBID= node108-1306098125-27009
PBS_JOBID= 13387976
Free space on /tmp= 73076 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/crafted/kullmann/VanDerWaerden/VanderWaerden3k/VanDerWaerden_2-3-15_218.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3394622-1306100471/watcher-3394622-1306100471 -o /tmp/evaluation-result-3394622-1306100471/solver-3394622-1306100471 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3394622-1306100471.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= afc70649cebcc24cd0a7b28def593295
RANDOM SEED=256465593

node108.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      28070780 kB
Buffers:        123432 kB
Cached:        3530352 kB
SwapCached:      16344 kB
Active:        1053028 kB
Inactive:      3594424 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      28070780 kB
SwapTotal:    67111528 kB
SwapFree:     53876544 kB
Dirty:             448 kB
Writeback:           0 kB
AnonPages:      988092 kB
Mapped:          12304 kB
Slab:            96184 kB
PageTables:      65472 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 134537988 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73076 MiB
End job on node108 at 2011-05-23 01:04:44