Trace number 3250680

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 4800.11 1201.89

General information on the benchmark

Namecrafted/kullmann/VanDerWaerden/
VanderWaerden3k/VanDerWaerden_2-3-19_349.cnf
MD5SUMfaed4f95ee485d18812f5d39a28df3e5
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables349
Number of clauses33487
Sum of the clauses size151837
Maximum clause length19
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 330276
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 53211

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3250680-1304094502/watcher-3250680-1304094502 -o /tmp/evaluation-result-3250680-1304094502/solver-3250680-1304094502 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3250680-1304094502.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.03 7.90 8.02 2/181 23348
/proc/meminfo: memFree=25376268/32951124 swapFree=67111524/67111528
[pid=23348] ppid=23346 vsize=9436 CPUtime=0 cores=1,3,5,7
/proc/23348/stat : 23348 (plingeling) R 23346 23348 22519 0 -1 4202496 454 1260 0 0 0 0 0 0 22 0 1 0 521048410 9662464 382 33554432000 4194304 4417332 140735657366256 18446744073709551615 232335892224 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/23348/statm: 2359 382 147 55 0 245 0

[startup+0.0129521 s]
/proc/loadavg: 7.03 7.90 8.02 2/181 23348
/proc/meminfo: memFree=25376268/32951124 swapFree=67111524/67111528
[pid=23348] ppid=23346 vsize=11012 CPUtime=0 cores=1,3,5,7
/proc/23348/stat : 23348 (plingeling) R 23346 23348 22519 0 -1 4202496 872 1260 0 0 0 0 0 0 22 0 1 0 521048410 11276288 777 33554432000 4194304 4417332 140735657366256 18446744073709551615 4232632 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/23348/statm: 2753 777 148 55 0 639 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 11012

[startup+0.151884 s]
/proc/loadavg: 7.03 7.90 8.02 2/181 23348
/proc/meminfo: memFree=25376268/32951124 swapFree=67111524/67111528
[pid=23348] ppid=23346 vsize=128624 CPUtime=0.45 cores=1,3,5,7
/proc/23348/stat : 23348 (plingeling) S 23346 23348 22519 0 -1 4202496 3777 1260 0 0 45 0 0 0 22 0 5 0 521048410 131710976 3539 33554432000 4194304 4417332 140735657366256 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/23348/statm: 32156 3539 179 55 0 30042 0
Current children cumulated CPU time (s) 0.45
Current children cumulated vsize (KiB) 128624

[startup+0.300856 s]
/proc/loadavg: 7.03 7.90 8.02 2/181 23348
/proc/meminfo: memFree=25376268/32951124 swapFree=67111524/67111528
[pid=23348] ppid=23346 vsize=126856 CPUtime=1.04 cores=1,3,5,7
/proc/23348/stat : 23348 (plingeling) S 23346 23348 22519 0 -1 4202496 4071 1260 0 0 104 0 0 0 22 0 5 0 521048410 129900544 3057 33554432000 4194304 4417332 140735657366256 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/23348/statm: 31714 3057 179 55 0 29600 0
Current children cumulated CPU time (s) 1.04
Current children cumulated vsize (KiB) 126856

[startup+0.700787 s]
/proc/loadavg: 7.03 7.90 8.02 2/181 23348
/proc/meminfo: memFree=25376268/32951124 swapFree=67111524/67111528
[pid=23348] ppid=23346 vsize=127424 CPUtime=2.64 cores=1,3,5,7
/proc/23348/stat : 23348 (plingeling) S 23346 23348 22519 0 -1 4202496 4276 1260 0 0 264 0 0 0 22 0 5 0 521048410 130482176 3262 33554432000 4194304 4417332 140735657366256 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/23348/statm: 31856 3262 179 55 0 29742 0
Current children cumulated CPU time (s) 2.64
Current children cumulated vsize (KiB) 127424

[startup+1.50065 s]
/proc/loadavg: 7.03 7.90 8.02 10/190 23362
/proc/meminfo: memFree=25354256/32951124 swapFree=67111524/67111528
[pid=23348] ppid=23346 vsize=132244 CPUtime=5.83 cores=1,3,5,7
/proc/23348/stat : 23348 (plingeling) S 23346 23348 22519 0 -1 4202496 5481 1260 0 0 583 0 0 0 22 0 5 0 521048410 135417856 4467 33554432000 4194304 4417332 140735657366256 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/23348/statm: 33061 4467 179 55 0 30947 0
[pid=23348/tid=23359] ppid=23346 vsize=132244 CPUtime=1.45 cores=1,3,5,7
/proc/23348/task/23359/stat : 23359 (plingeling) R 23346 23348 22519 0 -1 4202560 743 1260 0 0 145 0 0 0 20 0 5 0 521048414 135417856 4467 33554432000 4194304 4417332 140735657366256 18446744073709551615 4365846 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23348/tid=23360] ppid=23346 vsize=132244 CPUtime=1.45 cores=1,3,5,7
/proc/23348/task/23360/stat : 23360 (plingeling) R 23346 23348 22519 0 -1 4202560 429 1260 0 0 145 0 0 0 25 0 5 0 521048414 135417856 4467 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364734 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23348/tid=23361] ppid=23346 vsize=132244 CPUtime=1.44 cores=1,3,5,7
/proc/23348/task/23361/stat : 23361 (plingeling) R 23346 23348 22519 0 -1 4202560 344 1260 0 0 144 0 0 0 25 0 5 0 521048414 135417856 4467 33554432000 4194304 4417332 140735657366256 18446744073709551615 4366011 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23348/tid=23362] ppid=23346 vsize=132244 CPUtime=1.44 cores=1,3,5,7
/proc/23348/task/23362/stat : 23362 (plingeling) R 23346 23348 22519 0 -1 4202560 1274 1260 0 0 144 0 0 0 25 0 5 0 521048414 135417856 4467 33554432000 4194304 4417332 140735657366256 18446744073709551615 4365986 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 5.83
Current children cumulated vsize (KiB) 132244

[startup+3.10036 s]
/proc/loadavg: 7.03 7.90 8.02 10/190 23362
/proc/meminfo: memFree=25342768/32951124 swapFree=67111524/67111528
[pid=23348] ppid=23346 vsize=137760 CPUtime=12.23 cores=1,3,5,7
/proc/23348/stat : 23348 (plingeling) S 23346 23348 22519 0 -1 4202496 6860 1260 0 0 1223 0 0 0 22 0 5 0 521048410 141066240 5846 33554432000 4194304 4417332 140735657366256 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/23348/statm: 34440 5846 179 55 0 32326 0
[pid=23348/tid=23359] ppid=23346 vsize=137760 CPUtime=3.05 cores=1,3,5,7
/proc/23348/task/23359/stat : 23359 (plingeling) R 23346 23348 22519 0 -1 4202560 843 1260 0 0 305 0 0 0 25 0 5 0 521048414 141066240 5846 33554432000 4194304 4417332 140735657366256 18446744073709551615 4249222 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23348/tid=23360] ppid=23346 vsize=137760 CPUtime=3.05 cores=1,3,5,7
/proc/23348/task/23360/stat : 23360 (plingeling) R 23346 23348 22519 0 -1 4202560 429 1260 0 0 305 0 0 0 25 0 5 0 521048414 141066240 5846 33554432000 4194304 4417332 140735657366256 18446744073709551615 4366038 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23348/tid=23361] ppid=23346 vsize=137760 CPUtime=3.04 cores=1,3,5,7
/proc/23348/task/23361/stat : 23361 (plingeling) R 23346 23348 22519 0 -1 4202560 1028 1260 0 0 304 0 0 0 25 0 5 0 521048414 141066240 5846 33554432000 4194304 4417332 140735657366256 18446744073709551615 4366362 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23348/tid=23362] ppid=23346 vsize=137760 CPUtime=3.03 cores=1,3,5,7
/proc/23348/task/23362/stat : 23362 (plingeling) R 23346 23348 22519 0 -1 4202560 1869 1260 0 0 303 0 0 0 25 0 5 0 521048414 141066240 5846 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 12.23
Current children cumulated vsize (KiB) 137760

[startup+6.3008 s]
/proc/loadavg: 7.19 7.92 8.03 10/190 23362
/proc/meminfo: memFree=25329140/32951124 swapFree=67111524/67111528
[pid=23348] ppid=23346 vsize=145476 CPUtime=25.02 cores=1,3,5,7
/proc/23348/stat : 23348 (plingeling) S 23346 23348 22519 0 -1 4202496 9416 1260 0 0 2501 1 0 0 22 0 5 0 521048410 148967424 7775 33554432000 4194304 4417332 140735657366256 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/23348/statm: 36369 7775 179 55 0 34255 0
[pid=23348/tid=23359] ppid=23346 vsize=145476 CPUtime=6.25 cores=1,3,5,7
/proc/23348/task/23359/stat : 23359 (plingeling) R 23346 23348 22519 0 -1 4202560 1124 1260 0 0 625 0 0 0 25 0 5 0 521048414 148967424 7775 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364487 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23348/tid=23360] ppid=23346 vsize=145476 CPUtime=6.25 cores=1,3,5,7
/proc/23348/task/23360/stat : 23360 (plingeling) R 23346 23348 22519 0 -1 4202560 664 1260 0 0 625 0 0 0 25 0 5 0 521048414 148967424 7775 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364487 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23348/tid=23361] ppid=23346 vsize=145476 CPUtime=6.24 cores=1,3,5,7
/proc/23348/task/23361/stat : 23361 (plingeling) R 23346 23348 22519 0 -1 4202560 1561 1260 0 0 624 0 0 0 25 0 5 0 521048414 148967424 7775 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23348/tid=23362] ppid=23346 vsize=145476 CPUtime=6.22 cores=1,3,5,7
/proc/23348/task/23362/stat : 23362 (plingeling) R 23346 23348 22519 0 -1 4202560 3376 1260 0 0 622 0 0 0 25 0 5 0 521048414 148967424 7775 33554432000 4194304 4417332 140735657366256 18446744073709551615 4245694 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 25.02
Current children cumulated vsize (KiB) 145476

[startup+12.7007 s]
/proc/loadavg: 7.25 7.92 8.02 10/190 23362
/proc/meminfo: memFree=25301860/32951124 swapFree=67111524/67111528
[pid=23348] ppid=23346 vsize=159440 CPUtime=50.58 cores=1,3,5,7
/proc/23348/stat : 23348 (plingeling) S 23346 23348 22519 0 -1 4202496 19574 1260 0 0 5056 2 0 0 22 0 5 0 521048410 163266560 11266 33554432000 4194304 4417332 140735657366256 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/23348/statm: 39860 11266 179 55 0 37746 0
[pid=23348/tid=23359] ppid=23346 vsize=159440 CPUtime=12.65 cores=1,3,5,7
/proc/23348/task/23359/stat : 23359 (plingeling) R 23346 23348 22519 0 -1 4202560 3114 1260 0 0 1265 0 0 0 25 0 5 0 521048414 163266560 11266 33554432000 4194304 4417332 140735657366256 18446744073709551615 4251248 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23348/tid=23360] ppid=23346 vsize=159440 CPUtime=12.65 cores=1,3,5,7
/proc/23348/task/23360/stat : 23360 (plingeling) R 23346 23348 22519 0 -1 4202560 1176 1260 0 0 1265 0 0 0 25 0 5 0 521048414 163266560 11266 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364565 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23348/tid=23361] ppid=23346 vsize=159440 CPUtime=12.64 cores=1,3,5,7
/proc/23348/task/23361/stat : 23361 (plingeling) R 23346 23348 22519 0 -1 4202560 2368 1260 0 0 1264 0 0 0 25 0 5 0 521048414 163266560 11266 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364753 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23348/tid=23362] ppid=23346 vsize=159440 CPUtime=12.58 cores=1,3,5,7
/proc/23348/task/23362/stat : 23362 (plingeling) R 23346 23348 22519 0 -1 4202560 10225 1260 0 0 1257 1 0 0 25 0 5 0 521048414 163266560 11266 33554432000 4194304 4417332 140735657366256 18446744073709551615 4365977 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 50.58
Current children cumulated vsize (KiB) 159440

[startup+25.5004 s]
/proc/loadavg: 7.49 7.94 8.03 10/190 23362
/proc/meminfo: memFree=25277688/32951124 swapFree=67111524/67111528
[pid=23348] ppid=23346 vsize=163824 CPUtime=101.7 cores=1,3,5,7
/proc/23348/stat : 23348 (plingeling) S 23346 23348 22519 0 -1 4202496 37017 1260 0 0 10166 4 0 0 22 0 5 0 521048410 167755776 12391 33554432000 4194304 4417332 140735657366256 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/23348/statm: 40956 12391 179 55 0 38842 0
[pid=23348/tid=23359] ppid=23346 vsize=163824 CPUtime=25.45 cores=1,3,5,7
/proc/23348/task/23359/stat : 23359 (plingeling) R 23346 23348 22519 0 -1 4202560 5707 1260 0 0 2545 0 0 0 25 0 5 0 521048414 167755776 12391 33554432000 4194304 4417332 140735657366256 18446744073709551615 4259875 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23348/tid=23360] ppid=23346 vsize=163824 CPUtime=25.45 cores=1,3,5,7
/proc/23348/task/23360/stat : 23360 (plingeling) R 23346 23348 22519 0 -1 4202560 2735 1260 0 0 2545 0 0 0 25 0 5 0 521048414 167755776 12391 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364568 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23348/tid=23361] ppid=23346 vsize=163824 CPUtime=25.44 cores=1,3,5,7
/proc/23348/task/23361/stat : 23361 (plingeling) R 23346 23348 22519 0 -1 4202560 5458 1260 0 0 2543 1 0 0 25 0 5 0 521048414 167755776 12391 33554432000 4194304 4417332 140735657366256 18446744073709551615 4353618 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23348/tid=23362] ppid=23346 vsize=163824 CPUtime=25.3 cores=1,3,5,7
/proc/23348/task/23362/stat : 23362 (plingeling) R 23346 23348 22519 0 -1 4202560 20426 1260 0 0 2528 2 0 0 25 0 5 0 521048414 167755776 12391 33554432000 4194304 4417332 140735657366256 18446744073709551615 4366381 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 101.7
Current children cumulated vsize (KiB) 163824

[startup+51.1008 s]
/proc/loadavg: 7.72 7.96 8.03 9/190 23363
/proc/meminfo: memFree=25264260/32951124 swapFree=67111524/67111528
[pid=23348] ppid=23346 vsize=177116 CPUtime=203.96 cores=1,3,5,7
/proc/23348/stat : 23348 (plingeling) S 23346 23348 22519 0 -1 4202496 101646 1260 0 0 20382 14 0 0 22 0 5 0 521048410 181366784 15714 33554432000 4194304 4417332 140735657366256 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/23348/statm: 44279 15714 179 55 0 42165 0
[pid=23348/tid=23359] ppid=23346 vsize=177116 CPUtime=51.05 cores=1,3,5,7
/proc/23348/task/23359/stat : 23359 (plingeling) R 23346 23348 22519 0 -1 4202560 19110 1260 0 0 5103 2 0 0 25 0 5 0 521048414 181366784 15714 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23348/tid=23360] ppid=23346 vsize=177116 CPUtime=51.04 cores=1,3,5,7
/proc/23348/task/23360/stat : 23360 (plingeling) R 23346 23348 22519 0 -1 4202560 22438 1260 0 0 5101 3 0 0 25 0 5 0 521048414 181366784 15714 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23348/tid=23361] ppid=23346 vsize=177116 CPUtime=51.04 cores=1,3,5,7
/proc/23348/task/23361/stat : 23361 (plingeling) R 23346 23348 22519 0 -1 4202560 19461 1260 0 0 5101 3 0 0 25 0 5 0 521048414 181366784 15714 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23348/tid=23362] ppid=23346 vsize=177116 CPUtime=50.76 cores=1,3,5,7
/proc/23348/task/23362/stat : 23362 (plingeling) R 23346 23348 22519 0 -1 4202560 37946 1260 0 0 5072 4 0 0 25 0 5 0 521048414 181366784 15714 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364608 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 203.96
Current children cumulated vsize (KiB) 177116

[startup+102.307 s]
/proc/loadavg: 8.04 8.01 8.04 10/190 23364
/proc/meminfo: memFree=25242184/32951124 swapFree=67111524/67111528
[pid=23348] ppid=23346 vsize=186288 CPUtime=408.51 cores=1,3,5,7
/proc/23348/stat : 23348 (plingeling) S 23346 23348 22519 0 -1 4202496 241730 1260 0 0 40813 38 0 0 22 0 5 0 521048410 190758912 18010 33554432000 4194304 4417332 140735657366256 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/23348/statm: 46572 18010 179 55 0 44458 0
[pid=23348/tid=23359] ppid=23346 vsize=186288 CPUtime=102.25 cores=1,3,5,7
/proc/23348/task/23359/stat : 23359 (plingeling) R 23346 23348 22519 0 -1 4202560 46235 1260 0 0 10219 6 0 0 25 0 5 0 521048414 190758912 18010 33554432000 4194304 4417332 140735657366256 18446744073709551615 4365869 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23348/tid=23360] ppid=23346 vsize=186288 CPUtime=102.25 cores=1,3,5,7
/proc/23348/task/23360/stat : 23360 (plingeling) R 23346 23348 22519 0 -1 4202560 64622 1260 0 0 10215 10 0 0 25 0 5 0 521048414 190758912 18010 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23348/tid=23361] ppid=23346 vsize=186288 CPUtime=102.25 cores=1,3,5,7
/proc/23348/task/23361/stat : 23361 (plingeling) R 23346 23348 22519 0 -1 4202560 52039 1260 0 0 10217 8 0 0 25 0 5 0 521048414 190758912 18010 33554432000 4194304 4417332 140735657366256 18446744073709551615 4365894 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23348/tid=23362] ppid=23346 vsize=186288 CPUtime=101.68 cores=1,3,5,7
/proc/23348/task/23362/stat : 23362 (plingeling) R 23346 23348 22519 0 -1 4202560 76143 1260 0 0 10157 11 0 0 25 0 5 0 521048414 190758912 18010 33554432000 4194304 4417332 140735657366256 18446744073709551615 4321382 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 408.51
Current children cumulated vsize (KiB) 186288

[startup+162.301 s]
/proc/loadavg: 8.12 8.04 8.05 10/190 23366
/proc/meminfo: memFree=25204724/32951124 swapFree=67111524/67111528
[pid=23348] ppid=23346 vsize=209512 CPUtime=648.15 cores=1,3,5,7
/proc/23348/stat : 23348 (plingeling) S 23346 23348 22519 0 -1 4202496 408534 1260 0 0 64750 65 0 0 22 0 5 0 521048410 214540288 23816 33554432000 4194304 4417332 140735657366256 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/23348/statm: 52378 23816 179 55 0 50264 0
[pid=23348/tid=23359] ppid=23346 vsize=209512 CPUtime=162.25 cores=1,3,5,7
/proc/23348/task/23359/stat : 23359 (plingeling) R 23346 23348 22519 0 -1 4202560 86582 1260 0 0 16212 13 0 0 25 0 5 0 521048414 214540288 23816 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364544 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23348/tid=23360] ppid=23346 vsize=209512 CPUtime=162.24 cores=1,3,5,7
/proc/23348/task/23360/stat : 23360 (plingeling) R 23346 23348 22519 0 -1 4202560 109437 1260 0 0 16207 17 0 0 25 0 5 0 521048414 214540288 23816 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364713 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23348/tid=23361] ppid=23346 vsize=209512 CPUtime=162.24 cores=1,3,5,7
/proc/23348/task/23361/stat : 23361 (plingeling) R 23346 23348 22519 0 -1 4202560 93453 1260 0 0 16209 15 0 0 25 0 5 0 521048414 214540288 23816 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364713 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23348/tid=23362] ppid=23346 vsize=209512 CPUtime=161.35 cores=1,3,5,7
/proc/23348/task/23362/stat : 23362 (plingeling) R 23346 23348 22519 0 -1 4202560 116371 1260 0 0 16117 18 0 0 25 0 5 0 521048414 214540288 23816 33554432000 4194304 4417332 140735657366256 18446744073709551615 4365963 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 648.15
Current children cumulated vsize (KiB) 209512


################
# More data... #
################

/proc/meminfo: memFree=25086132/32951124 swapFree=67111524/67111528
[pid=23348] ppid=23346 vsize=262432 CPUtime=3044.88 cores=1,3,5,7
/proc/23348/stat : 23348 (plingeling) S 23346 23348 22519 0 -1 4202496 1701118 1260 0 0 304228 260 0 0 22 0 5 0 521048410 268730368 37014 33554432000 4194304 4417332 140735657366256 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/23348/statm: 65608 37014 179 55 0 63494 0
[pid=23348/tid=23359] ppid=23346 vsize=262432 CPUtime=762.27 cores=1,3,5,7
/proc/23348/task/23359/stat : 23359 (plingeling) R 23346 23348 22519 0 -1 4202560 400027 1260 0 0 76169 58 0 0 25 0 5 0 521048414 268730368 37014 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23348/tid=23360] ppid=23346 vsize=262432 CPUtime=762.26 cores=1,3,5,7
/proc/23348/task/23360/stat : 23360 (plingeling) R 23346 23348 22519 0 -1 4202560 438030 1260 0 0 76158 68 0 0 25 0 5 0 521048414 268730368 37014 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364479 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23348/tid=23361] ppid=23346 vsize=262432 CPUtime=762.26 cores=1,3,5,7
/proc/23348/task/23361/stat : 23361 (plingeling) R 23346 23348 22519 0 -1 4202560 433734 1260 0 0 76158 68 0 0 25 0 5 0 521048414 268730368 37014 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364536 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23348/tid=23362] ppid=23346 vsize=262432 CPUtime=758.01 cores=1,3,5,7
/proc/23348/task/23362/stat : 23362 (plingeling) R 23346 23348 22519 0 -1 4202560 426636 1260 0 0 75737 64 0 0 25 0 5 0 521048414 268730368 37014 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3044.88
Current children cumulated vsize (KiB) 262432

[startup+822.301 s]
/proc/loadavg: 8.07 8.13 8.09 10/190 23384
/proc/meminfo: memFree=25018172/32951124 swapFree=67111524/67111528
[pid=23348] ppid=23346 vsize=287556 CPUtime=3284.53 cores=1,3,5,7
/proc/23348/stat : 23348 (plingeling) S 23346 23348 22519 0 -1 4202496 1818725 1260 0 0 328178 275 0 0 22 0 5 0 521048410 294457344 43295 33554432000 4194304 4417332 140735657366256 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/23348/statm: 71889 43295 179 55 0 69775 0
[pid=23348/tid=23359] ppid=23346 vsize=287556 CPUtime=822.26 cores=1,3,5,7
/proc/23348/task/23359/stat : 23359 (plingeling) R 23346 23348 22519 0 -1 4202560 426827 1260 0 0 82165 61 0 0 25 0 5 0 521048414 294457344 43295 33554432000 4194304 4417332 140735657366256 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23348/tid=23360] ppid=23346 vsize=287556 CPUtime=822.26 cores=1,3,5,7
/proc/23348/task/23360/stat : 23360 (plingeling) R 23346 23348 22519 0 -1 4202560 460963 1260 0 0 82155 71 0 0 25 0 5 0 521048414 294457344 43295 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364544 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23348/tid=23361] ppid=23346 vsize=287556 CPUtime=822.26 cores=1,3,5,7
/proc/23348/task/23361/stat : 23361 (plingeling) R 23346 23348 22519 0 -1 4202560 472161 1260 0 0 82153 73 0 0 25 0 5 0 521048414 294457344 43295 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364479 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23348/tid=23362] ppid=23346 vsize=287556 CPUtime=817.68 cores=1,3,5,7
/proc/23348/task/23362/stat : 23362 (plingeling) R 23346 23348 22519 0 -1 4202560 456083 1260 0 0 81700 68 0 0 25 0 5 0 521048414 294457344 43295 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3284.53
Current children cumulated vsize (KiB) 287556

[startup+882.3 s]
/proc/loadavg: 8.12 8.13 8.09 10/190 23385
/proc/meminfo: memFree=25066396/32951124 swapFree=67111524/67111528
[pid=23348] ppid=23346 vsize=267244 CPUtime=3524.19 cores=1,3,5,7
/proc/23348/stat : 23348 (plingeling) S 23346 23348 22519 0 -1 4202496 1930110 1260 0 0 352127 292 0 0 22 0 5 0 521048410 273657856 38217 33554432000 4194304 4417332 140735657366256 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/23348/statm: 66811 38217 179 55 0 64697 0
[pid=23348/tid=23359] ppid=23346 vsize=267244 CPUtime=882.27 cores=1,3,5,7
/proc/23348/task/23359/stat : 23359 (plingeling) R 23346 23348 22519 0 -1 4202560 450268 1260 0 0 88162 65 0 0 25 0 5 0 521048414 273657856 38217 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364753 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23348/tid=23360] ppid=23346 vsize=267244 CPUtime=882.26 cores=1,3,5,7
/proc/23348/task/23360/stat : 23360 (plingeling) R 23346 23348 22519 0 -1 4202560 487423 1260 0 0 88150 76 0 0 25 0 5 0 521048414 273657856 38217 33554432000 4194304 4417332 140735657366256 18446744073709551615 4354401 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23348/tid=23361] ppid=23346 vsize=267244 CPUtime=882.26 cores=1,3,5,7
/proc/23348/task/23361/stat : 23361 (plingeling) R 23346 23348 22519 0 -1 4202560 504134 1260 0 0 88148 78 0 0 25 0 5 0 521048414 273657856 38217 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364479 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23348/tid=23362] ppid=23346 vsize=267244 CPUtime=877.34 cores=1,3,5,7
/proc/23348/task/23362/stat : 23362 (plingeling) R 23346 23348 22519 0 -1 4202560 485594 1260 0 0 87662 72 0 0 25 0 5 0 521048414 273657856 38217 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364652 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3524.19
Current children cumulated vsize (KiB) 267244

[startup+942.301 s]
/proc/loadavg: 8.14 8.13 8.09 9/190 23387
/proc/meminfo: memFree=25010456/32951124 swapFree=67111524/67111528
[pid=23348] ppid=23346 vsize=297112 CPUtime=3763.87 cores=1,3,5,7
/proc/23348/stat : 23348 (plingeling) S 23346 23348 22519 0 -1 4202496 2027863 1260 0 0 376081 306 0 0 22 0 5 0 521048410 304242688 45684 33554432000 4194304 4417332 140735657366256 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/23348/statm: 74278 45684 179 55 0 72164 0
[pid=23348/tid=23359] ppid=23346 vsize=297112 CPUtime=942.27 cores=1,3,5,7
/proc/23348/task/23359/stat : 23359 (plingeling) R 23346 23348 22519 0 -1 4202560 470358 1260 0 0 94159 68 0 0 25 0 5 0 521048414 304242688 45684 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23348/tid=23360] ppid=23346 vsize=297112 CPUtime=942.27 cores=1,3,5,7
/proc/23348/task/23360/stat : 23360 (plingeling) R 23346 23348 22519 0 -1 4202560 508264 1260 0 0 94148 79 0 0 25 0 5 0 521048414 304242688 45684 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364957 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23348/tid=23361] ppid=23346 vsize=297112 CPUtime=942.26 cores=1,3,5,7
/proc/23348/task/23361/stat : 23361 (plingeling) R 23346 23348 22519 0 -1 4202560 533970 1260 0 0 94144 82 0 0 25 0 5 0 521048414 304242688 45684 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23348/tid=23362] ppid=23346 vsize=297112 CPUtime=937.01 cores=1,3,5,7
/proc/23348/task/23362/stat : 23362 (plingeling) R 23346 23348 22519 0 -1 4202560 512580 1260 0 0 93625 76 0 0 25 0 5 0 521048414 304242688 45684 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3763.87
Current children cumulated vsize (KiB) 297112

[startup+1002.3 s]
/proc/loadavg: 8.19 8.15 8.10 10/190 23388
/proc/meminfo: memFree=25034976/32951124 swapFree=67111524/67111528
[pid=23348] ppid=23346 vsize=289252 CPUtime=4003.55 cores=1,3,5,7
/proc/23348/stat : 23348 (plingeling) S 23346 23348 22519 0 -1 4202496 2128478 1260 0 0 400032 323 0 0 22 0 5 0 521048410 296194048 43723 33554432000 4194304 4417332 140735657366256 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/23348/statm: 72313 43723 179 55 0 70199 0
[pid=23348/tid=23359] ppid=23346 vsize=289252 CPUtime=1002.27 cores=1,3,5,7
/proc/23348/task/23359/stat : 23359 (plingeling) R 23346 23348 22519 0 -1 4202560 497232 1260 0 0 100156 71 0 0 25 0 5 0 521048414 296194048 43723 33554432000 4194304 4417332 140735657366256 18446744073709551615 4366725 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23348/tid=23360] ppid=23346 vsize=289252 CPUtime=1002.26 cores=1,3,5,7
/proc/23348/task/23360/stat : 23360 (plingeling) R 23346 23348 22519 0 -1 4202560 538945 1260 0 0 100143 83 0 0 25 0 5 0 521048414 296194048 43723 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23348/tid=23361] ppid=23346 vsize=289252 CPUtime=1002.25 cores=1,3,5,7
/proc/23348/task/23361/stat : 23361 (plingeling) R 23346 23348 22519 0 -1 4202560 553949 1260 0 0 100140 85 0 0 25 0 5 0 521048414 296194048 43723 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23348/tid=23362] ppid=23346 vsize=289252 CPUtime=996.68 cores=1,3,5,7
/proc/23348/task/23362/stat : 23362 (plingeling) R 23346 23348 22519 0 -1 4202560 535661 1260 0 0 99587 81 0 0 25 0 5 0 521048414 296194048 43723 33554432000 4194304 4417332 140735657366256 18446744073709551615 4251081 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4003.55
Current children cumulated vsize (KiB) 289252

[startup+1062.3 s]
/proc/loadavg: 8.20 8.17 8.10 10/190 23390
/proc/meminfo: memFree=25029760/32951124 swapFree=67111524/67111528
[pid=23348] ppid=23346 vsize=324700 CPUtime=4243.22 cores=1,3,5,7
/proc/23348/stat : 23348 (plingeling) S 23346 23348 22519 0 -1 4202496 2228445 1260 0 0 423984 338 0 0 22 0 5 0 521048410 332492800 52585 33554432000 4194304 4417332 140735657366256 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/23348/statm: 81175 52585 179 55 0 79061 0
[pid=23348/tid=23359] ppid=23346 vsize=324700 CPUtime=1062.27 cores=1,3,5,7
/proc/23348/task/23359/stat : 23359 (plingeling) R 23346 23348 22519 0 -1 4202560 517350 1260 0 0 106153 74 0 0 25 0 5 0 521048414 332492800 52585 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23348/tid=23360] ppid=23346 vsize=324700 CPUtime=1062.27 cores=1,3,5,7
/proc/23348/task/23360/stat : 23360 (plingeling) R 23346 23348 22519 0 -1 4202560 564588 1260 0 0 106140 87 0 0 25 0 5 0 521048414 332492800 52585 33554432000 4194304 4417332 140735657366256 18446744073709551615 4365960 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23348/tid=23361] ppid=23346 vsize=324700 CPUtime=1062.26 cores=1,3,5,7
/proc/23348/task/23361/stat : 23361 (plingeling) R 23346 23348 22519 0 -1 4202560 581258 1260 0 0 106136 90 0 0 25 0 5 0 521048414 332492800 52585 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364742 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23348/tid=23362] ppid=23346 vsize=324700 CPUtime=1056.34 cores=1,3,5,7
/proc/23348/task/23362/stat : 23362 (plingeling) R 23346 23348 22519 0 -1 4202560 562558 1260 0 0 105550 84 0 0 25 0 5 0 521048414 332492800 52585 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364639 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4243.22
Current children cumulated vsize (KiB) 324700

[startup+1122.3 s]
/proc/loadavg: 8.16 8.16 8.10 10/190 23392
/proc/meminfo: memFree=24991176/32951124 swapFree=67111524/67111528
[pid=23348] ppid=23346 vsize=320920 CPUtime=4482.89 cores=1,3,5,7
/proc/23348/stat : 23348 (plingeling) S 23346 23348 22519 0 -1 4202496 2324611 1260 0 0 447936 353 0 0 22 0 5 0 521048410 328622080 51640 33554432000 4194304 4417332 140735657366256 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/23348/statm: 80230 51640 179 55 0 78116 0
[pid=23348/tid=23359] ppid=23346 vsize=320920 CPUtime=1122.27 cores=1,3,5,7
/proc/23348/task/23359/stat : 23359 (plingeling) R 23346 23348 22519 0 -1 4202560 541759 1260 0 0 112149 78 0 0 25 0 5 0 521048414 328622080 51640 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364487 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23348/tid=23360] ppid=23346 vsize=320920 CPUtime=1122.27 cores=1,3,5,7
/proc/23348/task/23360/stat : 23360 (plingeling) R 23346 23348 22519 0 -1 4202560 584518 1260 0 0 112136 91 0 0 25 0 5 0 521048414 328622080 51640 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364750 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23348/tid=23361] ppid=23346 vsize=320920 CPUtime=1122.26 cores=1,3,5,7
/proc/23348/task/23361/stat : 23361 (plingeling) R 23346 23348 22519 0 -1 4202560 606323 1260 0 0 112133 93 0 0 25 0 5 0 521048414 328622080 51640 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23348/tid=23362] ppid=23346 vsize=320920 CPUtime=1116.01 cores=1,3,5,7
/proc/23348/task/23362/stat : 23362 (plingeling) R 23346 23348 22519 0 -1 4202560 589320 1260 0 0 111512 89 0 0 25 0 5 0 521048414 328622080 51640 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4482.89
Current children cumulated vsize (KiB) 320920

[startup+1182.3 s]
/proc/loadavg: 8.20 8.17 8.10 10/190 23393
/proc/meminfo: memFree=24987820/32951124 swapFree=67111524/67111528
[pid=23348] ppid=23346 vsize=306712 CPUtime=4722.56 cores=1,3,5,7
/proc/23348/stat : 23348 (plingeling) S 23346 23348 22519 0 -1 4202496 2425898 1260 0 0 471887 369 0 0 22 0 5 0 521048410 314073088 48084 33554432000 4194304 4417332 140735657366256 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/23348/statm: 76678 48084 179 55 0 74564 0
[pid=23348/tid=23359] ppid=23346 vsize=306712 CPUtime=1182.27 cores=1,3,5,7
/proc/23348/task/23359/stat : 23359 (plingeling) R 23346 23348 22519 0 -1 4202560 563904 1260 0 0 118146 81 0 0 25 0 5 0 521048414 314073088 48084 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364942 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23348/tid=23360] ppid=23346 vsize=306712 CPUtime=1182.27 cores=1,3,5,7
/proc/23348/task/23360/stat : 23360 (plingeling) R 23346 23348 22519 0 -1 4202560 615700 1260 0 0 118132 95 0 0 25 0 5 0 521048414 314073088 48084 33554432000 4194304 4417332 140735657366256 18446744073709551615 4365975 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23348/tid=23361] ppid=23346 vsize=306712 CPUtime=1182.27 cores=1,3,5,7
/proc/23348/task/23361/stat : 23361 (plingeling) R 23346 23348 22519 0 -1 4202560 634299 1260 0 0 118128 99 0 0 25 0 5 0 521048414 314073088 48084 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23348/tid=23362] ppid=23346 vsize=306712 CPUtime=1175.67 cores=1,3,5,7
/proc/23348/task/23362/stat : 23362 (plingeling) R 23346 23348 22519 0 -1 4202560 609304 1260 0 0 117476 91 0 0 25 0 5 0 521048414 314073088 48084 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4722.56
Current children cumulated vsize (KiB) 306712



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.7 s]
/proc/loadavg: 8.22 8.18 8.11 5/187 23395
/proc/meminfo: memFree=25226912/32951124 swapFree=67111524/67111528
[pid=23348] ppid=23346 vsize=269984 CPUtime=4800.07 cores=1,3,5,7
/proc/23348/stat : 23348 (plingeling) S 23346 23348 22519 0 -1 4202496 2451977 1260 0 0 479635 372 0 0 22 0 5 0 521048410 276463616 38902 33554432000 4194304 4417332 140735657366256 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/23348/statm: 67496 38902 179 55 0 65382 0
[pid=23348/tid=23359] ppid=23346 vsize=269984 CPUtime=1201.68 cores=1,3,5,7
/proc/23348/task/23359/stat : 23359 (plingeling) R 23346 23348 22519 0 -1 4202560 571467 1260 0 0 120086 82 0 0 25 0 5 0 521048414 276463616 38902 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23348/tid=23360] ppid=23346 vsize=269984 CPUtime=1201.67 cores=1,3,5,7
/proc/23348/task/23360/stat : 23360 (plingeling) R 23346 23348 22519 0 -1 4202560 622676 1260 0 0 120071 96 0 0 25 0 5 0 521048414 276463616 38902 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364982 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23348/tid=23361] ppid=23346 vsize=269984 CPUtime=1201.67 cores=1,3,5,7
/proc/23348/task/23361/stat : 23361 (plingeling) R 23346 23348 22519 0 -1 4202560 641501 1260 0 0 120068 99 0 0 25 0 5 0 521048414 276463616 38902 33554432000 4194304 4417332 140735657366256 18446744073709551615 4364531 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23348/tid=23362] ppid=23346 vsize=269984 CPUtime=1194.97 cores=1,3,5,7
/proc/23348/task/23362/stat : 23362 (plingeling) R 23346 23348 22519 0 -1 4202560 613642 1260 0 0 119405 92 0 0 25 0 5 0 521048414 276463616 38902 33554432000 4194304 4417332 140735657366256 18446744073709551615 4245774 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4800.07
Current children cumulated vsize (KiB) 269984

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.81 s]
/proc/loadavg: 8.22 8.18 8.11 5/187 23395
/proc/meminfo: memFree=25226912/32951124 swapFree=67111524/67111528
[pid=23348] ppid=23346 vsize=0 CPUtime=4800.1 cores=1,3,5,7
/proc/23348/stat : 23348 (plingeling) Z 23346 23348 22519 0 -1 4203548 2451977 1260 0 0 479637 373 0 0 18 0 2 0 521048410 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 3 0 0 0
/proc/23348/statm: 0 0 0 0 0 0 0
[pid=23348/tid=23362] ppid=23346 vsize=0 CPUtime=1194.98 cores=1,3,5,7
/proc/23348/task/23362/stat : 23362 (plingeling) S 23346 23348 22519 0 -1 4203588 613642 1260 0 0 119405 93 0 0 25 0 2 0 521048414 0 0 33554432000 0 0 0 0 0 0 2147483391 4096 0 18446744073709551615 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4800.1
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1201.89
CPU time (s): 4800.11
CPU user time (s): 4796.37
CPU system time (s): 3.73643
CPU usage (%): 399.38
Max. virtual memory (cumulated for all children) (KiB): 338340

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4796.37
system time used= 3.73643
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2453237
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 129
involuntary context switches= 34411

runsolver used 1.82172 second user time and 4.74228 second system time

The end

Launcher Data 

Begin job on node132 at 2011-04-29 18:28:22
IDJOB=3250680
IDBENCH=83336
IDSOLVER=1605
FILE ID=node132/3250680-1304094502
RUNJOBID= node132-1304094502-23324
PBS_JOBID= 13170555
Free space on /tmp= 71872 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT11/crafted/kullmann/VanDerWaerden/VanderWaerden3k/VanDerWaerden_2-3-19_349.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3250680-1304094502/watcher-3250680-1304094502 -o /tmp/evaluation-result-3250680-1304094502/solver-3250680-1304094502 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3250680-1304094502.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= faed4f95ee485d18812f5d39a28df3e5
RANDOM SEED=989972996

node132.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      25376920 kB
Buffers:        773656 kB
Cached:        6229404 kB
SwapCached:          4 kB
Active:        2089116 kB
Inactive:      4987480 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      25376920 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:            1764 kB
Writeback:           0 kB
AnonPages:       74092 kB
Mapped:          15284 kB
Slab:           435872 kB
PageTables:       4992 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   216484 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71872 MiB
End job on node132 at 2011-04-29 18:48:26