Trace number 3394560

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40000.3 5010.37

General information on the benchmark

Namecrafted/kullmann/GreenTao/
GreenTaok1k2/GreenTao_2-3-6_2072.cnf
MD5SUMd2ef996f95bbf68a3c980c797dfdc364
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2071
Number of clauses159039
Sum of the clauses size479052
Maximum clause length6
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3158394
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 5645

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3394560-1306099698/watcher-3394560-1306099698 -o /tmp/evaluation-result-3394560-1306099698/solver-3394560-1306099698 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3394560-1306099698.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.47 7.99 8.07 1/451 4117
/proc/meminfo: memFree=29614288/32951124 swapFree=60688720/67111528
[pid=4117] ppid=4115 vsize=8728 CPUtime=0 cores=0-7
/proc/4117/stat : 4117 (plingeling) S 4115 4117 4091 0 -1 4202496 215 0 0 0 0 0 0 0 25 0 1 0 721567940 8937472 146 33554432000 4194304 4417452 140736396971856 18446744073709551615 232335892224 0 0 4096 0 18446744071562362840 0 0 17 0 0 0 0
/proc/4117/statm: 2182 146 122 55 0 68 0

[startup+0.0252251 s]
/proc/loadavg: 7.47 7.99 8.07 1/451 4117
/proc/meminfo: memFree=29614288/32951124 swapFree=60688720/67111528
[pid=4117] ppid=4115 vsize=8728 CPUtime=0 cores=0-7
/proc/4117/stat : 4117 (plingeling) S 4115 4117 4091 0 -1 4202496 215 0 0 0 0 0 0 0 25 0 1 0 721567940 8937472 146 33554432000 4194304 4417452 140736396971856 18446744073709551615 232335892224 0 0 4096 0 18446744071562362840 0 0 17 0 0 0 0
/proc/4117/statm: 2182 146 122 55 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8728

[startup+0.100213 s]
/proc/loadavg: 7.47 7.99 8.07 1/451 4117
/proc/meminfo: memFree=29614288/32951124 swapFree=60688720/67111528
[pid=4117] ppid=4115 vsize=9528 CPUtime=0 cores=0-7
/proc/4117/stat : 4117 (plingeling) R 4115 4117 4091 0 -1 4202496 443 1253 0 5 0 0 0 0 25 0 1 0 721567940 9756672 373 33554432000 4194304 4417452 140736396971856 18446744073709551615 4231393 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/4117/statm: 2382 373 148 55 0 268 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9528

[startup+0.300172 s]
/proc/loadavg: 7.47 7.99 8.07 1/451 4117
/proc/meminfo: memFree=29614288/32951124 swapFree=60688720/67111528
[pid=4117] ppid=4115 vsize=42772 CPUtime=0.2 cores=0-7
/proc/4117/stat : 4117 (plingeling) R 4115 4117 4091 0 -1 4202496 8836 1253 0 5 19 1 0 0 25 0 1 0 721567940 43798528 8709 33554432000 4194304 4417452 140736396971856 18446744073709551615 4231248 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/4117/statm: 10693 8709 149 55 0 8579 0
Current children cumulated CPU time (s) 0.2
Current children cumulated vsize (KiB) 42772

[startup+1.43594 s]
/proc/loadavg: 7.47 7.99 8.07 1/451 4117
/proc/meminfo: memFree=29614288/32951124 swapFree=60688720/67111528
[pid=4117] ppid=4115 vsize=325552 CPUtime=7.88 cores=0-7
/proc/4117/stat : 4117 (plingeling) S 4115 4117 4091 0 -1 4202496 44312 1253 0 5 783 5 0 0 18 0 9 0 721567940 333365248 26232 33554432000 4194304 4417452 140736396971856 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4117/statm: 81388 26232 179 55 0 79274 0
Current children cumulated CPU time (s) 7.88
Current children cumulated vsize (KiB) 325552

[startup+1.50093 s]
/proc/loadavg: 7.47 7.99 8.07 1/451 4117
/proc/meminfo: memFree=29614288/32951124 swapFree=60688720/67111528
[pid=4117] ppid=4115 vsize=325552 CPUtime=8.4 cores=0-7
/proc/4117/stat : 4117 (plingeling) S 4115 4117 4091 0 -1 4202496 44312 1253 0 5 835 5 0 0 18 0 9 0 721567940 333365248 26232 33554432000 4194304 4417452 140736396971856 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4117/statm: 81388 26232 179 55 0 79274 0
Current children cumulated CPU time (s) 8.4
Current children cumulated vsize (KiB) 325552

[startup+3.10064 s]
/proc/loadavg: 7.59 8.01 8.07 9/460 4131
/proc/meminfo: memFree=29491908/32951124 swapFree=60688720/67111528
[pid=4117] ppid=4115 vsize=341936 CPUtime=21.16 cores=0-7
/proc/4117/stat : 4117 (plingeling) S 4115 4117 4091 0 -1 4202496 48408 1253 0 5 2111 5 0 0 18 0 9 0 721567940 350142464 30328 33554432000 4194304 4417452 140736396971856 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4117/statm: 85484 30328 179 55 0 83370 0
[pid=4117/tid=4124] ppid=4115 vsize=341936 CPUtime=2.6 cores=0-7
/proc/4117/task/4124/stat : 4124 (plingeling) R 4115 4117 4091 0 -1 4202560 4527 1253 0 5 260 0 0 0 22 0 9 0 721567988 350142464 30328 33554432000 4194304 4417452 140736396971856 18446744073709551615 4337310 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4117/tid=4125] ppid=4115 vsize=341936 CPUtime=2.6 cores=0-7
/proc/4117/task/4125/stat : 4125 (plingeling) R 4115 4117 4091 0 -1 4202560 4393 1253 0 5 260 0 0 0 22 0 9 0 721567988 350142464 30328 33554432000 4194304 4417452 140736396971856 18446744073709551615 4249181 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4117/tid=4126] ppid=4115 vsize=341936 CPUtime=2.52 cores=0-7
/proc/4117/task/4126/stat : 4126 (plingeling) R 4115 4117 4091 0 -1 4202560 2916 1253 0 5 252 0 0 0 22 0 9 0 721567988 350142464 30328 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4117/tid=4127] ppid=4115 vsize=341936 CPUtime=2.61 cores=0-7
/proc/4117/task/4127/stat : 4127 (plingeling) R 4115 4117 4091 0 -1 4202560 38 1253 0 5 261 0 0 0 25 0 9 0 721567988 350142464 30328 33554432000 4194304 4417452 140736396971856 18446744073709551615 4365973 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4117/tid=4128] ppid=4115 vsize=341936 CPUtime=2.59 cores=0-7
/proc/4117/task/4128/stat : 4128 (plingeling) R 4115 4117 4091 0 -1 4202560 5373 1253 0 5 259 0 0 0 22 0 9 0 721567988 350142464 30328 33554432000 4194304 4417452 140736396971856 18446744073709551615 4249130 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4117/tid=4129] ppid=4115 vsize=341936 CPUtime=2.6 cores=0-7
/proc/4117/task/4129/stat : 4129 (plingeling) R 4115 4117 4091 0 -1 4202560 4373 1253 0 5 260 0 0 0 25 0 9 0 721567988 350142464 30328 33554432000 4194304 4417452 140736396971856 18446744073709551615 4249130 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4117/tid=4130] ppid=4115 vsize=341936 CPUtime=2.58 cores=0-7
/proc/4117/task/4130/stat : 4130 (plingeling) R 4115 4117 4091 0 -1 4202560 3958 1253 0 5 258 0 0 0 25 0 9 0 721567988 350142464 30328 33554432000 4194304 4417452 140736396971856 18446744073709551615 4365204 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4117/tid=4131] ppid=4115 vsize=341936 CPUtime=2.59 cores=0-7
/proc/4117/task/4131/stat : 4131 (plingeling) R 4115 4117 4091 0 -1 4202560 5408 1253 0 5 259 0 0 0 22 0 9 0 721567988 350142464 30328 33554432000 4194304 4417452 140736396971856 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 21.16
Current children cumulated vsize (KiB) 341936

[startup+6.30107 s]
/proc/loadavg: 7.59 8.01 8.07 9/460 4131
/proc/meminfo: memFree=29480500/32951124 swapFree=60688720/67111528
[pid=4117] ppid=4115 vsize=355428 CPUtime=46.7 cores=0-7
/proc/4117/stat : 4117 (plingeling) S 4115 4117 4091 0 -1 4202496 52007 1253 0 5 4664 6 0 0 18 0 9 0 721567940 363958272 33701 33554432000 4194304 4417452 140736396971856 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4117/statm: 88857 33701 179 55 0 86743 0
[pid=4117/tid=4124] ppid=4115 vsize=355428 CPUtime=5.8 cores=0-7
/proc/4117/task/4124/stat : 4124 (plingeling) R 4115 4117 4091 0 -1 4202560 4527 1253 0 5 580 0 0 0 25 0 9 0 721567988 363958272 33701 33554432000 4194304 4417452 140736396971856 18446744073709551615 4249110 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4117/tid=4125] ppid=4115 vsize=355428 CPUtime=5.8 cores=0-7
/proc/4117/task/4125/stat : 4125 (plingeling) R 4115 4117 4091 0 -1 4202560 4393 1253 0 5 580 0 0 0 25 0 9 0 721567988 363958272 33701 33554432000 4194304 4417452 140736396971856 18446744073709551615 4258553 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4117/tid=4126] ppid=4115 vsize=355428 CPUtime=5.67 cores=0-7
/proc/4117/task/4126/stat : 4126 (plingeling) R 4115 4117 4091 0 -1 4202560 4104 1253 0 5 567 0 0 0 25 0 9 0 721567988 363958272 33701 33554432000 4194304 4417452 140736396971856 18446744073709551615 18446744073699065939 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4117/tid=4127] ppid=4115 vsize=355428 CPUtime=5.8 cores=0-7
/proc/4117/task/4127/stat : 4127 (plingeling) R 4115 4117 4091 0 -1 4202560 1397 1253 0 5 580 0 0 0 25 0 9 0 721567988 363958272 33701 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4117/tid=4128] ppid=4115 vsize=355428 CPUtime=5.8 cores=0-7
/proc/4117/task/4128/stat : 4128 (plingeling) R 4115 4117 4091 0 -1 4202560 5373 1253 0 5 580 0 0 0 25 0 9 0 721567988 363958272 33701 33554432000 4194304 4417452 140736396971856 18446744073709551615 4258565 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4117/tid=4129] ppid=4115 vsize=355428 CPUtime=5.81 cores=0-7
/proc/4117/task/4129/stat : 4129 (plingeling) R 4115 4117 4091 0 -1 4202560 4629 1253 0 5 581 0 0 0 25 0 9 0 721567988 363958272 33701 33554432000 4194304 4417452 140736396971856 18446744073709551615 4249233 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4117/tid=4130] ppid=4115 vsize=355428 CPUtime=5.79 cores=0-7
/proc/4117/task/4130/stat : 4130 (plingeling) R 4115 4117 4091 0 -1 4202560 4561 1253 0 5 579 0 0 0 25 0 9 0 721567988 363958272 33701 33554432000 4194304 4417452 140736396971856 18446744073709551615 4250447 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4117/tid=4131] ppid=4115 vsize=355428 CPUtime=5.79 cores=0-7
/proc/4117/task/4131/stat : 4131 (plingeling) R 4115 4117 4091 0 -1 4202560 5601 1253 0 5 579 0 0 0 25 0 9 0 721567988 363958272 33701 33554432000 4194304 4417452 140736396971856 18446744073709551615 4249209 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 46.7
Current children cumulated vsize (KiB) 355428

[startup+12.7009 s]
/proc/loadavg: 7.66 8.01 8.07 9/460 4132
/proc/meminfo: memFree=29464548/32951124 swapFree=60688720/67111528
[pid=4117] ppid=4115 vsize=371864 CPUtime=97.86 cores=0-7
/proc/4117/stat : 4117 (plingeling) S 4115 4117 4091 0 -1 4202496 56116 1253 0 5 9779 7 0 0 18 0 9 0 721567940 380788736 37810 33554432000 4194304 4417452 140736396971856 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4117/statm: 92966 37810 179 55 0 90852 0
[pid=4117/tid=4124] ppid=4115 vsize=371864 CPUtime=12.22 cores=0-7
/proc/4117/task/4124/stat : 4124 (plingeling) R 4115 4117 4091 0 -1 4202560 4527 1253 0 5 1222 0 0 0 25 0 9 0 721567988 380788736 37810 33554432000 4194304 4417452 140736396971856 18446744073709551615 4249222 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4117/tid=4125] ppid=4115 vsize=371864 CPUtime=12.21 cores=0-7
/proc/4117/task/4125/stat : 4125 (plingeling) R 4115 4117 4091 0 -1 4202560 4939 1253 0 5 1221 0 0 0 25 0 9 0 721567988 380788736 37810 33554432000 4194304 4417452 140736396971856 18446744073709551615 4249110 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4117/tid=4126] ppid=4115 vsize=371864 CPUtime=11.99 cores=0-7
/proc/4117/task/4126/stat : 4126 (plingeling) R 4115 4117 4091 0 -1 4202560 5366 1253 0 5 1198 1 0 0 25 0 9 0 721567988 380788736 37810 33554432000 4194304 4417452 140736396971856 18446744073709551615 4249601 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4117/tid=4127] ppid=4115 vsize=371864 CPUtime=12.21 cores=0-7
/proc/4117/task/4127/stat : 4127 (plingeling) R 4115 4117 4091 0 -1 4202560 1881 1253 0 5 1221 0 0 0 25 0 9 0 721567988 380788736 37810 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4117/tid=4128] ppid=4115 vsize=371864 CPUtime=12.21 cores=0-7
/proc/4117/task/4128/stat : 4128 (plingeling) R 4115 4117 4091 0 -1 4202560 5373 1253 0 5 1221 0 0 0 25 0 9 0 721567988 380788736 37810 33554432000 4194304 4417452 140736396971856 18446744073709551615 4249181 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4117/tid=4129] ppid=4115 vsize=371864 CPUtime=12.22 cores=0-7
/proc/4117/task/4129/stat : 4129 (plingeling) R 4115 4117 4091 0 -1 4202560 5345 1253 0 5 1222 0 0 0 25 0 9 0 721567988 380788736 37810 33554432000 4194304 4417452 140736396971856 18446744073709551615 4249186 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4117/tid=4130] ppid=4115 vsize=371864 CPUtime=12.2 cores=0-7
/proc/4117/task/4130/stat : 4130 (plingeling) R 4115 4117 4091 0 -1 4202560 5219 1253 0 5 1220 0 0 0 25 0 9 0 721567988 380788736 37810 33554432000 4194304 4417452 140736396971856 18446744073709551615 4236319 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4117/tid=4131] ppid=4115 vsize=371864 CPUtime=12.21 cores=0-7
/proc/4117/task/4131/stat : 4131 (plingeling) R 4115 4117 4091 0 -1 4202560 6044 1253 0 5 1221 0 0 0 25 0 9 0 721567988 380788736 37810 33554432000 4194304 4417452 140736396971856 18446744073709551615 4249997 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 97.86
Current children cumulated vsize (KiB) 371864

[startup+25.5007 s]
/proc/loadavg: 7.71 8.01 8.07 9/460 4132
/proc/meminfo: memFree=29442912/32951124 swapFree=60688720/67111528
[pid=4117] ppid=4115 vsize=395104 CPUtime=200.02 cores=0-7
/proc/4117/stat : 4117 (plingeling) S 4115 4117 4091 0 -1 4202496 61926 1253 0 5 19994 8 0 0 18 0 9 0 721567940 404586496 43620 33554432000 4194304 4417452 140736396971856 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4117/statm: 98776 43620 179 55 0 96662 0
[pid=4117/tid=4124] ppid=4115 vsize=395104 CPUtime=25 cores=0-7
/proc/4117/task/4124/stat : 4124 (plingeling) R 4115 4117 4091 0 -1 4202560 4527 1253 0 5 2500 0 0 0 25 0 9 0 721567988 404586496 43620 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4117/tid=4125] ppid=4115 vsize=395104 CPUtime=25 cores=0-7
/proc/4117/task/4125/stat : 4125 (plingeling) R 4115 4117 4091 0 -1 4202560 5485 1253 0 5 2500 0 0 0 25 0 9 0 721567988 404586496 43620 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4117/tid=4126] ppid=4115 vsize=395104 CPUtime=24.59 cores=0-7
/proc/4117/task/4126/stat : 4126 (plingeling) R 4115 4117 4091 0 -1 4202560 5912 1253 0 5 2458 1 0 0 25 0 9 0 721567988 404586496 43620 33554432000 4194304 4417452 140736396971856 18446744073709551615 4366412 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4117/tid=4127] ppid=4115 vsize=395104 CPUtime=25 cores=0-7
/proc/4117/task/4127/stat : 4127 (plingeling) R 4115 4117 4091 0 -1 4202560 5480 1253 0 5 2499 1 0 0 25 0 9 0 721567988 404586496 43620 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4117/tid=4128] ppid=4115 vsize=395104 CPUtime=25 cores=0-7
/proc/4117/task/4128/stat : 4128 (plingeling) R 4115 4117 4091 0 -1 4202560 5373 1253 0 5 2500 0 0 0 25 0 9 0 721567988 404586496 43620 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4117/tid=4129] ppid=4115 vsize=395104 CPUtime=25.01 cores=0-7
/proc/4117/task/4129/stat : 4129 (plingeling) R 4115 4117 4091 0 -1 4202560 5646 1253 0 5 2501 0 0 0 25 0 9 0 721567988 404586496 43620 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4117/tid=4130] ppid=4115 vsize=395104 CPUtime=24.98 cores=0-7
/proc/4117/task/4130/stat : 4130 (plingeling) R 4115 4117 4091 0 -1 4202560 6037 1253 0 5 2498 0 0 0 25 0 9 0 721567988 404586496 43620 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4117/tid=4131] ppid=4115 vsize=395104 CPUtime=24.99 cores=0-7
/proc/4117/task/4131/stat : 4131 (plingeling) R 4115 4117 4091 0 -1 4202560 6044 1253 0 5 2499 0 0 0 25 0 9 0 721567988 404586496 43620 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 200.02
Current children cumulated vsize (KiB) 395104

[startup+51.1011 s]
/proc/loadavg: 7.93 8.04 8.08 9/460 4133
/proc/meminfo: memFree=29430224/32951124 swapFree=60688720/67111528
[pid=4117] ppid=4115 vsize=407796 CPUtime=404.42 cores=0-7
/proc/4117/stat : 4117 (plingeling) S 4115 4117 4091 0 -1 4202496 65099 1253 0 5 40434 8 0 0 18 0 9 0 721567940 417583104 46793 33554432000 4194304 4417452 140736396971856 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4117/statm: 101949 46793 179 55 0 99835 0
[pid=4117/tid=4124] ppid=4115 vsize=407796 CPUtime=50.6 cores=0-7
/proc/4117/task/4124/stat : 4124 (plingeling) R 4115 4117 4091 0 -1 4202560 5017 1253 0 5 5060 0 0 0 25 0 9 0 721567988 417583104 46793 33554432000 4194304 4417452 140736396971856 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4117/tid=4125] ppid=4115 vsize=407796 CPUtime=50.6 cores=0-7
/proc/4117/task/4125/stat : 4125 (plingeling) R 4115 4117 4091 0 -1 4202560 5485 1253 0 5 5060 0 0 0 25 0 9 0 721567988 417583104 46793 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364607 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4117/tid=4126] ppid=4115 vsize=407796 CPUtime=49.8 cores=0-7
/proc/4117/task/4126/stat : 4126 (plingeling) R 4115 4117 4091 0 -1 4202560 6555 1253 0 5 4979 1 0 0 25 0 9 0 721567988 417583104 46793 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4117/tid=4127] ppid=4115 vsize=407796 CPUtime=50.59 cores=0-7
/proc/4117/task/4127/stat : 4127 (plingeling) R 4115 4117 4091 0 -1 4202560 6362 1253 0 5 5058 1 0 0 25 0 9 0 721567988 417583104 46793 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4117/tid=4128] ppid=4115 vsize=407796 CPUtime=50.6 cores=0-7
/proc/4117/task/4128/stat : 4128 (plingeling) R 4115 4117 4091 0 -1 4202560 5518 1253 0 5 5060 0 0 0 25 0 9 0 721567988 417583104 46793 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4117/tid=4129] ppid=4115 vsize=407796 CPUtime=50.61 cores=0-7
/proc/4117/task/4129/stat : 4129 (plingeling) R 4115 4117 4091 0 -1 4202560 5646 1253 0 5 5061 0 0 0 25 0 9 0 721567988 417583104 46793 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4117/tid=4130] ppid=4115 vsize=407796 CPUtime=50.58 cores=0-7
/proc/4117/task/4130/stat : 4130 (plingeling) R 4115 4117 4091 0 -1 4202560 6173 1253 0 5 5058 0 0 0 25 0 9 0 721567988 417583104 46793 33554432000 4194304 4417452 140736396971856 18446744073709551615 4251238 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4117/tid=4131] ppid=4115 vsize=407796 CPUtime=50.59 cores=0-7
/proc/4117/task/4131/stat : 4131 (plingeling) R 4115 4117 4091 0 -1 4202560 6921 1253 0 5 5059 0 0 0 25 0 9 0 721567988 417583104 46793 33554432000 4194304 4417452 140736396971856 18446744073709551615 4258389 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 404.42
Current children cumulated vsize (KiB) 407796

[startup+102.301 s]
/proc/loadavg: 8.06 8.06 8.08 9/460 4134

################
# More data... #
################

/proc/4117/task/4127/stat : 4127 (plingeling) R 4115 4117 4091 0 -1 4202560 749428 1253 0 5 465900 155 0 0 25 0 9 0 721567988 1087877120 210439 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4117/tid=4128] ppid=4115 vsize=1062380 CPUtime=4661.89 cores=0-7
/proc/4117/task/4128/stat : 4128 (plingeling) R 4115 4117 4091 0 -1 4202560 871146 1253 0 5 466025 164 0 0 25 0 9 0 721567988 1087877120 210439 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4117/tid=4129] ppid=4115 vsize=1062380 CPUtime=4644.04 cores=0-7
/proc/4117/task/4129/stat : 4129 (plingeling) R 4115 4117 4091 0 -1 4202560 618122 1253 0 5 464281 123 0 0 25 0 9 0 721567988 1087877120 210439 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4117/tid=4130] ppid=4115 vsize=1062380 CPUtime=4661.75 cores=0-7
/proc/4117/task/4130/stat : 4130 (plingeling) R 4115 4117 4091 0 -1 4202560 904508 1253 0 5 466014 161 0 0 25 0 9 0 721567988 1087877120 210439 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4117/tid=4131] ppid=4115 vsize=1062380 CPUtime=4661.29 cores=0-7
/proc/4117/task/4131/stat : 4131 (plingeling) R 4115 4117 4091 0 -1 4202560 938587 1253 0 5 465959 170 0 0 25 0 9 0 721567988 1087877120 210439 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 37221.5
Current children cumulated vsize (KiB) 1062380

[startup+4722.3 s]
/proc/loadavg: 8.07 8.11 8.09 9/460 4269
/proc/meminfo: memFree=28730264/32951124 swapFree=60688720/67111528
[pid=4117] ppid=4115 vsize=1103200 CPUtime=37700.5 cores=0-7
/proc/4117/stat : 4117 (plingeling) S 4115 4117 4091 0 -1 4202496 6588381 1253 0 5 3768833 1221 0 0 18 0 9 0 721567940 1129676800 220644 33554432000 4194304 4417452 140736396971856 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4117/statm: 275800 220644 179 55 0 273686 0
[pid=4117/tid=4124] ppid=4115 vsize=1103200 CPUtime=4721.92 cores=0-7
/proc/4117/task/4124/stat : 4124 (plingeling) R 4115 4117 4091 0 -1 4202560 926772 1253 0 5 472027 165 0 0 25 0 9 0 721567988 1129676800 220644 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4117/tid=4125] ppid=4115 vsize=1103200 CPUtime=4721.89 cores=0-7
/proc/4117/task/4125/stat : 4125 (plingeling) R 4115 4117 4091 0 -1 4202560 811549 1253 0 5 472041 148 0 0 25 0 9 0 721567988 1129676800 220644 33554432000 4194304 4417452 140736396971856 18446744073709551615 4365058 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4117/tid=4126] ppid=4115 vsize=1103200 CPUtime=4667.7 cores=0-7
/proc/4117/task/4126/stat : 4126 (plingeling) R 4115 4117 4091 0 -1 4202560 714120 1253 0 5 466649 121 0 0 25 0 9 0 721567988 1129676800 220644 33554432000 4194304 4417452 140736396971856 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4117/tid=4127] ppid=4115 vsize=1103200 CPUtime=4720.54 cores=0-7
/proc/4117/task/4127/stat : 4127 (plingeling) R 4115 4117 4091 0 -1 4202560 749428 1253 0 5 471899 155 0 0 25 0 9 0 721567988 1129676800 220644 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4117/tid=4128] ppid=4115 vsize=1103200 CPUtime=4721.89 cores=0-7
/proc/4117/task/4128/stat : 4128 (plingeling) R 4115 4117 4091 0 -1 4202560 879468 1253 0 5 472023 166 0 0 25 0 9 0 721567988 1129676800 220644 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4117/tid=4129] ppid=4115 vsize=1103200 CPUtime=4703.1 cores=0-7
/proc/4117/task/4129/stat : 4129 (plingeling) R 4115 4117 4091 0 -1 4202560 622199 1253 0 5 470186 124 0 0 25 0 9 0 721567988 1129676800 220644 33554432000 4194304 4417452 140736396971856 18446744073709551615 4365088 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4117/tid=4130] ppid=4115 vsize=1103200 CPUtime=4721.76 cores=0-7
/proc/4117/task/4130/stat : 4130 (plingeling) R 4115 4117 4091 0 -1 4202560 913519 1253 0 5 472012 164 0 0 25 0 9 0 721567988 1129676800 220644 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4117/tid=4131] ppid=4115 vsize=1103200 CPUtime=4721.29 cores=0-7
/proc/4117/task/4131/stat : 4131 (plingeling) R 4115 4117 4091 0 -1 4202560 953904 1253 0 5 471956 173 0 0 25 0 9 0 721567988 1129676800 220644 33554432000 4194304 4417452 140736396971856 18446744073709551615 4365027 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 37700.5
Current children cumulated vsize (KiB) 1103200

[startup+4782.3 s]
/proc/loadavg: 8.09 8.10 8.09 9/460 4271
/proc/meminfo: memFree=28645072/32951124 swapFree=60688720/67111528
[pid=4117] ppid=4115 vsize=1188080 CPUtime=38179.6 cores=0-7
/proc/4117/stat : 4117 (plingeling) S 4115 4117 4091 0 -1 4202496 6683766 1253 0 5 3816723 1242 0 0 18 0 9 0 721567940 1216593920 241864 33554432000 4194304 4417452 140736396971856 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4117/statm: 297020 241864 179 55 0 294906 0
[pid=4117/tid=4124] ppid=4115 vsize=1188080 CPUtime=4781.93 cores=0-7
/proc/4117/task/4124/stat : 4124 (plingeling) R 4115 4117 4091 0 -1 4202560 941796 1253 0 5 478025 168 0 0 25 0 9 0 721567988 1216593920 241864 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4117/tid=4125] ppid=4115 vsize=1188080 CPUtime=4781.91 cores=0-7
/proc/4117/task/4125/stat : 4125 (plingeling) R 4115 4117 4091 0 -1 4202560 819943 1253 0 5 478041 150 0 0 25 0 9 0 721567988 1216593920 241864 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4117/tid=4126] ppid=4115 vsize=1188080 CPUtime=4727.71 cores=0-7
/proc/4117/task/4126/stat : 4126 (plingeling) R 4115 4117 4091 0 -1 4202560 724287 1253 0 5 472648 123 0 0 25 0 9 0 721567988 1216593920 241864 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364461 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4117/tid=4127] ppid=4115 vsize=1188080 CPUtime=4780.54 cores=0-7
/proc/4117/task/4127/stat : 4127 (plingeling) R 4115 4117 4091 0 -1 4202560 757769 1253 0 5 477896 158 0 0 25 0 9 0 721567988 1216593920 241864 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4117/tid=4128] ppid=4115 vsize=1188080 CPUtime=4781.9 cores=0-7
/proc/4117/task/4128/stat : 4128 (plingeling) R 4115 4117 4091 0 -1 4202560 887597 1253 0 5 478022 168 0 0 25 0 9 0 721567988 1216593920 241864 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4117/tid=4129] ppid=4115 vsize=1188080 CPUtime=4762.19 cores=0-7
/proc/4117/task/4129/stat : 4129 (plingeling) R 4115 4117 4091 0 -1 4202560 638269 1253 0 5 476092 127 0 0 25 0 9 0 721567988 1216593920 241864 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4117/tid=4130] ppid=4115 vsize=1188080 CPUtime=4781.78 cores=0-7
/proc/4117/task/4130/stat : 4130 (plingeling) R 4115 4117 4091 0 -1 4202560 931449 1253 0 5 478010 168 0 0 25 0 9 0 721567988 1216593920 241864 33554432000 4194304 4417452 140736396971856 18446744073709551615 4366099 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4117/tid=4131] ppid=4115 vsize=1188080 CPUtime=4781.31 cores=0-7
/proc/4117/task/4131/stat : 4131 (plingeling) R 4115 4117 4091 0 -1 4202560 965234 1253 0 5 477956 175 0 0 25 0 9 0 721567988 1216593920 241864 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364737 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38179.7
Current children cumulated vsize (KiB) 1188080

[startup+4842.3 s]
/proc/loadavg: 8.29 8.15 8.10 10/460 4272
/proc/meminfo: memFree=28671584/32951124 swapFree=60688720/67111528
[pid=4117] ppid=4115 vsize=1161340 CPUtime=38658.7 cores=0-7
/proc/4117/stat : 4117 (plingeling) S 4115 4117 4091 0 -1 4202496 6754591 1253 0 5 3864611 1258 0 0 18 0 9 0 721567940 1189212160 235179 33554432000 4194304 4417452 140736396971856 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4117/statm: 290335 235179 179 55 0 288221 0
[pid=4117/tid=4124] ppid=4115 vsize=1161340 CPUtime=4841.92 cores=0-7
/proc/4117/task/4124/stat : 4124 (plingeling) R 4115 4117 4091 0 -1 4202560 952227 1253 0 5 484022 170 0 0 25 0 9 0 721567988 1189212160 235179 33554432000 4194304 4417452 140736396971856 18446744073709551615 4366407 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4117/tid=4125] ppid=4115 vsize=1161340 CPUtime=4841.89 cores=0-7
/proc/4117/task/4125/stat : 4125 (plingeling) R 4115 4117 4091 0 -1 4202560 830298 1253 0 5 484037 152 0 0 25 0 9 0 721567988 1189212160 235179 33554432000 4194304 4417452 140736396971856 18446744073709551615 4365027 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4117/tid=4126] ppid=4115 vsize=1161340 CPUtime=4787.7 cores=0-7
/proc/4117/task/4126/stat : 4126 (plingeling) R 4115 4117 4091 0 -1 4202560 736261 1253 0 5 478644 126 0 0 25 0 9 0 721567988 1189212160 235179 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4117/tid=4127] ppid=4115 vsize=1161340 CPUtime=4840.52 cores=0-7
/proc/4117/task/4127/stat : 4127 (plingeling) R 4115 4117 4091 0 -1 4202560 757769 1253 0 5 483894 158 0 0 25 0 9 0 721567988 1189212160 235179 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4117/tid=4128] ppid=4115 vsize=1161340 CPUtime=4841.89 cores=0-7
/proc/4117/task/4128/stat : 4128 (plingeling) R 4115 4117 4091 0 -1 4202560 900835 1253 0 5 484018 171 0 0 25 0 9 0 721567988 1189212160 235179 33554432000 4194304 4417452 140736396971856 18446744073709551615 4259882 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4117/tid=4129] ppid=4115 vsize=1161340 CPUtime=4821.26 cores=0-7
/proc/4117/task/4129/stat : 4129 (plingeling) R 4115 4117 4091 0 -1 4202560 644689 1253 0 5 481998 128 0 0 25 0 9 0 721567988 1189212160 235179 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4117/tid=4130] ppid=4115 vsize=1161340 CPUtime=4841.76 cores=0-7
/proc/4117/task/4130/stat : 4130 (plingeling) R 4115 4117 4091 0 -1 4202560 943775 1253 0 5 484005 171 0 0 25 0 9 0 721567988 1189212160 235179 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4117/tid=4131] ppid=4115 vsize=1161340 CPUtime=4841.3 cores=0-7
/proc/4117/task/4131/stat : 4131 (plingeling) R 4115 4117 4091 0 -1 4202560 971315 1253 0 5 483954 176 0 0 25 0 9 0 721567988 1189212160 235179 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38658.7
Current children cumulated vsize (KiB) 1161340

[startup+4902.32 s]
/proc/loadavg: 8.18 8.13 8.10 10/460 4274
/proc/meminfo: memFree=28869088/32951124 swapFree=60688720/67111528
[pid=4117] ppid=4115 vsize=968560 CPUtime=39137.9 cores=0-7
/proc/4117/stat : 4117 (plingeling) S 4115 4117 4091 0 -1 4202496 6843638 1253 0 5 3912512 1276 0 0 18 0 9 0 721567940 991805440 186984 33554432000 4194304 4417452 140736396971856 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4117/statm: 242140 186984 179 55 0 240026 0
[pid=4117/tid=4124] ppid=4115 vsize=968560 CPUtime=4901.93 cores=0-7
/proc/4117/task/4124/stat : 4124 (plingeling) R 4115 4117 4091 0 -1 4202560 957371 1253 0 5 490022 171 0 0 25 0 9 0 721567988 991805440 186984 33554432000 4194304 4417452 140736396971856 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4117/tid=4125] ppid=4115 vsize=968560 CPUtime=4901.91 cores=0-7
/proc/4117/task/4125/stat : 4125 (plingeling) R 4115 4117 4091 0 -1 4202560 835659 1253 0 5 490037 154 0 0 25 0 9 0 721567988 991805440 186984 33554432000 4194304 4417452 140736396971856 18446744073709551615 4365949 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4117/tid=4126] ppid=4115 vsize=968560 CPUtime=4847.72 cores=0-7
/proc/4117/task/4126/stat : 4126 (plingeling) R 4115 4117 4091 0 -1 4202560 751016 1253 0 5 484643 129 0 0 25 0 9 0 721567988 991805440 186984 33554432000 4194304 4417452 140736396971856 18446744073709551615 4366102 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4117/tid=4127] ppid=4115 vsize=968560 CPUtime=4900.52 cores=0-7
/proc/4117/task/4127/stat : 4127 (plingeling) R 4115 4117 4091 0 -1 4202560 778955 1253 0 5 489889 163 0 0 25 0 9 0 721567988 991805440 186984 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4117/tid=4128] ppid=4115 vsize=968560 CPUtime=4901.91 cores=0-7
/proc/4117/task/4128/stat : 4128 (plingeling) R 4115 4117 4091 0 -1 4202560 911830 1253 0 5 490018 173 0 0 25 0 9 0 721567988 991805440 186984 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364453 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4117/tid=4129] ppid=4115 vsize=968560 CPUtime=4880.35 cores=0-7
/proc/4117/task/4129/stat : 4129 (plingeling) R 4115 4117 4091 0 -1 4202560 654614 1253 0 5 487905 130 0 0 25 0 9 0 721567988 991805440 186984 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4117/tid=4130] ppid=4115 vsize=968560 CPUtime=4901.77 cores=0-7
/proc/4117/task/4130/stat : 4130 (plingeling) R 4115 4117 4091 0 -1 4202560 956615 1253 0 5 490004 173 0 0 25 0 9 0 721567988 991805440 186984 33554432000 4194304 4417452 140736396971856 18446744073709551615 4366385 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4117/tid=4131] ppid=4115 vsize=968560 CPUtime=4901.32 cores=0-7
/proc/4117/task/4131/stat : 4131 (plingeling) R 4115 4117 4091 0 -1 4202560 980156 1253 0 5 489954 178 0 0 25 0 9 0 721567988 991805440 186984 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39137.9
Current children cumulated vsize (KiB) 968560

[startup+4962.3 s]
/proc/loadavg: 8.13 8.12 8.09 9/460 4276
/proc/meminfo: memFree=28825700/32951124 swapFree=60688720/67111528
[pid=4117] ppid=4115 vsize=1007664 CPUtime=39616.8 cores=0-7
/proc/4117/stat : 4117 (plingeling) S 4115 4117 4091 0 -1 4202496 6928216 1253 0 5 3960392 1290 0 0 18 0 9 0 721567940 1031847936 196760 33554432000 4194304 4417452 140736396971856 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4117/statm: 251916 196760 179 55 0 249802 0
[pid=4117/tid=4124] ppid=4115 vsize=1007664 CPUtime=4961.92 cores=0-7
/proc/4117/task/4124/stat : 4124 (plingeling) R 4115 4117 4091 0 -1 4202560 969032 1253 0 5 496019 173 0 0 25 0 9 0 721567988 1031847936 196760 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364548 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4117/tid=4125] ppid=4115 vsize=1007664 CPUtime=4961.9 cores=0-7
/proc/4117/task/4125/stat : 4125 (plingeling) R 4115 4117 4091 0 -1 4202560 852190 1253 0 5 496033 157 0 0 25 0 9 0 721567988 1031847936 196760 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364903 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4117/tid=4126] ppid=4115 vsize=1007664 CPUtime=4907.7 cores=0-7
/proc/4117/task/4126/stat : 4126 (plingeling) R 4115 4117 4091 0 -1 4202560 759941 1253 0 5 490640 130 0 0 25 0 9 0 721567988 1031847936 196760 33554432000 4194304 4417452 140736396971856 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4117/tid=4127] ppid=4115 vsize=1007664 CPUtime=4960.49 cores=0-7
/proc/4117/task/4127/stat : 4127 (plingeling) R 4115 4117 4091 0 -1 4202560 782353 1253 0 5 495885 164 0 0 25 0 9 0 721567988 1031847936 196760 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4117/tid=4128] ppid=4115 vsize=1007664 CPUtime=4961.89 cores=0-7
/proc/4117/task/4128/stat : 4128 (plingeling) R 4115 4117 4091 0 -1 4202560 918541 1253 0 5 496015 174 0 0 25 0 9 0 721567988 1031847936 196760 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4117/tid=4129] ppid=4115 vsize=1007664 CPUtime=4939.4 cores=0-7
/proc/4117/task/4129/stat : 4129 (plingeling) R 4115 4117 4091 0 -1 4202560 662047 1253 0 5 493810 130 0 0 25 0 9 0 721567988 1031847936 196760 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4117/tid=4130] ppid=4115 vsize=1007664 CPUtime=4961.75 cores=0-7
/proc/4117/task/4130/stat : 4130 (plingeling) R 4115 4117 4091 0 -1 4202560 972572 1253 0 5 496000 175 0 0 25 0 9 0 721567988 1031847936 196760 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4117/tid=4131] ppid=4115 vsize=1007664 CPUtime=4961.3 cores=0-7
/proc/4117/task/4131/stat : 4131 (plingeling) R 4115 4117 4091 0 -1 4202560 994118 1253 0 5 495949 181 0 0 25 0 9 0 721567988 1031847936 196760 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39616.8
Current children cumulated vsize (KiB) 1007664



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5010.3 s]
/proc/loadavg: 8.14 8.12 8.09 9/460 4277
/proc/meminfo: memFree=28821224/32951124 swapFree=60688720/67111528
[pid=4117] ppid=4115 vsize=1011836 CPUtime=40000.1 cores=0-7
/proc/4117/stat : 4117 (plingeling) S 4115 4117 4091 0 -1 4202496 6998300 1253 0 5 3998708 1303 0 0 18 0 9 0 721567940 1036120064 197803 33554432000 4194304 4417452 140736396971856 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4117/statm: 252959 197803 179 55 0 250845 0
[pid=4117/tid=4124] ppid=4115 vsize=1011836 CPUtime=5009.94 cores=0-7
/proc/4117/task/4124/stat : 4124 (plingeling) R 4115 4117 4091 0 -1 4202560 980181 1253 0 5 500819 175 0 0 25 0 9 0 721567988 1036120064 197803 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364548 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4117/tid=4125] ppid=4115 vsize=1011836 CPUtime=5009.91 cores=0-7
/proc/4117/task/4125/stat : 4125 (plingeling) R 4115 4117 4091 0 -1 4202560 861247 1253 0 5 500833 158 0 0 25 0 9 0 721567988 1036120064 197803 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4117/tid=4126] ppid=4115 vsize=1011836 CPUtime=4955.72 cores=0-7
/proc/4117/task/4126/stat : 4126 (plingeling) R 4115 4117 4091 0 -1 4202560 769986 1253 0 5 495440 132 0 0 25 0 9 0 721567988 1036120064 197803 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4117/tid=4127] ppid=4115 vsize=1011836 CPUtime=5008.49 cores=0-7
/proc/4117/task/4127/stat : 4127 (plingeling) R 4115 4117 4091 0 -1 4202560 787092 1253 0 5 500685 164 0 0 25 0 9 0 721567988 1036120064 197803 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4117/tid=4128] ppid=4115 vsize=1011836 CPUtime=5009.91 cores=0-7
/proc/4117/task/4128/stat : 4128 (plingeling) R 4115 4117 4091 0 -1 4202560 928201 1253 0 5 500815 176 0 0 25 0 9 0 721567988 1036120064 197803 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4117/tid=4129] ppid=4115 vsize=1011836 CPUtime=4986.66 cores=0-7
/proc/4117/task/4129/stat : 4129 (plingeling) R 4115 4117 4091 0 -1 4202560 672507 1253 0 5 498534 132 0 0 25 0 9 0 721567988 1036120064 197803 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4117/tid=4130] ppid=4115 vsize=1011836 CPUtime=5009.77 cores=0-7
/proc/4117/task/4130/stat : 4130 (plingeling) R 4115 4117 4091 0 -1 4202560 978641 1253 0 5 500800 177 0 0 25 0 9 0 721567988 1036120064 197803 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4117/tid=4131] ppid=4115 vsize=1011836 CPUtime=5009.32 cores=0-7
/proc/4117/task/4131/stat : 4131 (plingeling) R 4115 4117 4091 0 -1 4202560 1003023 1253 0 5 500749 183 0 0 25 0 9 0 721567988 1036120064 197803 33554432000 4194304 4417452 140736396971856 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 40000.1
Current children cumulated vsize (KiB) 1011836

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5010.37
CPU time (s): 40000.3
CPU user time (s): 39987.2
CPU system time (s): 13.081
CPU usage (%): 798.35
Max. virtual memory (cumulated for all children) (KiB): 1188080

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39987.2
system time used= 13.081
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 6999553
page faults= 5
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 403
involuntary context switches= 345844

runsolver used 22.1596 second user time and 55.2766 second system time

The end

Launcher Data 

Begin job on node132 at 2011-05-22 23:28:18
IDJOB=3394560
IDBENCH=83245
IDSOLVER=1851
FILE ID=node132/3394560-1306099698
RUNJOBID= node132-1306099697-4105
PBS_JOBID= 13387978
Free space on /tmp= 73264 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/crafted/kullmann/GreenTao/GreenTaok1k2/GreenTao_2-3-6_2072.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3394560-1306099698/watcher-3394560-1306099698 -o /tmp/evaluation-result-3394560-1306099698/solver-3394560-1306099698 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3394560-1306099698.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= d2ef996f95bbf68a3c980c797dfdc364
RANDOM SEED=1426184267

node132.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      29614568 kB
Buffers:          8936 kB
Cached:          93692 kB
SwapCached:      14568 kB
Active:          65548 kB
Inactive:      3093620 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      29614568 kB
SwapTotal:    67111528 kB
SwapFree:     60688720 kB
Dirty:            2752 kB
Writeback:           0 kB
AnonPages:     3049800 kB
Mapped:          18224 kB
Slab:            56472 kB
PageTables:      49708 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 109199352 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73264 MiB
End job on node132 at 2011-05-23 00:51:50