Trace number 3309348

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 40000.3 5003.97

General information on the benchmark

NameAPPLICATIONS/bitverif/
countbitssrl/countbitssrl064.cnf
MD5SUM4f9c883534eb89cad9b9cad9e6eccbd0
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables75103
Number of clauses225116
Sum of the clauses size525268
Maximum clause length3
Minimum clause length1
Number of clauses of size 12
Number of clauses of size 2150076
Number of clauses of size 375038
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3309348-1303997329/watcher-3309348-1303997329 -o /tmp/evaluation-result-3309348-1303997329/solver-3309348-1303997329 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3309348-1303997329.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.56 7.93 7.94 1/170 10444
/proc/meminfo: memFree=30546236/32951124 swapFree=67111364/67111528
[pid=10444] ppid=10442 vsize=10168 CPUtime=0 cores=0-7
/proc/10444/stat : 10444 (plingeling) R 10442 10444 10091 0 -1 4202496 634 1259 0 0 0 0 0 0 23 0 1 0 511333338 10412032 564 33554432000 4194304 4417332 140735575755008 18446744073709551615 253640553419 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/10444/statm: 2542 564 147 55 0 428 0

[startup+0.0807029 s]
/proc/loadavg: 7.56 7.93 7.94 1/170 10444
/proc/meminfo: memFree=30546236/32951124 swapFree=67111364/67111528
[pid=10444] ppid=10442 vsize=27548 CPUtime=0.06 cores=0-7
/proc/10444/stat : 10444 (plingeling) R 10442 10444 10091 0 -1 4202496 4981 1259 0 0 6 0 0 0 23 0 1 0 511333338 28209152 4910 33554432000 4194304 4417332 140735575755008 18446744073709551615 4264950 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/10444/statm: 6887 4910 148 55 0 4773 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 27548

[startup+0.100695 s]
/proc/loadavg: 7.56 7.93 7.94 1/170 10444
/proc/meminfo: memFree=30546236/32951124 swapFree=67111364/67111528
[pid=10444] ppid=10442 vsize=27548 CPUtime=0.08 cores=0-7
/proc/10444/stat : 10444 (plingeling) R 10442 10444 10091 0 -1 4202496 4981 1259 0 0 8 0 0 0 23 0 1 0 511333338 28209152 4910 33554432000 4194304 4417332 140735575755008 18446744073709551615 4262086 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/10444/statm: 6887 4910 148 55 0 4773 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 27548

[startup+0.300666 s]
/proc/loadavg: 7.56 7.93 7.94 1/170 10444
/proc/meminfo: memFree=30546236/32951124 swapFree=67111364/67111528
[pid=10444] ppid=10442 vsize=80536 CPUtime=0.29 cores=0-7
/proc/10444/stat : 10444 (plingeling) R 10442 10444 10091 0 -1 4202496 18244 1259 0 0 27 2 0 0 24 0 1 0 511333338 82468864 18157 33554432000 4194304 4417332 140735575755008 18446744073709551615 253640859392 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/10444/statm: 20134 18157 148 55 0 18020 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 80536

[startup+0.700556 s]
/proc/loadavg: 7.56 7.93 7.94 1/170 10444
/proc/meminfo: memFree=30546236/32951124 swapFree=67111364/67111528
[pid=10444] ppid=10442 vsize=315052 CPUtime=1.91 cores=0-7
/proc/10444/stat : 10444 (plingeling) S 10442 10444 10091 0 -1 4202496 43784 1259 0 0 185 6 0 0 18 0 9 0 511333338 322613248 28353 33554432000 4194304 4417332 140735575755008 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10444/statm: 78763 28353 171 55 0 76649 0
Current children cumulated CPU time (s) 1.91
Current children cumulated vsize (KiB) 315052

[startup+1.50041 s]
/proc/loadavg: 7.56 7.93 7.94 9/179 10458
/proc/meminfo: memFree=30424156/32951124 swapFree=67111364/67111528
[pid=10444] ppid=10442 vsize=325124 CPUtime=8.32 cores=0-7
/proc/10444/stat : 10444 (plingeling) S 10442 10444 10091 0 -1 4202496 57370 1259 0 0 823 9 0 0 18 0 9 0 511333338 332926976 31404 33554432000 4194304 4417332 140735575755008 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10444/statm: 81281 31404 174 55 0 79167 0
[pid=10444/tid=10451] ppid=10442 vsize=325124 CPUtime=0.97 cores=0-7
/proc/10444/task/10451/stat : 10451 (plingeling) R 10442 10444 10091 0 -1 4202560 3797 1259 0 0 97 0 0 0 19 0 9 0 511333390 332926976 31404 33554432000 4194304 4417332 140735575755008 18446744073709551615 4284284 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=10444/tid=10452] ppid=10442 vsize=325124 CPUtime=0.96 cores=0-7
/proc/10444/task/10452/stat : 10452 (plingeling) R 10442 10444 10091 0 -1 4202560 2923 1259 0 0 96 0 0 0 25 0 9 0 511333390 332926976 31404 33554432000 4194304 4417332 140735575755008 18446744073709551615 4294216 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=10444/tid=10453] ppid=10442 vsize=325124 CPUtime=0.97 cores=0-7
/proc/10444/task/10453/stat : 10453 (plingeling) R 10442 10444 10091 0 -1 4202560 3623 1259 0 0 97 0 0 0 25 0 9 0 511333390 332926976 31404 33554432000 4194304 4417332 140735575755008 18446744073709551615 4302003 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=10444/tid=10454] ppid=10442 vsize=325124 CPUtime=0.97 cores=0-7
/proc/10444/task/10454/stat : 10454 (plingeling) R 10442 10444 10091 0 -1 4202560 2885 1259 0 0 97 0 0 0 19 0 9 0 511333390 332926976 31404 33554432000 4194304 4417332 140735575755008 18446744073709551615 4264778 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=10444/tid=10455] ppid=10442 vsize=325124 CPUtime=0.97 cores=0-7
/proc/10444/task/10455/stat : 10455 (plingeling) R 10442 10444 10091 0 -1 4202560 2965 1259 0 0 97 0 0 0 19 0 9 0 511333390 332926976 31404 33554432000 4194304 4417332 140735575755008 18446744073709551615 4284289 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10444/tid=10456] ppid=10442 vsize=325124 CPUtime=0.96 cores=0-7
/proc/10444/task/10456/stat : 10456 (plingeling) R 10442 10444 10091 0 -1 4202560 2880 1259 0 0 96 0 0 0 21 0 9 0 511333390 332926976 31404 33554432000 4194304 4417332 140735575755008 18446744073709551615 4306537 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10444/tid=10457] ppid=10442 vsize=325124 CPUtime=0.96 cores=0-7
/proc/10444/task/10457/stat : 10457 (plingeling) R 10442 10444 10091 0 -1 4202560 6818 1259 0 0 96 0 0 0 19 0 9 0 511333391 332926976 31404 33554432000 4194304 4417332 140735575755008 18446744073709551615 4302061 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10444/tid=10458] ppid=10442 vsize=325124 CPUtime=0.96 cores=0-7
/proc/10444/task/10458/stat : 10458 (plingeling) R 10442 10444 10091 0 -1 4202560 3742 1259 0 0 96 0 0 0 21 0 9 0 511333391 332926976 31404 33554432000 4194304 4417332 140735575755008 18446744073709551615 4288920 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 8.32
Current children cumulated vsize (KiB) 325124

[startup+3.10112 s]
/proc/loadavg: 7.56 7.93 7.94 9/179 10458
/proc/meminfo: memFree=30397496/32951124 swapFree=67111364/67111528
[pid=10444] ppid=10442 vsize=353612 CPUtime=21.11 cores=0-7
/proc/10444/stat : 10444 (plingeling) S 10442 10444 10091 0 -1 4202496 86454 1259 0 0 2098 13 0 0 18 0 9 0 511333338 362098688 37817 33554432000 4194304 4417332 140735575755008 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10444/statm: 88403 37817 180 55 0 86289 0
[pid=10444/tid=10451] ppid=10442 vsize=353612 CPUtime=2.57 cores=0-7
/proc/10444/task/10451/stat : 10451 (plingeling) R 10442 10444 10091 0 -1 4202560 6414 1259 0 0 256 1 0 0 22 0 9 0 511333390 362098688 37817 33554432000 4194304 4417332 140735575755008 18446744073709551615 4346556 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=10444/tid=10452] ppid=10442 vsize=353612 CPUtime=2.55 cores=0-7
/proc/10444/task/10452/stat : 10452 (plingeling) R 10442 10444 10091 0 -1 4202560 7356 1259 0 0 254 1 0 0 25 0 9 0 511333390 362098688 37817 33554432000 4194304 4417332 140735575755008 18446744073709551615 4346414 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=10444/tid=10453] ppid=10442 vsize=353612 CPUtime=2.57 cores=0-7
/proc/10444/task/10453/stat : 10453 (plingeling) R 10442 10444 10091 0 -1 4202560 8883 1259 0 0 256 1 0 0 25 0 9 0 511333390 362098688 37817 33554432000 4194304 4417332 140735575755008 18446744073709551615 4236448 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=10444/tid=10454] ppid=10442 vsize=353612 CPUtime=2.57 cores=0-7
/proc/10444/task/10454/stat : 10454 (plingeling) R 10442 10444 10091 0 -1 4202560 2885 1259 0 0 257 0 0 0 22 0 9 0 511333390 362098688 37817 33554432000 4194304 4417332 140735575755008 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=10444/tid=10455] ppid=10442 vsize=353612 CPUtime=2.57 cores=0-7
/proc/10444/task/10455/stat : 10455 (plingeling) R 10442 10444 10091 0 -1 4202560 7152 1259 0 0 256 1 0 0 22 0 9 0 511333390 362098688 37817 33554432000 4194304 4417332 140735575755008 18446744073709551615 4346594 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10444/tid=10456] ppid=10442 vsize=353612 CPUtime=2.56 cores=0-7
/proc/10444/task/10456/stat : 10456 (plingeling) R 10442 10444 10091 0 -1 4202560 5380 1259 0 0 256 0 0 0 25 0 9 0 511333390 362098688 37817 33554432000 4194304 4417332 140735575755008 18446744073709551615 4346401 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10444/tid=10457] ppid=10442 vsize=353612 CPUtime=2.57 cores=0-7
/proc/10444/task/10457/stat : 10457 (plingeling) R 10442 10444 10091 0 -1 4202560 10992 1259 0 0 256 1 0 0 22 0 9 0 511333391 362098688 37817 33554432000 4194304 4417332 140735575755008 18446744073709551615 4270916 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10444/tid=10458] ppid=10442 vsize=353612 CPUtime=2.56 cores=0-7
/proc/10444/task/10458/stat : 10458 (plingeling) R 10442 10444 10091 0 -1 4202560 9655 1259 0 0 255 1 0 0 25 0 9 0 511333391 362098688 37817 33554432000 4194304 4417332 140735575755008 18446744073709551615 4346410 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 21.11
Current children cumulated vsize (KiB) 353612

[startup+6.30053 s]
/proc/loadavg: 7.59 7.93 7.94 9/179 10458
/proc/meminfo: memFree=30379268/32951124 swapFree=67111364/67111528
[pid=10444] ppid=10442 vsize=357352 CPUtime=46.68 cores=0-7
/proc/10444/stat : 10444 (plingeling) S 10442 10444 10091 0 -1 4202496 90318 1259 0 0 4655 13 0 0 18 0 9 0 511333338 365928448 41280 33554432000 4194304 4417332 140735575755008 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10444/statm: 89338 41280 180 55 0 87224 0
[pid=10444/tid=10451] ppid=10442 vsize=357352 CPUtime=5.77 cores=0-7
/proc/10444/task/10451/stat : 10451 (plingeling) R 10442 10444 10091 0 -1 4202560 7472 1259 0 0 576 1 0 0 25 0 9 0 511333390 365928448 41280 33554432000 4194304 4417332 140735575755008 18446744073709551615 4253091 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=10444/tid=10452] ppid=10442 vsize=357352 CPUtime=5.73 cores=0-7
/proc/10444/task/10452/stat : 10452 (plingeling) R 10442 10444 10091 0 -1 4202560 8067 1259 0 0 572 1 0 0 25 0 9 0 511333390 365928448 41280 33554432000 4194304 4417332 140735575755008 18446744073709551615 4252022 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=10444/tid=10453] ppid=10442 vsize=357352 CPUtime=5.77 cores=0-7
/proc/10444/task/10453/stat : 10453 (plingeling) R 10442 10444 10091 0 -1 4202560 8883 1259 0 0 576 1 0 0 25 0 9 0 511333390 365928448 41280 33554432000 4194304 4417332 140735575755008 18446744073709551615 4281668 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=10444/tid=10454] ppid=10442 vsize=357352 CPUtime=5.77 cores=0-7
/proc/10444/task/10454/stat : 10454 (plingeling) R 10442 10444 10091 0 -1 4202560 3340 1259 0 0 577 0 0 0 25 0 9 0 511333390 365928448 41280 33554432000 4194304 4417332 140735575755008 18446744073709551615 4245664 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=10444/tid=10455] ppid=10442 vsize=357352 CPUtime=5.77 cores=0-7
/proc/10444/task/10455/stat : 10455 (plingeling) R 10442 10444 10091 0 -1 4202560 7158 1259 0 0 576 1 0 0 25 0 9 0 511333390 365928448 41280 33554432000 4194304 4417332 140735575755008 18446744073709551615 4252117 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10444/tid=10456] ppid=10442 vsize=357352 CPUtime=5.76 cores=0-7
/proc/10444/task/10456/stat : 10456 (plingeling) R 10442 10444 10091 0 -1 4202560 5386 1259 0 0 576 0 0 0 25 0 9 0 511333390 365928448 41280 33554432000 4194304 4417332 140735575755008 18446744073709551615 4253198 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10444/tid=10457] ppid=10442 vsize=357352 CPUtime=5.77 cores=0-7
/proc/10444/task/10457/stat : 10457 (plingeling) R 10442 10444 10091 0 -1 4202560 10992 1259 0 0 576 1 0 0 25 0 9 0 511333391 365928448 41280 33554432000 4194304 4417332 140735575755008 18446744073709551615 4232580 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10444/tid=10458] ppid=10442 vsize=357352 CPUtime=5.76 cores=0-7
/proc/10444/task/10458/stat : 10458 (plingeling) R 10442 10444 10091 0 -1 4202560 11283 1259 0 0 575 1 0 0 25 0 9 0 511333391 365928448 41280 33554432000 4194304 4417332 140735575755008 18446744073709551615 4258401 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 46.68
Current children cumulated vsize (KiB) 357352

[startup+12.7004 s]
/proc/loadavg: 7.62 7.93 7.94 9/179 10458
/proc/meminfo: memFree=30377068/32951124 swapFree=67111364/67111528
[pid=10444] ppid=10442 vsize=360564 CPUtime=97.84 cores=0-7
/proc/10444/stat : 10444 (plingeling) S 10442 10444 10091 0 -1 4202496 91413 1259 0 0 9771 13 0 0 18 0 9 0 511333338 369217536 42084 33554432000 4194304 4417332 140735575755008 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10444/statm: 90141 42084 181 55 0 88027 0
[pid=10444/tid=10451] ppid=10442 vsize=360564 CPUtime=12.17 cores=0-7
/proc/10444/task/10451/stat : 10451 (plingeling) R 10442 10444 10091 0 -1 4202560 7489 1259 0 0 1216 1 0 0 25 0 9 0 511333390 369217536 42084 33554432000 4194304 4417332 140735575755008 18446744073709551615 4251104 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=10444/tid=10452] ppid=10442 vsize=360564 CPUtime=12.1 cores=0-7
/proc/10444/task/10452/stat : 10452 (plingeling) R 10442 10444 10091 0 -1 4202560 8067 1259 0 0 1209 1 0 0 25 0 9 0 511333390 369217536 42084 33554432000 4194304 4417332 140735575755008 18446744073709551615 4355019 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=10444/tid=10453] ppid=10442 vsize=360564 CPUtime=12.17 cores=0-7
/proc/10444/task/10453/stat : 10453 (plingeling) R 10442 10444 10091 0 -1 4202560 8923 1259 0 0 1216 1 0 0 25 0 9 0 511333390 369217536 42084 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364565 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=10444/tid=10454] ppid=10442 vsize=360564 CPUtime=12.17 cores=0-7
/proc/10444/task/10454/stat : 10454 (plingeling) R 10442 10444 10091 0 -1 4202560 3340 1259 0 0 1217 0 0 0 25 0 9 0 511333390 369217536 42084 33554432000 4194304 4417332 140735575755008 18446744073709551615 4281668 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=10444/tid=10455] ppid=10442 vsize=360564 CPUtime=12.17 cores=0-7
/proc/10444/task/10455/stat : 10455 (plingeling) R 10442 10444 10091 0 -1 4202560 7431 1259 0 0 1216 1 0 0 25 0 9 0 511333390 369217536 42084 33554432000 4194304 4417332 140735575755008 18446744073709551615 4365950 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10444/tid=10456] ppid=10442 vsize=360564 CPUtime=12.16 cores=0-7
/proc/10444/task/10456/stat : 10456 (plingeling) R 10442 10444 10091 0 -1 4202560 5539 1259 0 0 1216 0 0 0 25 0 9 0 511333390 369217536 42084 33554432000 4194304 4417332 140735575755008 18446744073709551615 4251028 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10444/tid=10457] ppid=10442 vsize=360564 CPUtime=12.17 cores=0-7
/proc/10444/task/10457/stat : 10457 (plingeling) R 10442 10444 10091 0 -1 4202560 11588 1259 0 0 1216 1 0 0 25 0 9 0 511333391 369217536 42084 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10444/tid=10458] ppid=10442 vsize=360564 CPUtime=12.16 cores=0-7
/proc/10444/task/10458/stat : 10458 (plingeling) R 10442 10444 10091 0 -1 4202560 11299 1259 0 0 1215 1 0 0 25 0 9 0 511333391 369217536 42084 33554432000 4194304 4417332 140735575755008 18446744073709551615 4275927 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 97.84
Current children cumulated vsize (KiB) 360564

[startup+25.501 s]
/proc/loadavg: 7.71 7.93 7.94 9/179 10459
/proc/meminfo: memFree=30372668/32951124 swapFree=67111364/67111528
[pid=10444] ppid=10442 vsize=367812 CPUtime=200.18 cores=0-7
/proc/10444/stat : 10444 (plingeling) S 10442 10444 10091 0 -1 4202496 94960 1259 0 0 20004 14 0 0 18 0 9 0 511333338 376639488 43896 33554432000 4194304 4417332 140735575755008 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10444/statm: 91953 43896 181 55 0 89839 0
[pid=10444/tid=10451] ppid=10442 vsize=367812 CPUtime=24.97 cores=0-7
/proc/10444/task/10451/stat : 10451 (plingeling) R 10442 10444 10091 0 -1 4202560 7504 1259 0 0 2496 1 0 0 25 0 9 0 511333390 376639488 43896 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364750 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=10444/tid=10452] ppid=10442 vsize=367812 CPUtime=24.83 cores=0-7
/proc/10444/task/10452/stat : 10452 (plingeling) R 10442 10444 10091 0 -1 4202560 8099 1259 0 0 2482 1 0 0 25 0 9 0 511333390 376639488 43896 33554432000 4194304 4417332 140735575755008 18446744073709551615 4261410 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=10444/tid=10453] ppid=10442 vsize=367812 CPUtime=24.97 cores=0-7
/proc/10444/task/10453/stat : 10453 (plingeling) R 10442 10444 10091 0 -1 4202560 9145 1259 0 0 2496 1 0 0 25 0 9 0 511333390 376639488 43896 33554432000 4194304 4417332 140735575755008 18446744073709551615 4352396 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=10444/tid=10454] ppid=10442 vsize=367812 CPUtime=24.97 cores=0-7
/proc/10444/task/10454/stat : 10454 (plingeling) R 10442 10444 10091 0 -1 4202560 3788 1259 0 0 2497 0 0 0 25 0 9 0 511333390 376639488 43896 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364753 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=10444/tid=10455] ppid=10442 vsize=367812 CPUtime=24.97 cores=0-7
/proc/10444/task/10455/stat : 10455 (plingeling) R 10442 10444 10091 0 -1 4202560 7931 1259 0 0 2496 1 0 0 25 0 9 0 511333390 376639488 43896 33554432000 4194304 4417332 140735575755008 18446744073709551615 4345860 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10444/tid=10456] ppid=10442 vsize=367812 CPUtime=24.96 cores=0-7
/proc/10444/task/10456/stat : 10456 (plingeling) R 10442 10444 10091 0 -1 4202560 6178 1259 0 0 2496 0 0 0 25 0 9 0 511333390 376639488 43896 33554432000 4194304 4417332 140735575755008 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10444/tid=10457] ppid=10442 vsize=367812 CPUtime=24.97 cores=0-7
/proc/10444/task/10457/stat : 10457 (plingeling) R 10442 10444 10091 0 -1 4202560 13010 1259 0 0 2496 1 0 0 25 0 9 0 511333391 376639488 43896 33554432000 4194304 4417332 140735575755008 18446744073709551615 4237273 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10444/tid=10458] ppid=10442 vsize=367812 CPUtime=24.96 cores=0-7
/proc/10444/task/10458/stat : 10458 (plingeling) R 10442 10444 10091 0 -1 4202560 11568 1259 0 0 2495 1 0 0 25 0 9 0 511333391 376639488 43896 33554432000 4194304 4417332 140735575755008 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 200.18
Current children cumulated vsize (KiB) 367812

[startup+51.1003 s]
/proc/loadavg: 7.81 7.94 7.94 9/179 10460
/proc/meminfo: memFree=30334040/32951124 swapFree=67111364/67111528
[pid=10444] ppid=10442 vsize=406292 CPUtime=404.84 cores=0-7
/proc/10444/stat : 10444 (plingeling) S 10442 10444 10091 0 -1 4202496 108805 1259 0 0 40468 16 0 0 18 0 9 0 511333338 416043008 53528 33554432000 4194304 4417332 140735575755008 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10444/statm: 101573 53528 181 55 0 99459 0
[pid=10444/tid=10451] ppid=10442 vsize=406292 CPUtime=50.57 cores=0-7
/proc/10444/task/10451/stat : 10451 (plingeling) R 10442 10444 10091 0 -1 4202560 8234 1259 0 0 5056 1 0 0 25 0 9 0 511333390 416043008 53528 33554432000 4194304 4417332 140735575755008 18446744073709551615 4365101 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=10444/tid=10452] ppid=10442 vsize=406292 CPUtime=50.29 cores=0-7
/proc/10444/task/10452/stat : 10452 (plingeling) R 10442 10444 10091 0 -1 4202560 9005 1259 0 0 5028 1 0 0 25 0 9 0 511333390 416043008 53528 33554432000 4194304 4417332 140735575755008 18446744073709551615 4251028 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=10444/tid=10453] ppid=10442 vsize=406292 CPUtime=50.57 cores=0-7

################
# More data... #
################

/proc/10444/task/10454/stat : 10454 (plingeling) R 10442 10444 10091 0 -1 4202560 1163019 1259 0 0 472012 180 0 0 25 0 9 0 511333390 1271820288 265611 33554432000 4194304 4417332 140735575755008 18446744073709551615 4353424 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=10444/tid=10455] ppid=10442 vsize=1242012 CPUtime=4721.76 cores=0-7
/proc/10444/task/10455/stat : 10455 (plingeling) R 10442 10444 10091 0 -1 4202560 811891 1259 0 0 472036 140 0 0 25 0 9 0 511333390 1271820288 265611 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10444/tid=10456] ppid=10442 vsize=1242012 CPUtime=4721.78 cores=0-7
/proc/10444/task/10456/stat : 10456 (plingeling) R 10442 10444 10091 0 -1 4202560 726105 1259 0 0 472050 128 0 0 25 0 9 0 511333390 1271820288 265611 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10444/tid=10457] ppid=10442 vsize=1242012 CPUtime=4721.92 cores=0-7
/proc/10444/task/10457/stat : 10457 (plingeling) R 10442 10444 10091 0 -1 4202560 642342 1259 0 0 472076 116 0 0 25 0 9 0 511333391 1271820288 265611 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10444/tid=10458] ppid=10442 vsize=1242012 CPUtime=4720.82 cores=0-7
/proc/10444/task/10458/stat : 10458 (plingeling) R 10442 10444 10091 0 -1 4202560 611911 1259 0 0 471985 97 0 0 25 0 9 0 511333391 1271820288 265611 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 37748.9
Current children cumulated vsize (KiB) 1242012

[startup+4782.3 s]
/proc/loadavg: 8.03 8.02 8.00 9/179 10598
/proc/meminfo: memFree=29404140/32951124 swapFree=67111364/67111528
[pid=10444] ppid=10442 vsize=1321024 CPUtime=38228.5 cores=0-7
/proc/10444/stat : 10444 (plingeling) S 10442 10444 10091 0 -1 4202496 6296367 1259 0 0 3821778 1073 0 0 18 0 9 0 511333338 1352728576 285364 33554432000 4194304 4417332 140735575755008 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10444/statm: 330256 285364 181 55 0 328142 0
[pid=10444/tid=10451] ppid=10442 vsize=1321024 CPUtime=4781.89 cores=0-7
/proc/10444/task/10451/stat : 10451 (plingeling) R 10442 10444 10091 0 -1 4202560 574678 1259 0 0 478082 107 0 0 25 0 9 0 511333390 1352728576 285364 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=10444/tid=10452] ppid=10442 vsize=1321024 CPUtime=4755.98 cores=0-7
/proc/10444/task/10452/stat : 10452 (plingeling) R 10442 10444 10091 0 -1 4202560 780767 1259 0 0 475465 133 0 0 25 0 9 0 511333390 1352728576 285364 33554432000 4194304 4417332 140735575755008 18446744073709551615 4251164 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=10444/tid=10453] ppid=10442 vsize=1321024 CPUtime=4781.89 cores=0-7
/proc/10444/task/10453/stat : 10453 (plingeling) R 10442 10444 10091 0 -1 4202560 931288 1259 0 0 478030 159 0 0 25 0 9 0 511333390 1352728576 285364 33554432000 4194304 4417332 140735575755008 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=10444/tid=10454] ppid=10442 vsize=1321024 CPUtime=4781.91 cores=0-7
/proc/10444/task/10454/stat : 10454 (plingeling) R 10442 10444 10091 0 -1 4202560 1173971 1259 0 0 478008 183 0 0 25 0 9 0 511333390 1352728576 285364 33554432000 4194304 4417332 140735575755008 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=10444/tid=10455] ppid=10442 vsize=1321024 CPUtime=4781.76 cores=0-7
/proc/10444/task/10455/stat : 10455 (plingeling) R 10442 10444 10091 0 -1 4202560 817866 1259 0 0 478034 142 0 0 25 0 9 0 511333390 1352728576 285364 33554432000 4194304 4417332 140735575755008 18446744073709551615 4365021 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10444/tid=10456] ppid=10442 vsize=1321024 CPUtime=4781.78 cores=0-7
/proc/10444/task/10456/stat : 10456 (plingeling) R 10442 10444 10091 0 -1 4202560 730387 1259 0 0 478049 129 0 0 25 0 9 0 511333390 1352728576 285364 33554432000 4194304 4417332 140735575755008 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10444/tid=10457] ppid=10442 vsize=1321024 CPUtime=4781.91 cores=0-7
/proc/10444/task/10457/stat : 10457 (plingeling) R 10442 10444 10091 0 -1 4202560 645406 1259 0 0 478075 116 0 0 25 0 9 0 511333391 1352728576 285364 33554432000 4194304 4417332 140735575755008 18446744073709551615 4365975 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10444/tid=10458] ppid=10442 vsize=1321024 CPUtime=4780.81 cores=0-7
/proc/10444/task/10458/stat : 10458 (plingeling) R 10442 10444 10091 0 -1 4202560 614267 1259 0 0 477983 98 0 0 25 0 9 0 511333391 1352728576 285364 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38228.5
Current children cumulated vsize (KiB) 1321024

[startup+4842.3 s]
/proc/loadavg: 8.01 8.02 8.00 9/179 10599
/proc/meminfo: memFree=29575840/32951124 swapFree=67111364/67111528
[pid=10444] ppid=10442 vsize=1138408 CPUtime=38708.2 cores=0-7
/proc/10444/stat : 10444 (plingeling) S 10442 10444 10091 0 -1 4202496 6335947 1259 0 0 3869739 1080 0 0 18 0 9 0 511333338 1165729792 242547 33554432000 4194304 4417332 140735575755008 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10444/statm: 284602 242547 181 55 0 282488 0
[pid=10444/tid=10451] ppid=10442 vsize=1138408 CPUtime=4841.89 cores=0-7
/proc/10444/task/10451/stat : 10451 (plingeling) R 10442 10444 10091 0 -1 4202560 577499 1259 0 0 484082 107 0 0 25 0 9 0 511333390 1165729792 242547 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364656 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=10444/tid=10452] ppid=10442 vsize=1138408 CPUtime=4815.65 cores=0-7
/proc/10444/task/10452/stat : 10452 (plingeling) R 10442 10444 10091 0 -1 4202560 783906 1259 0 0 481431 134 0 0 25 0 9 0 511333390 1165729792 242547 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364709 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=10444/tid=10453] ppid=10442 vsize=1138408 CPUtime=4841.89 cores=0-7
/proc/10444/task/10453/stat : 10453 (plingeling) R 10442 10444 10091 0 -1 4202560 936780 1259 0 0 484029 160 0 0 25 0 9 0 511333390 1165729792 242547 33554432000 4194304 4417332 140735575755008 18446744073709551615 4237202 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=10444/tid=10454] ppid=10442 vsize=1138408 CPUtime=4841.92 cores=0-7
/proc/10444/task/10454/stat : 10454 (plingeling) R 10442 10444 10091 0 -1 4202560 1178033 1259 0 0 484008 184 0 0 25 0 9 0 511333390 1165729792 242547 33554432000 4194304 4417332 140735575755008 18446744073709551615 4237015 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=10444/tid=10455] ppid=10442 vsize=1138408 CPUtime=4841.76 cores=0-7
/proc/10444/task/10455/stat : 10455 (plingeling) R 10442 10444 10091 0 -1 4202560 822416 1259 0 0 484034 142 0 0 25 0 9 0 511333390 1165729792 242547 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10444/tid=10456] ppid=10442 vsize=1138408 CPUtime=4841.78 cores=0-7
/proc/10444/task/10456/stat : 10456 (plingeling) R 10442 10444 10091 0 -1 4202560 734639 1259 0 0 484049 129 0 0 25 0 9 0 511333390 1165729792 242547 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364590 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10444/tid=10457] ppid=10442 vsize=1138408 CPUtime=4841.91 cores=0-7
/proc/10444/task/10457/stat : 10457 (plingeling) R 10442 10444 10091 0 -1 4202560 649436 1259 0 0 484074 117 0 0 25 0 9 0 511333391 1165729792 242547 33554432000 4194304 4417332 140735575755008 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10444/tid=10458] ppid=10442 vsize=1138408 CPUtime=4840.8 cores=0-7
/proc/10444/task/10458/stat : 10458 (plingeling) R 10442 10444 10091 0 -1 4202560 625501 1259 0 0 483981 99 0 0 25 0 9 0 511333391 1165729792 242547 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364683 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38708.2
Current children cumulated vsize (KiB) 1138408

[startup+4902.3 s]
/proc/loadavg: 8.00 8.01 8.00 9/179 10601
/proc/meminfo: memFree=29406324/32951124 swapFree=67111364/67111528
[pid=10444] ppid=10442 vsize=1307280 CPUtime=39187.9 cores=0-7
/proc/10444/stat : 10444 (plingeling) S 10442 10444 10091 0 -1 4202496 6392306 1259 0 0 3917700 1088 0 0 18 0 9 0 511333338 1338654720 284765 33554432000 4194304 4417332 140735575755008 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10444/statm: 326820 284765 181 55 0 324706 0
[pid=10444/tid=10451] ppid=10442 vsize=1307280 CPUtime=4901.89 cores=0-7
/proc/10444/task/10451/stat : 10451 (plingeling) R 10442 10444 10091 0 -1 4202560 585601 1259 0 0 490081 108 0 0 25 0 9 0 511333390 1338654720 284765 33554432000 4194304 4417332 140735575755008 18446744073709551615 4250936 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=10444/tid=10452] ppid=10442 vsize=1307280 CPUtime=4875.33 cores=0-7
/proc/10444/task/10452/stat : 10452 (plingeling) R 10442 10444 10091 0 -1 4202560 787853 1259 0 0 487399 134 0 0 25 0 9 0 511333390 1338654720 284765 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364808 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=10444/tid=10453] ppid=10442 vsize=1307280 CPUtime=4901.9 cores=0-7
/proc/10444/task/10453/stat : 10453 (plingeling) R 10442 10444 10091 0 -1 4202560 945269 1259 0 0 490028 162 0 0 25 0 9 0 511333390 1338654720 284765 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=10444/tid=10454] ppid=10442 vsize=1307280 CPUtime=4901.92 cores=0-7
/proc/10444/task/10454/stat : 10454 (plingeling) R 10442 10444 10091 0 -1 4202560 1189050 1259 0 0 490007 185 0 0 25 0 9 0 511333390 1338654720 284765 33554432000 4194304 4417332 140735575755008 18446744073709551615 4365097 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=10444/tid=10455] ppid=10442 vsize=1307280 CPUtime=4901.77 cores=0-7
/proc/10444/task/10455/stat : 10455 (plingeling) R 10442 10444 10091 0 -1 4202560 834923 1259 0 0 490032 145 0 0 25 0 9 0 511333390 1338654720 284765 33554432000 4194304 4417332 140735575755008 18446744073709551615 4356687 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10444/tid=10456] ppid=10442 vsize=1307280 CPUtime=4901.78 cores=0-7
/proc/10444/task/10456/stat : 10456 (plingeling) R 10442 10444 10091 0 -1 4202560 738211 1259 0 0 490048 130 0 0 25 0 9 0 511333390 1338654720 284765 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10444/tid=10457] ppid=10442 vsize=1307280 CPUtime=4901.91 cores=0-7
/proc/10444/task/10457/stat : 10457 (plingeling) R 10442 10444 10091 0 -1 4202560 651556 1259 0 0 490074 117 0 0 25 0 9 0 511333391 1338654720 284765 33554432000 4194304 4417332 140735575755008 18446744073709551615 4357466 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10444/tid=10458] ppid=10442 vsize=1307280 CPUtime=4900.79 cores=0-7
/proc/10444/task/10458/stat : 10458 (plingeling) R 10442 10444 10091 0 -1 4202560 632106 1259 0 0 489979 100 0 0 25 0 9 0 511333391 1338654720 284765 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39187.9
Current children cumulated vsize (KiB) 1307280

[startup+4962.3 s]
/proc/loadavg: 8.05 8.02 8.00 9/179 10602
/proc/meminfo: memFree=29352620/32951124 swapFree=67111364/67111528
[pid=10444] ppid=10442 vsize=1360632 CPUtime=39667.6 cores=0-7
/proc/10444/stat : 10444 (plingeling) S 10442 10444 10091 0 -1 4202496 6446125 1259 0 0 3965660 1095 0 0 18 0 9 0 511333338 1393287168 298103 33554432000 4194304 4417332 140735575755008 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10444/statm: 340158 298103 181 55 0 338044 0
[pid=10444/tid=10451] ppid=10442 vsize=1360632 CPUtime=4961.9 cores=0-7
/proc/10444/task/10451/stat : 10451 (plingeling) R 10442 10444 10091 0 -1 4202560 590570 1259 0 0 496081 109 0 0 25 0 9 0 511333390 1393287168 298103 33554432000 4194304 4417332 140735575755008 18446744073709551615 4353603 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=10444/tid=10452] ppid=10442 vsize=1360632 CPUtime=4935 cores=0-7
/proc/10444/task/10452/stat : 10452 (plingeling) R 10442 10444 10091 0 -1 4202560 799541 1259 0 0 493363 137 0 0 25 0 9 0 511333390 1393287168 298103 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=10444/tid=10453] ppid=10442 vsize=1360632 CPUtime=4961.9 cores=0-7
/proc/10444/task/10453/stat : 10453 (plingeling) R 10442 10444 10091 0 -1 4202560 945269 1259 0 0 496028 162 0 0 25 0 9 0 511333390 1393287168 298103 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364375 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=10444/tid=10454] ppid=10442 vsize=1360632 CPUtime=4961.92 cores=0-7
/proc/10444/task/10454/stat : 10454 (plingeling) R 10442 10444 10091 0 -1 4202560 1203553 1259 0 0 496006 186 0 0 25 0 9 0 511333390 1393287168 298103 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=10444/tid=10455] ppid=10442 vsize=1360632 CPUtime=4961.76 cores=0-7
/proc/10444/task/10455/stat : 10455 (plingeling) R 10442 10444 10091 0 -1 4202560 845355 1259 0 0 496030 146 0 0 25 0 9 0 511333390 1393287168 298103 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364656 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10444/tid=10456] ppid=10442 vsize=1360632 CPUtime=4961.78 cores=0-7
/proc/10444/task/10456/stat : 10456 (plingeling) R 10442 10444 10091 0 -1 4202560 741541 1259 0 0 496048 130 0 0 25 0 9 0 511333390 1393287168 298103 33554432000 4194304 4417332 140735575755008 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10444/tid=10457] ppid=10442 vsize=1360632 CPUtime=4961.92 cores=0-7
/proc/10444/task/10457/stat : 10457 (plingeling) R 10442 10444 10091 0 -1 4202560 658179 1259 0 0 496074 118 0 0 25 0 9 0 511333391 1393287168 298103 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10444/tid=10458] ppid=10442 vsize=1360632 CPUtime=4960.79 cores=0-7
/proc/10444/task/10458/stat : 10458 (plingeling) R 10442 10444 10091 0 -1 4202560 634380 1259 0 0 495979 100 0 0 25 0 9 0 511333391 1393287168 298103 33554432000 4194304 4417332 140735575755008 18446744073709551615 4237232 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39667.6
Current children cumulated vsize (KiB) 1360632



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5003.91 s]
/proc/loadavg: 8.08 8.03 8.00 9/179 10604
/proc/meminfo: memFree=29159044/32951124 swapFree=67111364/67111528
[pid=10444] ppid=10442 vsize=1553992 CPUtime=40000.2 cores=0-7
/proc/10444/stat : 10444 (plingeling) S 10442 10444 10091 0 -1 4202496 6501005 1259 0 0 3998919 1102 0 0 18 0 9 0 511333338 1591287808 346443 33554432000 4194304 4417332 140735575755008 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10444/statm: 388498 346443 181 55 0 386384 0
[pid=10444/tid=10451] ppid=10442 vsize=1553992 CPUtime=5003.51 cores=0-7
/proc/10444/task/10451/stat : 10451 (plingeling) R 10442 10444 10091 0 -1 4202560 592928 1259 0 0 500242 109 0 0 25 0 9 0 511333390 1591287808 346443 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=10444/tid=10452] ppid=10442 vsize=1553992 CPUtime=4976.38 cores=0-7
/proc/10444/task/10452/stat : 10452 (plingeling) R 10442 10444 10091 0 -1 4202560 802979 1259 0 0 497500 138 0 0 25 0 9 0 511333390 1591287808 346443 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=10444/tid=10453] ppid=10442 vsize=1553992 CPUtime=5003.51 cores=0-7
/proc/10444/task/10453/stat : 10453 (plingeling) R 10442 10444 10091 0 -1 4202560 955696 1259 0 0 500188 163 0 0 25 0 9 0 511333390 1591287808 346443 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364473 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=10444/tid=10454] ppid=10442 vsize=1553992 CPUtime=5003.53 cores=0-7
/proc/10444/task/10454/stat : 10454 (plingeling) R 10442 10444 10091 0 -1 4202560 1221346 1259 0 0 500165 188 0 0 25 0 9 0 511333390 1591287808 346443 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=10444/tid=10455] ppid=10442 vsize=1553992 CPUtime=5003.38 cores=0-7
/proc/10444/task/10455/stat : 10455 (plingeling) R 10442 10444 10091 0 -1 4202560 846664 1259 0 0 500191 147 0 0 25 0 9 0 511333390 1591287808 346443 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364519 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10444/tid=10456] ppid=10442 vsize=1553992 CPUtime=5003.39 cores=0-7
/proc/10444/task/10456/stat : 10456 (plingeling) R 10442 10444 10091 0 -1 4202560 748020 1259 0 0 500208 131 0 0 25 0 9 0 511333390 1591287808 346443 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10444/tid=10457] ppid=10442 vsize=1553992 CPUtime=5003.53 cores=0-7
/proc/10444/task/10457/stat : 10457 (plingeling) R 10442 10444 10091 0 -1 4202560 660552 1259 0 0 500235 118 0 0 25 0 9 0 511333391 1591287808 346443 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10444/tid=10458] ppid=10442 vsize=1553992 CPUtime=5002.4 cores=0-7
/proc/10444/task/10458/stat : 10458 (plingeling) R 10442 10444 10091 0 -1 4202560 645083 1259 0 0 500139 101 0 0 25 0 9 0 511333391 1591287808 346443 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 40000.2
Current children cumulated vsize (KiB) 1553992

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5003.91 s]
/proc/loadavg: 8.08 8.03 8.00 9/179 10604
/proc/meminfo: memFree=29159044/32951124 swapFree=67111364/67111528
[pid=10444] ppid=10442 vsize=1553992 CPUtime=40000.2 cores=0-7
/proc/10444/stat : 10444 (plingeling) S 10442 10444 10091 0 -1 4202496 6501005 1259 0 0 3998919 1102 0 0 18 0 9 0 511333338 1591287808 346443 33554432000 4194304 4417332 140735575755008 18446744073709551615 253652662245 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10444/statm: 388498 346443 181 55 0 386384 0
[pid=10444/tid=10451] ppid=10442 vsize=1553992 CPUtime=5003.51 cores=0-7
/proc/10444/task/10451/stat : 10451 (plingeling) R 10442 10444 10091 0 -1 4202560 592928 1259 0 0 500242 109 0 0 25 0 9 0 511333390 1591287808 346443 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=10444/tid=10452] ppid=10442 vsize=1553992 CPUtime=4976.38 cores=0-7
/proc/10444/task/10452/stat : 10452 (plingeling) R 10442 10444 10091 0 -1 4202560 802979 1259 0 0 497500 138 0 0 25 0 9 0 511333390 1591287808 346443 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=10444/tid=10453] ppid=10442 vsize=1553992 CPUtime=5003.51 cores=0-7
/proc/10444/task/10453/stat : 10453 (plingeling) R 10442 10444 10091 0 -1 4202560 955696 1259 0 0 500188 163 0 0 25 0 9 0 511333390 1591287808 346443 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364473 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=10444/tid=10454] ppid=10442 vsize=1553992 CPUtime=5003.53 cores=0-7
/proc/10444/task/10454/stat : 10454 (plingeling) R 10442 10444 10091 0 -1 4202560 1221346 1259 0 0 500165 188 0 0 25 0 9 0 511333390 1591287808 346443 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=10444/tid=10455] ppid=10442 vsize=1553992 CPUtime=5003.38 cores=0-7
/proc/10444/task/10455/stat : 10455 (plingeling) R 10442 10444 10091 0 -1 4202560 846664 1259 0 0 500191 147 0 0 25 0 9 0 511333390 1591287808 346443 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364519 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=10444/tid=10456] ppid=10442 vsize=1553992 CPUtime=5003.39 cores=0-7
/proc/10444/task/10456/stat : 10456 (plingeling) R 10442 10444 10091 0 -1 4202560 748020 1259 0 0 500208 131 0 0 25 0 9 0 511333390 1591287808 346443 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10444/tid=10457] ppid=10442 vsize=1553992 CPUtime=5003.53 cores=0-7
/proc/10444/task/10457/stat : 10457 (plingeling) R 10442 10444 10091 0 -1 4202560 660552 1259 0 0 500235 118 0 0 25 0 9 0 511333391 1591287808 346443 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10444/tid=10458] ppid=10442 vsize=1553992 CPUtime=5002.4 cores=0-7
/proc/10444/task/10458/stat : 10458 (plingeling) R 10442 10444 10091 0 -1 4202560 645083 1259 0 0 500139 101 0 0 25 0 9 0 511333391 1591287808 346443 33554432000 4194304 4417332 140735575755008 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 40000.2
Current children cumulated vsize (KiB) 1553992

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5003.97
CPU time (s): 40000.3
CPU user time (s): 39989.2
CPU system time (s): 11.0853
CPU usage (%): 799.371
Max. virtual memory (cumulated for all children) (KiB): 1553992

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39989.2
system time used= 11.0853
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 6502264
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 414
involuntary context switches= 190352

runsolver used 8.09377 second user time and 19.0561 second system time

The end

Launcher Data 

Begin job on node106 at 2011-04-28 15:28:49
IDJOB=3309348
IDBENCH=70781
IDSOLVER=1695
FILE ID=node106/3309348-1303997329
RUNJOBID= node106-1303994636-10105
PBS_JOBID= 13168780
Free space on /tmp= 72156 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT09/APPLICATIONS/bitverif/countbitssrl/countbitssrl064.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3309348-1303997329/watcher-3309348-1303997329 -o /tmp/evaluation-result-3309348-1303997329/solver-3309348-1303997329 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3309348-1303997329.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 4f9c883534eb89cad9b9cad9e6eccbd0
RANDOM SEED=1340903642

node106.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.20
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      30546640 kB
Buffers:        193184 kB
Cached:        1937288 kB
SwapCached:         88 kB
Active:         631556 kB
Inactive:      1562160 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      30546640 kB
SwapTotal:    67111528 kB
SwapFree:     67111364 kB
Dirty:            4040 kB
Writeback:           0 kB
AnonPages:       63244 kB
Mapped:          14480 kB
Slab:           144148 kB
PageTables:       3900 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   186828 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72156 MiB
End job on node106 at 2011-04-28 16:52:15