Trace number 3372167

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40000.5 5006.9

General information on the benchmark

Nameindustrial/IBM_FV_2002_13_rule_1/
SAT_dat.k100.cnf
MD5SUMcf1b8194ff6d413d396ac9e26bb031ef
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark164.562
Satisfiable
(Un)Satisfiability was proved
Number of variables50797
Number of clauses177980
Sum of the clauses size470252
Maximum clause length37
Minimum clause length1
Number of clauses of size 11318
Number of clauses of size 295086
Number of clauses of size 374860
Number of clauses of size 4486
Number of clauses of size 5945
Number of clauses of size over 55285

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3372167-1305436237/watcher-3372167-1305436237 -o /tmp/evaluation-result-3372167-1305436237/solver-3372167-1305436237 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3372167-1305436237.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.50 7.72 7.04 2/309 27795
/proc/meminfo: memFree=15350820/32950928 swapFree=67085488/67111528
[pid=27795] ppid=27793 vsize=8728 CPUtime=0 cores=0-7
/proc/27795/stat : 27795 (plingeling) R 27793 27795 27696 0 -1 4202496 254 1256 0 0 0 0 0 0 21 0 1 0 733854348 8937472 185 33554432000 4194304 4417452 140735876598192 18446744073709551615 231895490304 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/27795/statm: 2182 185 138 55 0 68 0

[startup+0.0268829 s]
/proc/loadavg: 7.50 7.72 7.04 2/309 27795
/proc/meminfo: memFree=15350820/32950928 swapFree=67085488/67111528
[pid=27795] ppid=27793 vsize=12012 CPUtime=0.02 cores=0-7
/proc/27795/stat : 27795 (plingeling) R 27793 27795 27696 0 -1 4202496 1063 1256 0 0 2 0 0 0 21 0 1 0 733854348 12300288 994 33554432000 4194304 4417452 140735876598192 18446744073709551615 231895109089 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/27795/statm: 3003 994 148 55 0 889 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 12012

[startup+0.100871 s]
/proc/loadavg: 7.50 7.72 7.04 2/309 27795
/proc/meminfo: memFree=15350820/32950928 swapFree=67085488/67111528
[pid=27795] ppid=27793 vsize=28376 CPUtime=0.08 cores=0-7
/proc/27795/stat : 27795 (plingeling) R 27793 27795 27696 0 -1 4202496 5189 1256 0 0 8 0 0 0 21 0 1 0 733854348 29057024 5118 33554432000 4194304 4417452 140735876598192 18446744073709551615 4231321 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/27795/statm: 7094 5118 149 55 0 4980 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 28376

[startup+0.300834 s]
/proc/loadavg: 7.50 7.72 7.04 2/309 27795
/proc/meminfo: memFree=15350820/32950928 swapFree=67085488/67111528
[pid=27795] ppid=27793 vsize=63620 CPUtime=0.29 cores=0-7
/proc/27795/stat : 27795 (plingeling) R 27793 27795 27696 0 -1 4202496 14000 1256 0 0 27 2 0 0 22 0 1 0 733854348 65146880 13929 33554432000 4194304 4417452 140735876598192 18446744073709551615 4231321 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/27795/statm: 15905 13929 149 55 0 13791 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 63620

[startup+0.700744 s]
/proc/loadavg: 7.50 7.72 7.04 2/309 27795
/proc/meminfo: memFree=15350820/32950928 swapFree=67085488/67111528
[pid=27795] ppid=27793 vsize=293208 CPUtime=2.33 cores=0-7
/proc/27795/stat : 27795 (plingeling) S 27793 27795 27696 0 -1 4202496 32373 1256 0 0 229 4 0 0 18 0 9 0 733854348 300244992 22361 33554432000 4194304 4417452 140735876598192 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/27795/statm: 73302 22361 175 55 0 71188 0
Current children cumulated CPU time (s) 2.33
Current children cumulated vsize (KiB) 293208

[startup+1.50054 s]
/proc/loadavg: 7.50 7.72 7.04 9/318 27809
/proc/meminfo: memFree=15261112/32950928 swapFree=67085488/67111528
[pid=27795] ppid=27793 vsize=300884 CPUtime=8.68 cores=0-7
/proc/27795/stat : 27795 (plingeling) S 27793 27795 27696 0 -1 4202496 47869 1256 0 0 862 6 0 0 18 0 9 0 733854348 308105216 21880 33554432000 4194304 4417452 140735876598192 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/27795/statm: 75221 21880 176 55 0 73107 0
[pid=27795/tid=27802] ppid=27793 vsize=300884 CPUtime=1.06 cores=0-7
/proc/27795/task/27802/stat : 27802 (plingeling) R 27793 27795 27696 0 -1 4202560 8095 1256 0 0 106 0 0 0 19 0 9 0 733854391 308105216 21880 33554432000 4194304 4417452 140735876598192 18446744073709551615 4305799 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=27795/tid=27803] ppid=27793 vsize=300884 CPUtime=1.04 cores=0-7
/proc/27795/task/27803/stat : 27803 (plingeling) R 27793 27795 27696 0 -1 4202560 4956 1256 0 0 104 0 0 0 25 0 9 0 733854391 308105216 21880 33554432000 4194304 4417452 140735876598192 18446744073709551615 4276477 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27795/tid=27804] ppid=27793 vsize=300884 CPUtime=0.77 cores=0-7
/proc/27795/task/27804/stat : 27804 (plingeling) R 27793 27795 27696 0 -1 4202560 679 1256 0 0 77 0 0 0 25 0 9 0 733854391 308105216 21880 33554432000 4194304 4417452 140735876598192 18446744073709551615 4289629 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=27795/tid=27805] ppid=27793 vsize=300884 CPUtime=1.07 cores=0-7
/proc/27795/task/27805/stat : 27805 (plingeling) R 27793 27795 27696 0 -1 4202560 1089 1256 0 0 107 0 0 0 19 0 9 0 733854391 308105216 21880 33554432000 4194304 4417452 140735876598192 18446744073709551615 4365959 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27795/tid=27806] ppid=27793 vsize=300884 CPUtime=1.06 cores=0-7
/proc/27795/task/27806/stat : 27806 (plingeling) R 27793 27795 27696 0 -1 4202560 1354 1256 0 0 106 0 0 0 19 0 9 0 733854391 308105216 21880 33554432000 4194304 4417452 140735876598192 18446744073709551615 4284626 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=27795/tid=27807] ppid=27793 vsize=300884 CPUtime=1.06 cores=0-7
/proc/27795/task/27807/stat : 27807 (plingeling) R 27793 27795 27696 0 -1 4202560 7371 1256 0 0 106 0 0 0 19 0 9 0 733854391 308105216 21880 33554432000 4194304 4417452 140735876598192 18446744073709551615 4277628 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27795/tid=27808] ppid=27793 vsize=300884 CPUtime=1.07 cores=0-7
/proc/27795/task/27808/stat : 27808 (plingeling) R 27793 27795 27696 0 -1 4202560 2042 1256 0 0 107 0 0 0 21 0 9 0 733854391 308105216 21880 33554432000 4194304 4417452 140735876598192 18446744073709551615 4277540 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27795/tid=27809] ppid=27793 vsize=300884 CPUtime=1.06 cores=0-7
/proc/27795/task/27809/stat : 27809 (plingeling) R 27793 27795 27696 0 -1 4202560 3524 1256 0 0 106 0 0 0 19 0 9 0 733854391 308105216 21880 33554432000 4194304 4417452 140735876598192 18446744073709551615 4275748 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 8.68
Current children cumulated vsize (KiB) 300884

[startup+3.10123 s]
/proc/loadavg: 7.50 7.72 7.04 9/318 27809
/proc/meminfo: memFree=15261360/32950928 swapFree=67085488/67111528
[pid=27795] ppid=27793 vsize=330220 CPUtime=21.47 cores=0-7
/proc/27795/stat : 27795 (plingeling) S 27793 27795 27696 0 -1 4202496 66668 1256 0 0 2138 9 0 0 18 0 9 0 733854348 338145280 29032 33554432000 4194304 4417452 140735876598192 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/27795/statm: 82555 29032 180 55 0 80441 0
[pid=27795/tid=27802] ppid=27793 vsize=330220 CPUtime=2.67 cores=0-7
/proc/27795/task/27802/stat : 27802 (plingeling) R 27793 27795 27696 0 -1 4202560 11179 1256 0 0 266 1 0 0 23 0 9 0 733854391 338145280 29032 33554432000 4194304 4417452 140735876598192 18446744073709551615 4253420 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=27795/tid=27803] ppid=27793 vsize=330220 CPUtime=2.64 cores=0-7
/proc/27795/task/27803/stat : 27803 (plingeling) R 27793 27795 27696 0 -1 4202560 6855 1256 0 0 264 0 0 0 25 0 9 0 733854391 338145280 29032 33554432000 4194304 4417452 140735876598192 18446744073709551615 4252931 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27795/tid=27804] ppid=27793 vsize=330220 CPUtime=2.36 cores=0-7
/proc/27795/task/27804/stat : 27804 (plingeling) R 27793 27795 27696 0 -1 4202560 857 1256 0 0 236 0 0 0 25 0 9 0 733854391 338145280 29032 33554432000 4194304 4417452 140735876598192 18446744073709551615 4377023 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=27795/tid=27805] ppid=27793 vsize=330220 CPUtime=2.67 cores=0-7
/proc/27795/task/27805/stat : 27805 (plingeling) R 27793 27795 27696 0 -1 4202560 2113 1256 0 0 267 0 0 0 22 0 9 0 733854391 338145280 29032 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27795/tid=27806] ppid=27793 vsize=330220 CPUtime=2.67 cores=0-7
/proc/27795/task/27806/stat : 27806 (plingeling) R 27793 27795 27696 0 -1 4202560 4131 1256 0 0 266 1 0 0 22 0 9 0 733854391 338145280 29032 33554432000 4194304 4417452 140735876598192 18446744073709551615 4252365 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=27795/tid=27807] ppid=27793 vsize=330220 CPUtime=2.66 cores=0-7
/proc/27795/task/27807/stat : 27807 (plingeling) R 27793 27795 27696 0 -1 4202560 14312 1256 0 0 265 1 0 0 22 0 9 0 733854391 338145280 29032 33554432000 4194304 4417452 140735876598192 18446744073709551615 4252475 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27795/tid=27808] ppid=27793 vsize=330220 CPUtime=2.67 cores=0-7
/proc/27795/task/27808/stat : 27808 (plingeling) R 27793 27795 27696 0 -1 4202560 3501 1256 0 0 267 0 0 0 25 0 9 0 733854391 338145280 29032 33554432000 4194304 4417452 140735876598192 18446744073709551615 4232479 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27795/tid=27809] ppid=27793 vsize=330220 CPUtime=2.66 cores=0-7
/proc/27795/task/27809/stat : 27809 (plingeling) R 27793 27795 27696 0 -1 4202560 4961 1256 0 0 266 0 0 0 23 0 9 0 733854391 338145280 29032 33554432000 4194304 4417452 140735876598192 18446744073709551615 4273384 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 21.47
Current children cumulated vsize (KiB) 330220

[startup+6.30064 s]
/proc/loadavg: 7.54 7.72 7.05 9/318 27809
/proc/meminfo: memFree=15226664/32950928 swapFree=67085488/67111528
[pid=27795] ppid=27793 vsize=348020 CPUtime=47.04 cores=0-7
/proc/27795/stat : 27795 (plingeling) S 27793 27795 27696 0 -1 4202496 72459 1256 0 0 4694 10 0 0 18 0 9 0 733854348 356372480 34482 33554432000 4194304 4417452 140735876598192 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/27795/statm: 87005 34482 180 55 0 84891 0
[pid=27795/tid=27802] ppid=27793 vsize=348020 CPUtime=5.87 cores=0-7
/proc/27795/task/27802/stat : 27802 (plingeling) R 27793 27795 27696 0 -1 4202560 12477 1256 0 0 586 1 0 0 25 0 9 0 733854391 356372480 34482 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=27795/tid=27803] ppid=27793 vsize=348020 CPUtime=5.85 cores=0-7
/proc/27795/task/27803/stat : 27803 (plingeling) R 27793 27795 27696 0 -1 4202560 7536 1256 0 0 584 1 0 0 25 0 9 0 733854391 356372480 34482 33554432000 4194304 4417452 140735876598192 18446744073709551615 4366227 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27795/tid=27804] ppid=27793 vsize=348020 CPUtime=5.52 cores=0-7
/proc/27795/task/27804/stat : 27804 (plingeling) R 27793 27795 27696 0 -1 4202560 857 1256 0 0 552 0 0 0 25 0 9 0 733854391 356372480 34482 33554432000 4194304 4417452 140735876598192 18446744073709551615 4236995 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=27795/tid=27805] ppid=27793 vsize=348020 CPUtime=5.87 cores=0-7
/proc/27795/task/27805/stat : 27805 (plingeling) R 27793 27795 27696 0 -1 4202560 3383 1256 0 0 587 0 0 0 25 0 9 0 733854391 356372480 34482 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364537 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27795/tid=27806] ppid=27793 vsize=348020 CPUtime=5.87 cores=0-7
/proc/27795/task/27806/stat : 27806 (plingeling) R 27793 27795 27696 0 -1 4202560 4131 1256 0 0 586 1 0 0 25 0 9 0 733854391 356372480 34482 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364583 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=27795/tid=27807] ppid=27793 vsize=348020 CPUtime=5.87 cores=0-7
/proc/27795/task/27807/stat : 27807 (plingeling) R 27793 27795 27696 0 -1 4202560 15597 1256 0 0 585 2 0 0 25 0 9 0 733854391 356372480 34482 33554432000 4194304 4417452 140735876598192 18446744073709551615 4365167 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27795/tid=27808] ppid=27793 vsize=348020 CPUtime=5.87 cores=0-7
/proc/27795/task/27808/stat : 27808 (plingeling) R 27793 27795 27696 0 -1 4202560 4071 1256 0 0 587 0 0 0 25 0 9 0 733854391 356372480 34482 33554432000 4194304 4417452 140735876598192 18446744073709551615 4353557 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27795/tid=27809] ppid=27793 vsize=348020 CPUtime=5.86 cores=0-7
/proc/27795/task/27809/stat : 27809 (plingeling) R 27793 27795 27696 0 -1 4202560 5648 1256 0 0 586 0 0 0 25 0 9 0 733854391 356372480 34482 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364466 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 47.04
Current children cumulated vsize (KiB) 348020

[startup+12.7004 s]
/proc/loadavg: 7.58 7.73 7.05 9/318 27809
/proc/meminfo: memFree=15182444/32950928 swapFree=67085488/67111528
[pid=27795] ppid=27793 vsize=377052 CPUtime=98.17 cores=0-7
/proc/27795/stat : 27795 (plingeling) S 27793 27795 27696 0 -1 4202496 91055 1256 0 0 9804 13 0 0 18 0 9 0 733854348 386101248 42376 33554432000 4194304 4417452 140735876598192 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/27795/statm: 94263 42376 180 55 0 92149 0
[pid=27795/tid=27802] ppid=27793 vsize=377052 CPUtime=12.27 cores=0-7
/proc/27795/task/27802/stat : 27802 (plingeling) R 27793 27795 27696 0 -1 4202560 14408 1256 0 0 1225 2 0 0 25 0 9 0 733854391 386101248 42376 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=27795/tid=27803] ppid=27793 vsize=377052 CPUtime=12.24 cores=0-7
/proc/27795/task/27803/stat : 27803 (plingeling) R 27793 27795 27696 0 -1 4202560 7938 1256 0 0 1223 1 0 0 25 0 9 0 733854391 386101248 42376 33554432000 4194304 4417452 140735876598192 18446744073709551615 4358025 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27795/tid=27804] ppid=27793 vsize=377052 CPUtime=11.85 cores=0-7
/proc/27795/task/27804/stat : 27804 (plingeling) R 27793 27795 27696 0 -1 4202560 1881 1256 0 0 1185 0 0 0 25 0 9 0 733854391 386101248 42376 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=27795/tid=27805] ppid=27793 vsize=377052 CPUtime=12.26 cores=0-7
/proc/27795/task/27805/stat : 27805 (plingeling) R 27793 27795 27696 0 -1 4202560 5944 1256 0 0 1226 0 0 0 25 0 9 0 733854391 386101248 42376 33554432000 4194304 4417452 140735876598192 18446744073709551615 4357554 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27795/tid=27806] ppid=27793 vsize=377052 CPUtime=12.26 cores=0-7
/proc/27795/task/27806/stat : 27806 (plingeling) R 27793 27795 27696 0 -1 4202560 9882 1256 0 0 1225 1 0 0 25 0 9 0 733854391 386101248 42376 33554432000 4194304 4417452 140735876598192 18446744073709551615 4253043 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=27795/tid=27807] ppid=27793 vsize=377052 CPUtime=12.26 cores=0-7
/proc/27795/task/27807/stat : 27807 (plingeling) R 27793 27795 27696 0 -1 4202560 20676 1256 0 0 1224 2 0 0 25 0 9 0 733854391 386101248 42376 33554432000 4194304 4417452 140735876598192 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27795/tid=27808] ppid=27793 vsize=377052 CPUtime=12.26 cores=0-7
/proc/27795/task/27808/stat : 27808 (plingeling) R 27793 27795 27696 0 -1 4202560 5904 1256 0 0 1226 0 0 0 25 0 9 0 733854391 386101248 42376 33554432000 4194304 4417452 140735876598192 18446744073709551615 4358214 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27795/tid=27809] ppid=27793 vsize=377052 CPUtime=12.26 cores=0-7
/proc/27795/task/27809/stat : 27809 (plingeling) R 27793 27795 27696 0 -1 4202560 5663 1256 0 0 1226 0 0 0 25 0 9 0 733854391 386101248 42376 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364572 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 98.17
Current children cumulated vsize (KiB) 377052

[startup+25.5011 s]
/proc/loadavg: 7.67 7.74 7.07 9/318 27809
/proc/meminfo: memFree=15140136/32950928 swapFree=67085488/67111528
[pid=27795] ppid=27793 vsize=418640 CPUtime=200.45 cores=0-7
/proc/27795/stat : 27795 (plingeling) S 27793 27795 27696 0 -1 4202496 122682 1256 0 0 20027 18 0 0 18 0 9 0 733854348 428687360 52659 33554432000 4194304 4417452 140735876598192 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/27795/statm: 104660 52659 180 55 0 102546 0
[pid=27795/tid=27802] ppid=27793 vsize=418640 CPUtime=25.06 cores=0-7
/proc/27795/task/27802/stat : 27802 (plingeling) R 27793 27795 27696 0 -1 4202560 20010 1256 0 0 2504 2 0 0 25 0 9 0 733854391 428687360 52659 33554432000 4194304 4417452 140735876598192 18446744073709551615 4366099 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=27795/tid=27803] ppid=27793 vsize=418640 CPUtime=25.05 cores=0-7
/proc/27795/task/27803/stat : 27803 (plingeling) R 27793 27795 27696 0 -1 4202560 18251 1256 0 0 2503 2 0 0 25 0 9 0 733854391 428687360 52659 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27795/tid=27804] ppid=27793 vsize=418640 CPUtime=24.52 cores=0-7
/proc/27795/task/27804/stat : 27804 (plingeling) R 27793 27795 27696 0 -1 4202560 2524 1256 0 0 2452 0 0 0 25 0 9 0 733854391 428687360 52659 33554432000 4194304 4417452 140735876598192 18446744073709551615 4258267 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=27795/tid=27805] ppid=27793 vsize=418640 CPUtime=25.07 cores=0-7
/proc/27795/task/27805/stat : 27805 (plingeling) R 27793 27795 27696 0 -1 4202560 7992 1256 0 0 2506 1 0 0 25 0 9 0 733854391 428687360 52659 33554432000 4194304 4417452 140735876598192 18446744073709551615 4237317 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27795/tid=27806] ppid=27793 vsize=418640 CPUtime=25.06 cores=0-7
/proc/27795/task/27806/stat : 27806 (plingeling) R 27793 27795 27696 0 -1 4202560 14889 1256 0 0 2504 2 0 0 25 0 9 0 733854391 428687360 52659 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364453 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=27795/tid=27807] ppid=27793 vsize=418640 CPUtime=25.07 cores=0-7
/proc/27795/task/27807/stat : 27807 (plingeling) R 27793 27795 27696 0 -1 4202560 27374 1256 0 0 2504 3 0 0 25 0 9 0 733854391 428687360 52659 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364548 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27795/tid=27808] ppid=27793 vsize=418640 CPUtime=25.06 cores=0-7
/proc/27795/task/27808/stat : 27808 (plingeling) R 27793 27795 27696 0 -1 4202560 6919 1256 0 0 2506 0 0 0 25 0 9 0 733854391 428687360 52659 33554432000 4194304 4417452 140735876598192 18446744073709551615 4275748 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27795/tid=27809] ppid=27793 vsize=418640 CPUtime=25.06 cores=0-7
/proc/27795/task/27809/stat : 27809 (plingeling) R 27793 27795 27696 0 -1 4202560 5964 1256 0 0 2506 0 0 0 25 0 9 0 733854391 428687360 52659 33554432000 4194304 4417452 140735876598192 18446744073709551615 4366486 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 200.45
Current children cumulated vsize (KiB) 418640

[startup+51.1013 s]
/proc/loadavg: 7.78 7.76 7.09 9/318 27810
/proc/meminfo: memFree=15083436/32950928 swapFree=67085488/67111528
[pid=27795] ppid=27793 vsize=477740 CPUtime=404.98 cores=0-7
/proc/27795/stat : 27795 (plingeling) S 27793 27795 27696 0 -1 4202496 189133 1256 0 0 40471 27 0 0 18 0 9 0 733854348 489205760 68874 33554432000 4194304 4417452 140735876598192 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/27795/statm: 119435 68874 180 55 0 117321 0
[pid=27795/tid=27802] ppid=27793 vsize=477740 CPUtime=50.67 cores=0-7
/proc/27795/task/27802/stat : 27802 (plingeling) R 27793 27795 27696 0 -1 4202560 33102 1256 0 0 5063 4 0 0 25 0 9 0 733854391 489205760 68874 33554432000 4194304 4417452 140735876598192 18446744073709551615 4250488 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=27795/tid=27803] ppid=27793 vsize=477740 CPUtime=50.64 cores=0-7
/proc/27795/task/27803/stat : 27803 (plingeling) R 27793 27795 27696 0 -1 4202560 28928 1256 0 0 5061 3 0 0 25 0 9 0 733854391 489205760 68874 33554432000 4194304 4417452 140735876598192 18446744073709551615 4250945 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27795/tid=27804] ppid=27793 vsize=477740 CPUtime=49.86 cores=0-7

################
# More data... #
################

/proc/27795/task/27805/stat : 27805 (plingeling) R 27793 27795 27696 0 -1 4202560 1088064 1256 0 0 466022 180 0 0 25 0 9 0 733854391 1822707712 394829 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27795/tid=27806] ppid=27793 vsize=1779988 CPUtime=4626.31 cores=0-7
/proc/27795/task/27806/stat : 27806 (plingeling) R 27793 27795 27696 0 -1 4202560 1196630 1256 0 0 462414 217 0 0 25 0 9 0 733854391 1822707712 394829 33554432000 4194304 4417452 140735876598192 18446744073709551615 4366079 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=27795/tid=27807] ppid=27793 vsize=1779988 CPUtime=4660.61 cores=0-7
/proc/27795/task/27807/stat : 27807 (plingeling) R 27793 27795 27696 0 -1 4202560 2887008 1256 0 0 465600 461 0 0 25 0 9 0 733854391 1822707712 394829 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27795/tid=27808] ppid=27793 vsize=1779988 CPUtime=4661.94 cores=0-7
/proc/27795/task/27808/stat : 27808 (plingeling) R 27793 27795 27696 0 -1 4202560 1407350 1256 0 0 465941 253 0 0 25 0 9 0 733854391 1822707712 394829 33554432000 4194304 4417452 140735876598192 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27795/tid=27809] ppid=27793 vsize=1779988 CPUtime=4661.96 cores=0-7
/proc/27795/task/27809/stat : 27809 (plingeling) R 27793 27795 27696 0 -1 4202560 1034090 1256 0 0 466021 175 0 0 25 0 9 0 733854391 1822707712 394829 33554432000 4194304 4417452 140735876598192 18446744073709551615 4236874 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 37247.7
Current children cumulated vsize (KiB) 1779988

[startup+4722.31 s]
/proc/loadavg: 8.01 8.03 8.00 9/318 27946
/proc/meminfo: memFree=13624960/32950928 swapFree=67085488/67111528
[pid=27795] ppid=27793 vsize=1924840 CPUtime=37727.2 cores=0-7
/proc/27795/stat : 27795 (plingeling) S 27793 27795 27696 0 -1 4202496 11497611 1256 0 0 3770745 1974 0 0 18 0 9 0 733854348 1971036160 431042 33554432000 4194304 4417452 140735876598192 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/27795/statm: 481210 431042 180 55 0 479096 0
[pid=27795/tid=27802] ppid=27793 vsize=1924840 CPUtime=4721.98 cores=0-7
/proc/27795/task/27802/stat : 27802 (plingeling) R 27793 27795 27696 0 -1 4202560 1356109 1256 0 0 471969 229 0 0 25 0 9 0 733854391 1971036160 431042 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=27795/tid=27803] ppid=27793 vsize=1924840 CPUtime=4721.69 cores=0-7
/proc/27795/task/27803/stat : 27803 (plingeling) R 27793 27795 27696 0 -1 4202560 1546532 1256 0 0 471891 278 0 0 25 0 9 0 733854391 1971036160 431042 33554432000 4194304 4417452 140735876598192 18446744073709551615 4365188 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27795/tid=27804] ppid=27793 vsize=1924840 CPUtime=4710.75 cores=0-7
/proc/27795/task/27804/stat : 27804 (plingeling) R 27793 27795 27696 0 -1 4202560 865012 1256 0 0 470922 153 0 0 25 0 9 0 733854391 1971036160 431042 33554432000 4194304 4417452 140735876598192 18446744073709551615 4365184 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=27795/tid=27805] ppid=27793 vsize=1924840 CPUtime=4722.04 cores=0-7
/proc/27795/task/27805/stat : 27805 (plingeling) R 27793 27795 27696 0 -1 4202560 1115692 1256 0 0 472016 188 0 0 25 0 9 0 733854391 1971036160 431042 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27795/tid=27806] ppid=27793 vsize=1924840 CPUtime=4685.72 cores=0-7
/proc/27795/task/27806/stat : 27806 (plingeling) R 27793 27795 27696 0 -1 4202560 1207581 1256 0 0 468353 219 0 0 25 0 9 0 733854391 1971036160 431042 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=27795/tid=27807] ppid=27793 vsize=1924840 CPUtime=4720.61 cores=0-7
/proc/27795/task/27807/stat : 27807 (plingeling) R 27793 27795 27696 0 -1 4202560 2928345 1256 0 0 471589 472 0 0 25 0 9 0 733854391 1971036160 431042 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27795/tid=27808] ppid=27793 vsize=1924840 CPUtime=4721.95 cores=0-7
/proc/27795/task/27808/stat : 27808 (plingeling) R 27793 27795 27696 0 -1 4202560 1425491 1256 0 0 471940 255 0 0 25 0 9 0 733854391 1971036160 431042 33554432000 4194304 4417452 140735876598192 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27795/tid=27809] ppid=27793 vsize=1924840 CPUtime=4721.97 cores=0-7
/proc/27795/task/27809/stat : 27809 (plingeling) R 27793 27795 27696 0 -1 4202560 1034090 1256 0 0 472022 175 0 0 25 0 9 0 733854391 1971036160 431042 33554432000 4194304 4417452 140735876598192 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 37727.2
Current children cumulated vsize (KiB) 1924840

[startup+4782.3 s]
/proc/loadavg: 8.00 8.02 8.00 9/318 27948
/proc/meminfo: memFree=13908668/32950928 swapFree=67085488/67111528
[pid=27795] ppid=27793 vsize=1641176 CPUtime=38206.5 cores=0-7
/proc/27795/stat : 27795 (plingeling) S 27793 27795 27696 0 -1 4202496 11627440 1256 0 0 3818653 1999 0 0 18 0 9 0 733854348 1680564224 360126 33554432000 4194304 4417452 140735876598192 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/27795/statm: 410294 360126 180 55 0 408180 0
[pid=27795/tid=27802] ppid=27793 vsize=1641176 CPUtime=4781.97 cores=0-7
/proc/27795/task/27802/stat : 27802 (plingeling) R 27793 27795 27696 0 -1 4202560 1365853 1256 0 0 477967 230 0 0 25 0 9 0 733854391 1680564224 360126 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=27795/tid=27803] ppid=27793 vsize=1641176 CPUtime=4781.68 cores=0-7
/proc/27795/task/27803/stat : 27803 (plingeling) R 27793 27795 27696 0 -1 4202560 1585728 1256 0 0 477882 286 0 0 25 0 9 0 733854391 1680564224 360126 33554432000 4194304 4417452 140735876598192 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27795/tid=27804] ppid=27793 vsize=1641176 CPUtime=4770.74 cores=0-7
/proc/27795/task/27804/stat : 27804 (plingeling) R 27793 27795 27696 0 -1 4202560 881304 1256 0 0 476919 155 0 0 25 0 9 0 733854391 1680564224 360126 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=27795/tid=27805] ppid=27793 vsize=1641176 CPUtime=4782.03 cores=0-7
/proc/27795/task/27805/stat : 27805 (plingeling) R 27793 27795 27696 0 -1 4202560 1133299 1256 0 0 478011 192 0 0 25 0 9 0 733854391 1680564224 360126 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27795/tid=27806] ppid=27793 vsize=1641176 CPUtime=4745.12 cores=0-7
/proc/27795/task/27806/stat : 27806 (plingeling) R 27793 27795 27696 0 -1 4202560 1207581 1256 0 0 474293 219 0 0 25 0 9 0 733854391 1680564224 360126 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=27795/tid=27807] ppid=27793 vsize=1641176 CPUtime=4780.59 cores=0-7
/proc/27795/task/27807/stat : 27807 (plingeling) R 27793 27795 27696 0 -1 4202560 2946797 1256 0 0 477584 475 0 0 25 0 9 0 733854391 1680564224 360126 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27795/tid=27808] ppid=27793 vsize=1641176 CPUtime=4781.94 cores=0-7
/proc/27795/task/27808/stat : 27808 (plingeling) R 27793 27795 27696 0 -1 4202560 1445829 1256 0 0 477936 258 0 0 25 0 9 0 733854391 1680564224 360126 33554432000 4194304 4417452 140735876598192 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27795/tid=27809] ppid=27793 vsize=1641176 CPUtime=4781.96 cores=0-7
/proc/27795/task/27809/stat : 27809 (plingeling) R 27793 27795 27696 0 -1 4202560 1042290 1256 0 0 478019 177 0 0 25 0 9 0 733854391 1680564224 360126 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 38206.5
Current children cumulated vsize (KiB) 1641176

[startup+4842.3 s]
/proc/loadavg: 8.09 8.05 8.01 9/318 27950
/proc/meminfo: memFree=13824200/32950928 swapFree=67085488/67111528
[pid=27795] ppid=27793 vsize=1724900 CPUtime=38685.9 cores=0-7
/proc/27795/stat : 27795 (plingeling) S 27793 27795 27696 0 -1 4202496 11775693 1256 0 0 3866566 2026 0 0 18 0 9 0 733854348 1766297600 381057 33554432000 4194304 4417452 140735876598192 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/27795/statm: 431225 381057 180 55 0 429111 0
[pid=27795/tid=27802] ppid=27793 vsize=1724900 CPUtime=4841.97 cores=0-7
/proc/27795/task/27802/stat : 27802 (plingeling) R 27793 27795 27696 0 -1 4202560 1367349 1256 0 0 483967 230 0 0 25 0 9 0 733854391 1766297600 381057 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=27795/tid=27803] ppid=27793 vsize=1724900 CPUtime=4841.69 cores=0-7
/proc/27795/task/27803/stat : 27803 (plingeling) R 27793 27795 27696 0 -1 4202560 1647937 1256 0 0 483873 296 0 0 25 0 9 0 733854391 1766297600 381057 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364737 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27795/tid=27804] ppid=27793 vsize=1724900 CPUtime=4830.75 cores=0-7
/proc/27795/task/27804/stat : 27804 (plingeling) R 27793 27795 27696 0 -1 4202560 900528 1256 0 0 482917 158 0 0 25 0 9 0 733854391 1766297600 381057 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=27795/tid=27805] ppid=27793 vsize=1724900 CPUtime=4842.02 cores=0-7
/proc/27795/task/27805/stat : 27805 (plingeling) R 27793 27795 27696 0 -1 4202560 1154035 1256 0 0 484006 196 0 0 25 0 9 0 733854391 1766297600 381057 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27795/tid=27806] ppid=27793 vsize=1724900 CPUtime=4804.52 cores=0-7
/proc/27795/task/27806/stat : 27806 (plingeling) R 27793 27795 27696 0 -1 4202560 1207581 1256 0 0 480233 219 0 0 25 0 9 0 733854391 1766297600 381057 33554432000 4194304 4417452 140735876598192 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=27795/tid=27807] ppid=27793 vsize=1724900 CPUtime=4840.58 cores=0-7
/proc/27795/task/27807/stat : 27807 (plingeling) R 27793 27795 27696 0 -1 4202560 2965259 1256 0 0 483581 477 0 0 25 0 9 0 733854391 1766297600 381057 33554432000 4194304 4417452 140735876598192 18446744073709551615 4346677 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27795/tid=27808] ppid=27793 vsize=1724900 CPUtime=4841.95 cores=0-7
/proc/27795/task/27808/stat : 27808 (plingeling) R 27793 27795 27696 0 -1 4202560 1467967 1256 0 0 483930 265 0 0 25 0 9 0 733854391 1766297600 381057 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27795/tid=27809] ppid=27793 vsize=1724900 CPUtime=4841.96 cores=0-7
/proc/27795/task/27809/stat : 27809 (plingeling) R 27793 27795 27696 0 -1 4202560 1046278 1256 0 0 484018 178 0 0 25 0 9 0 733854391 1766297600 381057 33554432000 4194304 4417452 140735876598192 18446744073709551615 4365106 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 38685.9
Current children cumulated vsize (KiB) 1724900

[startup+4902.3 s]
/proc/loadavg: 8.14 8.07 8.01 9/318 27951
/proc/meminfo: memFree=13847480/32950928 swapFree=67085488/67111528
[pid=27795] ppid=27793 vsize=1702968 CPUtime=39165.3 cores=0-7
/proc/27795/stat : 27795 (plingeling) S 27793 27795 27696 0 -1 4202496 11937222 1256 0 0 3914482 2051 0 0 18 0 9 0 733854348 1743839232 375574 33554432000 4194304 4417452 140735876598192 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/27795/statm: 425742 375574 180 55 0 423628 0
[pid=27795/tid=27802] ppid=27793 vsize=1702968 CPUtime=4901.98 cores=0-7
/proc/27795/task/27802/stat : 27802 (plingeling) R 27793 27795 27696 0 -1 4202560 1377821 1256 0 0 489966 232 0 0 25 0 9 0 733854391 1743839232 375574 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=27795/tid=27803] ppid=27793 vsize=1702968 CPUtime=4901.68 cores=0-7
/proc/27795/task/27803/stat : 27803 (plingeling) R 27793 27795 27696 0 -1 4202560 1681178 1256 0 0 489868 300 0 0 25 0 9 0 733854391 1743839232 375574 33554432000 4194304 4417452 140735876598192 18446744073709551615 4357283 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27795/tid=27804] ppid=27793 vsize=1702968 CPUtime=4890.75 cores=0-7
/proc/27795/task/27804/stat : 27804 (plingeling) R 27793 27795 27696 0 -1 4202560 935389 1256 0 0 488909 166 0 0 25 0 9 0 733854391 1743839232 375574 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=27795/tid=27805] ppid=27793 vsize=1702968 CPUtime=4902.03 cores=0-7
/proc/27795/task/27805/stat : 27805 (plingeling) R 27793 27795 27696 0 -1 4202560 1163009 1256 0 0 490005 198 0 0 25 0 9 0 733854391 1743839232 375574 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27795/tid=27806] ppid=27793 vsize=1702968 CPUtime=4863.92 cores=0-7
/proc/27795/task/27806/stat : 27806 (plingeling) R 27793 27795 27696 0 -1 4202560 1235566 1256 0 0 486170 222 0 0 25 0 9 0 733854391 1743839232 375574 33554432000 4194304 4417452 140735876598192 18446744073709551615 4357671 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=27795/tid=27807] ppid=27793 vsize=1702968 CPUtime=4900.58 cores=0-7
/proc/27795/task/27807/stat : 27807 (plingeling) R 27793 27795 27696 0 -1 4202560 3001511 1256 0 0 489575 483 0 0 25 0 9 0 733854391 1743839232 375574 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27795/tid=27808] ppid=27793 vsize=1702968 CPUtime=4901.94 cores=0-7
/proc/27795/task/27808/stat : 27808 (plingeling) R 27793 27795 27696 0 -1 4202560 1473723 1256 0 0 489929 265 0 0 25 0 9 0 733854391 1743839232 375574 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27795/tid=27809] ppid=27793 vsize=1702968 CPUtime=4901.96 cores=0-7
/proc/27795/task/27809/stat : 27809 (plingeling) R 27793 27795 27696 0 -1 4202560 1050266 1256 0 0 490017 179 0 0 25 0 9 0 733854391 1743839232 375574 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 39165.3
Current children cumulated vsize (KiB) 1702968

[startup+4962.3 s]
/proc/loadavg: 8.14 8.08 8.02 9/318 27953
/proc/meminfo: memFree=13653048/32950928 swapFree=67085488/67111528
[pid=27795] ppid=27793 vsize=1896592 CPUtime=39644.7 cores=0-7
/proc/27795/stat : 27795 (plingeling) S 27793 27795 27696 0 -1 4202496 12189389 1256 0 0 3962373 2098 0 0 18 0 9 0 733854348 1942110208 423980 33554432000 4194304 4417452 140735876598192 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/27795/statm: 474148 423980 180 55 0 472034 0
[pid=27795/tid=27802] ppid=27793 vsize=1896592 CPUtime=4961.97 cores=0-7
/proc/27795/task/27802/stat : 27802 (plingeling) R 27793 27795 27696 0 -1 4202560 1386064 1256 0 0 495964 233 0 0 25 0 9 0 733854391 1942110208 423980 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=27795/tid=27803] ppid=27793 vsize=1896592 CPUtime=4961.68 cores=0-7
/proc/27795/task/27803/stat : 27803 (plingeling) R 27793 27795 27696 0 -1 4202560 1720531 1256 0 0 495860 308 0 0 25 0 9 0 733854391 1942110208 423980 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27795/tid=27804] ppid=27793 vsize=1896592 CPUtime=4950.75 cores=0-7
/proc/27795/task/27804/stat : 27804 (plingeling) R 27793 27795 27696 0 -1 4202560 974571 1256 0 0 494902 173 0 0 25 0 9 0 733854391 1942110208 423980 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=27795/tid=27805] ppid=27793 vsize=1896592 CPUtime=4962.03 cores=0-7
/proc/27795/task/27805/stat : 27805 (plingeling) R 27793 27795 27696 0 -1 4202560 1171985 1256 0 0 496002 201 0 0 25 0 9 0 733854391 1942110208 423980 33554432000 4194304 4417452 140735876598192 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27795/tid=27806] ppid=27793 vsize=1896592 CPUtime=4923.32 cores=0-7
/proc/27795/task/27806/stat : 27806 (plingeling) R 27793 27795 27696 0 -1 4202560 1321815 1256 0 0 492096 236 0 0 25 0 9 0 733854391 1942110208 423980 33554432000 4194304 4417452 140735876598192 18446744073709551615 4310288 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=27795/tid=27807] ppid=27793 vsize=1896592 CPUtime=4960.57 cores=0-7
/proc/27795/task/27807/stat : 27807 (plingeling) R 27793 27795 27696 0 -1 4202560 3035664 1256 0 0 495566 491 0 0 25 0 9 0 733854391 1942110208 423980 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364657 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27795/tid=27808] ppid=27793 vsize=1896592 CPUtime=4961.94 cores=0-7
/proc/27795/task/27808/stat : 27808 (plingeling) R 27793 27795 27696 0 -1 4202560 1501758 1256 0 0 495925 269 0 0 25 0 9 0 733854391 1942110208 423980 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27795/tid=27809] ppid=27793 vsize=1896592 CPUtime=4961.96 cores=0-7
/proc/27795/task/27809/stat : 27809 (plingeling) R 27793 27795 27696 0 -1 4202560 1058242 1256 0 0 496015 181 0 0 25 0 9 0 733854391 1942110208 423980 33554432000 4194304 4417452 140735876598192 18446744073709551615 4357097 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 39644.7
Current children cumulated vsize (KiB) 1896592



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5006.8 s]
/proc/loadavg: 8.07 8.07 8.01 9/318 27954
/proc/meminfo: memFree=13755696/32950928 swapFree=67085488/67111528
[pid=27795] ppid=27793 vsize=1794036 CPUtime=40000.3 cores=0-7
/proc/27795/stat : 27795 (plingeling) S 27793 27795 27696 0 -1 4202496 12286466 1256 0 0 3997915 2117 0 0 18 0 9 0 733854348 1837092864 398341 33554432000 4194304 4417452 140735876598192 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/27795/statm: 448509 398341 180 55 0 446395 0
[pid=27795/tid=27802] ppid=27793 vsize=1794036 CPUtime=5006.48 cores=0-7
/proc/27795/task/27802/stat : 27802 (plingeling) R 27793 27795 27696 0 -1 4202560 1389230 1256 0 0 500415 233 0 0 25 0 9 0 733854391 1837092864 398341 33554432000 4194304 4417452 140735876598192 18446744073709551615 4365058 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=27795/tid=27803] ppid=27793 vsize=1794036 CPUtime=5006.2 cores=0-7
/proc/27795/task/27803/stat : 27803 (plingeling) R 27793 27795 27696 0 -1 4202560 1744273 1256 0 0 500310 310 0 0 25 0 9 0 733854391 1837092864 398341 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27795/tid=27804] ppid=27793 vsize=1794036 CPUtime=4995.26 cores=0-7
/proc/27795/task/27804/stat : 27804 (plingeling) R 27793 27795 27696 0 -1 4202560 994389 1256 0 0 499349 177 0 0 25 0 9 0 733854391 1837092864 398341 33554432000 4194304 4417452 140735876598192 18446744073709551615 4365181 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=27795/tid=27805] ppid=27793 vsize=1794036 CPUtime=5006.54 cores=0-7
/proc/27795/task/27805/stat : 27805 (plingeling) R 27793 27795 27696 0 -1 4202560 1191400 1256 0 0 500446 208 0 0 25 0 9 0 733854391 1837092864 398341 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27795/tid=27806] ppid=27793 vsize=1794036 CPUtime=4967.38 cores=0-7
/proc/27795/task/27806/stat : 27806 (plingeling) R 27793 27795 27696 0 -1 4202560 1339343 1256 0 0 496499 239 0 0 25 0 9 0 733854391 1837092864 398341 33554432000 4194304 4417452 140735876598192 18446744073709551615 4251063 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=27795/tid=27807] ppid=27793 vsize=1794036 CPUtime=5005.07 cores=0-7
/proc/27795/task/27807/stat : 27807 (plingeling) R 27793 27795 27696 0 -1 4202560 3035664 1256 0 0 500016 491 0 0 25 0 9 0 733854391 1837092864 398341 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27795/tid=27808] ppid=27793 vsize=1794036 CPUtime=5006.44 cores=0-7
/proc/27795/task/27808/stat : 27808 (plingeling) R 27793 27795 27696 0 -1 4202560 1513410 1256 0 0 500373 271 0 0 25 0 9 0 733854391 1837092864 398341 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27795/tid=27809] ppid=27793 vsize=1794036 CPUtime=5006.47 cores=0-7
/proc/27795/task/27809/stat : 27809 (plingeling) R 27793 27795 27696 0 -1 4202560 1059998 1256 0 0 500466 181 0 0 25 0 9 0 733854391 1837092864 398341 33554432000 4194304 4417452 140735876598192 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 40000.3
Current children cumulated vsize (KiB) 1794036

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5006.9
CPU time (s): 40000.5
CPU user time (s): 39979.2
CPU system time (s): 21.2538
CPU usage (%): 798.907
Max. virtual memory (cumulated for all children) (KiB): 2163928

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39979.2
system time used= 21.2538
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 12287722
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 573
involuntary context switches= 191986

runsolver used 14.1438 second user time and 35.9335 second system time

The end

Launcher Data 

Begin job on node114 at 2011-05-15 07:10:37
IDJOB=3372167
IDBENCH=20536
IDSOLVER=1851
FILE ID=node114/3372167-1305436237
RUNJOBID= node114-1305435362-27710
PBS_JOBID= 13324733
Free space on /tmp= 72996 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT07/industrial/IBM_FV_2002_13_rule_1/SAT_dat.k100.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3372167-1305436237/watcher-3372167-1305436237 -o /tmp/evaluation-result-3372167-1305436237/solver-3372167-1305436237 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3372167-1305436237.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= cf1b8194ff6d413d396ac9e26bb031ef
RANDOM SEED=1031711988

node114.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.73
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      15350984 kB
Buffers:        288672 kB
Cached:        6551176 kB
SwapCached:         96 kB
Active:       11987784 kB
Inactive:      5204452 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      15350984 kB
SwapTotal:    67111528 kB
SwapFree:     67085488 kB
Dirty:            4816 kB
Writeback:           0 kB
AnonPages:    10351792 kB
Mapped:          26272 kB
Slab:           305492 kB
PageTables:      36492 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 52431620 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72996 MiB
End job on node114 at 2011-05-15 08:34:06