Trace number 3309675

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 40000.5 5005.27

General information on the benchmark

Nameapplication/jarvisalo/smtqfbv-aigs/
smtlib-qfbv-aigs-VS3-benchmark-S2-tseitin.cnf
MD5SUMb967fca11e85ef5b614d3452b1c4b2ad
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables257030
Number of clauses769313
Sum of the clauses size1795061
Maximum clause length3
Minimum clause length1
Number of clauses of size 12
Number of clauses of size 2512874
Number of clauses of size 3256437
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3309675-1304007742/watcher-3309675-1304007742 -o /tmp/evaluation-result-3309675-1304007742/solver-3309675-1304007742 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3309675-1304007742.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.79 7.67 7.85 1/170 26976
/proc/meminfo: memFree=23257872/32951124 swapFree=67111348/67111528
[pid=26976] ppid=26974 vsize=12156 CPUtime=0 cores=0-7
/proc/26976/stat : 26976 (plingeling) R 26974 26976 26909 0 -1 4202496 1129 1261 0 0 0 0 0 0 21 0 1 0 591004705 12447744 1058 33554432000 4194304 4417332 140736761756672 18446744073709551615 252457759700 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/26976/statm: 3039 1058 146 55 0 925 0

[startup+0.034108 s]
/proc/loadavg: 6.79 7.67 7.85 1/170 26976
/proc/meminfo: memFree=23257872/32951124 swapFree=67111348/67111528
[pid=26976] ppid=26974 vsize=18836 CPUtime=0.02 cores=0-7
/proc/26976/stat : 26976 (plingeling) R 26974 26976 26909 0 -1 4202496 2803 1261 0 0 2 0 0 0 21 0 1 0 591004705 19288064 2732 33554432000 4194304 4417332 140736761756672 18446744073709551615 4264940 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/26976/statm: 4709 2732 148 55 0 2595 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 18836

[startup+0.101097 s]
/proc/loadavg: 6.79 7.67 7.85 1/170 26976
/proc/meminfo: memFree=23257872/32951124 swapFree=67111348/67111528
[pid=26976] ppid=26974 vsize=29708 CPUtime=0.08 cores=0-7
/proc/26976/stat : 26976 (plingeling) R 26974 26976 26909 0 -1 4202496 5513 1261 0 0 8 0 0 0 21 0 1 0 591004705 30420992 5418 33554432000 4194304 4417332 140736761756672 18446744073709551615 4260166 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/26976/statm: 7427 5418 148 55 0 5313 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 29708

[startup+0.301078 s]
/proc/loadavg: 6.79 7.67 7.85 1/170 26976
/proc/meminfo: memFree=23257872/32951124 swapFree=67111348/67111528
[pid=26976] ppid=26974 vsize=82740 CPUtime=0.29 cores=0-7
/proc/26976/stat : 26976 (plingeling) R 26974 26976 26909 0 -1 4202496 18808 1261 0 0 27 2 0 0 22 0 1 0 591004705 84725760 18708 33554432000 4194304 4417332 140736761756672 18446744073709551615 4209569 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/26976/statm: 20685 18708 148 55 0 18571 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 82740

[startup+0.701003 s]
/proc/loadavg: 6.79 7.67 7.85 1/170 26976
/proc/meminfo: memFree=23257872/32951124 swapFree=67111348/67111528
[pid=26976] ppid=26974 vsize=153412 CPUtime=0.69 cores=0-7
/proc/26976/stat : 26976 (plingeling) R 26974 26976 26909 0 -1 4202496 36476 1261 0 0 65 4 0 0 23 0 1 0 591004705 157093888 36376 33554432000 4194304 4417332 140736761756672 18446744073709551615 4274358 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/26976/statm: 38353 36376 148 55 0 36239 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 153412

[startup+1.50084 s]
/proc/loadavg: 6.79 7.67 7.85 2/171 26982
/proc/meminfo: memFree=23038640/32951124 swapFree=67111348/67111528
[pid=26976] ppid=26974 vsize=293344 CPUtime=1.49 cores=0-7
/proc/26976/stat : 26976 (plingeling) R 26974 26976 26909 0 -1 4202496 71487 1261 0 0 140 9 0 0 25 0 1 0 591004705 300384256 71359 33554432000 4194304 4417332 140736761756672 18446744073709551615 4259651 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/26976/statm: 73336 71359 148 55 0 71222 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 293344

[startup+3.10051 s]
/proc/loadavg: 6.97 7.69 7.86 9/179 26990
/proc/meminfo: memFree=22936896/32951124 swapFree=67111348/67111528
[pid=26976] ppid=26974 vsize=532416 CPUtime=11.53 cores=0-7
/proc/26976/stat : 26976 (plingeling) S 26974 26976 26909 0 -1 4202496 156209 1261 0 0 1128 25 0 0 18 0 9 0 591004705 545193984 100317 33554432000 4194304 4417332 140736761756672 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/26976/statm: 133104 100317 171 55 0 130990 0
[pid=26976/tid=26983] ppid=26974 vsize=532416 CPUtime=1.2 cores=0-7
/proc/26976/task/26983/stat : 26983 (plingeling) R 26974 26976 26909 0 -1 4202560 12500 1261 0 0 118 2 0 0 19 0 9 0 591004893 545193984 100317 33554432000 4194304 4417332 140736761756672 18446744073709551615 4286241 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26976/tid=26984] ppid=26974 vsize=532416 CPUtime=1.2 cores=0-7
/proc/26976/task/26984/stat : 26984 (plingeling) R 26974 26976 26909 0 -1 4202560 12501 1261 0 0 119 1 0 0 19 0 9 0 591004893 545193984 100317 33554432000 4194304 4417332 140736761756672 18446744073709551615 4284595 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26976/tid=26985] ppid=26974 vsize=532416 CPUtime=1.21 cores=0-7
/proc/26976/task/26985/stat : 26985 (plingeling) R 26974 26976 26909 0 -1 4202560 10491 1261 0 0 119 2 0 0 25 0 9 0 591004893 545193984 100317 33554432000 4194304 4417332 140736761756672 18446744073709551615 4281668 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26976/tid=26986] ppid=26974 vsize=532416 CPUtime=1.2 cores=0-7
/proc/26976/task/26986/stat : 26986 (plingeling) R 26974 26976 26909 0 -1 4202560 4483 1261 0 0 120 0 0 0 19 0 9 0 591004893 545193984 100317 33554432000 4194304 4417332 140736761756672 18446744073709551615 4353461 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=26976/tid=26987] ppid=26974 vsize=532416 CPUtime=1.2 cores=0-7
/proc/26976/task/26987/stat : 26987 (plingeling) R 26974 26976 26909 0 -1 4202560 10447 1261 0 0 119 1 0 0 25 0 9 0 591004893 545193984 100317 33554432000 4194304 4417332 140736761756672 18446744073709551615 4284341 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26976/tid=26988] ppid=26974 vsize=532416 CPUtime=1.2 cores=0-7
/proc/26976/task/26988/stat : 26988 (plingeling) R 26974 26976 26909 0 -1 4202560 9916 1261 0 0 119 1 0 0 22 0 9 0 591004893 545193984 100317 33554432000 4194304 4417332 140736761756672 18446744073709551615 4284341 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=26976/tid=26989] ppid=26974 vsize=532416 CPUtime=1.19 cores=0-7
/proc/26976/task/26989/stat : 26989 (plingeling) R 26974 26976 26909 0 -1 4202560 13924 1261 0 0 116 3 0 0 22 0 9 0 591004893 545193984 100317 33554432000 4194304 4417332 140736761756672 18446744073709551615 4284341 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26976/tid=26990] ppid=26974 vsize=532416 CPUtime=1.18 cores=0-7
/proc/26976/task/26990/stat : 26990 (plingeling) R 26974 26976 26909 0 -1 4202560 10444 1261 0 0 117 1 0 0 22 0 9 0 591004893 545193984 100317 33554432000 4194304 4417332 140736761756672 18446744073709551615 4284514 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 11.53
Current children cumulated vsize (KiB) 532416

[startup+6.30091 s]
/proc/loadavg: 6.97 7.69 7.86 9/179 26990
/proc/meminfo: memFree=22744340/32951124 swapFree=67111348/67111528
[pid=26976] ppid=26974 vsize=587272 CPUtime=37.09 cores=0-7
/proc/26976/stat : 26976 (plingeling) S 26974 26976 26909 0 -1 4202496 279110 1261 0 0 3663 46 0 0 18 0 9 0 591004705 601366528 113005 33554432000 4194304 4417332 140736761756672 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/26976/statm: 146818 113005 178 55 0 144704 0
[pid=26976/tid=26983] ppid=26974 vsize=587272 CPUtime=4.37 cores=0-7
/proc/26976/task/26983/stat : 26983 (plingeling) R 26974 26976 26909 0 -1 4202560 29281 1261 0 0 433 4 0 0 25 0 9 0 591004893 601366528 113005 33554432000 4194304 4417332 140736761756672 18446744073709551615 4346166 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26976/tid=26984] ppid=26974 vsize=587272 CPUtime=4.4 cores=0-7
/proc/26976/task/26984/stat : 26984 (plingeling) R 26974 26976 26909 0 -1 4202560 21270 1261 0 0 436 4 0 0 25 0 9 0 591004893 601366528 113005 33554432000 4194304 4417332 140736761756672 18446744073709551615 4320443 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26976/tid=26985] ppid=26974 vsize=587272 CPUtime=4.39 cores=0-7
/proc/26976/task/26985/stat : 26985 (plingeling) R 26974 26976 26909 0 -1 4202560 28460 1261 0 0 435 4 0 0 25 0 9 0 591004893 601366528 113005 33554432000 4194304 4417332 140736761756672 18446744073709551615 4233447 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26976/tid=26986] ppid=26974 vsize=587272 CPUtime=4.4 cores=0-7
/proc/26976/task/26986/stat : 26986 (plingeling) R 26974 26976 26909 0 -1 4202560 4485 1261 0 0 440 0 0 0 25 0 9 0 591004893 601366528 113005 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=26976/tid=26987] ppid=26974 vsize=587272 CPUtime=4.4 cores=0-7
/proc/26976/task/26987/stat : 26987 (plingeling) R 26974 26976 26909 0 -1 4202560 25719 1261 0 0 436 4 0 0 25 0 9 0 591004893 601366528 113005 33554432000 4194304 4417332 140736761756672 18446744073709551615 4346401 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26976/tid=26988] ppid=26974 vsize=587272 CPUtime=4.39 cores=0-7
/proc/26976/task/26988/stat : 26988 (plingeling) R 26974 26976 26909 0 -1 4202560 20075 1261 0 0 436 3 0 0 25 0 9 0 591004893 601366528 113005 33554432000 4194304 4417332 140736761756672 18446744073709551615 4267712 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=26976/tid=26989] ppid=26974 vsize=587272 CPUtime=4.39 cores=0-7
/proc/26976/task/26989/stat : 26989 (plingeling) R 26974 26976 26909 0 -1 4202560 43149 1261 0 0 431 8 0 0 25 0 9 0 591004893 601366528 113005 33554432000 4194304 4417332 140736761756672 18446744073709551615 4253462 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26976/tid=26990] ppid=26974 vsize=587272 CPUtime=4.38 cores=0-7
/proc/26976/task/26990/stat : 26990 (plingeling) R 26974 26976 26909 0 -1 4202560 35168 1261 0 0 434 4 0 0 25 0 9 0 591004893 601366528 113005 33554432000 4194304 4417332 140736761756672 18446744073709551615 4346307 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 37.09
Current children cumulated vsize (KiB) 587272

[startup+12.7007 s]
/proc/loadavg: 7.05 7.70 7.86 9/179 26990
/proc/meminfo: memFree=22718564/32951124 swapFree=67111348/67111528
[pid=26976] ppid=26974 vsize=635072 CPUtime=88.25 cores=0-7
/proc/26976/stat : 26976 (plingeling) S 26974 26976 26909 0 -1 4202496 307559 1261 0 0 8776 49 0 0 18 0 9 0 591004705 650313728 135167 33554432000 4194304 4417332 140736761756672 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/26976/statm: 158768 135167 180 55 0 156654 0
[pid=26976/tid=26983] ppid=26974 vsize=635072 CPUtime=10.74 cores=0-7
/proc/26976/task/26983/stat : 26983 (plingeling) R 26974 26976 26909 0 -1 4202560 35576 1261 0 0 1069 5 0 0 25 0 9 0 591004893 650313728 135167 33554432000 4194304 4417332 140736761756672 18446744073709551615 4253179 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26976/tid=26984] ppid=26974 vsize=635072 CPUtime=10.8 cores=0-7
/proc/26976/task/26984/stat : 26984 (plingeling) R 26974 26976 26909 0 -1 4202560 21275 1261 0 0 1076 4 0 0 25 0 9 0 591004893 650313728 135167 33554432000 4194304 4417332 140736761756672 18446744073709551615 4253320 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26976/tid=26985] ppid=26974 vsize=635072 CPUtime=10.8 cores=0-7
/proc/26976/task/26985/stat : 26985 (plingeling) R 26974 26976 26909 0 -1 4202560 32789 1261 0 0 1075 5 0 0 25 0 9 0 591004893 650313728 135167 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364346 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26976/tid=26986] ppid=26974 vsize=635072 CPUtime=10.8 cores=0-7
/proc/26976/task/26986/stat : 26986 (plingeling) R 26974 26976 26909 0 -1 4202560 4485 1261 0 0 1080 0 0 0 25 0 9 0 591004893 650313728 135167 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364599 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=26976/tid=26987] ppid=26974 vsize=635072 CPUtime=10.8 cores=0-7
/proc/26976/task/26987/stat : 26987 (plingeling) R 26974 26976 26909 0 -1 4202560 31170 1261 0 0 1075 5 0 0 25 0 9 0 591004893 650313728 135167 33554432000 4194304 4417332 140736761756672 18446744073709551615 4253336 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26976/tid=26988] ppid=26974 vsize=635072 CPUtime=10.79 cores=0-7
/proc/26976/task/26988/stat : 26988 (plingeling) R 26974 26976 26909 0 -1 4202560 25961 1261 0 0 1075 4 0 0 25 0 9 0 591004893 650313728 135167 33554432000 4194304 4417332 140736761756672 18446744073709551615 4253304 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=26976/tid=26989] ppid=26974 vsize=635072 CPUtime=10.79 cores=0-7
/proc/26976/task/26989/stat : 26989 (plingeling) R 26974 26976 26909 0 -1 4202560 43663 1261 0 0 1071 8 0 0 25 0 9 0 591004893 650313728 135167 33554432000 4194304 4417332 140736761756672 18446744073709551615 4366025 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26976/tid=26990] ppid=26974 vsize=635072 CPUtime=10.78 cores=0-7
/proc/26976/task/26990/stat : 26990 (plingeling) R 26974 26976 26909 0 -1 4202560 41137 1261 0 0 1073 5 0 0 25 0 9 0 591004893 650313728 135167 33554432000 4194304 4417332 140736761756672 18446744073709551615 4252903 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 88.25
Current children cumulated vsize (KiB) 635072

[startup+25.5013 s]
/proc/loadavg: 7.33 7.73 7.86 9/179 26990
/proc/meminfo: memFree=22710076/32951124 swapFree=67111348/67111528
[pid=26976] ppid=26974 vsize=653896 CPUtime=190.6 cores=0-7
/proc/26976/stat : 26976 (plingeling) S 26974 26976 26909 0 -1 4202496 312876 1261 0 0 19010 50 0 0 18 0 9 0 591004705 669589504 137559 33554432000 4194304 4417332 140736761756672 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/26976/statm: 163474 137559 180 55 0 161360 0
[pid=26976/tid=26983] ppid=26974 vsize=653896 CPUtime=23.48 cores=0-7
/proc/26976/task/26983/stat : 26983 (plingeling) R 26974 26976 26909 0 -1 4202560 35577 1261 0 0 2343 5 0 0 25 0 9 0 591004893 669589504 137559 33554432000 4194304 4417332 140736761756672 18446744073709551615 4249130 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26976/tid=26984] ppid=26974 vsize=653896 CPUtime=23.6 cores=0-7
/proc/26976/task/26984/stat : 26984 (plingeling) R 26974 26976 26909 0 -1 4202560 21276 1261 0 0 2356 4 0 0 25 0 9 0 591004893 669589504 137559 33554432000 4194304 4417332 140736761756672 18446744073709551615 4237189 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26976/tid=26985] ppid=26974 vsize=653896 CPUtime=23.6 cores=0-7
/proc/26976/task/26985/stat : 26985 (plingeling) R 26974 26976 26909 0 -1 4202560 36053 1261 0 0 2355 5 0 0 25 0 9 0 591004893 669589504 137559 33554432000 4194304 4417332 140736761756672 18446744073709551615 4265807 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26976/tid=26986] ppid=26974 vsize=653896 CPUtime=23.61 cores=0-7
/proc/26976/task/26986/stat : 26986 (plingeling) R 26974 26976 26909 0 -1 4202560 4485 1261 0 0 2361 0 0 0 25 0 9 0 591004893 669589504 137559 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=26976/tid=26987] ppid=26974 vsize=653896 CPUtime=23.6 cores=0-7
/proc/26976/task/26987/stat : 26987 (plingeling) R 26974 26976 26909 0 -1 4202560 31171 1261 0 0 2355 5 0 0 25 0 9 0 591004893 669589504 137559 33554432000 4194304 4417332 140736761756672 18446744073709551615 4344048 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26976/tid=26988] ppid=26974 vsize=653896 CPUtime=23.59 cores=0-7
/proc/26976/task/26988/stat : 26988 (plingeling) R 26974 26976 26909 0 -1 4202560 25962 1261 0 0 2355 4 0 0 25 0 9 0 591004893 669589504 137559 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364509 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=26976/tid=26989] ppid=26974 vsize=653896 CPUtime=23.59 cores=0-7
/proc/26976/task/26989/stat : 26989 (plingeling) R 26974 26976 26909 0 -1 4202560 43663 1261 0 0 2351 8 0 0 25 0 9 0 591004893 669589504 137559 33554432000 4194304 4417332 140736761756672 18446744073709551615 4236649 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26976/tid=26990] ppid=26974 vsize=653896 CPUtime=23.58 cores=0-7
/proc/26976/task/26990/stat : 26990 (plingeling) R 26974 26976 26909 0 -1 4202560 43186 1261 0 0 2353 5 0 0 25 0 9 0 591004893 669589504 137559 33554432000 4194304 4417332 140736761756672 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 190.6
Current children cumulated vsize (KiB) 653896

[startup+51.1006 s]
/proc/loadavg: 7.56 7.75 7.87 9/179 26991
/proc/meminfo: memFree=22713048/32951124 swapFree=67111348/67111528
[pid=26976] ppid=26974 vsize=652580 CPUtime=395.25 cores=0-7
/proc/26976/stat : 26976 (plingeling) S 26974 26976 26909 0 -1 4202496 313540 1261 0 0 39475 50 0 0 18 0 9 0 591004705 668241920 136745 33554432000 4194304 4417332 140736761756672 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/26976/statm: 163145 136745 181 55 0 161031 0
[pid=26976/tid=26983] ppid=26974 vsize=652580 CPUtime=48.95 cores=0-7
/proc/26976/task/26983/stat : 26983 (plingeling) R 26974 26976 26909 0 -1 4202560 35670 1261 0 0 4890 5 0 0 25 0 9 0 591004893 668241920 136745 33554432000 4194304 4417332 140736761756672 18446744073709551615 4252133 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26976/tid=26984] ppid=26974 vsize=652580 CPUtime=49.2 cores=0-7
/proc/26976/task/26984/stat : 26984 (plingeling) R 26974 26976 26909 0 -1 4202560 21372 1261 0 0 4916 4 0 0 25 0 9 0 591004893 668241920 136745 33554432000 4194304 4417332 140736761756672 18446744073709551615 4252539 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26976/tid=26985] ppid=26974 vsize=652580 CPUtime=49.2 cores=0-7
/proc/26976/task/26985/stat : 26985 (plingeling) R 26974 26976 26909 0 -1 4202560 36054 1261 0 0 4915 5 0 0 25 0 9 0 591004893 668241920 136745 33554432000 4194304 4417332 140736761756672 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26976/tid=26986] ppid=26974 vsize=652580 CPUtime=49.21 cores=0-7
/proc/26976/task/26986/stat : 26986 (plingeling) R 26974 26976 26909 0 -1 4202560 4485 1261 0 0 4921 0 0 0 25 0 9 0 591004893 668241920 136745 33554432000 4194304 4417332 140736761756672 18446744073709551615 4236689 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=26976/tid=26987] ppid=26974 vsize=652580 CPUtime=49.2 cores=0-7
/proc/26976/task/26987/stat : 26987 (plingeling) R 26974 26976 26909 0 -1 4202560 31264 1261 0 0 4915 5 0 0 25 0 9 0 591004893 668241920 136745 33554432000 4194304 4417332 140736761756672 18446744073709551615 4252475 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26976/tid=26988] ppid=26974 vsize=652580 CPUtime=49.19 cores=0-7
/proc/26976/task/26988/stat : 26988 (plingeling) R 26974 26976 26909 0 -1 4202560 26058 1261 0 0 4915 4 0 0 25 0 9 0 591004893 668241920 136745 33554432000 4194304 4417332 140736761756672 18446744073709551615 4253091 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=26976/tid=26989] ppid=26974 vsize=652580 CPUtime=49.19 cores=0-7
/proc/26976/task/26989/stat : 26989 (plingeling) R 26974 26976 26909 0 -1 4202560 43855 1261 0 0 4911 8 0 0 25 0 9 0 591004893 668241920 136745 33554432000 4194304 4417332 140736761756672 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26976/tid=26990] ppid=26974 vsize=652580 CPUtime=49.18 cores=0-7
/proc/26976/task/26990/stat : 26990 (plingeling) R 26974 26976 26909 0 -1 4202560 43279 1261 0 0 4913 5 0 0 25 0 9 0 591004893 668241920 136745 33554432000 4194304 4417332 140736761756672 18446744073709551615 4253162 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 395.25
Current children cumulated vsize (KiB) 652580

[startup+102.306 s]
/proc/loadavg: 7.87 7.80 7.88 9/179 26992

################
# More data... #
################

/proc/26976/task/26986/stat : 26986 (plingeling) R 26974 26976 26909 0 -1 4202560 53225 1261 0 0 466049 7 0 0 25 0 9 0 591004893 1292517376 289181 33554432000 4194304 4417332 140736761756672 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=26976/tid=26987] ppid=26974 vsize=1262224 CPUtime=4660.55 cores=0-7
/proc/26976/task/26987/stat : 26987 (plingeling) R 26974 26976 26909 0 -1 4202560 204067 1261 0 0 466023 32 0 0 25 0 9 0 591004893 1292517376 289181 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26976/tid=26988] ppid=26974 vsize=1262224 CPUtime=4659.32 cores=0-7
/proc/26976/task/26988/stat : 26988 (plingeling) R 26974 26976 26909 0 -1 4202560 288786 1261 0 0 465882 50 0 0 25 0 9 0 591004893 1292517376 289181 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=26976/tid=26989] ppid=26974 vsize=1262224 CPUtime=4659.86 cores=0-7
/proc/26976/task/26989/stat : 26989 (plingeling) R 26974 26976 26909 0 -1 4202560 182124 1261 0 0 465955 31 0 0 25 0 9 0 591004893 1292517376 289181 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364509 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26976/tid=26990] ppid=26974 vsize=1262224 CPUtime=4660.53 cores=0-7
/proc/26976/task/26990/stat : 26990 (plingeling) R 26974 26976 26909 0 -1 4202560 217734 1261 0 0 466024 29 0 0 25 0 9 0 591004893 1292517376 289181 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 37259
Current children cumulated vsize (KiB) 1262224

[startup+4722.31 s]
/proc/loadavg: 8.17 8.10 8.03 9/179 27131
/proc/meminfo: memFree=22199020/32951124 swapFree=67111348/67111528
[pid=26976] ppid=26974 vsize=1156896 CPUtime=37738.7 cores=0-7
/proc/26976/stat : 26976 (plingeling) S 26974 26976 26909 0 -1 4202496 1760762 1261 0 0 3773580 291 0 0 18 0 9 0 591004705 1184661504 262849 33554432000 4194304 4417332 140736761756672 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/26976/statm: 289224 262849 181 55 0 287110 0
[pid=26976/tid=26983] ppid=26974 vsize=1156896 CPUtime=4695.11 cores=0-7
/proc/26976/task/26983/stat : 26983 (plingeling) R 26974 26976 26909 0 -1 4202560 205203 1261 0 0 469477 34 0 0 25 0 9 0 591004893 1184661504 262849 33554432000 4194304 4417332 140736761756672 18446744073709551615 4245774 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26976/tid=26984] ppid=26974 vsize=1156896 CPUtime=4720.54 cores=0-7
/proc/26976/task/26984/stat : 26984 (plingeling) R 26974 26976 26909 0 -1 4202560 253281 1261 0 0 472005 49 0 0 25 0 9 0 591004893 1184661504 262849 33554432000 4194304 4417332 140736761756672 18446744073709551615 4353461 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26976/tid=26985] ppid=26974 vsize=1156896 CPUtime=4720.26 cores=0-7
/proc/26976/task/26985/stat : 26985 (plingeling) R 26974 26976 26909 0 -1 4202560 271117 1261 0 0 471983 43 0 0 25 0 9 0 591004893 1184661504 262849 33554432000 4194304 4417332 140736761756672 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26976/tid=26986] ppid=26974 vsize=1156896 CPUtime=4720.57 cores=0-7
/proc/26976/task/26986/stat : 26986 (plingeling) R 26974 26976 26909 0 -1 4202560 53225 1261 0 0 472050 7 0 0 25 0 9 0 591004893 1184661504 262849 33554432000 4194304 4417332 140736761756672 18446744073709551615 4354401 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=26976/tid=26987] ppid=26974 vsize=1156896 CPUtime=4720.56 cores=0-7
/proc/26976/task/26987/stat : 26987 (plingeling) R 26974 26976 26909 0 -1 4202560 210161 1261 0 0 472023 33 0 0 25 0 9 0 591004893 1184661504 262849 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364509 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26976/tid=26988] ppid=26974 vsize=1156896 CPUtime=4719.32 cores=0-7
/proc/26976/task/26988/stat : 26988 (plingeling) R 26974 26976 26909 0 -1 4202560 291978 1261 0 0 471882 50 0 0 25 0 9 0 591004893 1184661504 262849 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364565 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=26976/tid=26989] ppid=26974 vsize=1156896 CPUtime=4719.87 cores=0-7
/proc/26976/task/26989/stat : 26989 (plingeling) R 26974 26976 26909 0 -1 4202560 185650 1261 0 0 471955 32 0 0 25 0 9 0 591004893 1184661504 262849 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26976/tid=26990] ppid=26974 vsize=1156896 CPUtime=4720.54 cores=0-7
/proc/26976/task/26990/stat : 26990 (plingeling) R 26974 26976 26909 0 -1 4202560 218644 1261 0 0 472025 29 0 0 25 0 9 0 591004893 1184661504 262849 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 37738.7
Current children cumulated vsize (KiB) 1156896

[startup+4782.3 s]
/proc/loadavg: 8.13 8.09 8.03 9/179 27132
/proc/meminfo: memFree=22230748/32951124 swapFree=67111348/67111528
[pid=26976] ppid=26974 vsize=1136344 CPUtime=38218.4 cores=0-7
/proc/26976/stat : 26976 (plingeling) S 26974 26976 26909 0 -1 4202496 1793276 1261 0 0 3821539 297 0 0 18 0 9 0 591004705 1163616256 257711 33554432000 4194304 4417332 140736761756672 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/26976/statm: 284086 257711 181 55 0 281972 0
[pid=26976/tid=26983] ppid=26974 vsize=1136344 CPUtime=4754.78 cores=0-7
/proc/26976/task/26983/stat : 26983 (plingeling) R 26974 26976 26909 0 -1 4202560 209950 1261 0 0 475443 35 0 0 25 0 9 0 591004893 1163616256 257711 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26976/tid=26984] ppid=26974 vsize=1136344 CPUtime=4780.54 cores=0-7
/proc/26976/task/26984/stat : 26984 (plingeling) R 26974 26976 26909 0 -1 4202560 261018 1261 0 0 478003 51 0 0 25 0 9 0 591004893 1163616256 257711 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26976/tid=26985] ppid=26974 vsize=1136344 CPUtime=4780.26 cores=0-7
/proc/26976/task/26985/stat : 26985 (plingeling) R 26974 26976 26909 0 -1 4202560 275213 1261 0 0 477982 44 0 0 25 0 9 0 591004893 1163616256 257711 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364526 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26976/tid=26986] ppid=26974 vsize=1136344 CPUtime=4780.56 cores=0-7
/proc/26976/task/26986/stat : 26986 (plingeling) R 26974 26976 26909 0 -1 4202560 55592 1261 0 0 478049 7 0 0 25 0 9 0 591004893 1163616256 257711 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=26976/tid=26987] ppid=26974 vsize=1136344 CPUtime=4780.55 cores=0-7
/proc/26976/task/26987/stat : 26987 (plingeling) R 26974 26976 26909 0 -1 4202560 211858 1261 0 0 478022 33 0 0 25 0 9 0 591004893 1163616256 257711 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26976/tid=26988] ppid=26974 vsize=1136344 CPUtime=4779.31 cores=0-7
/proc/26976/task/26988/stat : 26988 (plingeling) R 26974 26976 26909 0 -1 4202560 292026 1261 0 0 477881 50 0 0 25 0 9 0 591004893 1163616256 257711 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364699 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=26976/tid=26989] ppid=26974 vsize=1136344 CPUtime=4779.86 cores=0-7
/proc/26976/task/26989/stat : 26989 (plingeling) R 26974 26976 26909 0 -1 4202560 195652 1261 0 0 477953 33 0 0 25 0 9 0 591004893 1163616256 257711 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26976/tid=26990] ppid=26974 vsize=1136344 CPUtime=4780.54 cores=0-7
/proc/26976/task/26990/stat : 26990 (plingeling) R 26974 26976 26909 0 -1 4202560 220464 1261 0 0 478025 29 0 0 25 0 9 0 591004893 1163616256 257711 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 38218.4
Current children cumulated vsize (KiB) 1136344

[startup+4842.3 s]
/proc/loadavg: 8.11 8.09 8.02 9/179 27134
/proc/meminfo: memFree=22261352/32951124 swapFree=67111348/67111528
[pid=26976] ppid=26974 vsize=1105840 CPUtime=38698 cores=0-7
/proc/26976/stat : 26976 (plingeling) S 26974 26976 26909 0 -1 4202496 1819769 1261 0 0 3869503 300 0 0 18 0 9 0 591004705 1132380160 250085 33554432000 4194304 4417332 140736761756672 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/26976/statm: 276460 250085 181 55 0 274346 0
[pid=26976/tid=26983] ppid=26974 vsize=1105840 CPUtime=4814.45 cores=0-7
/proc/26976/task/26983/stat : 26983 (plingeling) R 26974 26976 26909 0 -1 4202560 209950 1261 0 0 481410 35 0 0 25 0 9 0 591004893 1132380160 250085 33554432000 4194304 4417332 140736761756672 18446744073709551615 4352392 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26976/tid=26984] ppid=26974 vsize=1105840 CPUtime=4840.54 cores=0-7
/proc/26976/task/26984/stat : 26984 (plingeling) R 26974 26976 26909 0 -1 4202560 262776 1261 0 0 484003 51 0 0 25 0 9 0 591004893 1132380160 250085 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26976/tid=26985] ppid=26974 vsize=1105840 CPUtime=4840.26 cores=0-7
/proc/26976/task/26985/stat : 26985 (plingeling) R 26974 26976 26909 0 -1 4202560 276225 1261 0 0 483982 44 0 0 25 0 9 0 591004893 1132380160 250085 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26976/tid=26986] ppid=26974 vsize=1105840 CPUtime=4840.56 cores=0-7
/proc/26976/task/26986/stat : 26986 (plingeling) R 26974 26976 26909 0 -1 4202560 56387 1261 0 0 484049 7 0 0 25 0 9 0 591004893 1132380160 250085 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=26976/tid=26987] ppid=26974 vsize=1105840 CPUtime=4840.56 cores=0-7
/proc/26976/task/26987/stat : 26987 (plingeling) R 26974 26976 26909 0 -1 4202560 224523 1261 0 0 484021 35 0 0 25 0 9 0 591004893 1132380160 250085 33554432000 4194304 4417332 140736761756672 18446744073709551615 4365960 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26976/tid=26988] ppid=26974 vsize=1105840 CPUtime=4839.31 cores=0-7
/proc/26976/task/26988/stat : 26988 (plingeling) R 26974 26976 26909 0 -1 4202560 294713 1261 0 0 483880 51 0 0 25 0 9 0 591004893 1132380160 250085 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364568 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=26976/tid=26989] ppid=26974 vsize=1105840 CPUtime=4839.86 cores=0-7
/proc/26976/task/26989/stat : 26989 (plingeling) R 26974 26976 26909 0 -1 4202560 199250 1261 0 0 483953 33 0 0 25 0 9 0 591004893 1132380160 250085 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26976/tid=26990] ppid=26974 vsize=1105840 CPUtime=4840.54 cores=0-7
/proc/26976/task/26990/stat : 26990 (plingeling) R 26974 26976 26909 0 -1 4202560 224442 1261 0 0 484024 30 0 0 25 0 9 0 591004893 1132380160 250085 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 38698
Current children cumulated vsize (KiB) 1105840

[startup+4902.3 s]
/proc/loadavg: 8.10 8.09 8.02 9/179 27135
/proc/meminfo: memFree=22198340/32951124 swapFree=67111348/67111528
[pid=26976] ppid=26974 vsize=1168196 CPUtime=39177.7 cores=0-7
/proc/26976/stat : 26976 (plingeling) S 26974 26976 26909 0 -1 4202496 1841376 1261 0 0 3917470 303 0 0 18 0 9 0 591004705 1196232704 265674 33554432000 4194304 4417332 140736761756672 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/26976/statm: 292049 265674 181 55 0 289935 0
[pid=26976/tid=26983] ppid=26974 vsize=1168196 CPUtime=4874.14 cores=0-7
/proc/26976/task/26983/stat : 26983 (plingeling) R 26974 26976 26909 0 -1 4202560 212736 1261 0 0 487378 36 0 0 25 0 9 0 591004893 1196232704 265674 33554432000 4194304 4417332 140736761756672 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26976/tid=26984] ppid=26974 vsize=1168196 CPUtime=4900.55 cores=0-7
/proc/26976/task/26984/stat : 26984 (plingeling) R 26974 26976 26909 0 -1 4202560 270968 1261 0 0 490003 52 0 0 25 0 9 0 591004893 1196232704 265674 33554432000 4194304 4417332 140736761756672 18446744073709551615 4249022 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26976/tid=26985] ppid=26974 vsize=1168196 CPUtime=4900.25 cores=0-7
/proc/26976/task/26985/stat : 26985 (plingeling) R 26974 26976 26909 0 -1 4202560 278249 1261 0 0 489981 44 0 0 25 0 9 0 591004893 1196232704 265674 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26976/tid=26986] ppid=26974 vsize=1168196 CPUtime=4900.57 cores=0-7
/proc/26976/task/26986/stat : 26986 (plingeling) R 26974 26976 26909 0 -1 4202560 57977 1261 0 0 490050 7 0 0 25 0 9 0 591004893 1196232704 265674 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=26976/tid=26987] ppid=26974 vsize=1168196 CPUtime=4900.56 cores=0-7
/proc/26976/task/26987/stat : 26987 (plingeling) R 26974 26976 26909 0 -1 4202560 226794 1261 0 0 490021 35 0 0 25 0 9 0 591004893 1196232704 265674 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26976/tid=26988] ppid=26974 vsize=1168196 CPUtime=4899.3 cores=0-7
/proc/26976/task/26988/stat : 26988 (plingeling) R 26974 26976 26909 0 -1 4202560 295792 1261 0 0 489879 51 0 0 25 0 9 0 591004893 1196232704 265674 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=26976/tid=26989] ppid=26974 vsize=1168196 CPUtime=4899.87 cores=0-7
/proc/26976/task/26989/stat : 26989 (plingeling) R 26974 26976 26909 0 -1 4202560 201075 1261 0 0 489953 34 0 0 25 0 9 0 591004893 1196232704 265674 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26976/tid=26990] ppid=26974 vsize=1168196 CPUtime=4900.55 cores=0-7
/proc/26976/task/26990/stat : 26990 (plingeling) R 26974 26976 26909 0 -1 4202560 226282 1261 0 0 490024 31 0 0 25 0 9 0 591004893 1196232704 265674 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 39177.7
Current children cumulated vsize (KiB) 1168196

[startup+4962.3 s]
/proc/loadavg: 8.04 8.07 8.02 9/179 27137
/proc/meminfo: memFree=22195844/32951124 swapFree=67111348/67111528
[pid=26976] ppid=26974 vsize=1171328 CPUtime=39657.4 cores=0-7
/proc/26976/stat : 26976 (plingeling) S 26974 26976 26909 0 -1 4202496 1880587 1261 0 0 3965432 308 0 0 18 0 9 0 591004705 1199439872 266457 33554432000 4194304 4417332 140736761756672 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/26976/statm: 292832 266457 181 55 0 290718 0
[pid=26976/tid=26983] ppid=26974 vsize=1171328 CPUtime=4933.81 cores=0-7
/proc/26976/task/26983/stat : 26983 (plingeling) R 26974 26976 26909 0 -1 4202560 221863 1261 0 0 493344 37 0 0 25 0 9 0 591004893 1199439872 266457 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26976/tid=26984] ppid=26974 vsize=1171328 CPUtime=4960.55 cores=0-7
/proc/26976/task/26984/stat : 26984 (plingeling) R 26974 26976 26909 0 -1 4202560 280132 1261 0 0 496002 53 0 0 25 0 9 0 591004893 1199439872 266457 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26976/tid=26985] ppid=26974 vsize=1171328 CPUtime=4960.26 cores=0-7
/proc/26976/task/26985/stat : 26985 (plingeling) R 26974 26976 26909 0 -1 4202560 290657 1261 0 0 495980 46 0 0 25 0 9 0 591004893 1199439872 266457 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26976/tid=26986] ppid=26974 vsize=1171328 CPUtime=4960.56 cores=0-7
/proc/26976/task/26986/stat : 26986 (plingeling) R 26974 26976 26909 0 -1 4202560 58778 1261 0 0 496049 7 0 0 25 0 9 0 591004893 1199439872 266457 33554432000 4194304 4417332 140736761756672 18446744073709551615 4237290 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=26976/tid=26987] ppid=26974 vsize=1171328 CPUtime=4960.56 cores=0-7
/proc/26976/task/26987/stat : 26987 (plingeling) R 26974 26976 26909 0 -1 4202560 230661 1261 0 0 496020 36 0 0 25 0 9 0 591004893 1199439872 266457 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26976/tid=26988] ppid=26974 vsize=1171328 CPUtime=4959.29 cores=0-7
/proc/26976/task/26988/stat : 26988 (plingeling) R 26974 26976 26909 0 -1 4202560 297749 1261 0 0 495878 51 0 0 25 0 9 0 591004893 1199439872 266457 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=26976/tid=26989] ppid=26974 vsize=1171328 CPUtime=4959.87 cores=0-7
/proc/26976/task/26989/stat : 26989 (plingeling) R 26974 26976 26909 0 -1 4202560 202035 1261 0 0 495953 34 0 0 25 0 9 0 591004893 1199439872 266457 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364512 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26976/tid=26990] ppid=26974 vsize=1171328 CPUtime=4960.55 cores=0-7
/proc/26976/task/26990/stat : 26990 (plingeling) R 26974 26976 26909 0 -1 4202560 227209 1261 0 0 496024 31 0 0 25 0 9 0 591004893 1199439872 266457 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 39657.4
Current children cumulated vsize (KiB) 1171328



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5005.2 s]
/proc/loadavg: 8.02 8.06 8.01 9/179 27138
/proc/meminfo: memFree=22130232/32951124 swapFree=67111348/67111528
[pid=26976] ppid=26974 vsize=1236732 CPUtime=40000.4 cores=0-7
/proc/26976/stat : 26976 (plingeling) S 26974 26976 26909 0 -1 4202496 1901265 1261 0 0 3999731 311 0 0 18 0 9 0 591004705 1266413568 282808 33554432000 4194304 4417332 140736761756672 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/26976/statm: 309183 282808 181 55 0 307069 0
[pid=26976/tid=26983] ppid=26974 vsize=1236732 CPUtime=4976.48 cores=0-7
/proc/26976/task/26983/stat : 26983 (plingeling) R 26974 26976 26909 0 -1 4202560 223540 1261 0 0 497611 37 0 0 25 0 9 0 591004893 1266413568 282808 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364509 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26976/tid=26984] ppid=26974 vsize=1236732 CPUtime=5003.46 cores=0-7
/proc/26976/task/26984/stat : 26984 (plingeling) R 26974 26976 26909 0 -1 4202560 280132 1261 0 0 500293 53 0 0 25 0 9 0 591004893 1266413568 282808 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364811 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26976/tid=26985] ppid=26974 vsize=1236732 CPUtime=5003.16 cores=0-7
/proc/26976/task/26985/stat : 26985 (plingeling) R 26974 26976 26909 0 -1 4202560 292683 1261 0 0 500270 46 0 0 25 0 9 0 591004893 1266413568 282808 33554432000 4194304 4417332 140736761756672 18446744073709551615 4354410 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26976/tid=26986] ppid=26974 vsize=1236732 CPUtime=5003.47 cores=0-7
/proc/26976/task/26986/stat : 26986 (plingeling) R 26974 26976 26909 0 -1 4202560 60379 1261 0 0 500340 7 0 0 25 0 9 0 591004893 1266413568 282808 33554432000 4194304 4417332 140736761756672 18446744073709551615 4353461 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=26976/tid=26987] ppid=26974 vsize=1236732 CPUtime=5003.47 cores=0-7
/proc/26976/task/26987/stat : 26987 (plingeling) R 26974 26976 26909 0 -1 4202560 230661 1261 0 0 500311 36 0 0 25 0 9 0 591004893 1266413568 282808 33554432000 4194304 4417332 140736761756672 18446744073709551615 4365033 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26976/tid=26988] ppid=26974 vsize=1236732 CPUtime=5002.19 cores=0-7
/proc/26976/task/26988/stat : 26988 (plingeling) R 26974 26976 26909 0 -1 4202560 298836 1261 0 0 500168 51 0 0 25 0 9 0 591004893 1266413568 282808 33554432000 4194304 4417332 140736761756672 18446744073709551615 4365033 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=26976/tid=26989] ppid=26974 vsize=1236732 CPUtime=5002.78 cores=0-7
/proc/26976/task/26989/stat : 26989 (plingeling) R 26974 26976 26909 0 -1 4202560 203874 1261 0 0 500243 35 0 0 25 0 9 0 591004893 1266413568 282808 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26976/tid=26990] ppid=26974 vsize=1236732 CPUtime=5003.45 cores=0-7
/proc/26976/task/26990/stat : 26990 (plingeling) R 26974 26976 26909 0 -1 4202560 239657 1261 0 0 500313 32 0 0 25 0 9 0 591004893 1266413568 282808 33554432000 4194304 4417332 140736761756672 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 1236732

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5005.27
CPU time (s): 40000.5
CPU user time (s): 39997.4
CPU system time (s): 3.17552
CPU usage (%): 799.168
Max. virtual memory (cumulated for all children) (KiB): 1262224

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39997.4
system time used= 3.17552
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1902526
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 384
involuntary context switches= 194407

runsolver used 8.01078 second user time and 18.9761 second system time

The end

Launcher Data 

Begin job on node111 at 2011-04-28 18:22:22
IDJOB=3309675
IDBENCH=82459
IDSOLVER=1695
FILE ID=node111/3309675-1304007742
RUNJOBID= node111-1304007708-26923
PBS_JOBID= 13168839
Free space on /tmp= 70884 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT11/application/jarvisalo/smtqfbv-aigs/smtlib-qfbv-aigs-VS3-benchmark-S2-tseitin.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3309675-1304007742/watcher-3309675-1304007742 -o /tmp/evaluation-result-3309675-1304007742/solver-3309675-1304007742 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3309675-1304007742.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= b967fca11e85ef5b614d3452b1c4b2ad
RANDOM SEED=594793389

node111.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5428.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      23258160 kB
Buffers:       1274724 kB
Cached:        7870128 kB
SwapCached:        132 kB
Active:        3651824 kB
Inactive:      5556488 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      23258160 kB
SwapTotal:    67111528 kB
SwapFree:     67111348 kB
Dirty:           14404 kB
Writeback:           0 kB
AnonPages:       63844 kB
Mapped:          14568 kB
Slab:           416876 kB
PageTables:       4360 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   186292 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 70884 MiB
End job on node111 at 2011-04-28 19:45:50