Trace number 3703651

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
pwbo 2.0SAT (TO)1 1800.17 900.318

General information on the benchmark

Namenormalized-PB06/OPT-SMALLINT/web/www.ps.uni-sb.de/~walser/
benchmarks/radar/normalized-10:10:4.5:0.95:100.opb
MD5SUMb258fc1dc8cb1827e842a0b9558712c7
Bench CategoryOPT-SMALLINT (optimisation, small integers)
Best result obtained on this benchmarkOPT
Best value of the objective obtained on this benchmark0
Best CPU time to get the best result obtained on this benchmark0.039993
Has Objective FunctionYES
SatisfiableYES
(Un)Satisfiability was provedYES
Best value of the objective function 0
Optimality of the best value was proved YES
Number of variables435
Total number of constraints501
Number of constraints which are clauses403
Number of constraints which are cardinality constraints (but not clauses)98
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint1
Maximum length of a constraint16
Number of terms in the objective function 435
Biggest coefficient in the objective function 282
Number of bits for the biggest coefficient in the objective function 9
Sum of the numbers in the objective function 1168
Number of bits of the sum of numbers in the objective function 11
Biggest number in a constraint 282
Number of bits of the biggest number in a constraint 9
Biggest sum of numbers in a constraint 1168
Number of bits of the biggest sum of numbers11
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Quality of the solution as a function of time


Download the above graph as a PDF file / as an EPS file

Solver Data 

0.00/0.00	c ==============================[ Solver Version ]===============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  pwbo:                    2.0 beta - March, 2012                            |
0.00/0.00	c |                                                                             |
0.00/0.00	c Parsing opb file format. File HOME/instance-3703651-1338090738.opb.
0.00/0.00	c Instance file HOME/instance-3703651-1338090738.opb
0.00/0.00	c File size is 20180 bytes.
0.00/0.00	c Cost Offset: -282
0.00/0.00	c Highest Coefficient sum: 19
0.00/0.00	c ============================[ Problem Statistics ]=============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  Parsing time:          0.00         s                                      |
0.00/0.00	c |  Number Variables:      435                                                 |
0.00/0.00	c |  Number Clauses:        404                                                 |
0.00/0.00	c |  Number Cardinality:    97                                                  |
0.00/0.00	c |  Number PB Constraints: 0                                                   |
0.00/0.00	c |                                                                             |
0.00/0.00	c |  Problem Type:          Optimization                                        |
0.00/0.00	c |  Objective Function:    Cardinality Constraint                              |
0.00/0.00	c |                                                                             |
0.00/0.00	c ===============================================================================
1800.17/900.31	o 45
1800.17/900.31	o 31
1800.17/900.31	o 17
1800.17/900.31	o 16
1800.17/900.31	o 14
1800.17/900.31	o 11
1800.17/900.31	o 4
1800.17/900.31	o 3
1800.17/900.31	o 2
1800.17/900.31	o 1
1800.17/900.31	c ===============================================================================
1800.17/900.31	c Wall Clock Time: 900.31 s
1800.17/900.31	c Best Solution: 1
1800.17/900.31	s SATISFIABLE
1800.17/900.31	v x359 x235 -x63 x6 -x341 x236 -x400 x363 -x340 x240 x7 x399 -x361 x239 x8 x401 -x342 x237 x144 -x11 x434 -x362 x344 -x238 x143 -x9 -x417 -x366 x145 -x10 -x421 x345 x435 x358 x234 -x183 x62 -x3 -x337 x233 -x187 x5 x364 -x336 -x321 x244 x4 -x12 -x367 -x343 x402 -x365 x346 x416 -x410 x420 -x406 x146 -x405 -x387 x154 -x27 x150 -x31 x356 -x317 -x252 x247 -x182 x64 x360 -x256 x248 -x186 -x2 x357 x320 -x243 -x20 -x368 x338 x16 -x407 x339 -x241 -x161 -x68 -x15 -x409 -x165 x418 -x383 x350 x151 -x129 x422 x153 -x403 -x386 -x302 -x26 x149 -x30 -x316 -x274 -x251 -x245 x184 x65 -x17 x355 -x278 -x255 -x188 -x19 x376 x322 -x84 x69 -x412 -x408 x372 -x88 -x67 -x411 x371 x353 -x242 -x190 -x160 -x125 -x48 -x13 x354 -x191 x164 -x152 x419 -x382 x349 -x325 -x298 -x128 -x14 x423 x424 -x404 x388 -x347 -x301 -x28 x425 x147 -x32 -x373 x318 -x273 -x253 -x246 x185 x66 -x18 -x375 -x277 -x257 x189 x70 -x352 x323 x193 -x83 -x44 -x351 -x192 x87 -x378 x369 -x326 -x259 x162 -x124 -x47 -x413 -x324 -x260 x166 -x22 x414 -x384 -x370 x297 -x130 -x21 x415 x429 x389 -x348 x303 -x168 -x29 -x169 x148 -x33 -x374 x314 -x275 -x254 x181 x78 x319 -x279 -x258 x180 -x156 x74 x315 -x262 -x197 -x155 -x120 x85 x73 -x43 -x327 -x261 x89 -x293 -x281 -x221 x163 -x126 -x112 x49 -x377 -x282 -x225 x167 x432 -x379 x299 x171 -x131 -x91 x433 -x385 x170 -x92 -x23 x428 x381 x304 -x211 -x132 -x52 -x24 x390 -x133 -x25 -x276 -x250 -x200 x80 x77 -x39 x313 -x280 -x249 -x201 x335 x284 -x266 -x196 -x108 x86 -x71 -x45 x331 x283 -x157 -x119 x90 -x431 -x330 -x220 -x194 -x158 x121 -x111 x94 -x72 x50 -x430 -x292 -x224 x159 x127 -x93 -x294 -x207 x175 x123 x53 -x380 x300 -x134 -x51 x426 x398 x296 -x210 -x36 x394 x305 -x37 -x332 x272 -x269 -x198 x75 x334 x271 -x270 x79 -x38 x288 -x265 x107 x81 -x40 x82 -x46 -x328 -x263 -x222 -x195 -x178 -x113 x98 x42 -x226 x179 x122 x54 x395 -x329 -x206 -x174 x142 -x35 x397 x295 x138 -x34 -x427 -x312 x228 -x212 -x172 x137 -x116 x393 x308 x229 -x333 x289 -x267 -x199 x105 -x76 x291 -x216 x287 -x215 -x177 x109 x101 -x176 x102 -x41 x285 -x264 -x223 x204 x139 -x114 -x97 -x61 x396 x227 x141 -x58 -x309 x231 -x208 -x117 -x95 -x57 -x311 x230 -x115 x213 -x173 -x135 x391 x306 x290 -x268 x100 x104 x99 x103 -x60 -x217 -x140 x110 -x59 -x286 x218 x106 -x310 x219 x203 -x118 x232 x202 -x96 -x55 x209 x205 -x136 -x56 x392 x307 -x214 x1

Verifier Data 

OK	1

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3703651-1338090738/watcher-3703651-1338090738 -o /tmp/evaluation-result-3703651-1338090738/solver-3703651-1338090738 -C 1800 -W 1900 -M 15500 pwbo2.0 -file-format=opb -verbosity=0 HOME/instance-3703651-1338090738.opb 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 4.04 4.00 3.84 3/171 25469
/proc/meminfo: memFree=30432184/32873648 swapFree=7108/7108
[pid=25469] ppid=25467 vsize=163052 CPUtime=0 cores=1,3,5,7
/proc/25469/stat : 25469 (pwbo2.0) S 25467 25469 25163 0 -1 4202496 1119 0 0 0 0 0 0 0 20 0 3 0 275199271 166965248 1067 33554432000 4194304 5965102 140737089467632 140737089467152 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/25469/statm: 40763 1073 205 433 0 40327 0
[pid=25469/tid=25470] ppid=25467 vsize=163052 CPUtime=0 cores=1,3,5,7
/proc/25469/task/25470/stat : 25470 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 108 0 0 0 0 0 0 0 20 0 3 0 275199272 166965248 1079 33554432000 4194304 5965102 140737089467632 139959303789000 5131557 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=25469/tid=25471] ppid=25467 vsize=163052 CPUtime=0 cores=1,3,5,7
/proc/25469/task/25471/stat : 25471 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 196 0 0 0 0 0 0 0 20 0 3 0 275199272 166965248 1080 33554432000 4194304 5965102 140737089467632 139959293298872 5137146 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0

[startup+0.080634 s]
/proc/loadavg: 4.04 4.00 3.84 3/171 25469
/proc/meminfo: memFree=30432184/32873648 swapFree=7108/7108
[pid=25469] ppid=25467 vsize=172596 CPUtime=0.15 cores=1,3,5,7
/proc/25469/stat : 25469 (pwbo2.0) S 25467 25469 25163 0 -1 4202496 5508 0 0 0 14 1 0 0 20 0 3 0 275199271 176738304 5438 33554432000 4194304 5965102 140737089467632 140737089467152 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/25469/statm: 43149 5438 206 433 0 42713 0
[pid=25469/tid=25470] ppid=25467 vsize=172596 CPUtime=0.06 cores=1,3,5,7
/proc/25469/task/25470/stat : 25470 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 515 0 0 0 6 0 0 0 20 0 3 0 275199272 176738304 5438 33554432000 4194304 5965102 140737089467632 139959303788888 4221120 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=25469/tid=25471] ppid=25467 vsize=172596 CPUtime=0.06 cores=1,3,5,7
/proc/25469/task/25471/stat : 25471 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 4165 0 0 0 5 1 0 0 20 0 3 0 275199272 176738304 5438 33554432000 4194304 5965102 140737089467632 139959293297800 4400867 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 0.15
Current children cumulated vsize (KiB) 172596

[startup+0.100406 s]
/proc/loadavg: 4.04 4.00 3.84 3/171 25469
/proc/meminfo: memFree=30432184/32873648 swapFree=7108/7108
[pid=25469] ppid=25467 vsize=172596 CPUtime=0.19 cores=1,3,5,7
/proc/25469/stat : 25469 (pwbo2.0) S 25467 25469 25163 0 -1 4202496 5584 0 0 0 18 1 0 0 20 0 3 0 275199271 176738304 5489 33554432000 4194304 5965102 140737089467632 140737089467152 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/25469/statm: 43149 5489 208 433 0 42713 0
[pid=25469/tid=25470] ppid=25467 vsize=172596 CPUtime=0.08 cores=1,3,5,7
/proc/25469/task/25470/stat : 25470 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 569 0 0 0 8 0 0 0 20 0 3 0 275199272 176738304 5489 33554432000 4194304 5965102 140737089467632 139959303788888 4401556 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=25469/tid=25471] ppid=25467 vsize=172596 CPUtime=0.08 cores=1,3,5,7
/proc/25469/task/25471/stat : 25471 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 4187 0 0 0 7 1 0 0 20 0 3 0 275199272 176738304 5489 33554432000 4194304 5965102 140737089467632 139959293298920 4404320 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 172596

[startup+0.300301 s]
/proc/loadavg: 4.04 4.00 3.84 3/171 25469
/proc/meminfo: memFree=30432184/32873648 swapFree=7108/7108
[pid=25469] ppid=25467 vsize=172596 CPUtime=0.59 cores=1,3,5,7
/proc/25469/stat : 25469 (pwbo2.0) S 25467 25469 25163 0 -1 4202496 6001 0 0 0 58 1 0 0 20 0 3 0 275199271 176738304 5860 33554432000 4194304 5965102 140737089467632 140737089467152 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/25469/statm: 43149 5860 208 433 0 42713 0
[pid=25469/tid=25470] ppid=25467 vsize=172596 CPUtime=0.28 cores=1,3,5,7
/proc/25469/task/25470/stat : 25470 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 913 0 0 0 28 0 0 0 20 0 3 0 275199272 176738304 5860 33554432000 4194304 5965102 140737089467632 139959303788888 4221120 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=25469/tid=25471] ppid=25467 vsize=172596 CPUtime=0.28 cores=1,3,5,7
/proc/25469/task/25471/stat : 25471 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 4260 0 0 0 27 1 0 0 20 0 3 0 275199272 176738304 5860 33554432000 4194304 5965102 140737089467632 139959293298920 4400955 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 0.59
Current children cumulated vsize (KiB) 172596

[startup+0.700361 s]
/proc/loadavg: 4.04 4.00 3.84 3/171 25469
/proc/meminfo: memFree=30432184/32873648 swapFree=7108/7108
[pid=25469] ppid=25467 vsize=172596 CPUtime=1.39 cores=1,3,5,7
/proc/25469/stat : 25469 (pwbo2.0) S 25467 25469 25163 0 -1 4202496 6377 0 0 0 138 1 0 0 20 0 3 0 275199271 176738304 6231 33554432000 4194304 5965102 140737089467632 140737089467152 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/25469/statm: 43149 6231 208 433 0 42713 0
[pid=25469/tid=25470] ppid=25467 vsize=172596 CPUtime=0.68 cores=1,3,5,7
/proc/25469/task/25470/stat : 25470 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 1242 0 0 0 68 0 0 0 20 0 3 0 275199272 176738304 6231 33554432000 4194304 5965102 140737089467632 139959303788888 4390125 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=25469/tid=25471] ppid=25467 vsize=172596 CPUtime=0.68 cores=1,3,5,7
/proc/25469/task/25471/stat : 25471 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 4307 0 0 0 67 1 0 0 20 0 3 0 275199272 176738304 6231 33554432000 4194304 5965102 140737089467632 139959293298920 4400894 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 1.39
Current children cumulated vsize (KiB) 172596

[startup+1.50031 s]
/proc/loadavg: 4.04 4.00 3.84 5/181 25483
/proc/meminfo: memFree=30355472/32873648 swapFree=7108/7108
[pid=25469] ppid=25467 vsize=172596 CPUtime=2.98 cores=1,3,5,7
/proc/25469/stat : 25469 (pwbo2.0) S 25467 25469 25163 0 -1 4202496 7224 0 0 0 297 1 0 0 20 0 3 0 275199271 176738304 7052 33554432000 4194304 5965102 140737089467632 140737089467152 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/25469/statm: 43149 7052 208 433 0 42713 0
[pid=25469/tid=25470] ppid=25467 vsize=172596 CPUtime=1.48 cores=1,3,5,7
/proc/25469/task/25470/stat : 25470 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 1993 0 0 0 148 0 0 0 20 0 3 0 275199272 176738304 7052 33554432000 4194304 5965102 140737089467632 139959303789080 4401510 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=25469/tid=25471] ppid=25467 vsize=172596 CPUtime=1.48 cores=1,3,5,7
/proc/25469/task/25471/stat : 25471 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 4403 0 0 0 147 1 0 0 20 0 3 0 275199272 176738304 7052 33554432000 4194304 5965102 140737089467632 139959293298808 4374193 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 2.98
Current children cumulated vsize (KiB) 172596

[startup+3.10025 s]
/proc/loadavg: 4.04 4.00 3.84 5/181 25483
/proc/meminfo: memFree=30312444/32873648 swapFree=7108/7108
[pid=25469] ppid=25467 vsize=172596 CPUtime=6.18 cores=1,3,5,7
/proc/25469/stat : 25469 (pwbo2.0) S 25467 25469 25163 0 -1 4202496 7990 0 0 0 617 1 0 0 20 0 3 0 275199271 176738304 7818 33554432000 4194304 5965102 140737089467632 140737089467152 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/25469/statm: 43149 7818 208 433 0 42713 0
[pid=25469/tid=25470] ppid=25467 vsize=172596 CPUtime=3.07 cores=1,3,5,7
/proc/25469/task/25470/stat : 25470 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 2546 0 0 0 307 0 0 0 20 0 3 0 275199272 176738304 7818 33554432000 4194304 5965102 140737089467632 139959303788344 4404608 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=25469/tid=25471] ppid=25467 vsize=172596 CPUtime=3.07 cores=1,3,5,7
/proc/25469/task/25471/stat : 25471 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 4616 0 0 0 306 1 0 0 20 0 3 0 275199272 176738304 7818 33554432000 4194304 5965102 140737089467632 139959293299048 4400960 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 6.18
Current children cumulated vsize (KiB) 172596

[startup+6.30031 s]
/proc/loadavg: 4.04 4.00 3.84 5/181 25483
/proc/meminfo: memFree=30193644/32873648 swapFree=7108/7108
[pid=25469] ppid=25467 vsize=172812 CPUtime=12.58 cores=1,3,5,7
/proc/25469/stat : 25469 (pwbo2.0) S 25467 25469 25163 0 -1 4202496 10235 0 0 0 1256 2 0 0 20 0 3 0 275199271 176959488 10063 33554432000 4194304 5965102 140737089467632 140737089467152 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/25469/statm: 43203 10063 208 433 0 42767 0
[pid=25469/tid=25470] ppid=25467 vsize=172812 CPUtime=6.26 cores=1,3,5,7
/proc/25469/task/25470/stat : 25470 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 4338 0 0 0 625 1 0 0 20 0 3 0 275199272 176959488 10063 33554432000 4194304 5965102 140737089467632 139959303788536 5134569 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=25469/tid=25471] ppid=25467 vsize=172812 CPUtime=6.26 cores=1,3,5,7
/proc/25469/task/25471/stat : 25471 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 5069 0 0 0 625 1 0 0 20 0 3 0 275199272 176959488 10063 33554432000 4194304 5965102 140737089467632 139959293299000 4404407 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 12.58
Current children cumulated vsize (KiB) 172812

[startup+12.7003 s]
/proc/loadavg: 4.03 4.00 3.84 5/181 25483
/proc/meminfo: memFree=30084348/32873648 swapFree=7108/7108
[pid=25469] ppid=25467 vsize=173428 CPUtime=25.37 cores=1,3,5,7
/proc/25469/stat : 25469 (pwbo2.0) S 25467 25469 25163 0 -1 4202496 14119 0 0 0 2532 5 0 0 20 0 3 0 275199271 177590272 13964 33554432000 4194304 5965102 140737089467632 140737089467152 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/25469/statm: 43357 13964 208 433 0 42921 0
[pid=25469/tid=25470] ppid=25467 vsize=173428 CPUtime=12.64 cores=1,3,5,7
/proc/25469/task/25470/stat : 25470 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 7047 0 0 0 1262 2 0 0 20 0 3 0 275199272 177590272 13964 33554432000 4194304 5965102 140737089467632 139959303788584 4401058 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=25469/tid=25471] ppid=25467 vsize=173428 CPUtime=12.65 cores=1,3,5,7
/proc/25469/task/25471/stat : 25471 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 6244 0 0 0 1263 2 0 0 20 0 3 0 275199272 177590272 13964 33554432000 4194304 5965102 140737089467632 139959293299000 4393903 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 25.37
Current children cumulated vsize (KiB) 173428

[startup+25.5003 s]
/proc/loadavg: 4.02 4.00 3.85 5/181 25483
/proc/meminfo: memFree=29855088/32873648 swapFree=7108/7108
[pid=25469] ppid=25467 vsize=173184 CPUtime=50.97 cores=1,3,5,7
/proc/25469/stat : 25469 (pwbo2.0) S 25467 25469 25163 0 -1 4202496 18918 0 0 0 5088 9 0 0 20 0 3 0 275199271 177340416 18687 33554432000 4194304 5965102 140737089467632 140737089467152 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/25469/statm: 43296 18687 208 433 0 42860 0
[pid=25469/tid=25470] ppid=25467 vsize=173184 CPUtime=25.4 cores=1,3,5,7
/proc/25469/task/25470/stat : 25470 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 9816 0 0 0 2536 4 0 0 20 0 3 0 275199272 177340416 18687 33554432000 4194304 5965102 140737089467632 139959303789080 4394002 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=25469/tid=25471] ppid=25467 vsize=173184 CPUtime=25.41 cores=1,3,5,7
/proc/25469/task/25471/stat : 25471 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 8274 0 0 0 2537 4 0 0 20 0 3 0 275199272 177340416 18687 33554432000 4194304 5965102 140737089467632 139959293299000 4393720 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 50.97
Current children cumulated vsize (KiB) 173184

[startup+51.1004 s]
/proc/loadavg: 4.08 4.02 3.85 5/181 25483
/proc/meminfo: memFree=29579916/32873648 swapFree=7108/7108
[pid=25469] ppid=25467 vsize=173740 CPUtime=102.14 cores=1,3,5,7
/proc/25469/stat : 25469 (pwbo2.0) S 25467 25469 25163 0 -1 4202496 28119 0 0 0 10197 17 0 0 20 0 3 0 275199271 177909760 28657 33554432000 4194304 5965102 140737089467632 140737089467152 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/25469/statm: 43435 28657 208 433 0 42999 0
[pid=25469/tid=25470] ppid=25467 vsize=173740 CPUtime=50.94 cores=1,3,5,7
/proc/25469/task/25470/stat : 25470 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 15776 0 0 0 5086 8 0 0 20 0 3 0 275199272 177909760 28657 33554432000 4194304 5965102 140737089467632 139959303788344 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=25469/tid=25471] ppid=25467 vsize=173740 CPUtime=50.93 cores=1,3,5,7
/proc/25469/task/25471/stat : 25471 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 11515 0 0 0 5085 8 0 0 20 0 3 0 275199272 177909760 28657 33554432000 4194304 5965102 140737089467632 139959293298872 4404379 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 102.14
Current children cumulated vsize (KiB) 173740

[startup+102.307 s]
/proc/loadavg: 4.03 4.01 3.86 5/181 25483
/proc/meminfo: memFree=29165276/32873648 swapFree=7108/7108
[pid=25469] ppid=25467 vsize=240504 CPUtime=204.54 cores=1,3,5,7
/proc/25469/stat : 25469 (pwbo2.0) S 25467 25469 25163 0 -1 4202496 47179 0 0 0 20425 29 0 0 20 0 3 0 275199271 246276096 47717 33554432000 4194304 5965102 140737089467632 140737089467152 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/25469/statm: 60126 47717 208 433 0 59690 0
[pid=25469/tid=25470] ppid=25467 vsize=240504 CPUtime=102.01 cores=1,3,5,7
/proc/25469/task/25470/stat : 25470 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 31657 0 0 0 10183 18 0 0 20 0 3 0 275199272 246276096 47717 33554432000 4194304 5965102 140737089467632 139959303788344 4401582 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=25469/tid=25471] ppid=25467 vsize=240504 CPUtime=102.01 cores=1,3,5,7
/proc/25469/task/25471/stat : 25471 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 14694 0 0 0 10189 12 0 0 20 0 3 0 275199272 246276096 47717 33554432000 4194304 5965102 140737089467632 139959293299000 4400960 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 204.54
Current children cumulated vsize (KiB) 240504

[startup+162.3 s]
/proc/loadavg: 4.05 4.02 3.87 5/181 25483
/proc/meminfo: memFree=28713324/32873648 swapFree=7108/7108
[pid=25469] ppid=25467 vsize=373132 CPUtime=324.51 cores=1,3,5,7
/proc/25469/stat : 25469 (pwbo2.0) S 25467 25469 25163 0 -1 4202496 59868 0 0 0 32406 45 0 0 20 0 3 0 275199271 382087168 60406 33554432000 4194304 5965102 140737089467632 140737089467152 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/25469/statm: 93283 60406 208 433 0 92847 0
[pid=25469/tid=25470] ppid=25467 vsize=373132 CPUtime=161.85 cores=1,3,5,7
/proc/25469/task/25470/stat : 25470 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 34972 0 0 0 16162 23 0 0 20 0 3 0 275199272 382087168 60406 33554432000 4194304 5965102 140737089467632 139959303788344 4221120 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=25469/tid=25471] ppid=25467 vsize=373132 CPUtime=161.85 cores=1,3,5,7
/proc/25469/task/25471/stat : 25471 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 24068 0 0 0 16160 25 0 0 20 0 3 0 275199272 382087168 60406 33554432000 4194304 5965102 140737089467632 139959293299000 4404446 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 324.51
Current children cumulated vsize (KiB) 373132

[startup+222.3 s]
/proc/loadavg: 4.02 4.02 3.87 5/181 25483
/proc/meminfo: memFree=28526736/32873648 swapFree=7108/7108
[pid=25469] ppid=25467 vsize=372724 CPUtime=444.49 cores=1,3,5,7
/proc/25469/stat : 25469 (pwbo2.0) S 25467 25469 25163 0 -1 4202496 74104 0 0 0 44389 60 0 0 20 0 3 0 275199271 381669376 74414 33554432000 4194304 5965102 140737089467632 140737089467152 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/25469/statm: 93181 74414 208 433 0 92745 0
[pid=25469/tid=25470] ppid=25467 vsize=372724 CPUtime=221.7 cores=1,3,5,7
/proc/25469/task/25470/stat : 25470 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 41496 0 0 0 22140 30 0 0 20 0 3 0 275199272 381669376 74414 33554432000 4194304 5965102 140737089467632 139959303789496 4221120 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=25469/tid=25471] ppid=25467 vsize=372724 CPUtime=221.69 cores=1,3,5,7
/proc/25469/task/25471/stat : 25471 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 31780 0 0 0 22134 35 0 0 20 0 3 0 275199272 381669376 74414 33554432000 4194304 5965102 140737089467632 139959293299000 4400955 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 444.49
Current children cumulated vsize (KiB) 372724

################
# More data... #
################

[pid=25469/tid=25471] ppid=25467 vsize=439044 CPUtime=341.39 cores=1,3,5,7
/proc/25469/task/25471/stat : 25471 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 36078 0 0 0 34096 43 0 0 20 0 3 0 275199272 449581056 79366 33554432000 4194304 5965102 140737089467632 139959293299000 4401159 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 684.45
Current children cumulated vsize (KiB) 439044

[startup+402.3 s]
/proc/loadavg: 4.00 4.00 3.89 5/181 25484
/proc/meminfo: memFree=28116384/32873648 swapFree=7108/7108
[pid=25469] ppid=25467 vsize=504724 CPUtime=804.43 cores=1,3,5,7
/proc/25469/stat : 25469 (pwbo2.0) S 25467 25469 25163 0 -1 4202496 94082 0 0 0 80362 81 0 0 20 0 3 0 275199271 516837376 94385 33554432000 4194304 5965102 140737089467632 140737089467152 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/25469/statm: 126181 94385 208 433 0 125745 0
[pid=25469/tid=25470] ppid=25467 vsize=504724 CPUtime=401.24 cores=1,3,5,7
/proc/25469/task/25470/stat : 25470 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 51053 0 0 0 40084 40 0 0 20 0 3 0 275199272 516837376 94385 33554432000 4194304 5965102 140737089467632 139959303788536 4380351 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=25469/tid=25471] ppid=25467 vsize=504724 CPUtime=401.23 cores=1,3,5,7
/proc/25469/task/25471/stat : 25471 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 42201 0 0 0 40071 52 0 0 20 0 3 0 275199272 516837376 94385 33554432000 4194304 5965102 140737089467632 139959293299048 4400955 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 804.43
Current children cumulated vsize (KiB) 504724

[startup+462.3 s]
/proc/loadavg: 4.00 4.00 3.90 5/181 25484
/proc/meminfo: memFree=28095708/32873648 swapFree=7108/7108
[pid=25469] ppid=25467 vsize=504724 CPUtime=924.41 cores=1,3,5,7
/proc/25469/stat : 25469 (pwbo2.0) S 25467 25469 25163 0 -1 4202496 99012 0 0 0 92354 87 0 0 20 0 3 0 275199271 516837376 99313 33554432000 4194304 5965102 140737089467632 140737089467152 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/25469/statm: 126181 99313 208 433 0 125745 0
[pid=25469/tid=25470] ppid=25467 vsize=504724 CPUtime=461.08 cores=1,3,5,7
/proc/25469/task/25470/stat : 25470 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 55983 0 0 0 46060 48 0 0 20 0 3 0 275199272 516837376 99313 33554432000 4194304 5965102 140737089467632 139959303788344 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=25469/tid=25471] ppid=25467 vsize=504724 CPUtime=461.08 cores=1,3,5,7
/proc/25469/task/25471/stat : 25471 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 42201 0 0 0 46055 53 0 0 20 0 3 0 275199272 516837376 99313 33554432000 4194304 5965102 140737089467632 139959293299048 4400894 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 924.41
Current children cumulated vsize (KiB) 504724

[startup+522.3 s]
/proc/loadavg: 4.00 4.00 3.90 5/182 25487
/proc/meminfo: memFree=28066596/32873648 swapFree=7108/7108
[pid=25469] ppid=25467 vsize=504868 CPUtime=1044.4 cores=1,3,5,7
/proc/25469/stat : 25469 (pwbo2.0) S 25467 25469 25163 0 -1 4202496 102391 0 0 0 104349 91 0 0 20 0 3 0 275199271 516984832 102691 33554432000 4194304 5965102 140737089467632 140737089467152 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/25469/statm: 126217 102691 208 433 0 125781 0
[pid=25469/tid=25470] ppid=25467 vsize=504868 CPUtime=520.93 cores=1,3,5,7
/proc/25469/task/25470/stat : 25470 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 59349 0 0 0 52042 51 0 0 20 0 3 0 275199272 516984832 102691 33554432000 4194304 5965102 140737089467632 139959303788584 4221120 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=25469/tid=25471] ppid=25467 vsize=504868 CPUtime=520.92 cores=1,3,5,7
/proc/25469/task/25471/stat : 25471 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 42214 0 0 0 52039 53 0 0 20 0 3 0 275199272 516984832 102691 33554432000 4194304 5965102 140737089467632 139959293298760 4400894 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 1044.4
Current children cumulated vsize (KiB) 504868

[startup+582.3 s]
/proc/loadavg: 4.00 4.00 3.91 5/182 25511
/proc/meminfo: memFree=27982168/32873648 swapFree=7108/7108
[pid=25469] ppid=25467 vsize=570404 CPUtime=1164.38 cores=1,3,5,7
/proc/25469/stat : 25469 (pwbo2.0) S 25467 25469 25163 0 -1 4202496 112337 0 0 0 116336 102 0 0 20 0 3 0 275199271 584093696 112635 33554432000 4194304 5965102 140737089467632 140737089467152 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/25469/statm: 142601 112635 208 433 0 142165 0
[pid=25469/tid=25470] ppid=25467 vsize=570404 CPUtime=580.77 cores=1,3,5,7
/proc/25469/task/25470/stat : 25470 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 69280 0 0 0 58014 63 0 0 20 0 3 0 275199272 584093696 112635 33554432000 4194304 5965102 140737089467632 139959303788584 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=25469/tid=25471] ppid=25467 vsize=570404 CPUtime=580.77 cores=1,3,5,7
/proc/25469/task/25471/stat : 25471 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 42229 0 0 0 58023 54 0 0 20 0 3 0 275199272 584093696 112635 33554432000 4194304 5965102 140737089467632 139959293298760 4400978 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 1164.38
Current children cumulated vsize (KiB) 570404

[startup+642.3 s]
/proc/loadavg: 4.04 4.01 3.91 5/181 25511
/proc/meminfo: memFree=27964436/32873648 swapFree=7108/7108
[pid=25469] ppid=25467 vsize=572316 CPUtime=1284.33 cores=1,3,5,7
/proc/25469/stat : 25469 (pwbo2.0) S 25467 25469 25163 0 -1 4202496 115032 0 0 0 128328 105 0 0 20 0 3 0 275199271 586051584 116133 33554432000 4194304 5965102 140737089467632 140737089467152 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/25469/statm: 143079 116133 208 433 0 142643 0
[pid=25469/tid=25470] ppid=25467 vsize=572316 CPUtime=640.61 cores=1,3,5,7
/proc/25469/task/25470/stat : 25470 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 71903 0 0 0 63993 68 0 0 20 0 3 0 275199272 586051584 116133 33554432000 4194304 5965102 140737089467632 139959303788584 4220672 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=25469/tid=25471] ppid=25467 vsize=572316 CPUtime=640.61 cores=1,3,5,7
/proc/25469/task/25471/stat : 25471 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 42301 0 0 0 64006 55 0 0 20 0 3 0 275199272 586051584 116133 33554432000 4194304 5965102 140737089467632 139959293298760 4400960 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 1284.33
Current children cumulated vsize (KiB) 572316

[startup+702.3 s]
/proc/loadavg: 4.01 4.01 3.91 5/181 25511
/proc/meminfo: memFree=27936812/32873648 swapFree=7108/7108
[pid=25469] ppid=25467 vsize=570968 CPUtime=1404.26 cores=1,3,5,7
/proc/25469/stat : 25469 (pwbo2.0) S 25467 25469 25163 0 -1 4202496 121510 0 0 0 140313 113 0 0 20 0 3 0 275199271 584671232 122380 33554432000 4194304 5965102 140737089467632 140737089467152 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/25469/statm: 142742 122380 208 433 0 142306 0
[pid=25469/tid=25470] ppid=25467 vsize=570968 CPUtime=700.43 cores=1,3,5,7
/proc/25469/task/25470/stat : 25470 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 77571 0 0 0 69968 75 0 0 20 0 3 0 275199272 584671232 122380 33554432000 4194304 5965102 140737089467632 139959303789080 4401073 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=25469/tid=25471] ppid=25467 vsize=570968 CPUtime=700.42 cores=1,3,5,7
/proc/25469/task/25471/stat : 25471 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 43111 0 0 0 69985 57 0 0 20 0 3 0 275199272 584671232 122380 33554432000 4194304 5965102 140737089467632 139959293299000 4400955 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 1404.26
Current children cumulated vsize (KiB) 570968

[startup+762.307 s]
/proc/loadavg: 4.05 4.02 3.92 5/181 25511
/proc/meminfo: memFree=27918144/32873648 swapFree=7108/7108
[pid=25469] ppid=25467 vsize=570968 CPUtime=1524.24 cores=1,3,5,7
/proc/25469/stat : 25469 (pwbo2.0) S 25467 25469 25163 0 -1 4202496 125002 0 0 0 152307 117 0 0 20 0 3 0 275199271 584671232 126082 33554432000 4194304 5965102 140737089467632 140737089467152 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/25469/statm: 142742 126082 208 433 0 142306 0
[pid=25469/tid=25470] ppid=25467 vsize=570968 CPUtime=760.27 cores=1,3,5,7
/proc/25469/task/25470/stat : 25470 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 77623 0 0 0 75951 76 0 0 20 0 3 0 275199272 584671232 126082 33554432000 4194304 5965102 140737089467632 139959303789080 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=25469/tid=25471] ppid=25467 vsize=570968 CPUtime=760.27 cores=1,3,5,7
/proc/25469/task/25471/stat : 25471 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 46551 0 0 0 75965 62 0 0 20 0 3 0 275199272 584671232 126082 33554432000 4194304 5965102 140737089467632 139959293299000 4400960 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 1524.24
Current children cumulated vsize (KiB) 570968

[startup+822.3 s]
/proc/loadavg: 4.02 4.01 3.92 5/181 25511
/proc/meminfo: memFree=27870060/32873648 swapFree=7108/7108
[pid=25469] ppid=25467 vsize=570968 CPUtime=1644.2 cores=1,3,5,7
/proc/25469/stat : 25469 (pwbo2.0) S 25467 25469 25163 0 -1 4202496 125023 0 0 0 164303 117 0 0 20 0 3 0 275199271 584671232 126097 33554432000 4194304 5965102 140737089467632 140737089467152 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/25469/statm: 142742 126097 208 433 0 142306 0
[pid=25469/tid=25470] ppid=25467 vsize=570968 CPUtime=820.09 cores=1,3,5,7
/proc/25469/task/25470/stat : 25470 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 77640 0 0 0 81932 77 0 0 20 0 3 0 275199272 584671232 126097 33554432000 4194304 5965102 140737089467632 139959303789080 4401131 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=25469/tid=25471] ppid=25467 vsize=570968 CPUtime=820.11 cores=1,3,5,7
/proc/25469/task/25471/stat : 25471 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 46555 0 0 0 81948 63 0 0 20 0 3 0 275199272 584671232 126097 33554432000 4194304 5965102 140737089467632 139959293299000 4401668 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 1644.2
Current children cumulated vsize (KiB) 570968

[startup+882.3 s]
/proc/loadavg: 4.00 4.01 3.92 5/181 25512
/proc/meminfo: memFree=27858600/32873648 swapFree=7108/7108
[pid=25469] ppid=25467 vsize=571240 CPUtime=1764.16 cores=1,3,5,7
/proc/25469/stat : 25469 (pwbo2.0) S 25467 25469 25163 0 -1 4202496 125086 0 0 0 176299 117 0 0 20 0 3 0 275199271 584949760 126647 33554432000 4194304 5965102 140737089467632 140737089467152 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/25469/statm: 142810 126647 208 433 0 142374 0
[pid=25469/tid=25470] ppid=25467 vsize=571240 CPUtime=879.94 cores=1,3,5,7
/proc/25469/task/25470/stat : 25470 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 77641 0 0 0 87916 78 0 0 20 0 3 0 275199272 584949760 126647 33554432000 4194304 5965102 140737089467632 139959303789080 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=25469/tid=25471] ppid=25467 vsize=571240 CPUtime=879.94 cores=1,3,5,7
/proc/25469/task/25471/stat : 25471 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 46617 0 0 0 87930 64 0 0 20 0 3 0 275199272 584949760 126647 33554432000 4194304 5965102 140737089467632 139959293298760 4400960 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 1764.16
Current children cumulated vsize (KiB) 571240



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+900.3 s]
/proc/loadavg: 4.00 4.00 3.92 5/181 25512
/proc/meminfo: memFree=27840488/32873648 swapFree=7108/7108
[pid=25469] ppid=25467 vsize=571240 CPUtime=1800.17 cores=1,3,5,7
/proc/25469/stat : 25469 (pwbo2.0) S 25467 25469 25163 0 -1 4202496 127887 0 0 0 179896 121 0 0 20 0 3 0 275199271 584949760 129448 33554432000 4194304 5965102 140737089467632 140737089467152 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/25469/statm: 142810 129448 208 433 0 142374 0
[pid=25469/tid=25470] ppid=25467 vsize=571240 CPUtime=897.9 cores=1,3,5,7
/proc/25469/task/25470/stat : 25470 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 77750 0 0 0 89712 78 0 0 20 0 3 0 275199272 584949760 129448 33554432000 4194304 5965102 140737089467632 139959303789080 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=25469/tid=25471] ppid=25467 vsize=571240 CPUtime=897.9 cores=1,3,5,7
/proc/25469/task/25471/stat : 25471 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 49309 0 0 0 89722 68 0 0 20 0 3 0 275199272 584949760 129448 33554432000 4194304 5965102 140737089467632 139959293299000 4404383 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 1800.17
Current children cumulated vsize (KiB) 571240

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+900.3 s]
/proc/loadavg: 4.00 4.00 3.92 5/181 25512
/proc/meminfo: memFree=27840488/32873648 swapFree=7108/7108
[pid=25469] ppid=25467 vsize=571240 CPUtime=1800.17 cores=1,3,5,7
/proc/25469/stat : 25469 (pwbo2.0) S 25467 25469 25163 0 -1 4202496 127887 0 0 0 179896 121 0 0 20 0 3 0 275199271 584949760 129448 33554432000 4194304 5965102 140737089467632 140737089467152 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/25469/statm: 142810 129448 208 433 0 142374 0
[pid=25469/tid=25470] ppid=25467 vsize=571240 CPUtime=897.9 cores=1,3,5,7
/proc/25469/task/25470/stat : 25470 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 77750 0 0 0 89712 78 0 0 20 0 3 0 275199272 584949760 129448 33554432000 4194304 5965102 140737089467632 139959303789080 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=25469/tid=25471] ppid=25467 vsize=571240 CPUtime=897.9 cores=1,3,5,7
/proc/25469/task/25471/stat : 25471 (pwbo2.0) R 25467 25469 25163 0 -1 4202560 49309 0 0 0 89722 68 0 0 20 0 3 0 275199272 584949760 129448 33554432000 4194304 5965102 140737089467632 139959293299000 4404383 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 1800.17
Current children cumulated vsize (KiB) 571240

Child status: 0

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 25469 and gives
#  childrusage.ru_utime.tv_sec=1794
#  childrusage.ru_utime.tv_usec=348217
#  childrusage.ru_stime.tv_sec=1
#  childrusage.ru_stime.tv_usec=491773
# CPU time returned by wait4() is 1795.84
# while last known CPU time is 1800.17
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 900.318
CPU time (s): 1800.17
CPU user time (s): 1798.96
CPU system time (s): 1.21
CPU usage (%): 199.948
Max. virtual memory (cumulated for all children) (KiB): 572452

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1794.35
system time used= 1.49177
maximum resident set size= 517808
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 127891
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 352
involuntary context switches= 2196

runsolver used 1.64175 second user time and 5.37518 second system time

The end

Launcher Data 

Begin job on node123 at 2012-05-27 05:52:18
IDJOB=3703651
IDBENCH=2225
IDSOLVER=2279
FILE ID=node123/3703651-1338090738
RUNJOBID= node123-1338088025-25180
PBS_JOBID= 14624452
Free space on /tmp= 71692 MiB

SOLVER NAME= pwbo 2.0
BENCH NAME= PB06//final/normalized-PB06/OPT-SMALLINT/web/www.ps.uni-sb.de/~walser/benchmarks/radar/normalized-10:10:4.5:0.95:100.opb
COMMAND LINE= pwbo2.0 -file-format=opb -verbosity=0 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3703651-1338090738/watcher-3703651-1338090738 -o /tmp/evaluation-result-3703651-1338090738/solver-3703651-1338090738 -C 1800 -W 1900 -M 15500  pwbo2.0 -file-format=opb -verbosity=0 HOME/instance-3703651-1338090738.opb

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= b258fc1dc8cb1827e842a0b9558712c7
RANDOM SEED=628634099

node123.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.217
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.43
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.217
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.217
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.217
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.217
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.217
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.217
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.217
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873648 kB
MemFree:        30437904 kB
Buffers:          274088 kB
Cached:          1353760 kB
SwapCached:         1808 kB
Active:           671752 kB
Inactive:         967352 kB
Active(anon):       2468 kB
Inactive(anon):    10968 kB
Active(file):     669284 kB
Inactive(file):   956384 kB
Unevictable:        7108 kB
Mlocked:            7108 kB
SwapTotal:      67108856 kB
SwapFree:       67104820 kB
Dirty:              6704 kB
Writeback:            56 kB
AnonPages:         16460 kB
Mapped:            13484 kB
Shmem:               100 kB
Slab:             658244 kB
SReclaimable:     102404 kB
SUnreclaim:       555840 kB
KernelStack:        1432 kB
PageTables:         4056 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545680 kB
Committed_AS:     104360 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346400 kB
VmallocChunk:   34341988052 kB
HardwareCorrupted:     0 kB
AnonHugePages:      2048 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7680 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 71680 MiB
End job on node123 at 2012-05-27 06:07:20