Trace number 3190956

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio parSAT 349.83 87.6232

General information on the benchmark

Namerandom/large/
unif-k5-r20-v750-c15000-S976181807-056-UNKNOWN.cnf
MD5SUM3d45813e0b955709c303b5b4ba4955d9
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark13.204
Satisfiable
(Un)Satisfiability was proved
Number of variables750
Number of clauses15000
Sum of the clauses size75000
Maximum clause length5
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 515000
Number of clauses of size over 50

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 4 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] lingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	[5] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 4
0.00/0.00	Starting TNM 2009 on core(s) 6
0.00/0.00	Starting lingeling 276-6264d55-100731 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 6
349.83/87.62	s SATISFIABLE
349.83/87.62	v -1 2 3 -4 -5 -6 7 8 9 -10 11 -12 13 14 15 16 17 18 19 20 21 -22 -23 -24 25 -26 27 28 29 -30 31 -32 -33 34 -35 36 37 38 39 40 -41 -42 -43 44 -45 46 47 -48 -49 -50 -51 -52 53 -54 55 -56 -57 58 59 60 61 -62 63 64 -65 66 67 -68 69 70 -71 72 -73 74 -75 76 -77 78 79 -80 81 82 83 84 85 -86 87 88 -89 -90 91 92 -93 -94 -95 96 -97 -98 99 100 -101 -102 103 104 -105 -106 107 108 -109 -110 -111 -112 113 -114 -115 116 -117 118 119 -120 121 -122 123 -124 -125 126 -127 -128 129 130 131 132 -133 134 135 -136 -137 138 -139 140 141 142 143 144 145 146 147 148 -149 150 -151 -152 153 -154 155 156 157 158 159 160 161 -162 163 -164 165 166 -167 -168 169 170 -171 172 173 174 -175 176 177 -178 179 180 181 -182 183 184 -185 186 -187 -188 -189 -190 -191 -192 193 194 -195 196 197 -198 -199 200 -201 202 203 -204 -205 -206 207 -208 -209 210 -211 -212 213 214 215 -216 217 218 219 -220 221 222 -223 -224 -225 226 -227 -228 -229 -230 231 232 233 -234 235 -236 -237 238 -239 -240 241 242 -243 244 245 -246 -247 248 -249 250 -251 252 -253 -254 255 256 -257 258 -259 -260 261 262 -263 264 265 -266 267 268 269 270 -271 -272 273 274 275 -276 -277 -278 279 280 -281 282 283 284 -285 -286 -287 -288 289 290 -291 292 -293 294 295 -296 297 -298 -299 300 -301 -302 -303 304 -305 306 -307 308 -309 -310 311 -312 -313 314 -315 -316 -317 318 319 320 321 -322 -323 324 325 326 327 328 329 -330 -331 -332 -333 334 335 336 337 -338 -339 -340 -341 342 343 -344 -345 -346 -347 -348 -349 -350 351 -352 353 -354 -355 356 357 358 -359 360 -361 -362 -363 364 365 366 367 368 -369 370 -371 372 373 374 -375 -376 377 -378 379 380 381 -382 -383 384 385 386 387 -388 389 -390 391 392 -393 -394 -395 -396 -397 -398 -399 400 -401 -402 403 -404 -405 406 -407 408 -409 -410 -411 412 413 -414 415 416 -417 418 -419 -420 421 -422 -423 424 -425 -426 427 -428 429 -430 -431 -432 -433 434 -435 436 -437 -438 -439 440 441 -442 443 444 -445 -446 447 -448 -449 -450 451 -452 -453 -454 455 456 -457 -458 459 460 -461 462 463 464 465 -466 -467 -468 -469 470 -471 472 -473 -474 475 476 477 -478 479 480 481 -482 -483 -484 485 486 -487 488 -489 490 491 492 -493 494 -495 -496 497 -498 -499 500 -501 502 503 -504 505 -506 -507 -508 -509 510 511 -512 513 514 -515 516 517 -518 -519 520 521 -522 523 -524 -525 -526 -527 528 529 -530 531 -532 533 -534 535 -536 537 538 -539 540 -541 542 543 -544 -545 -546 547 -548 549 550 -551 552 553 -554 555 -556 -557 -558 -559 560 561 -562 -563 -564 -565 566 567 -568 569 570 -571 -572 573 574 575 576 -577 578 579 580 581 -582 583 584 -585 586 -587 -588 589 590 591 592 593 594 595 -596 597 -598 599 -600 601 -602 603 604 -605 -606 -607 -608 609 -610 -611 612 613 -614 615 616 -617 618 -619 -620 -621 -622 -623 -624 625 -626 -627 628 -629 -630 -631 632 633 634 635 636 637 -638 -639 640 641 642 643 644 -645 646 -647 648 -649 -650 -651 -652 -653 654 655 -656 -657 658 659 -660 -661 662 -663 -664 -665 -666 -667 668 669 670 -671 672 -673 -674 -675 676 -677 678 -679 680 -681 682 683 -684 685 686 687 -688 689 -690 -691 -692 693 -694 -695 -696 -697 698 699 700 -701 -702 703 704 705 706 707 708 -709 710 -711 712 -713 -714 715 -716 -717 -718 -719 720 -721 -722 -723 -724 -725 -726 727 728 729 730 -731 732 -733 -734 -735 736 -737 738 -739 740 -741 -742 -743 744 -745 746 747 -748 -749 750 
349.83/87.62	v 0 
349.83/87.62	solver [3] TNM 2009 answered
349.83/87.62	Done

Verifier Data 

OK

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3190956-1304048754/watcher-3190956-1304048754 -o /tmp/evaluation-result-3190956-1304048754/solver-3190956-1304048754 -C 4800 -W 1300 -M 15500 ppfolio HOME/instance-3190956-1304048754.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.73 4.01 3.99 3/178 28987
/proc/meminfo: memFree=31058592/32951124 swapFree=67111344/67111528
[pid=28987] ppid=28985 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28987/stat : 28987 (ppfolio) R 28985 28987 26347 0 -1 4202496 523 0 0 0 0 0 0 0 25 0 1 0 516473284 18763776 390 33554432000 4194304 4287625 140734697239536 18446744073709551615 271550557711 0 0 4096 65536 0 0 0 17 4 0 0 0
/proc/28987/statm: 4581 390 341 23 0 91 0

[startup+0.0688389 s]
/proc/loadavg: 3.73 4.01 3.99 3/178 28987
/proc/meminfo: memFree=31058592/32951124 swapFree=67111344/67111528
[pid=28987] ppid=28985 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28987/stat : 28987 (ppfolio) R 28985 28987 26347 0 -1 4202496 523 0 0 0 0 0 0 0 25 0 1 0 516473284 18763776 390 33554432000 4194304 4287625 140734697239536 18446744073709551615 271550557711 0 0 4096 65536 0 0 0 17 4 0 0 0
/proc/28987/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.100833 s]
/proc/loadavg: 3.73 4.01 3.99 3/178 28987
/proc/meminfo: memFree=31058592/32951124 swapFree=67111344/67111528
[pid=28987] ppid=28985 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28987/stat : 28987 (ppfolio) R 28985 28987 26347 0 -1 4202496 523 0 0 0 0 0 0 0 25 0 1 0 516473284 18763776 390 33554432000 4194304 4287625 140734697239536 18446744073709551615 271550557711 0 0 4096 65536 0 0 0 17 4 0 0 0
/proc/28987/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300796 s]
/proc/loadavg: 3.73 4.01 3.99 3/178 28987
/proc/meminfo: memFree=31058592/32951124 swapFree=67111344/67111528
[pid=28987] ppid=28985 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28987/stat : 28987 (ppfolio) S 28985 28987 26347 0 -1 4202496 523 0 0 0 0 0 0 0 25 0 1 0 516473284 18763776 390 33554432000 4194304 4287625 140734697239536 18446744073709551615 271550557711 0 0 4096 65536 3579261784109875199 0 0 17 4 0 0 0
/proc/28987/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700726 s]
/proc/loadavg: 3.73 4.01 3.99 3/178 28987
/proc/meminfo: memFree=31058592/32951124 swapFree=67111344/67111528
[pid=28987] ppid=28985 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28987/stat : 28987 (ppfolio) S 28985 28987 26347 0 -1 4202496 523 0 0 0 0 0 0 0 20 0 1 0 516473284 18763776 390 33554432000 4194304 4287625 140734697239536 18446744073709551615 271550557711 0 0 4096 65536 3579261784109875199 0 0 17 4 0 0 0
/proc/28987/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50058 s]
/proc/loadavg: 3.73 4.01 3.99 11/192 29004
/proc/meminfo: memFree=30999328/32951124 swapFree=67111344/67111528
[pid=28987] ppid=28985 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28987/stat : 28987 (ppfolio) S 28985 28987 26347 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 516473284 18763776 390 33554432000 4194304 4287625 140734697239536 18446744073709551615 271550557711 0 0 4096 65536 3579261784109875199 0 0 17 4 0 0 0
/proc/28987/statm: 4581 390 341 23 0 91 0
[pid=28988] ppid=28987 vsize=534036 CPUtime=1.49 cores=0
/proc/28988/stat : 28988 (cryptominisat) R 28987 28987 26347 0 -1 4202496 5200 0 0 0 149 0 0 0 20 0 1 0 516473285 546852864 2117 33554432000 4194304 4587354 140737080661376 18446744073709551615 4238593 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/28988/statm: 133509 2117 328 96 0 130071 0
[pid=28989] ppid=28987 vsize=19488 CPUtime=1.49 cores=4
/proc/28989/stat : 28989 (clasp) R 28987 28987 26347 0 -1 4202496 2355 0 0 0 149 0 0 0 25 0 1 0 516473285 19955712 2210 33554432000 4194304 4785740 140733612248464 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/28989/statm: 4872 2210 341 145 0 1917 0
[pid=28990] ppid=28987 vsize=458076 CPUtime=0.77 cores=6
/proc/28990/stat : 28990 (TNM) R 28987 28987 26347 0 -1 4202496 1006 0 0 0 77 0 0 0 25 0 1 0 516473285 469069824 902 33554432000 134512640 134534564 4287973440 18446744073709551615 134528877 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28990/statm: 114519 902 95 6 0 113185 0
[pid=28991] ppid=28987 vsize=5808 CPUtime=1.49 cores=2
/proc/28991/stat : 28991 (lingeling) R 28987 28987 26347 0 -1 4202496 1104 0 0 0 149 0 0 0 25 0 1 0 516473285 5947392 644 33554432000 4194304 4352236 140737232809808 18446744073709551615 4328550 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/28991/statm: 1452 644 127 39 0 532 0
[pid=28992] ppid=28987 vsize=11112 CPUtime=0.7 cores=6
/proc/28992/stat : 28992 (march_hi) R 28987 28987 26347 0 -1 4202496 3129 0 0 0 70 0 0 0 25 0 1 0 516473285 11378688 1009 33554432000 4194304 4331988 140737253861712 18446744073709551615 4239564 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28992/statm: 2778 1009 174 34 0 1220 0
Current children cumulated CPU time (s) 5.94
Current children cumulated vsize (KiB) 1046844

[startup+3.10029 s]
/proc/loadavg: 3.73 4.01 3.99 11/192 29004
/proc/meminfo: memFree=30993996/32951124 swapFree=67111344/67111528
[pid=28987] ppid=28985 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28987/stat : 28987 (ppfolio) S 28985 28987 26347 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 516473284 18763776 390 33554432000 4194304 4287625 140734697239536 18446744073709551615 271550557711 0 0 4096 65536 3579261784109875199 0 0 17 4 0 0 0
/proc/28987/statm: 4581 390 341 23 0 91 0
[pid=28988] ppid=28987 vsize=559932 CPUtime=3.08 cores=0
/proc/28988/stat : 28988 (cryptominisat) R 28987 28987 26347 0 -1 4202496 12278 0 0 0 307 1 0 0 25 0 1 0 516473285 573370368 7731 33554432000 4194304 4587354 140737080661376 18446744073709551615 4364768 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/28988/statm: 139983 7731 338 96 0 136545 0
[pid=28989] ppid=28987 vsize=21336 CPUtime=3.09 cores=4
/proc/28989/stat : 28989 (clasp) R 28987 28987 26347 0 -1 4202496 2801 0 0 0 309 0 0 0 25 0 1 0 516473285 21848064 2656 33554432000 4194304 4785740 140733612248464 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/28989/statm: 5334 2656 341 145 0 2379 0
[pid=28990] ppid=28987 vsize=458076 CPUtime=1.56 cores=6
/proc/28990/stat : 28990 (TNM) R 28987 28987 26347 0 -1 4202496 1006 0 0 0 156 0 0 0 25 0 1 0 516473285 469069824 902 33554432000 134512640 134534564 4287973440 18446744073709551615 134528726 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28990/statm: 114519 902 95 6 0 113185 0
[pid=28991] ppid=28987 vsize=6388 CPUtime=3.09 cores=2
/proc/28991/stat : 28991 (lingeling) R 28987 28987 26347 0 -1 4202496 1249 0 0 0 309 0 0 0 25 0 1 0 516473285 6541312 789 33554432000 4194304 4352236 140737232809808 18446744073709551615 4328351 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/28991/statm: 1597 789 127 39 0 677 0
[pid=28992] ppid=28987 vsize=11492 CPUtime=1.5 cores=6
/proc/28992/stat : 28992 (march_hi) R 28987 28987 26347 0 -1 4202496 3148 0 0 0 150 0 0 0 25 0 1 0 516473285 11767808 1028 33554432000 4194304 4331988 140737253861712 18446744073709551615 4265440 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28992/statm: 2873 1028 174 34 0 1315 0
Current children cumulated CPU time (s) 12.32
Current children cumulated vsize (KiB) 1075548

[startup+6.30071 s]
/proc/loadavg: 4.39 4.14 4.04 11/192 29004
/proc/meminfo: memFree=30942908/32951124 swapFree=67111344/67111528
[pid=28987] ppid=28985 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28987/stat : 28987 (ppfolio) S 28985 28987 26347 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 516473284 18763776 390 33554432000 4194304 4287625 140734697239536 18446744073709551615 271550557711 0 0 4096 65536 3579261784109875199 0 0 17 4 0 0 0
/proc/28987/statm: 4581 390 341 23 0 91 0
[pid=28988] ppid=28987 vsize=550584 CPUtime=6.29 cores=0
/proc/28988/stat : 28988 (cryptominisat) R 28987 28987 26347 0 -1 4202496 13629 0 0 0 628 1 0 0 25 0 1 0 516473285 563798016 7512 33554432000 4194304 4587354 140737080661376 18446744073709551615 4238673 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/28988/statm: 137646 7512 338 96 0 134208 0
[pid=28989] ppid=28987 vsize=23620 CPUtime=6.28 cores=4
/proc/28989/stat : 28989 (clasp) R 28987 28987 26347 0 -1 4202496 3365 0 0 0 628 0 0 0 25 0 1 0 516473285 24186880 3220 33554432000 4194304 4785740 140733612248464 18446744073709551615 4287028 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/28989/statm: 5905 3220 341 145 0 2950 0
[pid=28990] ppid=28987 vsize=458076 CPUtime=3.15 cores=6
/proc/28990/stat : 28990 (TNM) R 28987 28987 26347 0 -1 4202496 1006 0 0 0 315 0 0 0 25 0 1 0 516473285 469069824 902 33554432000 134512640 134534564 4287973440 18446744073709551615 134530632 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28990/statm: 114519 902 95 6 0 113185 0
[pid=28991] ppid=28987 vsize=7676 CPUtime=6.28 cores=2
/proc/28991/stat : 28991 (lingeling) R 28987 28987 26347 0 -1 4202496 1733 0 0 0 628 0 0 0 25 0 1 0 516473285 7860224 1113 33554432000 4194304 4352236 140737232809808 18446744073709551615 4328026 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/28991/statm: 1919 1113 127 39 0 999 0
[pid=28992] ppid=28987 vsize=12176 CPUtime=3.1 cores=6
/proc/28992/stat : 28992 (march_hi) R 28987 28987 26347 0 -1 4202496 3344 0 0 0 310 0 0 0 25 0 1 0 516473285 12468224 1224 33554432000 4194304 4331988 140737253861712 18446744073709551615 4228307 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28992/statm: 3044 1224 174 34 0 1486 0
Current children cumulated CPU time (s) 25.1
Current children cumulated vsize (KiB) 1070456

[startup+12.7006 s]
/proc/loadavg: 4.84 4.24 4.07 11/192 29004
/proc/meminfo: memFree=30921324/32951124 swapFree=67111344/67111528
[pid=28987] ppid=28985 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28987/stat : 28987 (ppfolio) S 28985 28987 26347 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 516473284 18763776 390 33554432000 4194304 4287625 140734697239536 18446744073709551615 271550557711 0 0 4096 65536 3579261784109875199 0 0 17 4 0 0 0
/proc/28987/statm: 4581 390 341 23 0 91 0
[pid=28988] ppid=28987 vsize=561224 CPUtime=12.68 cores=0
/proc/28988/stat : 28988 (cryptominisat) R 28987 28987 26347 0 -1 4202496 15566 0 0 0 1267 1 0 0 25 0 1 0 516473285 574693376 9427 33554432000 4194304 4587354 140737080661376 18446744073709551615 4388096 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/28988/statm: 140306 9427 338 96 0 136868 0
[pid=28989] ppid=28987 vsize=27092 CPUtime=12.69 cores=4
/proc/28989/stat : 28989 (clasp) R 28987 28987 26347 0 -1 4202496 4250 0 0 0 1268 1 0 0 25 0 1 0 516473285 27742208 4105 33554432000 4194304 4785740 140733612248464 18446744073709551615 4552550 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/28989/statm: 6773 4105 341 145 0 3818 0
[pid=28990] ppid=28987 vsize=458076 CPUtime=6.34 cores=6
/proc/28990/stat : 28990 (TNM) R 28987 28987 26347 0 -1 4202496 1006 0 0 0 634 0 0 0 25 0 1 0 516473285 469069824 902 33554432000 134512640 134534564 4287973440 18446744073709551615 134529777 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28990/statm: 114519 902 95 6 0 113185 0
[pid=28991] ppid=28987 vsize=8064 CPUtime=12.68 cores=2
/proc/28991/stat : 28991 (lingeling) R 28987 28987 26347 0 -1 4202496 2247 0 0 0 1268 0 0 0 25 0 1 0 516473285 8257536 1210 33554432000 4194304 4352236 140737232809808 18446744073709551615 4329086 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/28991/statm: 2016 1210 127 39 0 1096 0
[pid=28992] ppid=28987 vsize=13028 CPUtime=6.27 cores=6
/proc/28992/stat : 28992 (march_hi) R 28987 28987 26347 0 -1 4202496 3518 0 0 0 627 0 0 0 25 0 1 0 516473285 13340672 1398 33554432000 4194304 4331988 140737253861712 18446744073709551615 271549932596 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28992/statm: 3257 1398 175 34 0 1699 0
Current children cumulated CPU time (s) 50.66
Current children cumulated vsize (KiB) 1085808

[startup+25.5012 s]
/proc/loadavg: 5.98 4.52 4.16 11/192 29004
/proc/meminfo: memFree=30917256/32951124 swapFree=67111344/67111528
[pid=28987] ppid=28985 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28987/stat : 28987 (ppfolio) S 28985 28987 26347 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 516473284 18763776 390 33554432000 4194304 4287625 140734697239536 18446744073709551615 271550557711 0 0 4096 65536 3579261784109875199 0 0 17 4 0 0 0
/proc/28987/statm: 4581 390 341 23 0 91 0
[pid=28988] ppid=28987 vsize=562348 CPUtime=25.48 cores=0
/proc/28988/stat : 28988 (cryptominisat) R 28987 28987 26347 0 -1 4202496 16037 0 0 0 2547 1 0 0 25 0 1 0 516473285 575844352 9890 33554432000 4194304 4587354 140737080661376 18446744073709551615 4238593 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/28988/statm: 140587 9890 338 96 0 137149 0
[pid=28989] ppid=28987 vsize=31132 CPUtime=25.49 cores=4
/proc/28989/stat : 28989 (clasp) R 28987 28987 26347 0 -1 4202496 5202 0 0 0 2548 1 0 0 25 0 1 0 516473285 31879168 5057 33554432000 4194304 4785740 140733612248464 18446744073709551615 4287201 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/28989/statm: 7783 5057 341 145 0 4828 0
[pid=28990] ppid=28987 vsize=458076 CPUtime=12.74 cores=6
/proc/28990/stat : 28990 (TNM) R 28987 28987 26347 0 -1 4202496 1006 0 0 0 1274 0 0 0 25 0 1 0 516473285 469069824 902 33554432000 134512640 134534564 4287973440 18446744073709551615 134528991 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28990/statm: 114519 902 95 6 0 113185 0
[pid=28991] ppid=28987 vsize=8192 CPUtime=25.48 cores=2
/proc/28991/stat : 28991 (lingeling) R 28987 28987 26347 0 -1 4202496 2567 0 0 0 2548 0 0 0 25 0 1 0 516473285 8388608 1242 33554432000 4194304 4352236 140737232809808 18446744073709551615 4329086 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/28991/statm: 2048 1242 127 39 0 1128 0
[pid=28992] ppid=28987 vsize=14944 CPUtime=12.6 cores=6
/proc/28992/stat : 28992 (march_hi) R 28987 28987 26347 0 -1 4202496 3863 0 0 0 1259 1 0 0 25 0 1 0 516473285 15302656 1743 33554432000 4194304 4331988 140737253861712 18446744073709551615 4265523 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28992/statm: 3736 1743 175 34 0 2178 0
Current children cumulated CPU time (s) 101.79
Current children cumulated vsize (KiB) 1093016

[startup+51.1006 s]
/proc/loadavg: 7.43 4.97 4.32 12/192 29005
/proc/meminfo: memFree=30882812/32951124 swapFree=67111344/67111528
[pid=28987] ppid=28985 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28987/stat : 28987 (ppfolio) S 28985 28987 26347 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 516473284 18763776 390 33554432000 4194304 4287625 140734697239536 18446744073709551615 271550557711 0 0 4096 65536 3579261784109875199 0 0 17 4 0 0 0
/proc/28987/statm: 4581 390 341 23 0 91 0
[pid=28988] ppid=28987 vsize=579024 CPUtime=51.09 cores=0
/proc/28988/stat : 28988 (cryptominisat) R 28987 28987 26347 0 -1 4202496 20571 0 0 0 5107 2 0 0 25 0 1 0 516473285 592920576 14000 33554432000 4194304 4587354 140737080661376 18446744073709551615 4364908 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/28988/statm: 144756 14000 338 96 0 141318 0
[pid=28989] ppid=28987 vsize=37568 CPUtime=51.09 cores=4
/proc/28989/stat : 28989 (clasp) R 28987 28987 26347 0 -1 4202496 6725 0 0 0 5108 1 0 0 25 0 1 0 516473285 38469632 6580 33554432000 4194304 4785740 140733612248464 18446744073709551615 4287053 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/28989/statm: 9392 6580 341 145 0 6437 0
[pid=28990] ppid=28987 vsize=458076 CPUtime=25.44 cores=6
/proc/28990/stat : 28990 (TNM) R 28987 28987 26347 0 -1 4202496 1006 0 0 0 2544 0 0 0 25 0 1 0 516473285 469069824 902 33554432000 134512640 134534564 4287973440 18446744073709551615 134530619 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28990/statm: 114519 902 95 6 0 113185 0
[pid=28991] ppid=28987 vsize=8820 CPUtime=51.08 cores=2

################
# More data... #
################

/proc/28989/statm: 10422 7576 341 145 0 7467 0
[pid=28990] ppid=28987 vsize=458076 CPUtime=38.16 cores=6
/proc/28990/stat : 28990 (TNM) R 28987 28987 26347 0 -1 4202496 1006 0 0 0 3816 0 0 0 25 0 1 0 516473285 469069824 902 33554432000 134512640 134534564 4287973440 18446744073709551615 134528483 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28990/statm: 114519 902 95 6 0 113185 0
[pid=28991] ppid=28987 vsize=9184 CPUtime=76.68 cores=2
/proc/28991/stat : 28991 (lingeling) R 28987 28987 26347 0 -1 4202496 4248 0 0 0 7668 0 0 0 25 0 1 0 516473285 9404416 1489 33554432000 4194304 4352236 140737232809808 18446744073709551615 4234468 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/28991/statm: 2296 1489 127 39 0 1376 0
[pid=28992] ppid=28987 vsize=23136 CPUtime=38.1 cores=6
/proc/28992/stat : 28992 (march_hi) R 28987 28987 26347 0 -1 4202496 5224 0 0 0 3809 1 0 0 25 0 1 0 516473285 23691264 3104 33554432000 4194304 4331988 140737253861712 18446744073709551615 4265444 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28992/statm: 5784 3104 175 34 0 4226 0
Current children cumulated CPU time (s) 306.32
Current children cumulated vsize (KiB) 1105296

[startup+83.1008 s]
/proc/loadavg: 8.57 5.53 4.53 11/192 29006
/proc/meminfo: memFree=30864436/32951124 swapFree=67111344/67111528
[pid=28987] ppid=28985 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28987/stat : 28987 (ppfolio) S 28985 28987 26347 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 516473284 18763776 390 33554432000 4194304 4287625 140734697239536 18446744073709551615 271550557711 0 0 4096 65536 3579261784109875199 0 0 17 4 0 0 0
/proc/28987/statm: 4581 390 341 23 0 91 0
[pid=28988] ppid=28987 vsize=588000 CPUtime=83.08 cores=0
/proc/28988/stat : 28988 (cryptominisat) R 28987 28987 26347 0 -1 4202496 42600 0 0 0 8302 6 0 0 25 0 1 0 516473285 602112000 14204 33554432000 4194304 4587354 140737080661376 18446744073709551615 4388350 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/28988/statm: 147000 14204 338 96 0 143562 0
[pid=28989] ppid=28987 vsize=42628 CPUtime=83.08 cores=4
/proc/28989/stat : 28989 (clasp) R 28987 28987 26347 0 -1 4202496 7930 0 0 0 8307 1 0 0 25 0 1 0 516473285 43651072 7785 33554432000 4194304 4785740 140733612248464 18446744073709551615 4287186 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/28989/statm: 10657 7785 341 145 0 7702 0
[pid=28990] ppid=28987 vsize=458076 CPUtime=41.34 cores=6
/proc/28990/stat : 28990 (TNM) R 28987 28987 26347 0 -1 4202496 1006 0 0 0 4134 0 0 0 25 0 1 0 516473285 469069824 902 33554432000 134512640 134534564 4287973440 18446744073709551615 134528295 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28990/statm: 114519 902 95 6 0 113185 0
[pid=28991] ppid=28987 vsize=9184 CPUtime=83.08 cores=2
/proc/28991/stat : 28991 (lingeling) R 28987 28987 26347 0 -1 4202496 4248 0 0 0 8308 0 0 0 25 0 1 0 516473285 9404416 1489 33554432000 4194304 4352236 140737232809808 18446744073709551615 4328055 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/28991/statm: 2296 1489 127 39 0 1376 0
[pid=28992] ppid=28987 vsize=23136 CPUtime=41.28 cores=6
/proc/28992/stat : 28992 (march_hi) R 28987 28987 26347 0 -1 4202496 5393 0 0 0 4127 1 0 0 25 0 1 0 516473285 23691264 3273 33554432000 4194304 4331988 140737253861712 18446744073709551615 4247575 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28992/statm: 5784 3273 175 34 0 4226 0
Current children cumulated CPU time (s) 331.86
Current children cumulated vsize (KiB) 1139348

[startup+84.7075 s]
/proc/loadavg: 8.57 5.53 4.53 11/192 29006
/proc/meminfo: memFree=30867040/32951124 swapFree=67111344/67111528
[pid=28987] ppid=28985 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28987/stat : 28987 (ppfolio) S 28985 28987 26347 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 516473284 18763776 390 33554432000 4194304 4287625 140734697239536 18446744073709551615 271550557711 0 0 4096 65536 3579261784109875199 0 0 17 4 0 0 0
/proc/28987/statm: 4581 390 341 23 0 91 0
[pid=28988] ppid=28987 vsize=588000 CPUtime=84.69 cores=0
/proc/28988/stat : 28988 (cryptominisat) R 28987 28987 26347 0 -1 4202496 42600 0 0 0 8463 6 0 0 25 0 1 0 516473285 602112000 14204 33554432000 4194304 4587354 140737080661376 18446744073709551615 4365352 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/28988/statm: 147000 14204 338 96 0 143562 0
[pid=28989] ppid=28987 vsize=42628 CPUtime=84.69 cores=4
/proc/28989/stat : 28989 (clasp) R 28987 28987 26347 0 -1 4202496 7930 0 0 0 8468 1 0 0 25 0 1 0 516473285 43651072 7785 33554432000 4194304 4785740 140733612248464 18446744073709551615 4287204 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/28989/statm: 10657 7785 341 145 0 7702 0
[pid=28990] ppid=28987 vsize=458076 CPUtime=42.14 cores=6
/proc/28990/stat : 28990 (TNM) R 28987 28987 26347 0 -1 4202496 1006 0 0 0 4214 0 0 0 25 0 1 0 516473285 469069824 902 33554432000 134512640 134534564 4287973440 18446744073709551615 134530568 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28990/statm: 114519 902 95 6 0 113185 0
[pid=28991] ppid=28987 vsize=11232 CPUtime=84.69 cores=2
/proc/28991/stat : 28991 (lingeling) R 28987 28987 26347 0 -1 4202496 4760 0 0 0 8469 0 0 0 25 0 1 0 516473285 11501568 2001 33554432000 4194304 4352236 140737232809808 18446744073709551615 4328181 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/28991/statm: 2808 2001 127 39 0 1888 0
[pid=28992] ppid=28987 vsize=23136 CPUtime=42.08 cores=6
/proc/28992/stat : 28992 (march_hi) R 28987 28987 26347 0 -1 4202496 5432 0 0 0 4207 1 0 0 25 0 1 0 516473285 23691264 3312 33554432000 4194304 4331988 140737253861712 18446744073709551615 4308765 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28992/statm: 5784 3312 175 34 0 4226 0
Current children cumulated CPU time (s) 338.29
Current children cumulated vsize (KiB) 1141396

[startup+86.3012 s]
/proc/loadavg: 8.57 5.53 4.53 11/192 29006
/proc/meminfo: memFree=30861832/32951124 swapFree=67111344/67111528
[pid=28987] ppid=28985 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28987/stat : 28987 (ppfolio) S 28985 28987 26347 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 516473284 18763776 390 33554432000 4194304 4287625 140734697239536 18446744073709551615 271550557711 0 0 4096 65536 3579261784109875199 0 0 17 4 0 0 0
/proc/28987/statm: 4581 390 341 23 0 91 0
[pid=28988] ppid=28987 vsize=588000 CPUtime=86.28 cores=0
/proc/28988/stat : 28988 (cryptominisat) R 28987 28987 26347 0 -1 4202496 42600 0 0 0 8622 6 0 0 25 0 1 0 516473285 602112000 14204 33554432000 4194304 4587354 140737080661376 18446744073709551615 4364908 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/28988/statm: 147000 14204 338 96 0 143562 0
[pid=28989] ppid=28987 vsize=42628 CPUtime=86.28 cores=4
/proc/28989/stat : 28989 (clasp) R 28987 28987 26347 0 -1 4202496 7931 0 0 0 8627 1 0 0 25 0 1 0 516473285 43651072 7786 33554432000 4194304 4785740 140733612248464 18446744073709551615 4287050 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/28989/statm: 10657 7786 341 145 0 7702 0
[pid=28990] ppid=28987 vsize=458076 CPUtime=42.94 cores=6
/proc/28990/stat : 28990 (TNM) R 28987 28987 26347 0 -1 4202496 1006 0 0 0 4294 0 0 0 25 0 1 0 516473285 469069824 902 33554432000 134512640 134534564 4287973440 18446744073709551615 134530701 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28990/statm: 114519 902 95 6 0 113185 0
[pid=28991] ppid=28987 vsize=10336 CPUtime=86.28 cores=2
/proc/28991/stat : 28991 (lingeling) R 28987 28987 26347 0 -1 4202496 5048 0 0 0 8628 0 0 0 25 0 1 0 516473285 10584064 1777 33554432000 4194304 4352236 140737232809808 18446744073709551615 4328336 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/28991/statm: 2584 1777 127 39 0 1664 0
[pid=28992] ppid=28987 vsize=23136 CPUtime=42.86 cores=6
/proc/28992/stat : 28992 (march_hi) R 28987 28987 26347 0 -1 4202496 5470 0 0 0 4285 1 0 0 25 0 1 0 516473285 23691264 3350 33554432000 4194304 4331988 140737253861712 18446744073709551615 4246743 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28992/statm: 5784 3350 175 34 0 4226 0
Current children cumulated CPU time (s) 344.64
Current children cumulated vsize (KiB) 1140500

[startup+87.101 s]
/proc/loadavg: 8.57 5.53 4.53 11/192 29006
/proc/meminfo: memFree=30859472/32951124 swapFree=67111344/67111528
[pid=28987] ppid=28985 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28987/stat : 28987 (ppfolio) S 28985 28987 26347 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 516473284 18763776 390 33554432000 4194304 4287625 140734697239536 18446744073709551615 271550557711 0 0 4096 65536 3579261784109875199 0 0 17 4 0 0 0
/proc/28987/statm: 4581 390 341 23 0 91 0
[pid=28988] ppid=28987 vsize=588000 CPUtime=87.08 cores=0
/proc/28988/stat : 28988 (cryptominisat) R 28987 28987 26347 0 -1 4202496 42600 0 0 0 8702 6 0 0 25 0 1 0 516473285 602112000 14204 33554432000 4194304 4587354 140737080661376 18446744073709551615 4364784 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/28988/statm: 147000 14204 338 96 0 143562 0
[pid=28989] ppid=28987 vsize=42628 CPUtime=87.08 cores=4
/proc/28989/stat : 28989 (clasp) R 28987 28987 26347 0 -1 4202496 7936 0 0 0 8707 1 0 0 25 0 1 0 516473285 43651072 7791 33554432000 4194304 4785740 140733612248464 18446744073709551615 4553402 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/28989/statm: 10657 7791 341 145 0 7702 0
[pid=28990] ppid=28987 vsize=458076 CPUtime=43.34 cores=6
/proc/28990/stat : 28990 (TNM) R 28987 28987 26347 0 -1 4202496 1006 0 0 0 4334 0 0 0 25 0 1 0 516473285 469069824 902 33554432000 134512640 134534564 4287973440 18446744073709551615 134530712 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28990/statm: 114519 902 95 6 0 113185 0
[pid=28991] ppid=28987 vsize=10336 CPUtime=87.08 cores=2
/proc/28991/stat : 28991 (lingeling) R 28987 28987 26347 0 -1 4202496 5048 0 0 0 8708 0 0 0 25 0 1 0 516473285 10584064 1777 33554432000 4194304 4352236 140737232809808 18446744073709551615 4328345 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/28991/statm: 2584 1777 127 39 0 1664 0
[pid=28992] ppid=28987 vsize=23136 CPUtime=43.26 cores=6
/proc/28992/stat : 28992 (march_hi) R 28987 28987 26347 0 -1 4202496 5493 0 0 0 4325 1 0 0 25 0 1 0 516473285 23691264 3373 33554432000 4194304 4331988 140737253861712 18446744073709551615 4237287 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28992/statm: 5784 3373 175 34 0 4226 0
Current children cumulated CPU time (s) 347.84
Current children cumulated vsize (KiB) 1140500

[startup+87.501 s]
/proc/loadavg: 8.57 5.53 4.53 11/192 29006
/proc/meminfo: memFree=30859472/32951124 swapFree=67111344/67111528
[pid=28987] ppid=28985 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28987/stat : 28987 (ppfolio) S 28985 28987 26347 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 516473284 18763776 390 33554432000 4194304 4287625 140734697239536 18446744073709551615 271550557711 0 0 4096 65536 3579261784109875199 0 0 17 4 0 0 0
/proc/28987/statm: 4581 390 341 23 0 91 0
[pid=28988] ppid=28987 vsize=588000 CPUtime=87.48 cores=0
/proc/28988/stat : 28988 (cryptominisat) R 28987 28987 26347 0 -1 4202496 42600 0 0 0 8742 6 0 0 25 0 1 0 516473285 602112000 14204 33554432000 4194304 4587354 140737080661376 18446744073709551615 4364768 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/28988/statm: 147000 14204 338 96 0 143562 0
[pid=28989] ppid=28987 vsize=43068 CPUtime=87.48 cores=4
/proc/28989/stat : 28989 (clasp) R 28987 28987 26347 0 -1 4202496 8034 0 0 0 8747 1 0 0 25 0 1 0 516473285 44101632 7889 33554432000 4194304 4785740 140733612248464 18446744073709551615 4286998 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/28989/statm: 10767 7889 341 145 0 7812 0
[pid=28990] ppid=28987 vsize=458076 CPUtime=43.54 cores=6
/proc/28990/stat : 28990 (TNM) R 28987 28987 26347 0 -1 4202496 1006 0 0 0 4354 0 0 0 25 0 1 0 516473285 469069824 902 33554432000 134512640 134534564 4287973440 18446744073709551615 134528708 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28990/statm: 114519 902 95 6 0 113185 0
[pid=28991] ppid=28987 vsize=10336 CPUtime=87.48 cores=2
/proc/28991/stat : 28991 (lingeling) R 28987 28987 26347 0 -1 4202496 5048 0 0 0 8748 0 0 0 25 0 1 0 516473285 10584064 1777 33554432000 4194304 4352236 140737232809808 18446744073709551615 4328243 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/28991/statm: 2584 1777 127 39 0 1664 0
[pid=28992] ppid=28987 vsize=23136 CPUtime=43.46 cores=6
/proc/28992/stat : 28992 (march_hi) R 28987 28987 26347 0 -1 4202496 5504 0 0 0 4345 1 0 0 25 0 1 0 516473285 23691264 3384 33554432000 4194304 4331988 140737253861712 18446744073709551615 4299275 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28992/statm: 5784 3384 175 34 0 4226 0
Current children cumulated CPU time (s) 349.44
Current children cumulated vsize (KiB) 1140940

[startup+87.601 s]
/proc/loadavg: 8.57 5.53 4.53 11/192 29006
/proc/meminfo: memFree=30859472/32951124 swapFree=67111344/67111528
[pid=28987] ppid=28985 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28987/stat : 28987 (ppfolio) S 28985 28987 26347 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 516473284 18763776 390 33554432000 4194304 4287625 140734697239536 18446744073709551615 271550557711 0 0 4096 65536 3579261784109875199 0 0 17 4 0 0 0
/proc/28987/statm: 4581 390 341 23 0 91 0
[pid=28988] ppid=28987 vsize=588000 CPUtime=87.58 cores=0
/proc/28988/stat : 28988 (cryptominisat) R 28987 28987 26347 0 -1 4202496 42600 0 0 0 8752 6 0 0 25 0 1 0 516473285 602112000 14204 33554432000 4194304 4587354 140737080661376 18446744073709551615 4365352 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/28988/statm: 147000 14204 338 96 0 143562 0
[pid=28989] ppid=28987 vsize=43068 CPUtime=87.58 cores=4
/proc/28989/stat : 28989 (clasp) R 28987 28987 26347 0 -1 4202496 8034 0 0 0 8757 1 0 0 25 0 1 0 516473285 44101632 7889 33554432000 4194304 4785740 140733612248464 18446744073709551615 4287252 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/28989/statm: 10767 7889 341 145 0 7812 0
[pid=28990] ppid=28987 vsize=458076 CPUtime=43.6 cores=6
/proc/28990/stat : 28990 (TNM) R 28987 28987 26347 0 -1 4202496 1006 0 0 0 4360 0 0 0 25 0 1 0 516473285 469069824 902 33554432000 134512640 134534564 4287973440 18446744073709551615 134530606 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28990/statm: 114519 902 95 6 0 113185 0
[pid=28991] ppid=28987 vsize=10336 CPUtime=87.58 cores=2
/proc/28991/stat : 28991 (lingeling) R 28987 28987 26347 0 -1 4202496 5048 0 0 0 8758 0 0 0 25 0 1 0 516473285 10584064 1777 33554432000 4194304 4352236 140737232809808 18446744073709551615 4329086 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/28991/statm: 2584 1777 127 39 0 1664 0
[pid=28992] ppid=28987 vsize=23136 CPUtime=43.49 cores=6
/proc/28992/stat : 28992 (march_hi) R 28987 28987 26347 0 -1 4202496 5506 0 0 0 4348 1 0 0 25 0 1 0 516473285 23691264 3386 33554432000 4194304 4331988 140737253861712 18446744073709551615 4228243 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28992/statm: 5784 3386 175 34 0 4226 0
Current children cumulated CPU time (s) 349.83
Current children cumulated vsize (KiB) 1140940

Child status: 0

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 28987 and gives
#  childrusage.ru_utime.tv_sec=306
#  childrusage.ru_utime.tv_usec=379423
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=106983
# CPU time returned by wait4() is 306.486
# while last known CPU time is 349.83
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 87.6232
CPU time (s): 349.83
CPU user time (s): 349.75
CPU system time (s): 0.08
CPU usage (%): 399.244
Max. virtual memory (cumulated for all children) (KiB): 1142548

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 306.379
system time used= 0.106983
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 57249
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 242
involuntary context switches= 5048

runsolver used 0.12598 second user time and 0.354946 second system time

The end

Launcher Data 

Begin job on node133 at 2011-04-29 05:45:54
IDJOB=3190956
IDBENCH=82687
IDSOLVER=1612
FILE ID=node133/3190956-1304048754
RUNJOBID= node133-1304048754-28973
PBS_JOBID= 13170319
Free space on /tmp= 71896 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/random/large/unif-k5-r20-v750-c15000-S976181807-056-UNKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3190956-1304048754/watcher-3190956-1304048754 -o /tmp/evaluation-result-3190956-1304048754/solver-3190956-1304048754 -C 4800 -W 1300 -M 15500  ppfolio HOME/instance-3190956-1304048754.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 3d45813e0b955709c303b5b4ba4955d9
RANDOM SEED=1724203769

node133.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 6741.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      31059120 kB
Buffers:        703688 kB
Cached:         766248 kB
SwapCached:        124 kB
Active:         554692 kB
Inactive:       980260 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      31059120 kB
SwapTotal:    67111528 kB
SwapFree:     67111344 kB
Dirty:           10548 kB
Writeback:           0 kB
AnonPages:       64672 kB
Mapped:          14440 kB
Slab:           293672 kB
PageTables:       4728 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   190756 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71876 MiB
End job on node133 at 2011-04-29 05:47:22