Trace number 3219543

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio parSAT 10025 1254.08

General information on the benchmark

Namerandom/large/
unif-k5-r20-v1250-c25000-S1245360321-088-UNKNOWN.cnf
MD5SUM951e2fa38b08f2dfbb0dbb3e8ab83f8d
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark10.8733
Satisfiable
(Un)Satisfiability was proved
Number of variables1250
Number of clauses25000
Sum of the clauses size125000
Maximum clause length5
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 525000
Number of clauses of size over 50

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 8 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7
10025.05/1254.06	s SATISFIABLE
10025.05/1254.06	v 1 2 3 -4 -5 6 -7 8 9 -10 -11 12 -13 -14 15 -16 -17 -18 -19 -20 -21 22 -23 -24 -25 -26 27 28 -29 30 31 32 -33 -34 -35 36 37 38 39 -40 -41 -42 -43 44 -45 -46 47 -48 49 50 -51 52 -53 54 -55 -56 -57 58 -59 60 61 -62 63 64 -65 66 -67 -68 69 70 71 -72 -73 74 -75 -76 -77 78 79 -80 -81 82 83 -84 -85 -86 87 -88 89 90 91 92 93 -94 -95 96 -97 98 99 100 101 -102 103 104 -105 106 107 -108 109 -110 111 112 113 114 -115 116 117 118 -119 -120 121 122 -123 -124 125 -126 -127 -128 -129 130 131 -132 -133 134 -135 -136 137 138 139 140 141 142 -143 144 145 146 147 148 149 150 -151 -152 153 -154 -155 -156 -157 -158 159 160 -161 -162 -163 164 -165 166 167 168 -169 -170 171 -172 -173 -174 175 176 177 -178 179 180 -181 -182 183 -184 -185 186 187 -188 -189 -190 -191 192 -193 -194 195 -196 197 -198 -199 -200 201 -202 203 -204 -205 206 207 208 209 210 -211 -212 -213 -214 215 -216 217 218 -219 -220 -221 222 -223 -224 225 -226 227 -228 -229 -230 231 232 -233 -234 235 236 -237 238 -239 240 -241 -242 -243 244 -245 -246 -247 -248 249 250 -251 252 253 -254 255 -256 257 -258 259 260 261 -262 -263 264 265 -266 -267 268 269 270 271 -272 -273 -274 275 276 -277 278 279 280 -281 282 -283 -284 -285 286 287 -288 289 -290 291 -292 293 -294 -295 296 -297 -298 299 -300 301 -302 303 -304 -305 306 307 308 -309 310 311 -312 313 314 315 316 317 -318 -319 320 -321 322 323 -324 -325 -326 327 328 329 330 331 -332 -333 334 -335 336 -337 -338 -339 340 341 342 343 -344 -345 -346 347 348 349 350 351 -352 353 -354 355 356 357 358 359 360 361 -362 363 -364 -365 -366 367 368 369 -370 371 372 373 -374 -375 -376 377 -378 379 -380 381 382 383 -384 -385 -386 387 388 389 390 -391 -392 393 -394 -395 396 397 398 399 -400 401 402 403 -404 405 406 -407 408 409 -410 -411 -412 -413 414 -415 -416 417 -418 419 420 421 -422 -423 424 425 -426 -427 428 429 430 431 -432 -433 -434 -435 436 -437 -438 439 -440 441 442 -443 -444 445 446 -447 -448 -449 -450 -451 452 -453 454 -455 -456 457 -458 -459 -460 461 462 -463 464 465 466 -467 468 469 -470 471 -472 473 -474 -475 476 -477 478 -479 -480 -481 -482 -483 -484 -485 486 487 488 -489 490 -491 -492 -493 494 495 496 -497 -498 -499 -500 501 -502 503 -504 505 506 507 508 -509 510 511 512 513 514 -515 -516 -517 -518 519 -520 -521 522 -523 524 -525 -526 527 -528 -529 530 -531 -532 533 534 -535 536 -537 538 539 540 541 -542 -543 544 -545 -546 -547 548 549 -550 -551 -552 553 554 -555 556 557 -558 -559 -560 -561 562 -563 564 565 566 -567 568 569 570 -571 -572 -573 574 -575 576 -577 -578 579 580 -581 -582 583 584 585 -586 -587 588 589 -590 591 -592 593 -594 595 596 -597 -598 599 -600 -601 602 603 -604 605 606 607 -608 609 -610 -611 612 -613 614 615 616 -617 -618 -619 -620 621 -622 623 624 -625 -626 -627 628 629 -630 -631 632 633 634 -635 -636 -637 -638 639 -640 641 -642 -643 644 -645 646 -647 648 649 -650 -651 -652 -653 -654 -655 656 657 658 659 -660 -661 662 -663 -664 -665 666 667 -668 669 -670 -671 672 673 -674 675 676 -677 -678 -679 -680 -681 682 -683 684 -685 686 -687 -688 -689 -690 691 692 693 694 695 -696 -697 -698 -699 700 701 702 -703 -704 705 -706 707 -708 709 710 711 712 713 714 715 -716 -717 718 -719 720 -721 -722 723 -724 725 -726 -727 728 729 -730 -731 -732 733 734 -735 736 737 738 -739 740 741 742 -743 -744 745 -746 -747 748 749 -750 751 752 753 754 -755 -756 757 -758 -759 760 -761 762 763 -764 765 -766 767 -768 -769 -770 771 772 -773 -774 775 776 -777 -778 779 -780 781 -782 -783 -784 -785 -786 -787 788 -789 -790 -791 792 793 794 795 -796 -797 798 -799 800 -801 802 803 -804 -805 -806 -807 808 -809 810 -811 812 -813 814 -815 816 817 818 -819 820 821 -822 823 824 825 826 827 828 -829 -830 831 -832 833 834 -835 836 837 838 -839 840 841 842 843 -844 845 846 -847 -848 849 -850 -851 -852 -853 -854 855 856 -857 858 -859 860 -861 862 -863 864 -865 -866 867 -868 -869 -870 -871 -872 -873 -874 875 876 -877 -878 -879 880 -881 -882 883 884 885 886 887 888 889 890 891 -892 -893 894 -895 896 -897 898 899 900 901 902 -903 904 905 -906 907 -908 909 -910 -911 912 913 914 -915 -916 917 -918 919 -920 921 922 -923 -924 -925 926 927 928 -929 930 931 -932 -933 934 935 -936 937 -938 -939 940 941 -942 -943 944 945 946 -947 948 949 950 951 -952 953 954 -955 -956 957 -958 959 -960 961 -962 963 -964 965 -966 967 -968 969 -970 971 972 -973 974 -975 -976 977 -978 -979 -980 981 -982 -983 984 985 986 -987 -988 -989 -990 991 -992 -993 -994 -995 -996 -997 998 -999 -1000 -1001 1002 -1003 -1004 -1005 -1006 -1007 -1008 -1009 -1010 -1011 1012 1013 -1014 1015 1016 1017 1018 -1019 1020 -1021 -1022 1023 1024 -1025 -1026 1027 1028 1029 1030 1031 -1032 1033 1034 -1035 1036 1037 1038 -1039 1040 -1041 1042 -1043 1044 -1045 1046 -1047 1048 -1049 1050 -1051 -1052 -1053 1054 -1055 1056 -1057 1058 -1059 -1060 -1061 1062 -1063 -1064 -1065 1066 -1067 1068 1069 -1070 -1071 -1072 1073 -1074 -1075 -1076 -1077 1078 1079 -1080 1081 -1082 -1083 -1084 1085 -1086 1087 1088 -1089 1090 -1091 1092 1093 -1094 1095 1096 -1097 -1098 -1099 -1100 -1101 -1102 1103 1104 -1105 1106 1107 -1108 -1109 -1110 1111 1112 -1113 1114 -1115 -1116 1117 1118 1119 -1120 1121 1122 -1123 -1124 1125 -1126 1127 1128 1129 1130 -1131 1132 -1133 1134 -1135 -1136 -1137 -1138 -1139 -1140 1141 -1142 1143 -1144 1145 -1146 1147 -1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 -1158 -1159 -1160 1161 1162 -1163 -1164 -1165 -1166 -1167 -1168 1169 -1170 1171 1172 -1173 -1174 1175 1176 -1177 1178 1179 1180 1181 -1182 1183 1184 1185 1186 1187 1188 1189 -1190 1191 -1192 1193 1194 1195 1196 -1197 1198 1199 -1200 -1201 -1202 1203 -1204 -1205 -1206 1207 -1208 1209 -1210 1211 1212 1213 -1214 1215 -1216 1217 -1218 -1219 -1220 1221 -1222 -1223 -1224 -1225 -1226 1227 1228 1229 -1230 -1231 -1232 1233 1234 1235 1236 -1237 -1238 -1239 -1240 -1241 -1242 1243 -1244 1245 1246 -1247 1248 -1249 1250 
10025.05/1254.06	v 0 
10025.05/1254.07	solver [3] TNM 2009 answered
10025.05/1254.07	Done

Verifier Data 

OK

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3219543-1304150798/watcher-3219543-1304150798 -o /tmp/evaluation-result-3219543-1304150798/solver-3219543-1304150798 -C 40000 -W 5000 -M 31000 ppfolio HOME/instance-3219543-1304150798.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.07 8.08 8.00 1/170 11791
/proc/meminfo: memFree=23006888/32950928 swapFree=67111524/67111528
[pid=11791] ppid=11789 vsize=18324 CPUtime=0 cores=0-7
/proc/11791/stat : 11791 (ppfolio) S 11789 11791 11386 0 -1 4202496 519 0 0 0 0 0 0 0 25 0 1 0 526679420 18763776 389 33554432000 4194304 4287625 140736051481664 18446744073709551615 274473987599 0 0 4096 65536 3188134986662281215 0 0 17 6 0 0 0
/proc/11791/statm: 4581 389 341 23 0 91 0

[startup+0.0176569 s]
/proc/loadavg: 8.07 8.08 8.00 1/170 11791
/proc/meminfo: memFree=23006888/32950928 swapFree=67111524/67111528
[pid=11791] ppid=11789 vsize=18324 CPUtime=0 cores=0-7
/proc/11791/stat : 11791 (ppfolio) S 11789 11791 11386 0 -1 4202496 519 0 0 0 0 0 0 0 25 0 1 0 526679420 18763776 389 33554432000 4194304 4287625 140736051481664 18446744073709551615 274473987599 0 0 4096 65536 3188134986662281215 0 0 17 6 0 0 0
/proc/11791/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.176632 s]
/proc/loadavg: 8.07 8.08 8.00 1/170 11791
/proc/meminfo: memFree=23006888/32950928 swapFree=67111524/67111528
[pid=11791] ppid=11789 vsize=18324 CPUtime=0 cores=0-7
/proc/11791/stat : 11791 (ppfolio) R 11789 11791 11386 0 -1 4202496 519 0 0 0 0 0 0 0 24 0 1 0 526679420 18763776 389 33554432000 4194304 4287625 140736051481664 18446744073709551615 274473987599 0 0 4096 65536 0 0 0 17 6 0 0 0
/proc/11791/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.376597 s]
/proc/loadavg: 8.07 8.08 8.00 1/170 11791
/proc/meminfo: memFree=23006888/32950928 swapFree=67111524/67111528
[pid=11791] ppid=11789 vsize=18324 CPUtime=0 cores=0-7
/proc/11791/stat : 11791 (ppfolio) S 11789 11791 11386 0 -1 4202496 519 0 0 0 0 0 0 0 23 0 1 0 526679420 18763776 389 33554432000 4194304 4287625 140736051481664 18446744073709551615 274473987599 0 0 4096 65536 3188134986662281215 0 0 17 6 0 0 0
/proc/11791/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700542 s]
/proc/loadavg: 8.07 8.08 8.00 1/170 11791
/proc/meminfo: memFree=23006888/32950928 swapFree=67111524/67111528
[pid=11791] ppid=11789 vsize=18324 CPUtime=0 cores=0-7
/proc/11791/stat : 11791 (ppfolio) R 11789 11791 11386 0 -1 4202496 519 0 0 0 0 0 0 0 21 0 1 0 526679420 18763776 389 33554432000 4194304 4287625 140736051481664 18446744073709551615 274473987599 0 0 4096 65536 0 0 0 17 6 0 0 0
/proc/11791/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.5004 s]
/proc/loadavg: 8.07 8.08 8.00 9/180 11806
/proc/meminfo: memFree=22961176/32950928 swapFree=67111524/67111528
[pid=11791] ppid=11789 vsize=18324 CPUtime=0 cores=0-7
/proc/11791/stat : 11791 (ppfolio) S 11789 11791 11386 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 526679420 18763776 389 33554432000 4194304 4287625 140736051481664 18446744073709551615 274473987599 0 0 4096 65536 3188134986662281215 0 0 17 6 0 0 0
/proc/11791/statm: 4581 389 341 23 0 91 0
[pid=11792] ppid=11791 vsize=1143580 CPUtime=1.49 cores=0
/proc/11792/stat : 11792 (cryptominisat) R 11791 11791 11386 0 -1 4202496 5704 0 0 0 149 0 0 0 25 0 1 0 526679420 1171025920 3194 33554432000 4194304 4587354 140735233540624 18446744073709551615 4243833 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/11792/statm: 285895 3194 327 96 0 282457 0
[pid=11793] ppid=11791 vsize=22696 CPUtime=1.49 cores=1
/proc/11793/stat : 11793 (clasp) R 11791 11791 11386 0 -1 4202496 3233 0 0 0 149 0 0 0 20 0 1 0 526679420 23240704 3029 33554432000 4194304 4785740 140736440864784 18446744073709551615 4553380 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/11793/statm: 5674 3029 340 145 0 2719 0
[pid=11794] ppid=11791 vsize=459788 CPUtime=1.49 cores=2
/proc/11794/stat : 11794 (TNM) R 11791 11791 11386 0 -1 4202496 1542 0 0 0 149 0 0 0 25 0 1 0 526679420 470822912 1439 33554432000 134512640 134534564 4287138560 18446744073709551615 134528726 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/11794/statm: 114947 1439 95 6 0 113613 0
[pid=11795] ppid=11791 vsize=13880 CPUtime=1.49 cores=3
/proc/11795/stat : 11795 (march_hi) R 11791 11791 11386 0 -1 4202496 4928 0 0 0 149 0 0 0 25 0 1 0 526679420 14213120 1435 33554432000 4194304 4331988 140737335270144 18446744073709551615 4265456 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/11795/statm: 3470 1435 174 34 0 1912 0
[pid=11796] ppid=11791 vsize=125156 CPUtime=5.8 cores=4-7
/proc/11796/stat : 11796 (plingeling) S 11791 11791 11386 0 -1 4202496 4081 1252 0 0 580 0 0 0 19 0 5 0 526679420 128159744 3359 33554432000 4194304 4356188 140736076187824 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/11796/statm: 31289 3359 161 40 0 29833 0
[pid=11796/tid=11803] ppid=11791 vsize=125156 CPUtime=1.44 cores=4-7
/proc/11796/task/11803/stat : 11803 (plingeling) R 11791 11791 11386 0 -1 4202560 355 1252 0 0 144 0 0 0 23 0 5 0 526679424 128159744 3359 33554432000 4194304 4356188 140736076187824 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11796/tid=11804] ppid=11791 vsize=125156 CPUtime=1.44 cores=4-7
/proc/11796/task/11804/stat : 11804 (plingeling) R 11791 11791 11386 0 -1 4202560 348 1252 0 0 144 0 0 0 23 0 5 0 526679424 128159744 3359 33554432000 4194304 4356188 140736076187824 18446744073709551615 4281798 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11796/tid=11805] ppid=11791 vsize=125156 CPUtime=1.44 cores=4-7
/proc/11796/task/11805/stat : 11805 (plingeling) R 11791 11791 11386 0 -1 4202560 376 1252 0 0 144 0 0 0 23 0 5 0 526679424 128159744 3359 33554432000 4194304 4356188 140736076187824 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11796/tid=11806] ppid=11791 vsize=125156 CPUtime=1.42 cores=4-7
/proc/11796/task/11806/stat : 11806 (plingeling) R 11791 11791 11386 0 -1 4202560 354 1252 0 0 142 0 0 0 25 0 5 0 526679425 128159744 3359 33554432000 4194304 4356188 140736076187824 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 11.76
Current children cumulated vsize (KiB) 1783424

[startup+3.10112 s]
/proc/loadavg: 8.07 8.08 8.00 9/180 11806
/proc/meminfo: memFree=22958324/32950928 swapFree=67111524/67111528
[pid=11791] ppid=11789 vsize=18324 CPUtime=0 cores=0-7
/proc/11791/stat : 11791 (ppfolio) S 11789 11791 11386 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 526679420 18763776 389 33554432000 4194304 4287625 140736051481664 18446744073709551615 274473987599 0 0 4096 65536 3188134986662281215 0 0 17 6 0 0 0
/proc/11791/statm: 4581 389 341 23 0 91 0
[pid=11792] ppid=11791 vsize=1163184 CPUtime=3.09 cores=0
/proc/11792/stat : 11792 (cryptominisat) R 11791 11791 11386 0 -1 4202496 13718 0 0 0 307 2 0 0 25 0 1 0 526679420 1191100416 8254 33554432000 4194304 4587354 140735233540624 18446744073709551615 4388350 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/11792/statm: 290796 8254 337 96 0 287358 0
[pid=11793] ppid=11791 vsize=25828 CPUtime=3.09 cores=1
/proc/11793/stat : 11793 (clasp) R 11791 11791 11386 0 -1 4202496 4013 0 0 0 309 0 0 0 25 0 1 0 526679420 26447872 3809 33554432000 4194304 4785740 140736440864784 18446744073709551615 4286973 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/11793/statm: 6457 3809 340 145 0 3502 0
[pid=11794] ppid=11791 vsize=459788 CPUtime=3.09 cores=2
/proc/11794/stat : 11794 (TNM) R 11791 11791 11386 0 -1 4202496 1542 0 0 0 309 0 0 0 25 0 1 0 526679420 470822912 1439 33554432000 134512640 134534564 4287138560 18446744073709551615 134530598 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/11794/statm: 114947 1439 95 6 0 113613 0
[pid=11795] ppid=11791 vsize=14548 CPUtime=3.09 cores=3
/proc/11795/stat : 11795 (march_hi) R 11791 11791 11386 0 -1 4202496 5062 0 0 0 309 0 0 0 25 0 1 0 526679420 14897152 1569 33554432000 4194304 4331988 140737335270144 18446744073709551615 4210448 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/11795/statm: 3637 1569 174 34 0 2079 0
[pid=11796] ppid=11791 vsize=126836 CPUtime=12.2 cores=4-7
/proc/11796/stat : 11796 (plingeling) S 11791 11791 11386 0 -1 4202496 4504 1252 0 0 1220 0 0 0 19 0 5 0 526679420 129880064 3765 33554432000 4194304 4356188 140736076187824 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/11796/statm: 31709 3765 161 40 0 30253 0
[pid=11796/tid=11803] ppid=11791 vsize=126836 CPUtime=3.05 cores=4-7
/proc/11796/task/11803/stat : 11803 (plingeling) R 11791 11791 11386 0 -1 4202560 469 1252 0 0 305 0 0 0 25 0 5 0 526679424 129880064 3765 33554432000 4194304 4356188 140736076187824 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11796/tid=11804] ppid=11791 vsize=126836 CPUtime=3.04 cores=4-7
/proc/11796/task/11804/stat : 11804 (plingeling) R 11791 11791 11386 0 -1 4202560 437 1252 0 0 304 0 0 0 25 0 5 0 526679424 129880064 3765 33554432000 4194304 4356188 140736076187824 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11796/tid=11805] ppid=11791 vsize=126836 CPUtime=3.04 cores=4-7
/proc/11796/task/11805/stat : 11805 (plingeling) R 11791 11791 11386 0 -1 4202560 473 1252 0 0 304 0 0 0 25 0 5 0 526679424 129880064 3765 33554432000 4194304 4356188 140736076187824 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11796/tid=11806] ppid=11791 vsize=126836 CPUtime=3.01 cores=4-7
/proc/11796/task/11806/stat : 11806 (plingeling) R 11791 11791 11386 0 -1 4202560 477 1252 0 0 301 0 0 0 25 0 5 0 526679425 129880064 3765 33554432000 4194304 4356188 140736076187824 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 24.56
Current children cumulated vsize (KiB) 1808508

[startup+6.30056 s]
/proc/loadavg: 8.06 8.08 8.00 9/180 11806
/proc/meminfo: memFree=22950764/32950928 swapFree=67111524/67111528
[pid=11791] ppid=11789 vsize=18324 CPUtime=0 cores=0-7
/proc/11791/stat : 11791 (ppfolio) S 11789 11791 11386 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 526679420 18763776 389 33554432000 4194304 4287625 140736051481664 18446744073709551615 274473987599 0 0 4096 65536 3188134986662281215 0 0 17 6 0 0 0
/proc/11791/statm: 4581 389 341 23 0 91 0
[pid=11792] ppid=11791 vsize=1161588 CPUtime=6.29 cores=0
/proc/11792/stat : 11792 (cryptominisat) R 11791 11791 11386 0 -1 4202496 20700 0 0 0 626 3 0 0 25 0 1 0 526679420 1189466112 5989 33554432000 4194304 4587354 140735233540624 18446744073709551615 4365356 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/11792/statm: 290397 5989 337 96 0 286959 0
[pid=11793] ppid=11791 vsize=29240 CPUtime=6.28 cores=1
/proc/11793/stat : 11793 (clasp) R 11791 11791 11386 0 -1 4202496 4832 0 0 0 628 0 0 0 25 0 1 0 526679420 29941760 4628 33554432000 4194304 4785740 140736440864784 18446744073709551615 4286964 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/11793/statm: 7310 4628 340 145 0 4355 0
[pid=11794] ppid=11791 vsize=459788 CPUtime=6.29 cores=2
/proc/11794/stat : 11794 (TNM) R 11791 11791 11386 0 -1 4202496 1542 0 0 0 629 0 0 0 25 0 1 0 526679420 470822912 1439 33554432000 134512640 134534564 4287138560 18446744073709551615 134530573 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/11794/statm: 114947 1439 95 6 0 113613 0
[pid=11795] ppid=11791 vsize=15572 CPUtime=6.29 cores=3
/proc/11795/stat : 11795 (march_hi) R 11791 11791 11386 0 -1 4202496 5313 0 0 0 629 0 0 0 25 0 1 0 526679420 15945728 1820 33554432000 4194304 4331988 140737335270144 18446744073709551615 4287737 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/11795/statm: 3893 1820 175 34 0 2335 0
[pid=11796] ppid=11791 vsize=130156 CPUtime=24.98 cores=4-7
/proc/11796/stat : 11796 (plingeling) S 11791 11791 11386 0 -1 4202496 5342 1252 0 0 2497 1 0 0 19 0 5 0 526679420 133279744 4603 33554432000 4194304 4356188 140736076187824 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/11796/statm: 32539 4603 161 40 0 31083 0
[pid=11796/tid=11803] ppid=11791 vsize=130156 CPUtime=6.25 cores=4-7
/proc/11796/task/11803/stat : 11803 (plingeling) R 11791 11791 11386 0 -1 4202560 646 1252 0 0 625 0 0 0 25 0 5 0 526679424 133279744 4603 33554432000 4194304 4356188 140736076187824 18446744073709551615 4273853 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11796/tid=11804] ppid=11791 vsize=130156 CPUtime=6.24 cores=4-7
/proc/11796/task/11804/stat : 11804 (plingeling) R 11791 11791 11386 0 -1 4202560 655 1252 0 0 624 0 0 0 25 0 5 0 526679424 133279744 4603 33554432000 4194304 4356188 140736076187824 18446744073709551615 4324886 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11796/tid=11805] ppid=11791 vsize=130156 CPUtime=6.24 cores=4-7
/proc/11796/task/11805/stat : 11805 (plingeling) R 11791 11791 11386 0 -1 4202560 657 1252 0 0 624 0 0 0 25 0 5 0 526679424 133279744 4603 33554432000 4194304 4356188 140736076187824 18446744073709551615 4332165 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11796/tid=11806] ppid=11791 vsize=130156 CPUtime=6.19 cores=4-7
/proc/11796/task/11806/stat : 11806 (plingeling) R 11791 11791 11386 0 -1 4202560 736 1252 0 0 619 0 0 0 25 0 5 0 526679425 133279744 4603 33554432000 4194304 4356188 140736076187824 18446744073709551615 4238823 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 50.13
Current children cumulated vsize (KiB) 1814668

[startup+12.7004 s]
/proc/loadavg: 8.06 8.07 8.00 9/180 11806
/proc/meminfo: memFree=22927672/32950928 swapFree=67111524/67111528
[pid=11791] ppid=11789 vsize=18324 CPUtime=0 cores=0-7
/proc/11791/stat : 11791 (ppfolio) S 11789 11791 11386 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 526679420 18763776 389 33554432000 4194304 4287625 140736051481664 18446744073709551615 274473987599 0 0 4096 65536 3188134986662281215 0 0 17 6 0 0 0
/proc/11791/statm: 4581 389 341 23 0 91 0
[pid=11792] ppid=11791 vsize=1163496 CPUtime=12.69 cores=0
/proc/11792/stat : 11792 (cryptominisat) R 11791 11791 11386 0 -1 4202496 25702 0 0 0 1265 4 0 0 25 0 1 0 526679420 1191419904 6639 33554432000 4194304 4587354 140735233540624 18446744073709551615 4238600 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/11792/statm: 290874 6639 337 96 0 287436 0
[pid=11793] ppid=11791 vsize=34368 CPUtime=12.69 cores=1
/proc/11793/stat : 11793 (clasp) R 11791 11791 11386 0 -1 4202496 6098 0 0 0 1268 1 0 0 25 0 1 0 526679420 35192832 5894 33554432000 4194304 4785740 140736440864784 18446744073709551615 4287028 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/11793/statm: 8592 5894 340 145 0 5637 0
[pid=11794] ppid=11791 vsize=459788 CPUtime=12.69 cores=2
/proc/11794/stat : 11794 (TNM) R 11791 11791 11386 0 -1 4202496 1542 0 0 0 1269 0 0 0 25 0 1 0 526679420 470822912 1439 33554432000 134512640 134534564 4287138560 18446744073709551615 134528511 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/11794/statm: 114947 1439 95 6 0 113613 0
[pid=11795] ppid=11791 vsize=16848 CPUtime=12.68 cores=3
/proc/11795/stat : 11795 (march_hi) R 11791 11791 11386 0 -1 4202496 5560 0 0 0 1268 0 0 0 25 0 1 0 526679420 17252352 2067 33554432000 4194304 4331988 140737335270144 18446744073709551615 4263314 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/11795/statm: 4212 2067 175 34 0 2654 0
[pid=11796] ppid=11791 vsize=131040 CPUtime=50.54 cores=4-7
/proc/11796/stat : 11796 (plingeling) S 11791 11791 11386 0 -1 4202496 6612 1252 0 0 5053 1 0 0 19 0 5 0 526679420 134184960 4832 33554432000 4194304 4356188 140736076187824 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/11796/statm: 32760 4832 161 40 0 31304 0
[pid=11796/tid=11803] ppid=11791 vsize=131040 CPUtime=12.64 cores=4-7
/proc/11796/task/11803/stat : 11803 (plingeling) R 11791 11791 11386 0 -1 4202560 1001 1252 0 0 1264 0 0 0 25 0 5 0 526679424 134184960 4832 33554432000 4194304 4356188 140736076187824 18446744073709551615 4282114 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11796/tid=11804] ppid=11791 vsize=131040 CPUtime=12.64 cores=4-7
/proc/11796/task/11804/stat : 11804 (plingeling) R 11791 11791 11386 0 -1 4202560 911 1252 0 0 1264 0 0 0 25 0 5 0 526679424 134184960 4832 33554432000 4194304 4356188 140736076187824 18446744073709551615 4331964 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11796/tid=11805] ppid=11791 vsize=131040 CPUtime=12.64 cores=4-7
/proc/11796/task/11805/stat : 11805 (plingeling) R 11791 11791 11386 0 -1 4202560 978 1252 0 0 1264 0 0 0 25 0 5 0 526679424 134184960 4832 33554432000 4194304 4356188 140736076187824 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11796/tid=11806] ppid=11791 vsize=131040 CPUtime=12.55 cores=4-7
/proc/11796/task/11806/stat : 11806 (plingeling) R 11791 11791 11386 0 -1 4202560 1074 1252 0 0 1255 0 0 0 25 0 5 0 526679425 134184960 4832 33554432000 4194304 4356188 140736076187824 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 101.29
Current children cumulated vsize (KiB) 1823864

[startup+25.5012 s]
/proc/loadavg: 8.04 8.07 8.00 9/180 11806
/proc/meminfo: memFree=22926760/32950928 swapFree=67111524/67111528
[pid=11791] ppid=11789 vsize=18324 CPUtime=0 cores=0-7
/proc/11791/stat : 11791 (ppfolio) S 11789 11791 11386 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 526679420 18763776 389 33554432000 4194304 4287625 140736051481664 18446744073709551615 274473987599 0 0 4096 65536 3188134986662281215 0 0 17 6 0 0 0
/proc/11791/statm: 4581 389 341 23 0 91 0
[pid=11792] ppid=11791 vsize=1145776 CPUtime=25.49 cores=0
/proc/11792/stat : 11792 (cryptominisat) R 11791 11791 11386 0 -1 4202496 39719 0 0 0 2543 6 0 0 25 0 1 0 526679420 1173274624 5147 33554432000 4194304 4587354 140735233540624 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0

################
# More data... #
################

[pid=11791] ppid=11789 vsize=18324 CPUtime=0 cores=0-7
/proc/11791/stat : 11791 (ppfolio) S 11789 11791 11386 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 526679420 18763776 389 33554432000 4194304 4287625 140736051481664 18446744073709551615 274473987599 0 0 4096 65536 3188134986662281215 0 0 17 6 0 0 0
/proc/11791/statm: 4581 389 341 23 0 91 0
[pid=11792] ppid=11791 vsize=1183064 CPUtime=1251.12 cores=0
/proc/11792/stat : 11792 (cryptominisat) R 11791 11791 11386 0 -1 4202496 676882 0 0 0 124919 193 0 0 25 0 1 0 526679420 1211457536 19835 33554432000 4194304 4587354 140735233540624 18446744073709551615 4232120 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/11792/statm: 295766 19835 337 96 0 292328 0
[pid=11793] ppid=11791 vsize=147452 CPUtime=1251.11 cores=1
/proc/11793/stat : 11793 (clasp) R 11791 11791 11386 0 -1 4202496 32709 0 0 0 125103 8 0 0 25 0 1 0 526679420 150990848 32424 33554432000 4194304 4785740 140736440864784 18446744073709551615 4286941 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/11793/statm: 36863 32424 340 145 0 33908 0
[pid=11794] ppid=11791 vsize=459788 CPUtime=1250.99 cores=2
/proc/11794/stat : 11794 (TNM) R 11791 11791 11386 0 -1 4202496 1542 0 0 0 125099 0 0 0 25 0 1 0 526679420 470822912 1439 33554432000 134512640 134534564 4287138560 18446744073709551615 134530504 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/11794/statm: 114947 1439 95 6 0 113613 0
[pid=11795] ppid=11791 vsize=216760 CPUtime=1251.11 cores=3
/proc/11795/stat : 11795 (march_hi) R 11791 11791 11386 0 -1 4202496 54118 0 0 0 125079 32 0 0 25 0 1 0 526679420 221962240 50625 33554432000 4194304 4331988 140737335270144 18446744073709551615 4226274 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/11795/statm: 54190 50625 175 34 0 52632 0
[pid=11796] ppid=11791 vsize=177004 CPUtime=4997.47 cores=4-7
/proc/11796/stat : 11796 (plingeling) S 11791 11791 11386 0 -1 4202496 125927 1252 0 0 499726 21 0 0 19 0 5 0 526679420 181252096 16323 33554432000 4194304 4356188 140736076187824 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/11796/statm: 44251 16323 161 40 0 42795 0
[pid=11796/tid=11803] ppid=11791 vsize=177004 CPUtime=1251.07 cores=4-7
/proc/11796/task/11803/stat : 11803 (plingeling) R 11791 11791 11386 0 -1 4202560 30622 1252 0 0 125102 5 0 0 25 0 5 0 526679424 181252096 16323 33554432000 4194304 4356188 140736076187824 18446744073709551615 4331035 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11796/tid=11804] ppid=11791 vsize=177004 CPUtime=1251.07 cores=4-7
/proc/11796/task/11804/stat : 11804 (plingeling) R 11791 11791 11386 0 -1 4202560 30895 1252 0 0 125102 5 0 0 25 0 5 0 526679424 181252096 16323 33554432000 4194304 4356188 140736076187824 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11796/tid=11805] ppid=11791 vsize=177004 CPUtime=1251.03 cores=4-7
/proc/11796/task/11805/stat : 11805 (plingeling) R 11791 11791 11386 0 -1 4202560 30002 1252 0 0 125099 4 0 0 25 0 5 0 526679424 181252096 16323 33554432000 4194304 4356188 140736076187824 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11796/tid=11806] ppid=11791 vsize=177004 CPUtime=1244.22 cores=4-7
/proc/11796/task/11806/stat : 11806 (plingeling) R 11791 11791 11386 0 -1 4202560 31760 1252 0 0 124417 5 0 0 25 0 5 0 526679425 181252096 16323 33554432000 4194304 4356188 140736076187824 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 10001.8
Current children cumulated vsize (KiB) 2202392

[startup+1252.7 s]
/proc/loadavg: 8.07 8.06 8.00 9/180 11839
/proc/meminfo: memFree=22574080/32950928 swapFree=67111524/67111528
[pid=11791] ppid=11789 vsize=18324 CPUtime=0 cores=0-7
/proc/11791/stat : 11791 (ppfolio) S 11789 11791 11386 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 526679420 18763776 389 33554432000 4194304 4287625 140736051481664 18446744073709551615 274473987599 0 0 4096 65536 3188134986662281215 0 0 17 6 0 0 0
/proc/11791/statm: 4581 389 341 23 0 91 0
[pid=11792] ppid=11791 vsize=1167912 CPUtime=1252.71 cores=0
/proc/11792/stat : 11792 (cryptominisat) R 11791 11791 11386 0 -1 4202496 679118 0 0 0 125078 193 0 0 25 0 1 0 526679420 1195941888 9033 33554432000 4194304 4587354 140735233540624 18446744073709551615 4238690 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/11792/statm: 291978 9033 337 96 0 288540 0
[pid=11793] ppid=11791 vsize=147452 CPUtime=1252.71 cores=1
/proc/11793/stat : 11793 (clasp) R 11791 11791 11386 0 -1 4202496 32709 0 0 0 125263 8 0 0 25 0 1 0 526679420 150990848 32424 33554432000 4194304 4785740 140736440864784 18446744073709551615 4286953 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/11793/statm: 36863 32424 340 145 0 33908 0
[pid=11794] ppid=11791 vsize=459788 CPUtime=1252.59 cores=2
/proc/11794/stat : 11794 (TNM) R 11791 11791 11386 0 -1 4202496 1542 0 0 0 125259 0 0 0 25 0 1 0 526679420 470822912 1439 33554432000 134512640 134534564 4287138560 18446744073709551615 134528979 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/11794/statm: 114947 1439 95 6 0 113613 0
[pid=11795] ppid=11791 vsize=216760 CPUtime=1252.72 cores=3
/proc/11795/stat : 11795 (march_hi) R 11791 11791 11386 0 -1 4202496 54176 0 0 0 125240 32 0 0 25 0 1 0 526679420 221962240 50683 33554432000 4194304 4331988 140737335270144 18446744073709551615 4298878 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/11795/statm: 54190 50683 175 34 0 52632 0
[pid=11796] ppid=11791 vsize=177132 CPUtime=5003.87 cores=4-7
/proc/11796/stat : 11796 (plingeling) S 11791 11791 11386 0 -1 4202496 125959 1252 0 0 500366 21 0 0 19 0 5 0 526679420 181383168 16355 33554432000 4194304 4356188 140736076187824 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/11796/statm: 44283 16355 161 40 0 42827 0
[pid=11796/tid=11803] ppid=11791 vsize=177132 CPUtime=1252.68 cores=4-7
/proc/11796/task/11803/stat : 11803 (plingeling) R 11791 11791 11386 0 -1 4202560 30654 1252 0 0 125263 5 0 0 25 0 5 0 526679424 181383168 16355 33554432000 4194304 4356188 140736076187824 18446744073709551615 4331319 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11796/tid=11804] ppid=11791 vsize=177132 CPUtime=1252.67 cores=4-7
/proc/11796/task/11804/stat : 11804 (plingeling) R 11791 11791 11386 0 -1 4202560 30895 1252 0 0 125262 5 0 0 25 0 5 0 526679424 181383168 16355 33554432000 4194304 4356188 140736076187824 18446744073709551615 4331337 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11796/tid=11805] ppid=11791 vsize=177132 CPUtime=1252.63 cores=4-7
/proc/11796/task/11805/stat : 11805 (plingeling) R 11791 11791 11386 0 -1 4202560 30002 1252 0 0 125259 4 0 0 25 0 5 0 526679424 181383168 16355 33554432000 4194304 4356188 140736076187824 18446744073709551615 4330964 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11796/tid=11806] ppid=11791 vsize=177132 CPUtime=1245.82 cores=4-7
/proc/11796/task/11806/stat : 11806 (plingeling) R 11791 11791 11386 0 -1 4202560 31760 1252 0 0 124577 5 0 0 25 0 5 0 526679425 181383168 16355 33554432000 4194304 4356188 140736076187824 18446744073709551615 4331165 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 10014.6
Current children cumulated vsize (KiB) 2187368

[startup+1253.5 s]
/proc/loadavg: 8.07 8.06 8.00 9/180 11839
/proc/meminfo: memFree=22571600/32950928 swapFree=67111524/67111528
[pid=11791] ppid=11789 vsize=18324 CPUtime=0 cores=0-7
/proc/11791/stat : 11791 (ppfolio) S 11789 11791 11386 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 526679420 18763776 389 33554432000 4194304 4287625 140736051481664 18446744073709551615 274473987599 0 0 4096 65536 3188134986662281215 0 0 17 6 0 0 0
/proc/11791/statm: 4581 389 341 23 0 91 0
[pid=11792] ppid=11791 vsize=1167912 CPUtime=1253.51 cores=0
/proc/11792/stat : 11792 (cryptominisat) R 11791 11791 11386 0 -1 4202496 679541 0 0 0 125158 193 0 0 25 0 1 0 526679420 1195941888 9455 33554432000 4194304 4587354 140735233540624 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/11792/statm: 291978 9455 337 96 0 288540 0
[pid=11793] ppid=11791 vsize=147452 CPUtime=1253.51 cores=1
/proc/11793/stat : 11793 (clasp) R 11791 11791 11386 0 -1 4202496 32709 0 0 0 125343 8 0 0 25 0 1 0 526679420 150990848 32424 33554432000 4194304 4785740 140736440864784 18446744073709551615 4549845 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/11793/statm: 36863 32424 340 145 0 33908 0
[pid=11794] ppid=11791 vsize=459788 CPUtime=1253.39 cores=2
/proc/11794/stat : 11794 (TNM) R 11791 11791 11386 0 -1 4202496 1542 0 0 0 125339 0 0 0 25 0 1 0 526679420 470822912 1439 33554432000 134512640 134534564 4287138560 18446744073709551615 134530659 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/11794/statm: 114947 1439 95 6 0 113613 0
[pid=11795] ppid=11791 vsize=216760 CPUtime=1253.52 cores=3
/proc/11795/stat : 11795 (march_hi) R 11791 11791 11386 0 -1 4202496 54203 0 0 0 125320 32 0 0 25 0 1 0 526679420 221962240 50710 33554432000 4194304 4331988 140737335270144 18446744073709551615 4299269 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/11795/statm: 54190 50710 175 34 0 52632 0
[pid=11796] ppid=11791 vsize=177132 CPUtime=5007.06 cores=4-7
/proc/11796/stat : 11796 (plingeling) S 11791 11791 11386 0 -1 4202496 125959 1252 0 0 500685 21 0 0 19 0 5 0 526679420 181383168 16355 33554432000 4194304 4356188 140736076187824 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/11796/statm: 44283 16355 161 40 0 42827 0
[pid=11796/tid=11803] ppid=11791 vsize=177132 CPUtime=1253.48 cores=4-7
/proc/11796/task/11803/stat : 11803 (plingeling) R 11791 11791 11386 0 -1 4202560 30654 1252 0 0 125343 5 0 0 25 0 5 0 526679424 181383168 16355 33554432000 4194304 4356188 140736076187824 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11796/tid=11804] ppid=11791 vsize=177132 CPUtime=1253.47 cores=4-7
/proc/11796/task/11804/stat : 11804 (plingeling) R 11791 11791 11386 0 -1 4202560 30895 1252 0 0 125342 5 0 0 25 0 5 0 526679424 181383168 16355 33554432000 4194304 4356188 140736076187824 18446744073709551615 4210581 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11796/tid=11805] ppid=11791 vsize=177132 CPUtime=1253.43 cores=4-7
/proc/11796/task/11805/stat : 11805 (plingeling) R 11791 11791 11386 0 -1 4202560 30002 1252 0 0 125339 4 0 0 25 0 5 0 526679424 181383168 16355 33554432000 4194304 4356188 140736076187824 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11796/tid=11806] ppid=11791 vsize=177132 CPUtime=1246.61 cores=4-7
/proc/11796/task/11806/stat : 11806 (plingeling) R 11791 11791 11386 0 -1 4202560 31760 1252 0 0 124656 5 0 0 25 0 5 0 526679425 181383168 16355 33554432000 4194304 4356188 140736076187824 18446744073709551615 4331074 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 10021
Current children cumulated vsize (KiB) 2187368

[startup+1253.9 s]
/proc/loadavg: 8.07 8.06 8.00 9/180 11839
/proc/meminfo: memFree=22568996/32950928 swapFree=67111524/67111528
[pid=11791] ppid=11789 vsize=18324 CPUtime=0 cores=0-7
/proc/11791/stat : 11791 (ppfolio) S 11789 11791 11386 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 526679420 18763776 389 33554432000 4194304 4287625 140736051481664 18446744073709551615 274473987599 0 0 4096 65536 3188134986662281215 0 0 17 6 0 0 0
/proc/11791/statm: 4581 389 341 23 0 91 0
[pid=11792] ppid=11791 vsize=1167912 CPUtime=1253.93 cores=0
/proc/11792/stat : 11792 (cryptominisat) R 11791 11791 11386 0 -1 4202496 679756 0 0 0 125199 194 0 0 25 0 1 0 526679420 1195941888 9670 33554432000 4194304 4587354 140735233540624 18446744073709551615 274473664267 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/11792/statm: 291978 9670 337 96 0 288540 0
[pid=11793] ppid=11791 vsize=147452 CPUtime=1253.92 cores=1
/proc/11793/stat : 11793 (clasp) R 11791 11791 11386 0 -1 4202496 32709 0 0 0 125384 8 0 0 25 0 1 0 526679420 150990848 32424 33554432000 4194304 4785740 140736440864784 18446744073709551615 4287241 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/11793/statm: 36863 32424 340 145 0 33908 0
[pid=11794] ppid=11791 vsize=459788 CPUtime=1253.8 cores=2
/proc/11794/stat : 11794 (TNM) R 11791 11791 11386 0 -1 4202496 1542 0 0 0 125380 0 0 0 25 0 1 0 526679420 470822912 1439 33554432000 134512640 134534564 4287138560 18446744073709551615 134528944 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/11794/statm: 114947 1439 95 6 0 113613 0
[pid=11795] ppid=11791 vsize=216760 CPUtime=1253.92 cores=3
/proc/11795/stat : 11795 (march_hi) R 11791 11791 11386 0 -1 4202496 54219 0 0 0 125360 32 0 0 25 0 1 0 526679420 221962240 50726 33554432000 4194304 4331988 140737335270144 18446744073709551615 4210448 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/11795/statm: 54190 50726 175 34 0 52632 0
[pid=11796] ppid=11791 vsize=177132 CPUtime=5008.67 cores=4-7
/proc/11796/stat : 11796 (plingeling) S 11791 11791 11386 0 -1 4202496 125959 1252 0 0 500846 21 0 0 19 0 5 0 526679420 181383168 16355 33554432000 4194304 4356188 140736076187824 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/11796/statm: 44283 16355 161 40 0 42827 0
[pid=11796/tid=11803] ppid=11791 vsize=177132 CPUtime=1253.88 cores=4-7
/proc/11796/task/11803/stat : 11803 (plingeling) R 11791 11791 11386 0 -1 4202560 30654 1252 0 0 125383 5 0 0 25 0 5 0 526679424 181383168 16355 33554432000 4194304 4356188 140736076187824 18446744073709551615 4210549 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11796/tid=11804] ppid=11791 vsize=177132 CPUtime=1253.88 cores=4-7
/proc/11796/task/11804/stat : 11804 (plingeling) R 11791 11791 11386 0 -1 4202560 30895 1252 0 0 125383 5 0 0 25 0 5 0 526679424 181383168 16355 33554432000 4194304 4356188 140736076187824 18446744073709551615 4274258 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11796/tid=11805] ppid=11791 vsize=177132 CPUtime=1253.83 cores=4-7
/proc/11796/task/11805/stat : 11805 (plingeling) R 11791 11791 11386 0 -1 4202560 30002 1252 0 0 125379 4 0 0 25 0 5 0 526679424 181383168 16355 33554432000 4194304 4356188 140736076187824 18446744073709551615 4332020 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11796/tid=11806] ppid=11791 vsize=177132 CPUtime=1247.01 cores=4-7
/proc/11796/task/11806/stat : 11806 (plingeling) R 11791 11791 11386 0 -1 4202560 31760 1252 0 0 124696 5 0 0 25 0 5 0 526679425 181383168 16355 33554432000 4194304 4356188 140736076187824 18446744073709551615 4330964 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 10024.2
Current children cumulated vsize (KiB) 2187368

[startup+1254.01 s]
/proc/loadavg: 8.07 8.06 8.00 9/180 11839
/proc/meminfo: memFree=22568996/32950928 swapFree=67111524/67111528
[pid=11791] ppid=11789 vsize=18324 CPUtime=0 cores=0-7
/proc/11791/stat : 11791 (ppfolio) S 11789 11791 11386 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 526679420 18763776 389 33554432000 4194304 4287625 140736051481664 18446744073709551615 274473987599 0 0 4096 65536 3188134986662281215 0 0 17 6 0 0 0
/proc/11791/statm: 4581 389 341 23 0 91 0
[pid=11792] ppid=11791 vsize=1167912 CPUtime=1254.03 cores=0
/proc/11792/stat : 11792 (cryptominisat) R 11791 11791 11386 0 -1 4202496 679810 0 0 0 125209 194 0 0 25 0 1 0 526679420 1195941888 9724 33554432000 4194304 4587354 140735233540624 18446744073709551615 4238752 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/11792/statm: 291978 9724 337 96 0 288540 0
[pid=11793] ppid=11791 vsize=147452 CPUtime=1254.02 cores=1
/proc/11793/stat : 11793 (clasp) R 11791 11791 11386 0 -1 4202496 32709 0 0 0 125394 8 0 0 25 0 1 0 526679420 150990848 32424 33554432000 4194304 4785740 140736440864784 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/11793/statm: 36863 32424 340 145 0 33908 0
[pid=11794] ppid=11791 vsize=459788 CPUtime=1253.9 cores=2
/proc/11794/stat : 11794 (TNM) R 11791 11791 11386 0 -1 4202496 1542 0 0 0 125390 0 0 0 25 0 1 0 526679420 470822912 1439 33554432000 134512640 134534564 4287138560 18446744073709551615 134528737 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/11794/statm: 114947 1439 95 6 0 113613 0
[pid=11795] ppid=11791 vsize=216760 CPUtime=1254.02 cores=3
/proc/11795/stat : 11795 (march_hi) R 11791 11791 11386 0 -1 4202496 54223 0 0 0 125370 32 0 0 25 0 1 0 526679420 221962240 50730 33554432000 4194304 4331988 140737335270144 18446744073709551615 4298550 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/11795/statm: 54190 50730 175 34 0 52632 0
[pid=11796] ppid=11791 vsize=177260 CPUtime=5009.08 cores=4-7
/proc/11796/stat : 11796 (plingeling) S 11791 11791 11386 0 -1 4202496 125991 1252 0 0 500887 21 0 0 19 0 5 0 526679420 181514240 16387 33554432000 4194304 4356188 140736076187824 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/11796/statm: 44315 16387 161 40 0 42859 0
[pid=11796/tid=11803] ppid=11791 vsize=177260 CPUtime=1253.98 cores=4-7
/proc/11796/task/11803/stat : 11803 (plingeling) R 11791 11791 11386 0 -1 4202560 30654 1252 0 0 125393 5 0 0 25 0 5 0 526679424 181514240 16387 33554432000 4194304 4356188 140736076187824 18446744073709551615 4330999 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11796/tid=11804] ppid=11791 vsize=177260 CPUtime=1253.98 cores=4-7
/proc/11796/task/11804/stat : 11804 (plingeling) R 11791 11791 11386 0 -1 4202560 30895 1252 0 0 125393 5 0 0 25 0 5 0 526679424 181514240 16387 33554432000 4194304 4356188 140736076187824 18446744073709551615 4281798 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11796/tid=11805] ppid=11791 vsize=177260 CPUtime=1253.93 cores=4-7
/proc/11796/task/11805/stat : 11805 (plingeling) R 11791 11791 11386 0 -1 4202560 30002 1252 0 0 125389 4 0 0 25 0 5 0 526679424 181514240 16387 33554432000 4194304 4356188 140736076187824 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11796/tid=11806] ppid=11791 vsize=177260 CPUtime=1247.11 cores=4-7
/proc/11796/task/11806/stat : 11806 (plingeling) R 11791 11791 11386 0 -1 4202560 31792 1252 0 0 124706 5 0 0 25 0 5 0 526679425 181514240 16387 33554432000 4194304 4356188 140736076187824 18446744073709551615 4331035 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 10025
Current children cumulated vsize (KiB) 2187496

Child status: 0

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 11791 and gives
#  childrusage.ru_utime.tv_sec=5013
#  childrusage.ru_utime.tv_usec=890772
#  childrusage.ru_stime.tv_sec=2
#  childrusage.ru_stime.tv_usec=389636
# CPU time returned by wait4() is 5016.28
# while last known CPU time is 10025
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1254.08
CPU time (s): 10025
CPU user time (s): 10022.5
CPU system time (s): 2.55
CPU usage (%): 799.395
Max. virtual memory (cumulated for all children) (KiB): 3315700

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 5013.89
system time used= 2.38964
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 768875
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 608
involuntary context switches= 23004

runsolver used 1.9737 second user time and 4.87326 second system time

The end

Launcher Data 

Begin job on node125 at 2011-04-30 10:06:38
IDJOB=3219543
IDBENCH=82607
IDSOLVER=1638
FILE ID=node125/3219543-1304150798
RUNJOBID= node125-1304145216-11400
PBS_JOBID= 13170872
Free space on /tmp= 73020 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/random/large/unif-k5-r20-v1250-c25000-S1245360321-088-UNKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3219543-1304150798/watcher-3219543-1304150798 -o /tmp/evaluation-result-3219543-1304150798/solver-3219543-1304150798 -C 40000 -W 5000 -M 31000  ppfolio HOME/instance-3219543-1304150798.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 951e2fa38b08f2dfbb0dbb3e8ab83f8d
RANDOM SEED=2093178487

node125.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5266.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      23007044 kB
Buffers:       2132424 kB
Cached:        7221944 kB
SwapCached:          4 kB
Active:        2559444 kB
Inactive:      6858328 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      23007044 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:           10496 kB
Writeback:           0 kB
AnonPages:       63536 kB
Mapped:          14764 kB
Slab:           461168 kB
PageTables:       3988 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   183824 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73012 MiB
End job on node125 at 2011-04-30 10:27:32