Trace number 3728718

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
pwbo 2.02MSAT (TO) 1800.03 900.326

General information on the benchmark

Name/PARTIAL-SMALLINT-LIN/wcsp/coloring/
normalized-le450-5a-3_wcsp.wbo
MD5SUM72c9ce0449d9d860fc68cf7a2819c6b0
Bench CategoryPARTIAL-SMALLINT-LIN (both soft and hard constraints, small integers, linear constraints)
Best result obtained on this benchmarkMSAT
Best cost obtained on this benchmark1554
Best CPU time to get the best result obtained on this benchmark1800.08
Max-Satisfiable
Max-(Un)Satisfiability was proved
Best value of the cost
Optimality of the best cost was proved
Number of variables1350
Total number of constraints17592
Number of soft constraints17142
Number of constraints which are clauses17142
Number of constraints which are cardinality constraints (but not clauses)450
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint2
Maximum length of a constraint3
Top cost 5715
Min constraint cost 1
Max constraint cost 1
Sum of constraints costs 17142
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 4
Number of bits of the biggest sum of numbers3
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Quality of the solution as a function of time


Download the above graph as a PDF file / as an EPS file

Solver Data 

0.00/0.00	c ==============================[ Solver Version ]===============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  pwbo:                     2.02 beta - May, 2012                            |
0.00/0.00	c |                                                                             |
0.00/0.00	c Parsing wbo file format. File HOME/instance-3728718-1338675222.wbo.
0.00/0.00	c Instance file HOME/instance-3728718-1338675222.wbo
0.00/0.00	c File size is 500231 bytes.
0.00/0.04	c Highest Coefficient sum: 4
0.00/0.04	c ============================[ Problem Statistics ]=============================
0.00/0.04	c |                                                                             |
0.00/0.04	c |  Parsing time:          0.05         s                                      |
0.00/0.04	c |  Number Variables:      18492                                               |
0.00/0.04	c |  Number Clauses:        17592                                               |
0.00/0.04	c |  Number Cardinality:    450                                                 |
0.00/0.04	c |  Number PB Constraints: 0                                                   |
0.00/0.04	c |                                                                             |
0.00/0.06	c |  Problem Type:          Optimization                                        |
0.00/0.06	c |  Objective Function:    Cardinality Constraint                              |
0.00/0.06	c |                                                                             |
0.00/0.06	c ===============================================================================
1800.03/900.31	o 1708
1800.03/900.31	o 1699
1800.03/900.31	o 1667
1800.03/900.31	o 1571
1800.03/900.31	c ===============================================================================
1800.03/900.31	c Wall Clock Time: 900.31 s
1800.03/900.31	c Best Solution: 1571
1800.03/900.31	s SATISFIABLE
1800.03/900.31	v x1 -x2 -x3 -x4 -x5 x6 -x7 -x8 x9 x10 -x11 -x12 x13 -x14 -x15 -x16 -x17 x18 -x19 -x20 x21 x22 -x23 -x24 x25 -x26 -x27 -x28 x29 -x30 -x31 x32 -x33 -x34 x35 -x36 x37 -x38 -x39 -x40 -x41 x42 -x43 -x44 x45 x46 -x47 -x48 x49 -x50 -x51 -x52 x53 -x54 -x55 -x56 x57 -x58 x59 -x60 -x61 x62 -x63 -x64 -x65 x66 -x67 -x68 x69 -x70 -x71 x72 -x73 -x74 x75 -x76 x77 -x78 x79 -x80 -x81 x82 -x83 -x84 -x85 -x86 x87 x88 -x89 -x90 -x91 -x92 x93 -x94 x95 -x96 x97 -x98 -x99 x100 -x101 -x102 -x103 x104 -x105 x106 -x107 -x108 -x109 x110 -x111 -x112 x113 -x114 -x115 -x116 x117 x118 -x119 -x120 -x121 -x122 x123 -x124 -x125 x126 -x127 -x128 x129 -x130 x131 -x132 x133 -x134 -x135 -x136 -x137 x138 -x139 -x140 x141 x142 -x143 -x144 -x145 x146 -x147 -x148 x149 -x150 -x151 -x152 x153 x154 -x155 -x156 x157 -x158 -x159 -x160 -x161 x162 -x163 x164 -x165 -x166 -x167 x168 -x169 -x170 x171 x172 -x173 -x174 -x175 x176 -x177 -x178 x179 -x180 -x181 -x182 x183 -x184 x185 -x186 -x187 -x188 x189 -x190 -x191 x192 -x193 x194 -x195 -x196 -x197 x198 -x199 x200 -x201 -x202 -x203 x204 -x205 x206 -x207 x208 -x209 -x210 -x211 x212 -x213 x214 -x215 -x216 -x217 x218 -x219 -x220 -x221 x222 -x223 x224 -x225 -x226 x227 -x228 -x229 -x230 x231 x232 -x233 -x234 x235 -x236 -x237 x238 -x239 -x240 -x241 -x242 x243 -x244 x245 -x246 x247 -x248 -x249 -x250 x251 -x252 -x253 -x254 x255 x256 -x257 -x258 -x259 -x260 x261 -x262 x263 -x264 -x265 x266 -x267 -x268 x269 -x270 -x271 x272 -x273 -x274 x275 -x276 -x277 -x278 x279 -x280 -x281 x282 x283 -x284 -x285 -x286 -x287 x288 -x289 x290 -x291 -x292 -x293 x294 x295 -x296 -x297 -x298 x299 -x300 x301 -x302 -x303 -x304 -x305 x306 x307 -x308 -x309 -x310 -x311 x312 -x313 -x314 x315 x316 -x317 -x318 -x319 -x320 x321 -x322 -x323 x324 -x325 -x326 x327 -x328 -x329 x330 -x331 x332 -x333 x334 -x335 -x336 -x337 x338 -x339 -x340 -x341 x342 x343 -x344 -x345 -x346 x347 -x348 -x349 x350 -x351 -x352 -x353 x354 x355 -x356 -x357 x358 -x359 -x360 x361 -x362 -x363 -x364 -x365 x366 -x367 -x368 x369 -x370 -x371 x372 -x373 x374 -x375 -x376 x377 -x378 -x379 -x380 x381 -x382 -x383 x384 -x385 -x386 x387 -x388 x389 -x390 x391 -x392 -x393 -x394 -x395 x396 -x397 -x398 x399 x400 -x401 -x402 x403 -x404 -x405 -x406 x407 -x408 -x409 -x410 x411 -x412 -x413 x414 -x415 x416 -x417 -x418 x419 -x420 -x421 -x422 x423 -x424 x425 -x426 x427 -x428 -x429 -x430 -x431 x432 -x433 -x434 x435 -x436 x437 -x438 -x439 x440 -x441 x442 -x443 -x444 -x445 x446 -x447 -x448 x449 -x450 -x451 -x452 x453 -x454 -x455 x456 -x457 x458 -x459 -x460 -x461 x462 x463 -x464 -x465 x466 -x467 -x468 x469 -x470 -x471 -x472 -x473 x474 x475 -x476 -x477 x478 -x479 -x480 -x481 x482 -x483 -x484 x485 -x486 x487 -x488 -x489 x490 -x491 -x492 -x493 -x494 x495 -x496 x497 -x498 -x499 -x500 x501 -x502 -x503 x504 -x505 -x506 x507 x508 -x509 -x510 -x511 -x512 x513 -x514 x515 -x516 -x517 -x518 x519 x520 -x521 -x522 -x523 x524 -x525 x526 -x527 -x528 x529 -x530 -x531 -x532 -x533 x534 -x535 -x536 x537 -x538 x539 -x540 -x541 x542 -x543 -x544 -x545 x546 -x547 -x548 x549 -x550 x551 -x552 x553 -x554 -x555 -x556 -x557 x558 -x559 -x560 x561 x562 -x563 -x564 -x565 -x566 x567 -x568 -x569 x570 -x571 x572 -x573 -x574 x575 -x576 x577 -x578 -x579 -x580 -x581 x582 x583 -x584 -x585 x586 -x587 -x588 x589 -x590 -x591 -x592 -x593 x594 -x595 x596 -x597 -x598 x599 -x600 -x601 x602 -x603 -x604 -x605 x606 -x607 x608 -x609 x610 -x611 -x612 x613 -x614 -x615 x616 -x617 -x618 -x619 x620 -x621 -x622 x623 -x624 x625 -x626 -x627 x628 -x629 -x630 x631 -x632 -x633 -x634 -x635 x636 -x637 x638 -x639 x640 -x641 -x642 -x643 -x644 x645 -x646 x647 -x648 x649 -x650 -x651 -x652 x653 -x654 -x655 x656 -x657 -x658 -x659 x660 -x661 x662 -x663 -x664 -x665 x666 -x667 -x668 x669 -x670 -x671 x672 -x673 x674 -x675 x676 -x677 -x678 -x679 x680 -x681 -x682 -x683 x684 x685 -x686 -x687 -x688 x689 -x690 x691 -x692 -x693 -x694 x695 -x696 -x697 x698 -x699 -x700 x701 -x702 x703 -x704 -x705 -x706 x707 -x708 -x709 x710 -x711 -x712 -x713 x714 -x715 x716 -x717 -x718 x719 -x720 x721 -x722 -x723 -x724 -x725 x726 -x727 -x728 x729 -x730 -x731 x732 -x733 x734 -x735 x736 -x737 -x738 x739 -x740 -x741 -x742 -x743 x744 x745 -x746 -x747 x748 -x749 -x750 -x751 x752 -x753 -x754 -x755 x756 x757 -x758 -x759 -x760 -x761 x762 -x763 -x764 x765 x766 -x767 -x768 -x769 -x770 x771 x772 -x773 -x774 -x775 x776 -x777 -x778 x779 -x780 x781 -x782 -x783 -x784 -x785 x786 -x787 x788 -x789 x790 -x791 -x792 x793 -x794 -x795 x796 -x797 -x798 -x799 -x800 x801 x802 -x803 -x804 x805 -x806 -x807 x808 -x809 -x810 -x811 x812 -x813 x814 -x815 -x816 -x817 x818 -x819 -x820 x821 -x822 x823 -x824 -x825 -x826 -x827 x828 -x829 x830 -x831 -x832 x833 -x834 -x835 -x836 x837 -x838 x839 -x840 -x841 x842 -x843 -x844 -x845 x846 -x847 -x848 x849 -x850 -x851 x852 -x853 -x854 x855 -x856 -x857 x858 -x859 -x860 x861 -x862 x863 -x864 -x865 x866 -x867 -x868 x869 -x870 -x871 -x872 x873 -x874 -x875 x876 x877 -x878 -x879 -x880 x881 -x882 -x883 x884 -x885 x886 -x887 -x888 -x889 -x890 x891 x892 -x893 -x894 -x895 -x896 x897 x898 -x899 -x900 -x901 -x902 x903 -x904 -x905 x906 x907 -x908 -x909 -x910 -x911 x912 x913 -x914 -x915 -x916 x917 -x918 -x919 -x920 x921 x922 -x923 -x924 -x925 x926 -x927 -x928 -x929 x930 -x931 x932 -x933 x934 -x935 -x936 x937 -x938 -x939 -x940 x941 -x942 -x943 x944 -x945 x946 -x947 -x948 -x949 x950 -x951 -x952 -x953 x954 -x955 -x956 x957 -x958 x959 -x960 x961 -x962 -x963 x964 -x965 -x966 -x967 x968 -x969 -x970 -x971 x972 -x973 x974 -x975 -x976 -x977 x978 -x979 x980 -x981 -x982 -x983 x984 x985 -x986 -x987 -x988 -x989 x990 -x991 -x992 x993 -x994 x995 -x996 -x997 -x998 x999 x1000 -x1001 -x1002 x1003 -x1004 -x1005 -x1006 x1007 -x1008 x1009 -x1010 -x1011 x1012 -x1013 -x1014 x1015 -x1016 -x1017 -x1018 x1019 -x1020 -x1021 x1022 -x1023 -x1024 x1025 -x1026 -x1027 x1028 -x1029 -x1030 x1031 -x1032 -x1033 -x1034 x1035 -x1036 x1037 -x1038 -x1039 x1040 -x1041 -x1042 -x1043 x1044 x1045 -x1046 -x1047 -x1048 -x1049 x1050 x1051 -x1052 -x1053 -x1054 x1055 -x1056 -x1057 x1058 -x1059 x1060 -x1061 -x1062 -x1063 -x1064 x1065 x1066 -x1067 -x1068 x1069 -x1070 -x1071 -x1072 x1073 -x1074 -x1075 x1076 -x1077 x1078 -x1079 -x1080 -x1081 x1082 -x1083 x1084 -x1085 -x1086 -x1087 x1088 -x1089 x1090 -x1091 -x1092 -x1093 -x1094 x1095 -x1096 -x1097 x1098 -x1099 x1100 -x1101 -x1102 -x1103 x1104 -x1105 -x1106 x1107 x1108 -x1109 -x1110 -x1111 -x1112 x1113 -x1114 x1115 -x1116 -x1117 -x1118 x1119 -x1120 x1121 -x1122 x1123 -x1124 -x1125 -x1126 -x1127 x1128 -x1129 x1130 -x1131 x1132 -x1133 -x1134 x1135 -x1136 -x1137 x1138 -x1139 -x1140 x1141 -x1142 -x1143 -x1144 x1145 -x1146 x1147 -x1148 -x1149 -x1150 -x1151 x1152 -x1153 -x1154 x1155 x1156 -x1157 -x1158 -x1159 x1160 -x1161 -x1162 x1163 -x1164 -x1165 -x1166 x1167 -x1168 -x1169 x1170 x1171 -x1172 -x1173 -x1174 -x1175 x1176 -x1177 x1178 -x1179 -x1180 x1181 -x1182 x1183 -x1184 -x1185 -x1186 -x1187 x1188 -x1189 -x1190 x1191 -x1192 x1193 -x1194 -x1195 -x1196 x1197 -x1198 x1199 -x1200 -x1201 x1202 -x1203 -x1204 x1205 -x1206 -x1207 -x1208 x1209 -x1210 -x1211 x1212 -x1213 x1214 -x1215 x1216 -x1217 -x1218 -x1219 -x1220 x1221 -x1222 -x1223 x1224 -x1225 x1226 -x1227 -x1228 x1229 -x1230 -x1231 x1232 -x1233 -x1234 x1235 -x1236 x1237 -x1238 -x1239 -x1240 -x1241 x1242 -x1243 -x1244 x1245 -x1246 x1247 -x1248 -x1249 x1250 -x1251 x1252 -x1253 -x1254 -x1255 x1256 -x1257 -x1258 x1259 -x1260 -x1261 x1262 -x1263 -x1264 -x1265 x1266 -x1267 x1268 -x1269 -x1270 x1271 -x1272 -x1273 -x1274 x1275 -x1276 x1277 -x1278 -x1279 x1280 -x1281 x1282 -x1283 -x1284 x1285 -x1286 -x1287 -x1288 x1289 -x1290 -x1291 -x1292 x1293 -x1294 x1295 -x1296 -x1297 -x1298 x1299 -x1300 x1301 -x1302 x1303 -x1304 -x1305 x1306 -x1307 -x1308 -x1309 x1310 -x1311 -x1312 -x1313 x1314 -x1315 -x1316 x1317 -x1318 -x1319 x1320 -x1321 -x1322 x1323 -x1324 x1325 -x1326 -x1327 x1328 -x1329 -x1330 x1331 -x1332 x1333 -x1334 -x1335 -x1336 x1337 -x1338 -x1339 -x1340 x1341 -x1342 -x1343 x1344 x1345 -x1346 -x1347 -x1348 x1349 -x1350

Verifier Data 

OK	1571

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3728718-1338675222/watcher-3728718-1338675222 -o /tmp/evaluation-result-3728718-1338675222/solver-3728718-1338675222 -C 1800 -W 1900 -M 15500 pwbo2.02 -file-format=wbo -verbosity=0 HOME/instance-3728718-1338675222.wbo 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.84 3.28 2.28 5/178 31274
/proc/meminfo: memFree=27362756/32873844 swapFree=7112/7112
[pid=31274] ppid=31272 vsize=5756 CPUtime=0 cores=1,3,5,7
/proc/31274/stat : 31274 (pwbo2.02) R 31272 31274 30511 0 -1 4202496 632 0 0 0 0 0 0 0 20 0 1 0 333647882 5894144 585 33554432000 4194304 5964478 140734669827248 140735152961096 4395529 0 0 4096 16387 0 0 0 17 5 0 0 0 0 0
/proc/31274/statm: 1439 586 171 433 0 880 0

[startup+0.098378 s]
/proc/loadavg: 3.84 3.28 2.28 5/178 31274
/proc/meminfo: memFree=27362756/32873844 swapFree=7112/7112
[pid=31274] ppid=31272 vsize=177824 CPUtime=0.13 cores=1,3,5,7
/proc/31274/stat : 31274 (pwbo2.02) S 31272 31274 30511 0 -1 4202496 9182 0 0 0 11 2 0 0 20 0 3 0 333647882 182091776 8503 33554432000 4194304 5964478 140734669827248 140734669826768 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/31274/statm: 44456 8512 205 433 0 44020 0
Current children cumulated CPU time (s) 0.13
Current children cumulated vsize (KiB) 177824

[startup+0.100252 s]
/proc/loadavg: 3.84 3.28 2.28 5/178 31274
/proc/meminfo: memFree=27362756/32873844 swapFree=7112/7112
[pid=31274] ppid=31272 vsize=178544 CPUtime=0.13 cores=1,3,5,7
/proc/31274/stat : 31274 (pwbo2.02) S 31272 31274 30511 0 -1 4202496 9492 0 0 0 11 2 0 0 20 0 3 0 333647882 182829056 8813 33554432000 4194304 5964478 140734669827248 140734669826768 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/31274/statm: 44636 8819 205 433 0 44200 0
Current children cumulated CPU time (s) 0.13
Current children cumulated vsize (KiB) 178544

[startup+0.300165 s]
/proc/loadavg: 3.84 3.28 2.28 5/178 31274
/proc/meminfo: memFree=27362756/32873844 swapFree=7112/7112
[pid=31274] ppid=31272 vsize=219540 CPUtime=0.52 cores=1,3,5,7
/proc/31274/stat : 31274 (pwbo2.02) S 31272 31274 30511 0 -1 4202496 28237 0 0 0 45 7 0 0 20 0 3 0 333647882 224808960 28147 33554432000 4194304 5964478 140734669827248 140734669826768 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/31274/statm: 54885 28153 207 433 0 54449 0
Current children cumulated CPU time (s) 0.52
Current children cumulated vsize (KiB) 219540

[startup+0.700164 s]
/proc/loadavg: 3.84 3.28 2.28 5/178 31274
/proc/meminfo: memFree=27362756/32873844 swapFree=7112/7112
[pid=31274] ppid=31272 vsize=373940 CPUtime=1.32 cores=1,3,5,7
/proc/31274/stat : 31274 (pwbo2.02) S 31272 31274 30511 0 -1 4202496 58326 0 0 0 115 17 0 0 20 0 3 0 333647882 382914560 68117 33554432000 4194304 5964478 140734669827248 140734669826768 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/31274/statm: 93485 68117 207 433 0 93049 0
Current children cumulated CPU time (s) 1.32
Current children cumulated vsize (KiB) 373940

[startup+1.5004 s]
/proc/loadavg: 3.84 3.28 2.28 5/181 31277
/proc/meminfo: memFree=26973432/32873844 swapFree=7112/7112
[pid=31274] ppid=31272 vsize=677252 CPUtime=2.91 cores=1,3,5,7
/proc/31274/stat : 31274 (pwbo2.02) S 31272 31274 30511 0 -1 4202496 96434 0 0 0 259 32 0 0 20 0 3 0 333647882 693506048 119768 33554432000 4194304 5964478 140734669827248 140734669826768 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/31274/statm: 169313 119768 207 433 0 168877 0
[pid=31274/tid=31276] ppid=31272 vsize=677252 CPUtime=1.42 cores=1,3,5,7
/proc/31274/task/31276/stat : 31276 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 11193 0 0 0 139 3 0 0 20 0 3 0 333647887 693506048 119768 33554432000 4194304 5964478 140734669827248 140340999952920 4222416 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=31274/tid=31277] ppid=31272 vsize=677252 CPUtime=1.42 cores=1,3,5,7
/proc/31274/task/31277/stat : 31277 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 82426 0 0 0 113 29 0 0 20 0 3 0 333647887 693506048 119768 33554432000 4194304 5964478 140734669827248 140340989462888 4400760 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 2.91
Current children cumulated vsize (KiB) 677252

[startup+3.10045 s]
/proc/loadavg: 3.84 3.28 2.28 5/181 31277
/proc/meminfo: memFree=26875240/32873844 swapFree=7112/7112
[pid=31274] ppid=31272 vsize=683744 CPUtime=6.11 cores=1,3,5,7
/proc/31274/stat : 31274 (pwbo2.02) S 31272 31274 30511 0 -1 4202496 104454 0 0 0 577 34 0 0 20 0 3 0 333647882 700153856 122350 33554432000 4194304 5964478 140734669827248 140734669826768 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/31274/statm: 170936 122350 207 433 0 170500 0
[pid=31274/tid=31276] ppid=31272 vsize=683744 CPUtime=3.02 cores=1,3,5,7
/proc/31274/task/31276/stat : 31276 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 18184 0 0 0 298 4 0 0 20 0 3 0 333647887 700153856 122350 33554432000 4194304 5964478 140734669827248 140340999952920 5139166 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=31274/tid=31277] ppid=31272 vsize=683744 CPUtime=3.02 cores=1,3,5,7
/proc/31274/task/31277/stat : 31277 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 83455 0 0 0 272 30 0 0 20 0 3 0 333647887 700153856 122350 33554432000 4194304 5964478 140734669827248 140340989462760 4393005 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 6.11
Current children cumulated vsize (KiB) 683744

[startup+6.30044 s]
/proc/loadavg: 3.85 3.29 2.29 5/181 31277
/proc/meminfo: memFree=26855648/32873844 swapFree=7112/7112
[pid=31274] ppid=31272 vsize=683744 CPUtime=12.51 cores=1,3,5,7
/proc/31274/stat : 31274 (pwbo2.02) S 31272 31274 30511 0 -1 4202496 116229 0 0 0 1215 36 0 0 20 0 3 0 333647882 700153856 124826 33554432000 4194304 5964478 140734669827248 140734669826768 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/31274/statm: 170936 124826 207 433 0 170500 0
[pid=31274/tid=31276] ppid=31272 vsize=683744 CPUtime=6.21 cores=1,3,5,7
/proc/31274/task/31276/stat : 31276 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 29803 0 0 0 616 5 0 0 20 0 3 0 333647887 700153856 124826 33554432000 4194304 5964478 140734669827248 140340999953224 4399854 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=31274/tid=31277] ppid=31272 vsize=683744 CPUtime=6.21 cores=1,3,5,7
/proc/31274/task/31277/stat : 31277 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 83611 0 0 0 591 30 0 0 20 0 3 0 333647887 700153856 124826 33554432000 4194304 5964478 140734669827248 140340989462760 4399915 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 12.51
Current children cumulated vsize (KiB) 683744

[startup+12.7004 s]
/proc/loadavg: 3.86 3.30 2.29 5/181 31277
/proc/meminfo: memFree=26823248/32873844 swapFree=7112/7112
[pid=31274] ppid=31272 vsize=683744 CPUtime=25.29 cores=1,3,5,7
/proc/31274/stat : 31274 (pwbo2.02) S 31272 31274 30511 0 -1 4202496 131189 0 0 0 2490 39 0 0 20 0 3 0 333647882 700153856 129773 33554432000 4194304 5964478 140734669827248 140734669826768 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/31274/statm: 170936 129773 207 433 0 170500 0
[pid=31274/tid=31276] ppid=31272 vsize=683744 CPUtime=12.59 cores=1,3,5,7
/proc/31274/task/31276/stat : 31276 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 44134 0 0 0 1251 8 0 0 20 0 3 0 333647887 700153856 129773 33554432000 4194304 5964478 140734669827248 140340999952728 4399920 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=31274/tid=31277] ppid=31272 vsize=683744 CPUtime=12.6 cores=1,3,5,7
/proc/31274/task/31277/stat : 31277 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 84240 0 0 0 1230 30 0 0 20 0 3 0 333647887 700153856 129773 33554432000 4194304 5964478 140734669827248 140340989462760 4400087 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 25.29
Current children cumulated vsize (KiB) 683744

[startup+25.5004 s]
/proc/loadavg: 3.89 3.33 2.32 5/181 31277
/proc/meminfo: memFree=26790704/32873844 swapFree=7112/7112
[pid=31274] ppid=31272 vsize=688072 CPUtime=50.9 cores=1,3,5,7
/proc/31274/stat : 31274 (pwbo2.02) S 31272 31274 30511 0 -1 4202496 135085 0 0 0 5049 41 0 0 20 0 3 0 333647882 704585728 132122 33554432000 4194304 5964478 140734669827248 140734669826768 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/31274/statm: 172018 132122 207 433 0 171582 0
[pid=31274/tid=31276] ppid=31272 vsize=688072 CPUtime=25.35 cores=1,3,5,7
/proc/31274/task/31276/stat : 31276 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 47749 0 0 0 2526 9 0 0 20 0 3 0 333647887 704585728 132122 33554432000 4194304 5964478 140734669827248 140340999953336 4387951 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=31274/tid=31277] ppid=31272 vsize=688072 CPUtime=25.36 cores=1,3,5,7
/proc/31274/task/31277/stat : 31277 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 84521 0 0 0 2506 30 0 0 20 0 3 0 333647887 704585728 132122 33554432000 4194304 5964478 140734669827248 140340989462760 4403343 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 50.9
Current children cumulated vsize (KiB) 688072

[startup+51.1005 s]
/proc/loadavg: 3.93 3.39 2.36 5/181 31277
/proc/meminfo: memFree=26724564/32873844 swapFree=7112/7112
[pid=31274] ppid=31272 vsize=688072 CPUtime=102.08 cores=1,3,5,7
/proc/31274/stat : 31274 (pwbo2.02) S 31272 31274 30511 0 -1 4202496 135617 0 0 0 10167 41 0 0 20 0 3 0 333647882 704585728 132654 33554432000 4194304 5964478 140734669827248 140734669826768 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/31274/statm: 172018 132654 207 433 0 171582 0
[pid=31274/tid=31276] ppid=31272 vsize=688072 CPUtime=50.88 cores=1,3,5,7
/proc/31274/task/31276/stat : 31276 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 47750 0 0 0 5079 9 0 0 20 0 3 0 333647887 704585728 132654 33554432000 4194304 5964478 140734669827248 140340999953336 4400519 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=31274/tid=31277] ppid=31272 vsize=688072 CPUtime=50.9 cores=1,3,5,7
/proc/31274/task/31277/stat : 31277 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 85052 0 0 0 5059 31 0 0 20 0 3 0 333647887 704585728 132654 33554432000 4194304 5964478 140734669827248 140340989462744 4392918 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 102.08
Current children cumulated vsize (KiB) 688072

[startup+102.307 s]
/proc/loadavg: 3.97 3.48 2.45 5/181 31277
/proc/meminfo: memFree=26706196/32873844 swapFree=7112/7112
[pid=31274] ppid=31272 vsize=749280 CPUtime=204.47 cores=1,3,5,7
/proc/31274/stat : 31274 (pwbo2.02) S 31272 31274 30511 0 -1 4202496 141564 0 0 0 20401 46 0 0 20 0 3 0 333647882 767262720 137702 33554432000 4194304 5964478 140734669827248 140734669826768 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/31274/statm: 187320 137702 207 433 0 186884 0
[pid=31274/tid=31276] ppid=31272 vsize=749280 CPUtime=101.95 cores=1,3,5,7
/proc/31274/task/31276/stat : 31276 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 53205 0 0 0 10182 13 0 0 20 0 3 0 333647887 767262720 137702 33554432000 4194304 5964478 140734669827248 140340999953336 4403406 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=31274/tid=31277] ppid=31272 vsize=749280 CPUtime=101.97 cores=1,3,5,7
/proc/31274/task/31277/stat : 31277 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 85544 0 0 0 10164 33 0 0 20 0 3 0 333647887 767262720 137702 33554432000 4194304 5964478 140734669827248 140340989462840 4399854 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 204.47
Current children cumulated vsize (KiB) 749280

[startup+162.3 s]
/proc/loadavg: 3.99 3.57 2.54 5/181 31277
/proc/meminfo: memFree=26695064/32873844 swapFree=7112/7112
[pid=31274] ppid=31272 vsize=749280 CPUtime=324.45 cores=1,3,5,7
/proc/31274/stat : 31274 (pwbo2.02) S 31272 31274 30511 0 -1 4202496 144304 0 0 0 32394 51 0 0 20 0 3 0 333647882 767262720 140442 33554432000 4194304 5964478 140734669827248 140734669826768 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/31274/statm: 187320 140442 207 433 0 186884 0
[pid=31274/tid=31276] ppid=31272 vsize=749280 CPUtime=161.79 cores=1,3,5,7
/proc/31274/task/31276/stat : 31276 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 55607 0 0 0 16161 18 0 0 20 0 3 0 333647887 767262720 140442 33554432000 4194304 5964478 140734669827248 140340999952184 4409285 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=31274/tid=31277] ppid=31272 vsize=749280 CPUtime=161.81 cores=1,3,5,7
/proc/31274/task/31277/stat : 31277 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 85882 0 0 0 16145 36 0 0 20 0 3 0 333647887 767262720 140442 33554432000 4194304 5964478 140734669827248 140340989462648 4400678 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 324.45
Current children cumulated vsize (KiB) 749280

[startup+222.3 s]
/proc/loadavg: 3.99 3.65 2.63 5/181 31278
/proc/meminfo: memFree=26589320/32873844 swapFree=7112/7112
[pid=31274] ppid=31272 vsize=749280 CPUtime=444.42 cores=1,3,5,7
/proc/31274/stat : 31274 (pwbo2.02) S 31272 31274 30511 0 -1 4202496 149288 0 0 0 44385 57 0 0 20 0 3 0 333647882 767262720 145426 33554432000 4194304 5964478 140734669827248 140734669826768 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/31274/statm: 187320 145426 207 433 0 186884 0
[pid=31274/tid=31276] ppid=31272 vsize=749280 CPUtime=221.64 cores=1,3,5,7
/proc/31274/task/31276/stat : 31276 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 60158 0 0 0 22139 25 0 0 20 0 3 0 333647887 767262720 145426 33554432000 4194304 5964478 140734669827248 140340999952184 4399920 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=31274/tid=31277] ppid=31272 vsize=749280 CPUtime=221.66 cores=1,3,5,7
/proc/31274/task/31277/stat : 31277 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 86315 0 0 0 22129 37 0 0 20 0 3 0 333647887 767262720 145426 33554432000 4194304 5964478 140734669827248 140340989462888 4399920 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 444.42
Current children cumulated vsize (KiB) 749280

[startup+282.3 s]
/proc/loadavg: 3.99 3.71 2.71 5/181 31278
/proc/meminfo: memFree=26519692/32873844 swapFree=7112/7112
[pid=31274] ppid=31272 vsize=817160 CPUtime=564.4 cores=1,3,5,7
/proc/31274/stat : 31274 (pwbo2.02) S 31272 31274 30511 0 -1 4202496 158206 0 0 0 56370 70 0 0 20 0 3 0 333647882 836771840 154336 33554432000 4194304 5964478 140734669827248 140734669826768 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/31274/statm: 204290 154336 207 433 0 203854 0
[pid=31274/tid=31276] ppid=31272 vsize=817160 CPUtime=281.48 cores=1,3,5,7
/proc/31274/task/31276/stat : 31276 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 68085 0 0 0 28110 38 0 0 20 0 3 0 333647887 836771840 154336 33554432000 4194304 5964478 140734669827248 140340999952184 4399920 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=31274/tid=31277] ppid=31272 vsize=817160 CPUtime=281.5 cores=1,3,5,7
/proc/31274/task/31277/stat : 31277 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 87306 0 0 0 28111 39 0 0 20 0 3 0 333647887 836771840 154336 33554432000 4194304 5964478 140734669827248 140340989462888 4399810 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 564.4
Current children cumulated vsize (KiB) 817160

[startup+342.3 s]
/proc/loadavg: 3.99 3.76 2.79 5/181 31278
/proc/meminfo: memFree=26373464/32873844 swapFree=7112/7112
[pid=31274] ppid=31272 vsize=817160 CPUtime=684.35 cores=1,3,5,7
/proc/31274/stat : 31274 (pwbo2.02) S 31272 31274 30511 0 -1 4202496 163511 0 0 0 68358 77 0 0 20 0 3 0 333647882 836771840 160254 33554432000 4194304 5964478 140734669827248 140734669826768 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/31274/statm: 204290 160254 207 433 0 203854 0

################
# More data... #
################

[pid=31274/tid=31277] ppid=31272 vsize=817160 CPUtime=341.34 cores=1,3,5,7
/proc/31274/task/31277/stat : 31277 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 87760 0 0 0 34093 41 0 0 20 0 3 0 333647887 836771840 160254 33554432000 4194304 5964478 140734669827248 140340989462888 4399920 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 684.35
Current children cumulated vsize (KiB) 817160

[startup+402.3 s]
/proc/loadavg: 3.99 3.80 2.86 5/182 31281
/proc/meminfo: memFree=26165376/32873844 swapFree=7112/7112
[pid=31274] ppid=31272 vsize=817292 CPUtime=804.31 cores=1,3,5,7
/proc/31274/stat : 31274 (pwbo2.02) S 31272 31274 30511 0 -1 4202496 171857 0 0 0 80341 90 0 0 20 0 3 0 333647882 836907008 168568 33554432000 4194304 5964478 140734669827248 140734669826768 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/31274/statm: 204323 168568 207 433 0 203887 0
[pid=31274/tid=31276] ppid=31272 vsize=817292 CPUtime=401.13 cores=1,3,5,7
/proc/31274/task/31276/stat : 31276 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 80866 0 0 0 40054 59 0 0 20 0 3 0 333647887 836907008 168568 33554432000 4194304 5964478 140734669827248 140340999952424 4399920 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=31274/tid=31277] ppid=31272 vsize=817292 CPUtime=401.17 cores=1,3,5,7
/proc/31274/task/31277/stat : 31277 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 88176 0 0 0 40075 42 0 0 20 0 3 0 333647887 836907008 168568 33554432000 4194304 5964478 140734669827248 140340989462888 4400010 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 804.31
Current children cumulated vsize (KiB) 817292

[startup+462.3 s]
/proc/loadavg: 3.99 3.83 2.93 5/181 31281
/proc/meminfo: memFree=26019568/32873844 swapFree=7112/7112
[pid=31274] ppid=31272 vsize=819992 CPUtime=924.29 cores=1,3,5,7
/proc/31274/stat : 31274 (pwbo2.02) S 31272 31274 30511 0 -1 4202496 173062 0 0 0 92337 92 0 0 20 0 3 0 333647882 839671808 170264 33554432000 4194304 5964478 140734669827248 140734669826768 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/31274/statm: 204998 170264 207 433 0 204562 0
[pid=31274/tid=31276] ppid=31272 vsize=819992 CPUtime=460.98 cores=1,3,5,7
/proc/31274/task/31276/stat : 31276 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 81474 0 0 0 46038 60 0 0 20 0 3 0 333647887 839671808 170264 33554432000 4194304 5964478 140734669827248 140340999952184 4399915 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=31274/tid=31277] ppid=31272 vsize=819992 CPUtime=461 cores=1,3,5,7
/proc/31274/task/31277/stat : 31277 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 88773 0 0 0 46056 44 0 0 20 0 3 0 333647887 839671808 170264 33554432000 4194304 5964478 140734669827248 140340989462888 4392863 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 924.29
Current children cumulated vsize (KiB) 819992

[startup+522.3 s]
/proc/loadavg: 2.79 3.52 2.88 3/173 31307
/proc/meminfo: memFree=28040020/32873844 swapFree=7112/7112
[pid=31274] ppid=31272 vsize=885796 CPUtime=1044.27 cores=1,3,5,7
/proc/31274/stat : 31274 (pwbo2.02) S 31272 31274 30511 0 -1 4202496 178418 0 0 0 104325 102 0 0 20 0 3 0 333647882 907055104 175732 33554432000 4194304 5964478 140734669827248 140734669826768 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/31274/statm: 221449 175732 207 433 0 221013 0
[pid=31274/tid=31276] ppid=31272 vsize=885796 CPUtime=520.82 cores=1,3,5,7
/proc/31274/task/31276/stat : 31276 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 86378 0 0 0 52012 70 0 0 20 0 3 0 333647887 907055104 175732 33554432000 4194304 5964478 140734669827248 140340999952376 4399920 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=31274/tid=31277] ppid=31272 vsize=885796 CPUtime=520.85 cores=1,3,5,7
/proc/31274/task/31277/stat : 31277 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 89225 0 0 0 52040 45 0 0 20 0 3 0 333647887 907055104 175732 33554432000 4194304 5964478 140734669827248 140340989462840 4392882 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1044.27
Current children cumulated vsize (KiB) 885796

[startup+582.3 s]
/proc/loadavg: 2.29 3.24 2.82 3/173 31307
/proc/meminfo: memFree=28008684/32873844 swapFree=7112/7112
[pid=31274] ppid=31272 vsize=886068 CPUtime=1164.23 cores=1,3,5,7
/proc/31274/stat : 31274 (pwbo2.02) S 31272 31274 30511 0 -1 4202496 186253 0 0 0 116307 116 0 0 20 0 3 0 333647882 907333632 183538 33554432000 4194304 5964478 140734669827248 140734669826768 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/31274/statm: 221517 183538 207 433 0 221081 0
[pid=31274/tid=31276] ppid=31272 vsize=886068 CPUtime=580.66 cores=1,3,5,7
/proc/31274/task/31276/stat : 31276 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 93702 0 0 0 57982 84 0 0 20 0 3 0 333647887 907333632 183538 33554432000 4194304 5964478 140734669827248 140340999952376 4399920 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=31274/tid=31277] ppid=31272 vsize=886068 CPUtime=580.67 cores=1,3,5,7
/proc/31274/task/31277/stat : 31277 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 89736 0 0 0 58021 46 0 0 20 0 3 0 333647887 907333632 183538 33554432000 4194304 5964478 140734669827248 140340989462888 4392918 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1164.23
Current children cumulated vsize (KiB) 886068

[startup+642.3 s]
/proc/loadavg: 2.10 3.02 2.77 3/173 31307
/proc/meminfo: memFree=27983484/32873844 swapFree=7112/7112
[pid=31274] ppid=31272 vsize=952004 CPUtime=1284.2 cores=1,3,5,7
/proc/31274/stat : 31274 (pwbo2.02) S 31272 31274 30511 0 -1 4202496 192386 0 0 0 128294 126 0 0 20 0 3 0 333647882 974852096 189745 33554432000 4194304 5964478 140734669827248 140734669826768 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/31274/statm: 238001 189745 207 433 0 237565 0
[pid=31274/tid=31276] ppid=31272 vsize=952004 CPUtime=640.51 cores=1,3,5,7
/proc/31274/task/31276/stat : 31276 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 99286 0 0 0 63957 94 0 0 20 0 3 0 333647887 974852096 189745 33554432000 4194304 5964478 140734669827248 140340999952376 4379296 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=31274/tid=31277] ppid=31272 vsize=952004 CPUtime=640.51 cores=1,3,5,7
/proc/31274/task/31277/stat : 31277 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 90285 0 0 0 64003 48 0 0 20 0 3 0 333647887 974852096 189745 33554432000 4194304 5964478 140734669827248 140340989462888 4399920 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1284.2
Current children cumulated vsize (KiB) 952004

[startup+702.3 s]
/proc/loadavg: 2.04 2.83 2.72 3/173 31307
/proc/meminfo: memFree=27979872/32873844 swapFree=7112/7112
[pid=31274] ppid=31272 vsize=952136 CPUtime=1404.14 cores=1,3,5,7
/proc/31274/stat : 31274 (pwbo2.02) S 31272 31274 30511 0 -1 4202496 193030 0 0 0 140288 126 0 0 20 0 3 0 333647882 974987264 190616 33554432000 4194304 5964478 140734669827248 140734669826768 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/31274/statm: 238034 190616 207 433 0 237598 0
[pid=31274/tid=31276] ppid=31272 vsize=952136 CPUtime=700.33 cores=1,3,5,7
/proc/31274/task/31276/stat : 31276 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 99332 0 0 0 69938 95 0 0 20 0 3 0 333647887 974987264 190616 33554432000 4194304 5964478 140734669827248 140340999952184 4392863 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=31274/tid=31277] ppid=31272 vsize=952136 CPUtime=700.34 cores=1,3,5,7
/proc/31274/task/31277/stat : 31277 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 90883 0 0 0 69984 50 0 0 20 0 3 0 333647887 974987264 190616 33554432000 4194304 5964478 140734669827248 140340989462888 4403343 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1404.14
Current children cumulated vsize (KiB) 952136

[startup+762.307 s]
/proc/loadavg: 2.01 2.67 2.67 3/173 31307
/proc/meminfo: memFree=27973384/32873844 swapFree=7112/7112
[pid=31274] ppid=31272 vsize=952400 CPUtime=1524.11 cores=1,3,5,7
/proc/31274/stat : 31274 (pwbo2.02) S 31272 31274 30511 0 -1 4202496 193620 0 0 0 152284 127 0 0 20 0 3 0 333647882 975257600 192226 33554432000 4194304 5964478 140734669827248 140734669826768 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/31274/statm: 238100 192226 207 433 0 237664 0
[pid=31274/tid=31276] ppid=31272 vsize=952400 CPUtime=760.17 cores=1,3,5,7
/proc/31274/task/31276/stat : 31276 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 99559 0 0 0 75921 96 0 0 20 0 3 0 333647887 975257600 192226 33554432000 4194304 5964478 140734669827248 140340999952184 4399920 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=31274/tid=31277] ppid=31272 vsize=952400 CPUtime=760.2 cores=1,3,5,7
/proc/31274/task/31277/stat : 31277 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 91246 0 0 0 75968 52 0 0 20 0 3 0 333647887 975257600 192226 33554432000 4194304 5964478 140734669827248 140340989462648 4392918 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1524.11
Current children cumulated vsize (KiB) 952400

[startup+822.3 s]
/proc/loadavg: 2.06 2.56 2.63 3/174 31309
/proc/meminfo: memFree=27915012/32873844 swapFree=7112/7112
[pid=31274] ppid=31272 vsize=950232 CPUtime=1644.05 cores=1,3,5,7
/proc/31274/stat : 31274 (pwbo2.02) S 31272 31274 30511 0 -1 4202496 208501 0 0 0 164265 140 0 0 20 0 3 0 333647882 973037568 206643 33554432000 4194304 5964478 140734669827248 140734669826768 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/31274/statm: 237558 206643 207 433 0 237122 0
[pid=31274/tid=31276] ppid=31272 vsize=950232 CPUtime=819.97 cores=1,3,5,7
/proc/31274/task/31276/stat : 31276 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 113791 0 0 0 81888 109 0 0 20 0 3 0 333647887 973037568 206643 33554432000 4194304 5964478 140734669827248 140340999953336 4410877 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=31274/tid=31277] ppid=31272 vsize=950232 CPUtime=820.04 cores=1,3,5,7
/proc/31274/task/31277/stat : 31277 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 91895 0 0 0 81950 54 0 0 20 0 3 0 333647887 973037568 206643 33554432000 4194304 5964478 140734669827248 140340989462840 4399915 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1644.05
Current children cumulated vsize (KiB) 950232

[startup+882.3 s]
/proc/loadavg: 2.02 2.46 2.59 3/173 31309
/proc/meminfo: memFree=27913532/32873844 swapFree=7112/7112
[pid=31274] ppid=31272 vsize=950500 CPUtime=1764.03 cores=1,3,5,7
/proc/31274/stat : 31274 (pwbo2.02) S 31272 31274 30511 0 -1 4202496 208930 0 0 0 176262 141 0 0 20 0 3 0 333647882 973312000 207071 33554432000 4194304 5964478 140734669827248 140734669826768 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/31274/statm: 237625 207071 207 433 0 237189 0
[pid=31274/tid=31276] ppid=31272 vsize=950500 CPUtime=879.82 cores=1,3,5,7
/proc/31274/task/31276/stat : 31276 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 113791 0 0 0 87872 110 0 0 20 0 3 0 333647887 973312000 207071 33554432000 4194304 5964478 140734669827248 140340999953336 4399920 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=31274/tid=31277] ppid=31272 vsize=950500 CPUtime=879.88 cores=1,3,5,7
/proc/31274/task/31277/stat : 31277 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 92324 0 0 0 87932 56 0 0 20 0 3 0 333647887 973312000 207071 33554432000 4194304 5964478 140734669827248 140340989462840 4403343 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1764.03
Current children cumulated vsize (KiB) 950500



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+900.3 s]
/proc/loadavg: 2.01 2.43 2.57 3/173 31309
/proc/meminfo: memFree=27912788/32873844 swapFree=7112/7112
[pid=31274] ppid=31272 vsize=950636 CPUtime=1800.03 cores=1,3,5,7
/proc/31274/stat : 31274 (pwbo2.02) S 31272 31274 30511 0 -1 4202496 209071 0 0 0 179862 141 0 0 20 0 3 0 333647882 973451264 207212 33554432000 4194304 5964478 140734669827248 140734669826768 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/31274/statm: 237659 207212 207 433 0 237223 0
[pid=31274/tid=31276] ppid=31272 vsize=950636 CPUtime=897.77 cores=1,3,5,7
/proc/31274/task/31276/stat : 31276 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 113791 0 0 0 89667 110 0 0 20 0 3 0 333647887 973451264 207212 33554432000 4194304 5964478 140734669827248 140340999953336 4389085 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=31274/tid=31277] ppid=31272 vsize=950636 CPUtime=897.84 cores=1,3,5,7
/proc/31274/task/31277/stat : 31277 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 92465 0 0 0 89728 56 0 0 20 0 3 0 333647887 973451264 207212 33554432000 4194304 5964478 140734669827248 140340989462648 4399854 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1800.03
Current children cumulated vsize (KiB) 950636

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+900.3 s]
/proc/loadavg: 2.01 2.43 2.57 3/173 31309
/proc/meminfo: memFree=27912788/32873844 swapFree=7112/7112
[pid=31274] ppid=31272 vsize=950636 CPUtime=1800.03 cores=1,3,5,7
/proc/31274/stat : 31274 (pwbo2.02) S 31272 31274 30511 0 -1 4202496 209071 0 0 0 179862 141 0 0 20 0 3 0 333647882 973451264 207212 33554432000 4194304 5964478 140734669827248 140734669826768 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/31274/statm: 237659 207212 207 433 0 237223 0
[pid=31274/tid=31276] ppid=31272 vsize=950636 CPUtime=897.77 cores=1,3,5,7
/proc/31274/task/31276/stat : 31276 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 113791 0 0 0 89667 110 0 0 20 0 3 0 333647887 973451264 207212 33554432000 4194304 5964478 140734669827248 140340999953336 4389085 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=31274/tid=31277] ppid=31272 vsize=950636 CPUtime=897.84 cores=1,3,5,7
/proc/31274/task/31277/stat : 31277 (pwbo2.02) R 31272 31274 30511 0 -1 4202560 92465 0 0 0 89728 56 0 0 20 0 3 0 333647887 973451264 207212 33554432000 4194304 5964478 140734669827248 140340989462648 4399854 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1800.03
Current children cumulated vsize (KiB) 950636

Child status: 0

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 31274 and gives
#  childrusage.ru_utime.tv_sec=1794
#  childrusage.ru_utime.tv_usec=17267
#  childrusage.ru_stime.tv_sec=1
#  childrusage.ru_stime.tv_usec=696742
# CPU time returned by wait4() is 1795.71
# while last known CPU time is 1800.03
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 900.326
CPU time (s): 1800.03
CPU user time (s): 1798.62
CPU system time (s): 1.41
CPU usage (%): 199.931
Max. virtual memory (cumulated for all children) (KiB): 952664

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1794.02
system time used= 1.69674
maximum resident set size= 828856
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 209073
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1697
involuntary context switches= 3590

runsolver used 2.16167 second user time and 5.2342 second system time

The end

Launcher Data 

Begin job on node119 at 2012-06-03 00:13:42
IDJOB=3728718
IDBENCH=79661
IDSOLVER=2316
FILE ID=node119/3728718-1338675222
RUNJOBID= node119-1338674785-31040
PBS_JOBID= 14636522
Free space on /tmp= 70892 MiB

SOLVER NAME= pwbo 2.02
BENCH NAME= PB10/normalized-PB10/PARTIAL-SMALLINT-LIN/wcsp/coloring/normalized-le450-5a-3_wcsp.wbo
COMMAND LINE= pwbo2.02 -file-format=wbo -verbosity=0 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3728718-1338675222/watcher-3728718-1338675222 -o /tmp/evaluation-result-3728718-1338675222/solver-3728718-1338675222 -C 1800 -W 1900 -M 15500  pwbo2.02 -file-format=wbo -verbosity=0 HOME/instance-3728718-1338675222.wbo

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 72c9ce0449d9d860fc68cf7a2819c6b0
RANDOM SEED=723179836

node119.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.943
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.88
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.943
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.943
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.93
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.943
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.943
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.93
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.943
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.943
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.943
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        27363036 kB
Buffers:          389484 kB
Cached:          2919072 kB
SwapCached:         1936 kB
Active:          2808096 kB
Inactive:        1879200 kB
Active(anon):    1374092 kB
Inactive(anon):     6636 kB
Active(file):    1434004 kB
Inactive(file):  1872564 kB
Unevictable:        7112 kB
Mlocked:            7112 kB
SwapTotal:      67108856 kB
SwapFree:       67100224 kB
Dirty:              5832 kB
Writeback:             0 kB
AnonPages:       1383964 kB
Mapped:            12512 kB
Shmem:                 0 kB
Slab:             682192 kB
SReclaimable:     123796 kB
SUnreclaim:       558396 kB
KernelStack:        1448 kB
PageTables:         6616 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     173600 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:    405504 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70896 MiB
End job on node119 at 2012-06-03 00:28:44