Trace number 3736879

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
PB07: PB-clasp 2007-04-10SAT (TO)43 1802.06 1802.63

General information on the benchmark

Namenormalized-PB07/OPT-SMALLINT-NLC/submittedPB07/
manquinho/mds/normalized-mds_500_25_2.opb
MD5SUMe4d0ce6c299ae1a05f607b170c68aad6
Bench CategoryOPT-SMALLINT-NLC (optimisation, small integers, non linear constraints)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark28
Best CPU time to get the best result obtained on this benchmark1796.77
Has Objective FunctionYES
SatisfiableYES
(Un)Satisfiability was provedYES
Best value of the objective function 33
Optimality of the best value was proved NO
Number of variables500
Total number of constraints500
Number of constraints which are clauses0
Number of constraints which are cardinality constraints (but not clauses)0
Number of constraints which are nor clauses,nor cardinality constraints500
Minimum length of a constraint26
Maximum length of a constraint46
Number of terms in the objective function 500
Biggest coefficient in the objective function 1
Number of bits for the biggest coefficient in the objective function 1
Sum of the numbers in the objective function 500
Number of bits of the sum of numbers in the objective function 9
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 500
Number of bits of the biggest sum of numbers9
Number of products (including duplicates)15670
Sum of products size (including duplicates)31340
Number of different products15670
Sum of products size31340

Quality of the solution as a function of time


Download the above graph as a PDF file / as an EPS file

Solver Data 

1.28/1.75	o 54
2.17/2.25	o 53
2.24/2.73	o 52
3.27/3.45	o 51
3.30/4.06	o 49
4.38/4.65	o 47
5.39/5.69	o 46
5.49/6.27	o 45
6.89/7.16	o 44
19.98/20.29	sh: line 1: 25140 Done                    cat HOME/temp/instance-3736879-1338733117.opb.op
19.98/20.29	     25141 Terminated              | clasp-1.0 --trans=no > HOME/temp/instance-3736879-1338733117.opb.clasp
226.52/226.84	o 43
1800.02/1800.61	sh: line 1: 25278 Done                    cat HOME/temp/instance-3736879-1338733117.opb.op
1800.02/1800.61	     25279 Terminated              | clasp-1.0 --trans=no > HOME/temp/instance-3736879-1338733117.opb.clasp
1800.02/1800.62	c Somebody sent me a SIGTERM
1800.02/1800.62	s SATISFIABLE
1800.02/1800.62	v x45 x58 x85 x102 x104 x106 x108 x120 x125 x194 x197 x205 x218 x237 x239 x257 x266 x270 x309 x316 x337 x347 x348 x350 x370 x375 x385 x397 x399 x403 x412 x423 x426 x446 x455 x459 x466 x472 x479 x481 x485 x486 x496 x524 x529 x531 x537 x540 x544 x555 x565 x588 x592 x600 x607 x628 x630 x634 x642 x643 x645 x646 x649 x653 x689 x730 x745 x781 x782 x792 x804 x813 x824 x835 x853 x861 x871 x880 x887 x891 x908 x952 x956 x971 x977 x987 x992 x999 x1005 x1043 x1045 x1049 x1058 x1067 x1077 x1078 x1080 x1085 x1100 x1102 x1117 x1128 x1133 x1140 x1156 x1166 x1167 x1175 x1202 x1233 x1275 x1282 x1293 x1295 x1302 x1325 x1330 x1331 x1336 x1338 x1347 x1366 x1374 x1381 x1382 x1385 x1412 x1429 x1449 x1451 x1501 x1528 x1530 x1537 x1547 x1559 x1568 x1589 x1593 x1602 x1609 x1610 x1612 x1633 x1660 x1672 x1678 x1682 x1685 x1745 x1764 x1776 x1781 x1787 x1829 x1832 x1835 x1849 x1865 x1877 x1881 x1884 x1897 x1904 x1906 x1913 x1920 x1930 x1953 x1972 x1975 x1983 x2013 x2036 x2042 x2064 x2107 x2129 x2151 x2152 x2158 x2165 x2176 x2193 x2199 x2213 x2221 x2277 x2283 x2284 x2312 x2319 x2338 x2340 x2350 x2364 x2365 x2373 x2378 x2385 x2396 x2398 x2413 x2418 x2439 x2474 x2513 x2517 x2518 x2536 x2539 x2543 x2556 x2562 x2574 x2576 x2588 x2598 x2599 x2604 x2625 x2631 x2647 x2693 x2774 x2778 x2781 x2804 x2809 x2812 x2825 x2831 x2838 x2842 x2862 x2883 x2889 x2895 x2897 x2910 x2922 x2923 x2926 x2935 x2964 x2976 x2977 x3010 x3011 x3041 x3058 x3060 x3089 x3105 x3109 x3112 x3122 x3136 x3146 x3147 x3155 x3194 x3208 x3214 x3218 x3224 x3250 x3271 x3273 x3279 x3284 x3285 x3286 x3290 x3293 x3308 x3322 x3373 x3407 x3408 x3417 x3431 x3456 x3467 x3496 x3517 x3523 x3534 x3543 x3557 x3558 x3570 x3578 x3590 x3625 x3650 x3664 x3666 x3672 x3708 x3760 x3779 x3800 x3810 x3816 x3820 x3838 x3840 x3869 x3878 x3889 x3902 x3907 x3934 x3944 x3946 x3970 x3984 x3987 x4001 x4017 x4029 x4037 x4051 x4060 x4063 x4075 x4082 x4087 x4091 x4093 x4094 x4098 x4099 x4108 x4114 x4126 x4134 x4169 x4175 x4178 x4195 x4216 x4225 x4241 x4242 x4263 x4292 x4300 x4305 x4316 x4357 x4361 x4368 x4370 x4433 x4438 x4439 x4444 x4459 x4460 x4461 x4515 x4520 x4578 x4582 x4598 x4626 x4635 x4663 x4666 x4677 x4678 x4694 x4719 x4722 x4750 x4761 x4771 x4807 x4820 x4839 x4841 x4852 x4865 x4891 x4934 x4935 x4942 x4951 x4952 x5032 x5040 x5050 x5053 x5055 x5087 x5095 x5112 x5124 x5127 x5136 x5184 x5186 x5187 x5198 x5200 x5201 x5210 x5222 x5246 x5249 x5274 x5276 x5284 x5285 x5292 x5321 x5328 x5340 x5342 x5357 x5369 x5374 x5378 x5391 x5410 x5412 x5421 x5431 x5435 x5445 x5452 x5458 x5462 x5487 x5495 x5499 x5504 x5511 x5524 x5527 x5538 x5542 x5546 x5578 x5590 x5600 x5612 x5638 x5667 x5668 x5671 x5687 x5724 x5749 x5764 x5777 x5802 x5819 x5828 x5831 x5842 x5843 x5846 x5848 x5872 x5878 x5881 x5885 x5906 x5915 x5926 x5930 x5933 x5947 x5950 x5958 x5966 x5968 x5988 x5995 x5999 x6061 x6080 x6086 x6088 x6090 x6111 x6159 x6162 x6176 x6190 x6205 x6234 x6245 x6246 x6250 x6264 x6276 x6279 x6284 x6285 x6315 x6366 x6388 x6390 x6392 x6396 x6409 x6412 x6426 x6455 x6463 x6486 x6494 x6500 x6502 x6520 x6534 x6542 x6549 x6563 x6566 x6587 x6588 x6589 x6601 x6602 x6603 x6609 x6611 x6614 x6615 x6636 x6639 x6643 x6680 x6681 x6702 x6713 x6731 x6732 x6755 x6758 x6769 x6777 x6796 x6840 x6845 x6873 x6874 x6898 x6914 x6924 x6928 x6936 x6939 x6943 x6953 x6954 x6968 x6984 x6997 x7010 x7022 x7030 x7041 x7044 x7045 x7046 x7066 x7081 x7082 x7092 x7103 x7111 x7123 x7127 x7142 x7144 x7147 x7152 x7178 x7217 x7219 x7225 x7248 x7264 x7270 x7278 x7289 x7304 x7329 x7333 x7342 x7346 x7347 x7350 x7360 x7362 x7390 x7398 x7400 x7417 x7429 x7431 x7469 x7472 x7494 x7533 x7582 x7627 x7629 x7642 x7650 x7658 x7660 x7661 x7677 x7678 x7683 x7686 x7727 x7730 x7732 x7736 x7744 x7773 x7778 x7787 x7820 x7879 x7890 x7894 x7903 x7904 x7908 x7919 x7921 x7923 x7926 x7928 x7946 x7971 x7988 x7994 x7999 x8012 x8014 x8019 x8026 x8050 x8079 x8092 x8113 x8156 x8182 x8184 x8192 x8202 x8236 x8296 x8311 x8335 x8341 x8364 x8394 x8404 x8410 x8427 x8432 x8467 x8476 x8481 x8482 x8491 x8497 x8504 x8519 x8526 x8531 x8545 x8554 x8561 x8573 x8583 x8587 x8598 x8610 x8614 x8623 x8634 x8636 x8651 x8663 x8669 x8678 x8680 x8686 x8718 x8739 x8743 x8746 x8761 x8767 x8775 x8780 x8788 x8810 x8829 x8841 x8842 x8843 x8844 x8887 x8896 x8902 x8903 x8948 x8949 x8950 x8961 x8985 x9022 x9032 x9033 x9034 x9039 x9044 x9053 x9057 x9065 x9074 x9114 x9123 x9126 x9133 x9152 x9179 x9182 x9192 x9201 x9208 x9241 x9260 x9262 x9270 x9274 x9296 x9314 x9317 x9318 x9327 x9357 x9358 x9366 x9391 x9415 x9426 x9445 x9496 x9506 x9521 x9558 x9574 x9582 x9600 x9602 x9604 x9606 x9612 x9620 x9622 x9629 x9632 x9637 x9644 x9656 x9657 x9687 x9692 x9699 x9707 x9708 x9731 x9783 x9818 x9833 x9845 x9846 x9852 x9857 x9915 x9922 x9935 x9938 x9961 x9963 x9991 x10006 x10018 x10022 x10025 x10028 x10071 x10086 x10108 x10124 x10132 x10137 x10157 x10160 x10164 x10169 x10173 x10174 x10185 x10194 x10208 x10234 x10260 x10265 x10274 x10279 x10306 x10334 x10335 x10363 x10368 x10381 x10390 x10420 x10433 x10435 x10448 x10452 x10473 x10510 x10520 x10528 x10532 x10533 x10552 x10564 x10567 x10570 x10586 x10593 x10600 x10610 x10617 x10620 x10621 x10629 x10657 x10666 x10682 x10699 x10740 x10760 x10769 x10775 x10783 x10788 x10789 x10806 x10828 x10856 x10874 x10911 x10913 x10923 x10925 x10927 x10931 x10932 x10936 x10942 x10965 x10970 x10975 x11003 x11005 x11055 x11085 x11089 x11102 x11108 x11124 x11141 x11174 x11179 x11189 x11195 x11241 x11246 x11272 x11286 x11300 x11306 x11317 x11319 x11327 x11349 x11354 x11366 x11371 x11395 x11402 x11403 x11405 x11423 x11428 x11436 x11437 x11438 x11447 x11453 x11455 x11466 x11479 x11487 x11493 x11494 x11496 x11507 x11537 x11538 x11548 x11552 x11553 x11556 x11586 x11615 x11639 x11643 x11659 x11665 x11675 x11700 x11718 x11726 x11732 x11738 x11741 x11753 x11763 x11776 x11781 x11795 x11817 x11859 x11878 x11898 x11913 x11931 x11933 x11941 x11946 x11982 x11990 x11991 x11999 x12016 x12017 x12026 x12045 x12049 x12051 x12073 x12093 x12169 x12182 x12191 x12235 x12242 x12246 x12272 x12287 x12320 x12323 x12326 x12338 x12340 x12349 x12377 x12401 x12433 x12449 x12456 x12480 x12486 x12508 x12513 x12516 x12517 x12523 x12525 x12564 x12587 x12598 x12620 x12641 x12663 x12672 x12689 x12706 x12712 x12726 x12732 x12790 x12819 x12831 x12845 x12858 x12915 x12927 x12948 x12964 x13002 x13021 x13022 x13024 x13035 x13038 x13064 x13073 x13083 x13097 x13103 x13105 x13109 x13131 x13143 x13177 x13179 x13182 x13183 x13192 x13225 x13235 x13262 x13292 x13296 x13307 x13326 x13351 x13362 x13363 x13378 x13393 x13405 x13423 x13424 x13431 x13434 x13435 x13460 x13496 x13547 x13548 x13592 x13598 x13610 x13617 x13672 x13683 x13685 x13706 x13725 x13740 x13744 x13748 x13753 x13775 x13786 x13815 x13822 x13840 x13890 x13892 x13907 x13908 x13939 x13957 x13958 x13962 x13974 x13976 x13977 x13986 x14009 x14015 x14016 x14020 x14038 x14050 x14056 x14062 x14078 x14104 x14105 x14106 x14113 x14129 x14132 x14140 x14141 x14149 x14150 x14157 x14205 x14206 x14216 x14230 x14264 x14293 x14296 x14323 x14328 x14336 x14353 x14358 x14363 x14386 x14415 x14424 x14434 x14448 x14467 x14495 x14507 x14525 x14528 x14545 x14553 x14572 x14587 x14592 x14605 x14606 x14611 x14618 x14627 x14638 x14642 x14680 x14700 x14718 x14727 x14732 x14738 x14806 x14808 x14810 x14820 x14858 x14870 x14891 x14893 x14900 x14906 x14913 x14920 x14939 x14944 x14945 x14981 x15037 x15040 x15041 x15048 x15063 x15068 x15078 x15088 x15143 x15162 x15171 x15179 x15203 x15212 x15223 x15241 x15250 x15275 x15276 x15277 x15319 x15326 x15344 x15356 x15363 x15381 x15395 x15397 x15402 x15405 x15411 x15413 x15428 x15430 x15434 x15468 x15499 x15503 x15511 x15512 x15513 x15526 x15556 x15562 x15570 x15588 x15614 x15657 x15710 x15717 x15723 x15737 x15741 x15748 x15763 x15842 x15867 x15889 x15908 x15929 x15944 x15958 x15961 x15966 x15976 x15985 x16008 x16031 x16037 x16038 x16039 x16042 x16081 x16101 x16106 x16134 x16139 x16166 
1800.02/1800.62	pb2clasp1: no process killed
1801.99/1802.62	sh: line 1: 25313 Done                    cat HOME/temp/instance-3736879-1338733117.opb.op
1801.99/1802.62	     25314 Killed                  | clasp-1.0 --trans=no > HOME/temp/instance-3736879-1338733117.opb.clasp

Verifier Data 

OK	43

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3736879-1338733117/watcher-3736879-1338733117 -o /tmp/evaluation-result-3736879-1338733117/solver-3736879-1338733117 -C 1800 -W 1900 -M 15500 pbclasp instance-3736879-1338733117.opb HOME/temp 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.06 2.05 2.00 3/176 25059
/proc/meminfo: memFree=23395004/32873844 swapFree=7112/7112
[pid=25059] ppid=25057 vsize=25872 CPUtime=0 cores=1,3,5,7
/proc/25059/stat : 25059 (pbclasp) R 25057 25059 27890 0 -1 4202496 610 0 0 0 0 0 0 0 20 0 1 0 339440293 26492928 540 33554432000 4194304 4198148 140737232253056 140733376936472 235374350228 0 0 4224 16384 0 0 0 17 5 0 0 0 0 0
/proc/25059/statm: 6468 540 355 1 0 263 0

[startup+0.075823 s]
/proc/loadavg: 2.06 2.05 2.00 3/176 25059
/proc/meminfo: memFree=23395004/32873844 swapFree=7112/7112
[pid=25059] ppid=25057 vsize=26268 CPUtime=0 cores=1,3,5,7
/proc/25059/stat : 25059 (pbclasp) S 25057 25059 27890 0 -1 4202496 733 0 0 0 0 0 0 0 20 0 1 0 339440293 26898432 646 33554432000 4194304 4198148 140737232253056 140737232251944 235378107632 0 0 4224 16384 18446744071580380219 0 0 17 5 0 0 0 0 0
/proc/25059/statm: 6567 646 369 1 0 362 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 26268

[startup+0.100206 s]
/proc/loadavg: 2.06 2.05 2.00 3/176 25059
/proc/meminfo: memFree=23395004/32873844 swapFree=7112/7112
[pid=25059] ppid=25057 vsize=26268 CPUtime=0 cores=1,3,5,7
/proc/25059/stat : 25059 (pbclasp) S 25057 25059 27890 0 -1 4202496 733 0 0 0 0 0 0 0 20 0 1 0 339440293 26898432 646 33554432000 4194304 4198148 140737232253056 140737232251944 235378107632 0 0 4224 16384 18446744071580380219 0 0 17 5 0 0 0 0 0
/proc/25059/statm: 6567 646 369 1 0 362 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 26268

[startup+0.300178 s]
/proc/loadavg: 2.06 2.05 2.00 3/176 25059
/proc/meminfo: memFree=23395004/32873844 swapFree=7112/7112
[pid=25059] ppid=25057 vsize=26268 CPUtime=0 cores=1,3,5,7
/proc/25059/stat : 25059 (pbclasp) S 25057 25059 27890 0 -1 4202496 733 0 0 0 0 0 0 0 20 0 1 0 339440293 26898432 646 33554432000 4194304 4198148 140737232253056 140737232251944 235378107632 0 0 4224 16384 18446744071580380219 0 0 17 5 0 0 0 0 0
/proc/25059/statm: 6567 646 369 1 0 362 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 26268

[startup+0.700177 s]
/proc/loadavg: 2.06 2.05 2.00 3/176 25059
/proc/meminfo: memFree=23395004/32873844 swapFree=7112/7112
[pid=25059] ppid=25057 vsize=26268 CPUtime=0 cores=1,3,5,7
/proc/25059/stat : 25059 (pbclasp) S 25057 25059 27890 0 -1 4202496 733 0 0 0 0 0 0 0 20 0 1 0 339440293 26898432 646 33554432000 4194304 4198148 140737232253056 140737232251944 235378107632 0 0 4224 16384 18446744071580380219 0 0 17 5 0 0 0 0 0
/proc/25059/statm: 6567 646 369 1 0 362 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 26268

[startup+1.50022 s]
/proc/loadavg: 2.05 2.05 2.00 3/179 25062
/proc/meminfo: memFree=23392764/32873844 swapFree=7112/7112
[pid=25059] ppid=25057 vsize=26268 CPUtime=1.28 cores=1,3,5,7
/proc/25059/stat : 25059 (pbclasp) S 25057 25059 27890 0 -1 4202496 829 2090 0 0 1 0 126 1 20 0 1 0 339440293 26898432 675 33554432000 4194304 4198148 140737232253056 140737232251984 235378110494 0 0 4230 16384 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/25059/statm: 6567 675 387 1 0 362 0
Current children cumulated CPU time (s) 1.28
Current children cumulated vsize (KiB) 26268

[startup+3.10026 s]
/proc/loadavg: 2.05 2.05 2.00 3/179 25077
/proc/meminfo: memFree=23369172/32873844 swapFree=7112/7112
[pid=25059] ppid=25057 vsize=26268 CPUtime=2.73 cores=1,3,5,7
/proc/25059/stat : 25059 (pbclasp) S 25057 25059 27890 0 -1 4202496 1222 26105 0 0 3 0 262 8 20 0 1 0 339440293 26898432 692 33554432000 4194304 4198148 140737232253056 140737232251984 235378110494 0 0 4230 16384 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/25059/statm: 6567 692 387 1 0 362 0
Current children cumulated CPU time (s) 2.73
Current children cumulated vsize (KiB) 26268

[startup+6.30015 s]
/proc/loadavg: 2.05 2.05 2.00 3/179 25117
/proc/meminfo: memFree=23368620/32873844 swapFree=7112/7112
[pid=25059] ppid=25057 vsize=26400 CPUtime=6.08 cores=1,3,5,7
/proc/25059/stat : 25059 (pbclasp) S 25057 25059 27890 0 -1 4202496 1926 66150 0 0 7 1 582 18 20 0 1 0 339440293 27033600 696 33554432000 4194304 4198148 140737232253056 140737232251984 235378110494 0 0 4230 16384 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/25059/statm: 6600 696 387 1 0 395 0
Current children cumulated CPU time (s) 6.08
Current children cumulated vsize (KiB) 26400

[startup+12.7003 s]
/proc/loadavg: 2.04 2.05 2.00 3/179 25141
/proc/meminfo: memFree=23366932/32873844 swapFree=7112/7112
[pid=25059] ppid=25057 vsize=26400 CPUtime=6.97 cores=1,3,5,7
/proc/25059/stat : 25059 (pbclasp) S 25057 25059 27890 0 -1 4202496 2064 74158 0 0 8 1 668 20 20 0 1 0 339440293 27033600 696 33554432000 4194304 4198148 140737232253056 140737232251984 235378110494 0 0 4230 16384 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/25059/statm: 6600 696 387 1 0 395 0
[pid=25139] ppid=25059 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/25139/stat : 25139 (sh) S 25059 25059 27890 0 -1 4202496 372 221 0 0 0 0 0 0 20 0 1 0 339441010 9367552 264 33554432000 4194304 5064252 140735882328096 140735882326352 235374536270 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/25139/statm: 2287 264 223 213 0 72 0
[pid=25141] ppid=25139 vsize=27632 CPUtime=5.52 cores=1,3,5,7
/proc/25141/stat : 25141 (clasp-1.0) R 25139 25059 27890 0 -1 4202496 6868 0 0 0 551 1 0 0 20 0 1 0 339441010 28295168 6329 33554432000 134512640 134847168 4289767968 4289766652 134715930 0 0 4096 0 0 0 0 17 5 0 0 0 0 0
/proc/25141/statm: 6908 6329 270 82 0 6149 0
Current children cumulated CPU time (s) 12.49
Current children cumulated vsize (KiB) 63180

[startup+25.5003 s]
/proc/loadavg: 1.96 2.03 2.00 3/179 25267
/proc/meminfo: memFree=23378848/32873844 swapFree=7112/7112
[pid=25059] ppid=25057 vsize=26400 CPUtime=20.1 cores=1,3,5,7
/proc/25059/stat : 25059 (pbclasp) S 25057 25059 27890 0 -1 4202496 2160 85504 0 0 8 1 1977 24 20 0 1 0 339440293 27033600 696 33554432000 4194304 4198148 140737232253056 140737232251984 235378110494 0 0 4230 16384 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/25059/statm: 6600 696 387 1 0 395 0
[pid=25158] ppid=25059 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/25158/stat : 25158 (sh) S 25059 25059 27890 0 -1 4202496 371 219 0 0 0 0 0 0 20 0 1 0 339442324 9367552 264 33554432000 4194304 5064252 140734696835584 140734696833840 235374536270 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/25158/statm: 2287 264 223 213 0 72 0
[pid=25160] ppid=25158 vsize=27364 CPUtime=5.17 cores=1,3,5,7
/proc/25160/stat : 25160 (clasp-1.0) R 25158 25059 27890 0 -1 4202496 6803 0 0 0 516 1 0 0 20 0 1 0 339442324 28020736 6275 33554432000 134512640 134847168 4291164336 4291162612 134792592 0 0 4096 0 0 0 0 17 1 0 0 0 0 0
/proc/25160/statm: 6841 6275 270 82 0 6082 0
Current children cumulated CPU time (s) 25.27
Current children cumulated vsize (KiB) 62912

[startup+51.1004 s]
/proc/loadavg: 1.97 2.03 2.00 3/178 25267
/proc/meminfo: memFree=23337380/32873844 swapFree=7112/7112
[pid=25059] ppid=25057 vsize=26400 CPUtime=20.1 cores=1,3,5,7
/proc/25059/stat : 25059 (pbclasp) S 25057 25059 27890 0 -1 4202496 2160 85504 0 0 8 1 1977 24 20 0 1 0 339440293 27033600 696 33554432000 4194304 4198148 140737232253056 140737232251984 235378110494 0 0 4230 16384 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/25059/statm: 6600 696 387 1 0 395 0
[pid=25158] ppid=25059 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/25158/stat : 25158 (sh) S 25059 25059 27890 0 -1 4202496 371 219 0 0 0 0 0 0 20 0 1 0 339442324 9367552 264 33554432000 4194304 5064252 140734696835584 140734696833840 235374536270 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/25158/statm: 2287 264 223 213 0 72 0
[pid=25160] ppid=25158 vsize=52628 CPUtime=30.76 cores=1,3,5,7
/proc/25160/stat : 25160 (clasp-1.0) R 25158 25059 27890 0 -1 4202496 13085 0 0 0 3073 3 0 0 20 0 1 0 339442324 53891072 12555 33554432000 134512640 134847168 4291164336 4291163020 134715805 0 0 4096 0 0 0 0 17 1 0 0 0 0 0
/proc/25160/statm: 13157 12555 270 82 0 12398 0
Current children cumulated CPU time (s) 50.86
Current children cumulated vsize (KiB) 88176

[startup+102.307 s]
/proc/loadavg: 1.99 2.02 2.00 3/180 25271
/proc/meminfo: memFree=23317600/32873844 swapFree=7112/7112
[pid=25059] ppid=25057 vsize=26400 CPUtime=20.1 cores=1,3,5,7
/proc/25059/stat : 25059 (pbclasp) S 25057 25059 27890 0 -1 4202496 2160 85504 0 0 8 1 1977 24 20 0 1 0 339440293 27033600 696 33554432000 4194304 4198148 140737232253056 140737232251984 235378110494 0 0 4230 16384 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/25059/statm: 6600 696 387 1 0 395 0
[pid=25158] ppid=25059 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/25158/stat : 25158 (sh) S 25059 25059 27890 0 -1 4202496 371 219 0 0 0 0 0 0 20 0 1 0 339442324 9367552 264 33554432000 4194304 5064252 140734696835584 140734696833840 235374536270 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/25158/statm: 2287 264 223 213 0 72 0
[pid=25160] ppid=25158 vsize=63492 CPUtime=81.95 cores=1,3,5,7
/proc/25160/stat : 25160 (clasp-1.0) R 25158 25059 27890 0 -1 4202496 15763 0 0 0 8191 4 0 0 20 0 1 0 339442324 65015808 15272 33554432000 134512640 134847168 4291164336 4291162964 134616350 0 0 4096 0 0 0 0 17 1 0 0 0 0 0
/proc/25160/statm: 15873 15272 270 82 0 15114 0
Current children cumulated CPU time (s) 102.05
Current children cumulated vsize (KiB) 99040

[startup+162.3 s]
/proc/loadavg: 1.99 2.02 2.00 3/179 25271
/proc/meminfo: memFree=23296316/32873844 swapFree=7112/7112
[pid=25059] ppid=25057 vsize=26400 CPUtime=20.1 cores=1,3,5,7
/proc/25059/stat : 25059 (pbclasp) S 25057 25059 27890 0 -1 4202496 2160 85504 0 0 8 1 1977 24 20 0 1 0 339440293 27033600 696 33554432000 4194304 4198148 140737232253056 140737232251984 235378110494 0 0 4230 16384 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/25059/statm: 6600 696 387 1 0 395 0
[pid=25158] ppid=25059 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/25158/stat : 25158 (sh) S 25059 25059 27890 0 -1 4202496 371 219 0 0 0 0 0 0 20 0 1 0 339442324 9367552 264 33554432000 4194304 5064252 140734696835584 140734696833840 235374536270 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/25158/statm: 2287 264 223 213 0 72 0
[pid=25160] ppid=25158 vsize=76252 CPUtime=141.94 cores=1,3,5,7
/proc/25160/stat : 25160 (clasp-1.0) R 25158 25059 27890 0 -1 4202496 18971 0 0 0 14188 6 0 0 20 0 1 0 339442324 78082048 18510 33554432000 134512640 134847168 4291164336 4291163028 134616619 0 0 4096 0 0 0 0 17 1 0 0 0 0 0
/proc/25160/statm: 19063 18510 270 82 0 18304 0
Current children cumulated CPU time (s) 162.04
Current children cumulated vsize (KiB) 111800

[startup+222.3 s]
/proc/loadavg: 1.99 2.01 2.00 3/179 25271
/proc/meminfo: memFree=23284780/32873844 swapFree=7112/7112
[pid=25059] ppid=25057 vsize=26400 CPUtime=20.1 cores=1,3,5,7
/proc/25059/stat : 25059 (pbclasp) S 25057 25059 27890 0 -1 4202496 2160 85504 0 0 8 1 1977 24 20 0 1 0 339440293 27033600 696 33554432000 4194304 4198148 140737232253056 140737232251984 235378110494 0 0 4230 16384 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/25059/statm: 6600 696 387 1 0 395 0
[pid=25158] ppid=25059 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/25158/stat : 25158 (sh) S 25059 25059 27890 0 -1 4202496 371 219 0 0 0 0 0 0 20 0 1 0 339442324 9367552 264 33554432000 4194304 5064252 140734696835584 140734696833840 235374536270 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/25158/statm: 2287 264 223 213 0 72 0
[pid=25160] ppid=25158 vsize=83456 CPUtime=201.92 cores=1,3,5,7
/proc/25160/stat : 25160 (clasp-1.0) R 25158 25059 27890 0 -1 4202496 20717 0 0 0 20186 6 0 0 20 0 1 0 339442324 85458944 20262 33554432000 134512640 134847168 4291164336 4291163068 134740921 0 0 4096 0 0 0 0 17 1 0 0 0 0 0
/proc/25160/statm: 20864 20262 270 82 0 20105 0
Current children cumulated CPU time (s) 222.02
Current children cumulated vsize (KiB) 119004

[startup+282.3 s]
/proc/loadavg: 1.99 2.01 2.00 3/179 25279
/proc/meminfo: memFree=23296224/32873844 swapFree=7112/7112
[pid=25059] ppid=25057 vsize=26400 CPUtime=226.58 cores=1,3,5,7
/proc/25059/stat : 25059 (pbclasp) S 25057 25059 27890 0 -1 4202496 2299 108473 0 0 8 1 22617 32 20 0 1 0 339440293 27033600 696 33554432000 4194304 4198148 140737232253056 140737232251984 235378110494 0 0 4230 16384 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/25059/statm: 6600 696 387 1 0 395 0
[pid=25277] ppid=25059 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/25277/stat : 25277 (sh) S 25059 25059 27890 0 -1 4202496 371 220 0 0 0 0 0 0 20 0 1 0 339462979 9367552 264 33554432000 4194304 5064252 140736320283136 140736320281392 235374536270 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/25277/statm: 2287 264 223 213 0 72 0
[pid=25279] ppid=25277 vsize=66752 CPUtime=55.41 cores=1,3,5,7
/proc/25279/stat : 25279 (clasp-1.0) R 25277 25059 27890 0 -1 4202496 16588 0 0 0 5537 4 0 0 20 0 1 0 339462979 68354048 16048 33554432000 134512640 134847168 4289362992 4289361632 134782452 0 0 4096 0 0 0 0 17 1 0 0 0 0 0
/proc/25279/statm: 16688 16048 272 82 0 15929 0
Current children cumulated CPU time (s) 281.99
Current children cumulated vsize (KiB) 102300

[startup+342.3 s]
/proc/loadavg: 1.99 2.00 2.00 3/179 25279
/proc/meminfo: memFree=23275164/32873844 swapFree=7112/7112
[pid=25059] ppid=25057 vsize=26400 CPUtime=226.58 cores=1,3,5,7

################
# More data... #
################

/proc/loadavg: 2.08 2.02 2.01 3/179 25290
/proc/meminfo: memFree=23179772/32873844 swapFree=7112/7112
[pid=25059] ppid=25057 vsize=26400 CPUtime=226.58 cores=1,3,5,7
/proc/25059/stat : 25059 (pbclasp) S 25057 25059 27890 0 -1 4202496 2299 108473 0 0 8 1 22617 32 20 0 1 0 339440293 27033600 696 33554432000 4194304 4198148 140737232253056 140737232251984 235378110494 0 0 4230 16384 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/25059/statm: 6600 696 387 1 0 395 0
[pid=25277] ppid=25059 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/25277/stat : 25277 (sh) S 25059 25059 27890 0 -1 4202496 371 220 0 0 0 0 0 0 20 0 1 0 339462979 9367552 264 33554432000 4194304 5064252 140736320283136 140736320281392 235374536270 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/25277/statm: 2287 264 223 213 0 72 0
[pid=25279] ppid=25277 vsize=150660 CPUtime=1195.19 cores=1,3,5,7
/proc/25279/stat : 25279 (clasp-1.0) R 25277 25059 27890 0 -1 4202496 37174 0 0 0 119507 12 0 0 20 0 1 0 339462979 154275840 37032 33554432000 134512640 134847168 4289362992 4289361712 134740930 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/25279/statm: 37665 37032 272 82 0 36906 0
Current children cumulated CPU time (s) 1421.77
Current children cumulated vsize (KiB) 186208

[startup+1482.3 s]
/proc/loadavg: 2.03 2.01 2.00 3/179 25290
/proc/meminfo: memFree=23177352/32873844 swapFree=7112/7112
[pid=25059] ppid=25057 vsize=26400 CPUtime=226.58 cores=1,3,5,7
/proc/25059/stat : 25059 (pbclasp) S 25057 25059 27890 0 -1 4202496 2299 108473 0 0 8 1 22617 32 20 0 1 0 339440293 27033600 696 33554432000 4194304 4198148 140737232253056 140737232251984 235378110494 0 0 4230 16384 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/25059/statm: 6600 696 387 1 0 395 0
[pid=25277] ppid=25059 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/25277/stat : 25277 (sh) S 25059 25059 27890 0 -1 4202496 371 220 0 0 0 0 0 0 20 0 1 0 339462979 9367552 264 33554432000 4194304 5064252 140736320283136 140736320281392 235374536270 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/25277/statm: 2287 264 223 213 0 72 0
[pid=25279] ppid=25277 vsize=150660 CPUtime=1255.18 cores=1,3,5,7
/proc/25279/stat : 25279 (clasp-1.0) R 25277 25059 27890 0 -1 4202496 37174 0 0 0 125506 12 0 0 20 0 1 0 339462979 154275840 37032 33554432000 134512640 134847168 4289362992 4289361684 134616619 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/25279/statm: 37665 37032 272 82 0 36906 0
Current children cumulated CPU time (s) 1481.76
Current children cumulated vsize (KiB) 186208

[startup+1542.3 s]
/proc/loadavg: 2.04 2.02 2.00 3/179 25290
/proc/meminfo: memFree=23170464/32873844 swapFree=7112/7112
[pid=25059] ppid=25057 vsize=26400 CPUtime=226.58 cores=1,3,5,7
/proc/25059/stat : 25059 (pbclasp) S 25057 25059 27890 0 -1 4202496 2299 108473 0 0 8 1 22617 32 20 0 1 0 339440293 27033600 696 33554432000 4194304 4198148 140737232253056 140737232251984 235378110494 0 0 4230 16384 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/25059/statm: 6600 696 387 1 0 395 0
[pid=25277] ppid=25059 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/25277/stat : 25277 (sh) S 25059 25059 27890 0 -1 4202496 371 220 0 0 0 0 0 0 20 0 1 0 339462979 9367552 264 33554432000 4194304 5064252 140736320283136 140736320281392 235374536270 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/25277/statm: 2287 264 223 213 0 72 0
[pid=25279] ppid=25277 vsize=156008 CPUtime=1315.17 cores=1,3,5,7
/proc/25279/stat : 25279 (clasp-1.0) R 25277 25059 27890 0 -1 4202496 38429 0 0 0 131504 13 0 0 20 0 1 0 339462979 159752192 38287 33554432000 134512640 134847168 4289362992 4289361684 134616619 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/25279/statm: 39002 38287 272 82 0 38243 0
Current children cumulated CPU time (s) 1541.75
Current children cumulated vsize (KiB) 191556

[startup+1602.3 s]
/proc/loadavg: 2.01 2.02 2.00 3/179 25291
/proc/meminfo: memFree=23168844/32873844 swapFree=7112/7112
[pid=25059] ppid=25057 vsize=26400 CPUtime=226.58 cores=1,3,5,7
/proc/25059/stat : 25059 (pbclasp) S 25057 25059 27890 0 -1 4202496 2299 108473 0 0 8 1 22617 32 20 0 1 0 339440293 27033600 696 33554432000 4194304 4198148 140737232253056 140737232251984 235378110494 0 0 4230 16384 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/25059/statm: 6600 696 387 1 0 395 0
[pid=25277] ppid=25059 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/25277/stat : 25277 (sh) S 25059 25059 27890 0 -1 4202496 371 220 0 0 0 0 0 0 20 0 1 0 339462979 9367552 264 33554432000 4194304 5064252 140736320283136 140736320281392 235374536270 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/25277/statm: 2287 264 223 213 0 72 0
[pid=25279] ppid=25277 vsize=156812 CPUtime=1375.16 cores=1,3,5,7
/proc/25279/stat : 25279 (clasp-1.0) R 25277 25059 27890 0 -1 4202496 38655 0 0 0 137503 13 0 0 20 0 1 0 339462979 160575488 38630 33554432000 134512640 134847168 4289362992 4289361684 134616619 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/25279/statm: 39203 38630 272 82 0 38444 0
Current children cumulated CPU time (s) 1601.74
Current children cumulated vsize (KiB) 192360

[startup+1662.3 s]
/proc/loadavg: 2.00 2.01 2.00 3/179 25299
/proc/meminfo: memFree=23225280/32873844 swapFree=7112/7112
[pid=25059] ppid=25057 vsize=26400 CPUtime=226.58 cores=1,3,5,7
/proc/25059/stat : 25059 (pbclasp) S 25057 25059 27890 0 -1 4202496 2299 108473 0 0 8 1 22617 32 20 0 1 0 339440293 27033600 696 33554432000 4194304 4198148 140737232253056 140737232251984 235378110494 0 0 4230 16384 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/25059/statm: 6600 696 387 1 0 395 0
[pid=25277] ppid=25059 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/25277/stat : 25277 (sh) S 25059 25059 27890 0 -1 4202496 371 220 0 0 0 0 0 0 20 0 1 0 339462979 9367552 264 33554432000 4194304 5064252 140736320283136 140736320281392 235374536270 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/25277/statm: 2287 264 223 213 0 72 0
[pid=25279] ppid=25277 vsize=156812 CPUtime=1435.15 cores=1,3,5,7
/proc/25279/stat : 25279 (clasp-1.0) R 25277 25059 27890 0 -1 4202496 38655 0 0 0 143502 13 0 0 20 0 1 0 339462979 160575488 38630 33554432000 134512640 134847168 4289362992 4289361616 134784667 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/25279/statm: 39203 38630 272 82 0 38444 0
Current children cumulated CPU time (s) 1661.73
Current children cumulated vsize (KiB) 192360

[startup+1722.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/179 25299
/proc/meminfo: memFree=23219484/32873844 swapFree=7112/7112
[pid=25059] ppid=25057 vsize=26400 CPUtime=226.58 cores=1,3,5,7
/proc/25059/stat : 25059 (pbclasp) S 25057 25059 27890 0 -1 4202496 2299 108473 0 0 8 1 22617 32 20 0 1 0 339440293 27033600 696 33554432000 4194304 4198148 140737232253056 140737232251984 235378110494 0 0 4230 16384 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/25059/statm: 6600 696 387 1 0 395 0
[pid=25277] ppid=25059 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/25277/stat : 25277 (sh) S 25059 25059 27890 0 -1 4202496 371 220 0 0 0 0 0 0 20 0 1 0 339462979 9367552 264 33554432000 4194304 5064252 140736320283136 140736320281392 235374536270 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/25277/statm: 2287 264 223 213 0 72 0
[pid=25279] ppid=25277 vsize=156812 CPUtime=1495.14 cores=1,3,5,7
/proc/25279/stat : 25279 (clasp-1.0) R 25277 25059 27890 0 -1 4202496 38655 0 0 0 149501 13 0 0 20 0 1 0 339462979 160575488 38630 33554432000 134512640 134847168 4289362992 4289361684 134616619 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/25279/statm: 39203 38630 272 82 0 38444 0
Current children cumulated CPU time (s) 1721.72
Current children cumulated vsize (KiB) 192360

[startup+1782.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/179 25299
/proc/meminfo: memFree=23215764/32873844 swapFree=7112/7112
[pid=25059] ppid=25057 vsize=26400 CPUtime=226.58 cores=1,3,5,7
/proc/25059/stat : 25059 (pbclasp) S 25057 25059 27890 0 -1 4202496 2299 108473 0 0 8 1 22617 32 20 0 1 0 339440293 27033600 696 33554432000 4194304 4198148 140737232253056 140737232251984 235378110494 0 0 4230 16384 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/25059/statm: 6600 696 387 1 0 395 0
[pid=25277] ppid=25059 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/25277/stat : 25277 (sh) S 25059 25059 27890 0 -1 4202496 371 220 0 0 0 0 0 0 20 0 1 0 339462979 9367552 264 33554432000 4194304 5064252 140736320283136 140736320281392 235374536270 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/25277/statm: 2287 264 223 213 0 72 0
[pid=25279] ppid=25277 vsize=155832 CPUtime=1555.13 cores=1,3,5,7
/proc/25279/stat : 25279 (clasp-1.0) R 25277 25059 27890 0 -1 4202496 38655 0 0 0 155500 13 0 0 20 0 1 0 339462979 159571968 38390 33554432000 134512640 134847168 4289362992 4289361568 134714504 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/25279/statm: 38958 38390 272 82 0 38199 0
Current children cumulated CPU time (s) 1781.71
Current children cumulated vsize (KiB) 191380



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1800.6 s]
/proc/loadavg: 2.00 2.00 2.00 3/179 25299
/proc/meminfo: memFree=23211796/32873844 swapFree=7112/7112
[pid=25059] ppid=25057 vsize=26400 CPUtime=226.58 cores=1,3,5,7
/proc/25059/stat : 25059 (pbclasp) S 25057 25059 27890 0 -1 4202496 2299 108473 0 0 8 1 22617 32 20 0 1 0 339440293 27033600 696 33554432000 4194304 4198148 140737232253056 140737232251984 235378110494 0 0 4230 16384 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/25059/statm: 6600 696 387 1 0 395 0
[pid=25277] ppid=25059 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/25277/stat : 25277 (sh) S 25059 25059 27890 0 -1 4202496 371 220 0 0 0 0 0 0 20 0 1 0 339462979 9367552 264 33554432000 4194304 5064252 140736320283136 140736320281392 235374536270 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/25277/statm: 2287 264 223 213 0 72 0
[pid=25279] ppid=25277 vsize=155832 CPUtime=1573.44 cores=1,3,5,7
/proc/25279/stat : 25279 (clasp-1.0) R 25277 25059 27890 0 -1 4202496 38655 0 0 0 157331 13 0 0 20 0 1 0 339462979 159571968 38390 33554432000 134512640 134847168 4289362992 4289361684 134616619 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/25279/statm: 38958 38390 272 82 0 38199 0
Current children cumulated CPU time (s) 1800.02
Current children cumulated vsize (KiB) 191380

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)

Sending SIGKILL to -25059

Solver just ended. Dumping a history of the last processes samples

[startup+1801.4 s]
/proc/loadavg: 2.00 2.00 2.00 3/179 25299
/proc/meminfo: memFree=23211796/32873844 swapFree=7112/7112
[pid=25059] ppid=25057 vsize=26400 CPUtime=1800.05 cores=1,3,5,7
/proc/25059/stat : 25059 (pbclasp) S 25057 25059 27890 0 -1 4202496 2463 149898 0 0 8 1 179949 47 20 0 1 0 339440293 27033600 696 33554432000 4194304 4198148 140737232253056 140737232251984 235378110494 0 0 4230 16384 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/25059/statm: 6600 696 387 1 0 395 0
Current children cumulated CPU time (s) 1800.05
Current children cumulated vsize (KiB) 26400

[startup+1802.2 s]
/proc/loadavg: 2.00 2.00 2.00 3/180 25318
/proc/meminfo: memFree=23377416/32873844 swapFree=7112/7112
[pid=25059] ppid=25057 vsize=26400 CPUtime=1800.05 cores=1,3,5,7
/proc/25059/stat : 25059 (pbclasp) S 25057 25059 27890 0 -1 4202496 2463 149898 0 0 8 1 179949 47 20 0 1 0 339440293 27033600 696 33554432000 4194304 4198148 140737232253056 140737232251984 235378110494 0 0 4230 16384 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/25059/statm: 6600 696 387 1 0 395 0
[pid=25311] ppid=25059 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/25311/stat : 25311 (sh) S 25059 25059 27890 0 -1 4202496 370 219 0 0 0 0 0 0 20 0 1 0 339620356 9367552 263 33554432000 4194304 5064252 140736246972208 140736246970464 235374536270 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/25311/statm: 2287 263 223 213 0 72 0
[pid=25314] ppid=25311 vsize=24556 CPUtime=1.54 cores=1,3,5,7
/proc/25314/stat : 25314 (clasp-1.0) R 25311 25059 27890 0 -1 4202496 6061 0 0 0 154 0 0 0 20 0 1 0 339620357 25145344 5523 33554432000 134512640 134847168 4290866960 4290865728 134617488 0 0 4096 0 0 0 0 17 1 0 0 0 0 0
/proc/25314/statm: 6139 5523 272 82 0 5380 0
Current children cumulated CPU time (s) 1801.59
Current children cumulated vsize (KiB) 60104

[startup+1802.6 s]
/proc/loadavg: 2.00 2.00 2.00 3/180 25318
/proc/meminfo: memFree=23377416/32873844 swapFree=7112/7112
[pid=25059] ppid=25057 vsize=26400 CPUtime=1800.05 cores=1,3,5,7
/proc/25059/stat : 25059 (pbclasp) S 25057 25059 27890 0 -1 4202496 2463 149898 0 0 8 1 179949 47 20 0 1 0 339440293 27033600 696 33554432000 4194304 4198148 140737232253056 140737232251984 235378110494 0 0 4230 16384 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/25059/statm: 6600 696 387 1 0 395 0
[pid=25311] ppid=25059 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/25311/stat : 25311 (sh) S 25059 25059 27890 0 -1 4202496 370 219 0 0 0 0 0 0 20 0 1 0 339620356 9367552 263 33554432000 4194304 5064252 140736246972208 140736246970464 235374536270 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/25311/statm: 2287 263 223 213 0 72 0
[pid=25314] ppid=25311 vsize=24556 CPUtime=1.94 cores=1,3,5,7
/proc/25314/stat : 25314 (clasp-1.0) R 25311 25059 27890 0 -1 4202496 6061 0 0 0 194 0 0 0 20 0 1 0 339620357 25145344 5523 33554432000 134512640 134847168 4290866960 4290865536 134714688 0 0 4096 0 0 0 0 17 1 0 0 0 0 0
/proc/25314/statm: 6139 5523 272 82 0 5380 0
Current children cumulated CPU time (s) 1801.99
Current children cumulated vsize (KiB) 60104

Child status: 0
Real time (s): 1802.63
CPU time (s): 1802.06
CPU user time (s): 1801.56
CPU system time (s): 0.497924
CPU usage (%): 99.9686
Max. virtual memory (cumulated for all children) (KiB): 192360

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1801.56
system time used= 0.497924
maximum resident set size= 154520
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 159272
page faults= 0
swaps= 0
block input operations= 0
block output operations= 42008
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 5041
involuntary context switches= 2347

runsolver used 3.70744 second user time and 8.85765 second system time

The end

Launcher Data 

Begin job on node104 at 2012-06-03 16:18:37
IDJOB=3736879
IDBENCH=48140
IDSOLVER=2317
FILE ID=node104/3736879-1338733117
RUNJOBID= node104-1338713258-27907
PBS_JOBID= 14636727
Free space on /tmp= 66432 MiB

SOLVER NAME= PB07: PB-clasp 2007-04-10
BENCH NAME= PB07/normalized-PB07/OPT-SMALLINT-NLC/submittedPB07/manquinho/mds/normalized-mds_500_25_2.opb
COMMAND LINE= pbclasp BENCHNAMENOPATH TMPDIR/temp
CONVERSION SCRIPT= PBconversionToLinear BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3736879-1338733117/watcher-3736879-1338733117 -o /tmp/evaluation-result-3736879-1338733117/solver-3736879-1338733117 -C 1800 -W 1900 -M 15500  pbclasp instance-3736879-1338733117.opb HOME/temp

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= e4d0ce6c299ae1a05f607b170c68aad6
RANDOM SEED=816181548

node104.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.266
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.53
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.266
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.266
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.266
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.266
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.266
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.266
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.266
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        23396432 kB
Buffers:         1011768 kB
Cached:          7424752 kB
SwapCached:          708 kB
Active:          5255712 kB
Inactive:        3214244 kB
Active(anon):      24404 kB
Inactive(anon):     8092 kB
Active(file):    5231308 kB
Inactive(file):  3206152 kB
Unevictable:        7112 kB
Mlocked:            7112 kB
SwapTotal:      67108856 kB
SwapFree:       67105164 kB
Dirty:               704 kB
Writeback:            20 kB
AnonPages:         36768 kB
Mapped:            16224 kB
Shmem:                 8 kB
Slab:             868032 kB
SReclaimable:     310036 kB
SUnreclaim:       557996 kB
KernelStack:        1496 kB
PageTables:         4388 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     137360 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:     16384 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

CONVERSION COMMAND LINE= PBconversionToLinear HOME/instance-3736879-1338733117.opb

Free space on /tmp at the end= 66428 MiB
End job on node104 at 2012-06-03 16:48:40