Trace number 3707683

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
bsolo 3.2? 1798.01 1798.61

General information on the benchmark

Name/DEC-SMALLINT-LIN/sroussel/
ShortestPathNG/normalized-NG_K192.opb
MD5SUM83a21ae30c37a390d06f7aafb55d6d81
Bench CategoryDEC-SMALLINT-LIN (no optimisation, small integers, linear constraints)
Best result obtained on this benchmark
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Has Objective FunctionNO
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables22848
Total number of constraints23234
Number of constraints which are clauses23042
Number of constraints which are cardinality constraints (but not clauses)192
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint1
Maximum length of a constraint192
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 193
Number of bits of the biggest sum of numbers8
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.00	c bsolo beta version 3.2 - 23/05/2010 : 1630 GMT
0.00/0.00	c Developed by Vasco Manquinho and José Santos IST/UTL - INESC-ID
0.00/0.00	c type "bsolo -h" for help
0.00/0.00	c Time Limit set via environment variable PBTIMEOUT to 1800
0.00/0.00	c Learning Strategy: Cardinality Constraint Learning
0.00/0.00	c Time Limit set to 1800
0.00/0.00	c Instance file HOME/instance-3707683-1338234233.opb
0.00/0.00	c File size is 1585017 bytes.
0.09/0.12	c Highest Coefficient sum: 193
0.09/0.12	c Parsing was ok!!
0.09/0.12	c Total parsing time: 0.126 s
0.09/0.13	c Var: 22848 Constr: 23234 23042/192/0 Lit: 149530 Watch. Lit: 68930
0.09/0.13	c Obj. Vars: 0 (0 % of total variables)
11.09/11.10	c Pre-processing Time: 11.102 s
11.09/11.10	c    Confl   Vars     Ctrs     Lits  LPC   W.Lits     Max Learnt Conf/s Time
11.09/11.10	c        0  21277    21769   142892    6    65580  195625      0      0 11.10
11.09/11.10	c Switching off lower bounding mode.
1797.99/1798.60	c      100  21277    21877   154987    7    66285  215188    108      8 11.20
1797.99/1798.60	c      250  21277    22051   180928    8    67663  236707    282     21 11.44
1797.99/1798.60	c      475  21277    22337   199404    8    70767  260378    568     39 11.89
1797.99/1798.60	c      813  21277    22777   222585    9    73935  286416   1008     64 12.60
1797.99/1798.60	c     1321  21277    23427   304155   12    78221  315058   1658     96 13.66
1797.99/1798.60	c     2084  21277    24352   408122   16    84908  346564   2583    135 15.39
1797.99/1798.60	c     3230  21277    25788   612419   23    92398  381221   4019    184 17.52
1797.99/1798.60	c     4951  21277    28005   878582   31   107002  419344   6236    219 22.50
1797.99/1798.60	c     7536  21277    31084  1442221   46   125237  461279   9315    263 28.56
1797.99/1798.60	c    11416  21277    35740  2610986   73   147014  507407  13971    294 38.78
1797.99/1798.60	c    17241  21277    42547  4125256   96   179318  558148  20778    294 58.59
1797.99/1798.60	c    25978  21277    52716  7695017  145   224058  613963  30947    274 94.57
1797.99/1798.60	c    39088  21277    68005 12503107  183   296496  675360  46236    248 157.44
1797.99/1798.60	c    58758  21277    91044 19823399  217   389635  742897  69275    209 280.55
1797.99/1798.60	c    88269  21277   125508 36475190  290   537290  817187 103739    178 494.25
1797.99/1798.60	c   132539  21277   176868 57510457  325   766939  898906 155099    134 984.90
1797.99/1798.60	c Reducing formula size due to memory constraints
1797.99/1798.60	c Memory Limit: 1000.00 MB  Used: 1001.39 MB
1797.99/1798.60	c   198950  21277    50871  5821597  114   178697   60755  29102    124 1601.71
1797.99/1798.60	c Resources Exceeded
1797.99/1798.60	c  Rst    Confl    NCB      Dec     Ctrs     Lits Learnt Conf/s Time
1797.99/1798.60	c   17   239345 161896 21774551    50519  6668590  28750    133 1798.00
1797.99/1798.60	s UNKNOWN
1797.99/1798.60	c Total time: 1798 s

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3707683-1338234233/watcher-3707683-1338234233 -o /tmp/evaluation-result-3707683-1338234233/solver-3707683-1338234233 -C 1800 -W 1900 -M 15500 bsolo -l2 -t1800 HOME/instance-3707683-1338234233.opb 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.00 2.00 2.00 3/176 4512
/proc/meminfo: memFree=30323944/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=20440 CPUtime=0 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 377 0 0 0 0 0 0 0 20 0 1 0 289548788 20930560 331 33554432000 134512640 137138936 4293651968 4293650496 134517511 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 5110 331 179 642 0 4079 0

[startup+0.0545809 s]
/proc/loadavg: 2.00 2.00 2.00 3/176 4512
/proc/meminfo: memFree=30323944/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=15552 CPUtime=0.05 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 1256 0 0 0 5 0 0 0 20 0 1 0 289548788 15925248 1170 33554432000 134512640 137138936 4293651968 4293650360 136506715 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 3888 1170 339 642 0 2857 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 15552

[startup+0.100219 s]
/proc/loadavg: 2.00 2.00 2.00 3/176 4512
/proc/meminfo: memFree=30323944/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=16912 CPUtime=0.09 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 1735 0 0 0 9 0 0 0 20 0 1 0 289548788 17317888 1649 33554432000 134512640 137138936 4293651968 4293650496 134517592 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 4228 1649 474 642 0 3197 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 16912

[startup+0.300225 s]
/proc/loadavg: 2.00 2.00 2.00 3/176 4512
/proc/meminfo: memFree=30323944/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=19660 CPUtime=0.29 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 2844 0 0 0 29 0 0 0 20 0 1 0 289548788 20131840 2371 33554432000 134512640 137138936 4293651968 4293376304 134542128 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 4915 2371 184 642 0 4271 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 19660

[startup+0.700206 s]
/proc/loadavg: 2.00 2.00 2.00 3/176 4512
/proc/meminfo: memFree=30323944/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=19660 CPUtime=0.69 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 2844 0 0 0 69 0 0 0 20 0 1 0 289548788 20131840 2371 33554432000 134512640 137138936 4293651968 4293376492 136570369 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 4915 2371 184 642 0 4271 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 19660

[startup+1.50018 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 4513
/proc/meminfo: memFree=30315124/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=19660 CPUtime=1.49 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 2846 0 0 0 149 0 0 0 20 0 1 0 289548788 20131840 2373 33554432000 134512640 137138936 4293651968 4293376492 136570369 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 4915 2373 184 642 0 4271 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 19660

[startup+3.10019 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 4513
/proc/meminfo: memFree=30314860/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=19660 CPUtime=3.09 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 2846 0 0 0 309 0 0 0 20 0 1 0 289548788 20131840 2373 33554432000 134512640 137138936 4293651968 4293376492 136570369 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 4915 2373 184 642 0 4271 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 19660

[startup+6.3002 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 4513
/proc/meminfo: memFree=30314636/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=19660 CPUtime=6.29 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 2847 0 0 0 628 1 0 0 20 0 1 0 289548788 20131840 2374 33554432000 134512640 137138936 4293651968 4293376492 136570369 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 4915 2374 184 642 0 4271 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 19660

[startup+12.7001 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 4513
/proc/meminfo: memFree=30312592/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=21512 CPUtime=12.69 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 3592 0 0 0 1258 11 0 0 20 0 1 0 289548788 22028288 2894 33554432000 134512640 137138936 4293651968 4293650644 136511125 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 5378 2894 192 642 0 4734 0
Current children cumulated CPU time (s) 12.69
Current children cumulated vsize (KiB) 21512

[startup+25.5002 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 4513
/proc/meminfo: memFree=30300432/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=34716 CPUtime=25.49 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 6887 0 0 0 2496 53 0 0 20 0 1 0 289548788 35549184 6189 33554432000 134512640 137138936 4293651968 4293650680 136504212 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 8679 6189 192 642 0 8035 0
Current children cumulated CPU time (s) 25.49
Current children cumulated vsize (KiB) 34716

[startup+51.1002 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 4513
/proc/meminfo: memFree=30269328/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=65804 CPUtime=51.08 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 14636 0 0 0 5013 95 0 0 20 0 1 0 289548788 67383296 13938 33554432000 134512640 137138936 4293651968 4293650896 134542662 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 16451 13938 192 642 0 15807 0
Current children cumulated CPU time (s) 51.08
Current children cumulated vsize (KiB) 65804

[startup+102.306 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 4513
/proc/meminfo: memFree=30206272/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=127516 CPUtime=102.25 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 30100 0 0 0 10083 142 0 0 20 0 1 0 289548788 130576384 29370 33554432000 134512640 137138936 4293651968 4293650824 136501664 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 31879 29370 192 642 0 31235 0
Current children cumulated CPU time (s) 102.25
Current children cumulated vsize (KiB) 127516

[startup+162.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 4514
/proc/meminfo: memFree=30142068/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=190900 CPUtime=162.19 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 45986 0 0 0 16025 194 0 0 20 0 1 0 289548788 195481600 45216 33554432000 134512640 137138936 4293651968 4293651072 134543427 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 47725 45216 192 642 0 47081 0
Current children cumulated CPU time (s) 162.19
Current children cumulated vsize (KiB) 190900

[startup+222.3 s]
/proc/loadavg: 2.07 2.02 2.00 3/177 4552
/proc/meminfo: memFree=31100940/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=233192 CPUtime=222.17 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 56572 0 0 0 21978 239 0 0 20 0 1 0 289548788 238788608 55789 33554432000 134512640 137138936 4293651968 4293650896 134542725 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 58298 55789 192 642 0 57654 0
Current children cumulated CPU time (s) 222.17
Current children cumulated vsize (KiB) 233192

[startup+282.3 s]
/proc/loadavg: 2.02 2.01 2.00 3/177 4552
/proc/meminfo: memFree=30971488/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=282616 CPUtime=282.16 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 68936 0 0 0 27942 274 0 0 20 0 1 0 289548788 289398784 68153 33554432000 134512640 137138936 4293651968 4293650896 134542325 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 70654 68153 192 642 0 70010 0
Current children cumulated CPU time (s) 282.16
Current children cumulated vsize (KiB) 282616

[startup+342.3 s]
/proc/loadavg: 2.01 2.00 2.00 3/177 4552
/proc/meminfo: memFree=30841844/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=343072 CPUtime=342.15 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 84043 0 0 0 33907 308 0 0 20 0 1 0 289548788 351305728 83259 33554432000 134512640 137138936 4293651968 4293650896 134542516 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 85768 83259 192 642 0 85124 0
Current children cumulated CPU time (s) 342.15
Current children cumulated vsize (KiB) 343072

[startup+402.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/178 4555
/proc/meminfo: memFree=30727184/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=395828 CPUtime=402.1 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 97278 0 0 0 39876 334 0 0 20 0 1 0 289548788 405327872 96448 33554432000 134512640 137138936 4293651968 4293650896 134542316 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 98957 96448 192 642 0 98313 0
Current children cumulated CPU time (s) 402.1
Current children cumulated vsize (KiB) 395828

[startup+462.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 4556
/proc/meminfo: memFree=30584372/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=469124 CPUtime=462.05 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 115662 0 0 0 45851 354 0 0 20 0 1 0 289548788 480382976 114786 33554432000 134512640 137138936 4293651968 4293650896 134542495 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 117281 114786 192 642 0 116637 0
Current children cumulated CPU time (s) 462.05
Current children cumulated vsize (KiB) 469124

[startup+522.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 4556
/proc/meminfo: memFree=30494748/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=511308 CPUtime=522.02 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 126238 0 0 0 51829 373 0 0 20 0 1 0 289548788 523579392 125319 33554432000 134512640 137138936 4293651968 4293650896 134542516 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 127827 125319 192 642 0 127183 0
Current children cumulated CPU time (s) 522.02
Current children cumulated vsize (KiB) 511308

[startup+582.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 4556
/proc/meminfo: memFree=30404996/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=541576 CPUtime=582.01 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 133814 0 0 0 57809 392 0 0 20 0 1 0 289548788 554573824 132888 33554432000 134512640 137138936 4293651968 4293650820 136501667 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 135394 132888 192 642 0 134750 0
Current children cumulated CPU time (s) 582.01
Current children cumulated vsize (KiB) 541576

[startup+642.3 s]

################
# More data... #
################

/proc/meminfo: memFree=29482832/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=848356 CPUtime=1121.93 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 210510 0 0 0 111646 547 0 0 20 0 1 0 289548788 868716544 209584 33554432000 134512640 137138936 4293651968 4293650896 134542316 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 212089 209584 192 642 0 211445 0
Current children cumulated CPU time (s) 1121.93
Current children cumulated vsize (KiB) 848356

[startup+1182.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 4584
/proc/meminfo: memFree=29449900/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=881224 CPUtime=1181.92 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 218727 0 0 0 117628 564 0 0 20 0 1 0 289548788 902373376 217801 33554432000 134512640 137138936 4293651968 4293651084 136570369 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 220306 217801 192 642 0 219662 0
Current children cumulated CPU time (s) 1181.92
Current children cumulated vsize (KiB) 881224

[startup+1242.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 4584
/proc/meminfo: memFree=29414188/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=916932 CPUtime=1241.91 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 227654 0 0 0 123617 574 0 0 20 0 1 0 289548788 938938368 226728 33554432000 134512640 137138936 4293651968 4293650392 134606847 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 229233 226728 192 642 0 228589 0
Current children cumulated CPU time (s) 1241.91
Current children cumulated vsize (KiB) 916932

[startup+1302.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 4584
/proc/meminfo: memFree=29379220/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=952516 CPUtime=1301.9 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 236550 0 0 0 129601 589 0 0 20 0 1 0 289548788 975376384 235624 33554432000 134512640 137138936 4293651968 4293650896 134542199 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 238129 235624 192 642 0 237485 0
Current children cumulated CPU time (s) 1301.9
Current children cumulated vsize (KiB) 952516

[startup+1362.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 4585
/proc/meminfo: memFree=29348344/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=985112 CPUtime=1361.89 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 244709 0 0 0 135586 603 0 0 20 0 1 0 289548788 1008754688 243783 33554432000 134512640 137138936 4293651968 4293651084 136570369 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 246278 243783 192 642 0 245634 0
Current children cumulated CPU time (s) 1361.89
Current children cumulated vsize (KiB) 985112

[startup+1422.31 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 4585
/proc/meminfo: memFree=29305564/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=1026708 CPUtime=1421.89 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 255098 0 0 0 141573 616 0 0 20 0 1 0 289548788 1051348992 254172 33554432000 134512640 137138936 4293651968 4293650896 134542187 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 256677 254172 192 642 0 256033 0
Current children cumulated CPU time (s) 1421.89
Current children cumulated vsize (KiB) 1026708

[startup+1482.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 4585
/proc/meminfo: memFree=29292048/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=1039900 CPUtime=1481.87 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 258589 0 0 0 147526 661 0 0 20 0 1 0 289548788 1064857600 257470 33554432000 134512640 137138936 4293651968 4293650648 136501664 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 259975 257470 192 642 0 259331 0
Current children cumulated CPU time (s) 1481.87
Current children cumulated vsize (KiB) 1039900

[startup+1542.3 s]
/proc/loadavg: 2.04 2.01 2.00 3/177 4585
/proc/meminfo: memFree=29291308/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=1039900 CPUtime=1541.85 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 258614 0 0 0 153477 708 0 0 20 0 1 0 289548788 1064857600 257470 33554432000 134512640 137138936 4293651968 4293650896 134542233 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 259975 257470 192 642 0 259331 0
Current children cumulated CPU time (s) 1541.85
Current children cumulated vsize (KiB) 1039900

[startup+1602.3 s]
/proc/loadavg: 2.01 2.01 2.00 3/178 4588
/proc/meminfo: memFree=29291480/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=1039900 CPUtime=1601.81 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 258653 0 0 0 159428 753 0 0 20 0 1 0 289548788 1064857600 257470 33554432000 134512640 137138936 4293651968 4293651084 136570369 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 259975 257470 192 642 0 259331 0
Current children cumulated CPU time (s) 1601.81
Current children cumulated vsize (KiB) 1039900

[startup+1662.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 4589
/proc/meminfo: memFree=29291516/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=1039900 CPUtime=1661.77 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 258695 0 0 0 165367 810 0 0 20 0 1 0 289548788 1064857600 257470 33554432000 134512640 137138936 4293651968 4293650392 134606987 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 259975 257470 192 642 0 259331 0
Current children cumulated CPU time (s) 1661.77
Current children cumulated vsize (KiB) 1039900

[startup+1722.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 4589
/proc/meminfo: memFree=29291296/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=1039900 CPUtime=1721.73 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 258729 0 0 0 171340 833 0 0 20 0 1 0 289548788 1064857600 257470 33554432000 134512640 137138936 4293651968 4293650896 134542295 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 259975 257470 192 642 0 259331 0
Current children cumulated CPU time (s) 1721.73
Current children cumulated vsize (KiB) 1039900

[startup+1782.3 s]
/proc/loadavg: 2.03 2.01 2.00 3/177 4589
/proc/meminfo: memFree=29291696/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=1039900 CPUtime=1781.7 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 258768 0 0 0 177319 851 0 0 20 0 1 0 289548788 1064857600 257470 33554432000 134512640 137138936 4293651968 4293650720 134578220 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 259975 257470 192 642 0 259331 0
Current children cumulated CPU time (s) 1781.7
Current children cumulated vsize (KiB) 1039900

Solver just ended. Dumping a history of the last processes samples

[startup+1791.91 s]
/proc/loadavg: 2.02 2.01 2.00 3/177 4589
/proc/meminfo: memFree=29291804/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=1039900 CPUtime=1791.3 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 258776 0 0 0 178271 859 0 0 20 0 1 0 289548788 1064857600 257470 33554432000 134512640 137138936 4293651968 4293651084 136570369 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 259975 257470 192 642 0 259331 0
Current children cumulated CPU time (s) 1791.3
Current children cumulated vsize (KiB) 1039900

[startup+1795.1 s]
/proc/loadavg: 2.02 2.01 2.00 3/177 4589
/proc/meminfo: memFree=29291804/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=1039900 CPUtime=1794.5 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 258776 0 0 0 178586 864 0 0 20 0 1 0 289548788 1064857600 257470 33554432000 134512640 137138936 4293651968 4293650720 134578403 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 259975 257470 192 642 0 259331 0
Current children cumulated CPU time (s) 1794.5
Current children cumulated vsize (KiB) 1039900

[startup+1796.7 s]
/proc/loadavg: 2.02 2.01 2.00 3/177 4589
/proc/meminfo: memFree=29291928/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=1039900 CPUtime=1796.09 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 258776 0 0 0 178743 866 0 0 20 0 1 0 289548788 1064857600 257470 33554432000 134512640 137138936 4293651968 4293650720 134576176 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 259975 257470 192 642 0 259331 0
Current children cumulated CPU time (s) 1796.09
Current children cumulated vsize (KiB) 1039900

[startup+1797.5 s]
/proc/loadavg: 2.02 2.01 2.00 3/177 4589
/proc/meminfo: memFree=29291928/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=1039900 CPUtime=1796.89 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 258776 0 0 0 178822 867 0 0 20 0 1 0 289548788 1064857600 257470 33554432000 134512640 137138936 4293651968 4293651084 136570369 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 259975 257470 192 642 0 259331 0
Current children cumulated CPU time (s) 1796.89
Current children cumulated vsize (KiB) 1039900

[startup+1798.3 s]
/proc/loadavg: 2.02 2.01 2.00 3/177 4589
/proc/meminfo: memFree=29291928/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=1039900 CPUtime=1797.69 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 258776 0 0 0 178901 868 0 0 20 0 1 0 289548788 1064857600 257470 33554432000 134512640 137138936 4293651968 4293650896 134542516 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 259975 257470 192 642 0 259331 0
Current children cumulated CPU time (s) 1797.69
Current children cumulated vsize (KiB) 1039900

[startup+1798.51 s]
/proc/loadavg: 2.02 2.01 2.00 3/177 4589
/proc/meminfo: memFree=29291928/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=1039900 CPUtime=1797.9 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 258776 0 0 0 178922 868 0 0 20 0 1 0 289548788 1064857600 257470 33554432000 134512640 137138936 4293651968 4293651072 134543496 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 259975 257470 192 642 0 259331 0
Current children cumulated CPU time (s) 1797.9
Current children cumulated vsize (KiB) 1039900

[startup+1798.6 s]
/proc/loadavg: 2.02 2.01 2.00 3/177 4589
/proc/meminfo: memFree=29291928/32873648 swapFree=7116/7116
[pid=4512] ppid=4510 vsize=1039900 CPUtime=1797.99 cores=1,3,5,7
/proc/4512/stat : 4512 (bsolo) R 4510 4512 4099 0 -1 4202496 258776 0 0 0 178931 868 0 0 20 0 1 0 289548788 1064857600 257470 33554432000 134512640 137138936 4293651968 4293651084 136570369 0 0 4096 16384 0 0 0 17 5 0 0 0 0 0
/proc/4512/statm: 259975 257470 192 642 0 259331 0
Current children cumulated CPU time (s) 1797.99
Current children cumulated vsize (KiB) 1039900

Child status: 0
Real time (s): 1798.61
CPU time (s): 1798.01
CPU user time (s): 1789.32
CPU system time (s): 8.69368
CPU usage (%): 99.9667
Max. virtual memory (cumulated for all children) (KiB): 1040672

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1789.32
system time used= 8.69368
maximum resident set size= 1030652
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 258776
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 427
involuntary context switches= 1834

runsolver used 4.93925 second user time and 18.5462 second system time

The end

Launcher Data 

Begin job on node123 at 2012-05-28 21:43:53
IDJOB=3707683
IDBENCH=90163
IDSOLVER=2286
FILE ID=node123/3707683-1338234233
RUNJOBID= node123-1338227645-4116
PBS_JOBID= 14624628
Free space on /tmp= 71676 MiB

SOLVER NAME= bsolo 3.2
BENCH NAME= PB12/normalized-PB12/DEC-SMALLINT-LIN/sroussel/ShortestPathNG/normalized-NG_K192.opb
COMMAND LINE= bsolo -l2 -tTIMEOUT BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3707683-1338234233/watcher-3707683-1338234233 -o /tmp/evaluation-result-3707683-1338234233/solver-3707683-1338234233 -C 1800 -W 1900 -M 15500  bsolo -l2 -t1800 HOME/instance-3707683-1338234233.opb

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 83a21ae30c37a390d06f7aafb55d6d81
RANDOM SEED=1290942900

node123.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.217
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.43
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.217
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.217
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.217
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.217
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.217
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.217
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.217
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873648 kB
MemFree:        30324704 kB
Buffers:          368072 kB
Cached:           298608 kB
SwapCached:          972 kB
Active:          1381700 kB
Inactive:         323256 kB
Active(anon):    1030160 kB
Inactive(anon):     8356 kB
Active(file):     351540 kB
Inactive(file):   314900 kB
Unevictable:        7116 kB
Mlocked:            7116 kB
SwapTotal:      67108856 kB
SwapFree:       67102960 kB
Dirty:              5444 kB
Writeback:             0 kB
AnonPages:       1044364 kB
Mapped:            11032 kB
Shmem:                 0 kB
Slab:             702800 kB
SReclaimable:     146920 kB
SUnreclaim:       555880 kB
KernelStack:        1456 kB
PageTables:         5992 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545680 kB
Committed_AS:    1162744 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346400 kB
VmallocChunk:   34341988052 kB
HardwareCorrupted:     0 kB
AnonHugePages:    950272 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7680 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 71672 MiB
End job on node123 at 2012-05-28 22:13:52