Trace number 3742347

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
PB07: bsolo 3.0.17? (TO) 1800.08 1800.57

General information on the benchmark

Name/DEC-SMALLINT-LIN/sroussel/
ShortestPathNG/normalized-NG_K192.opb
MD5SUM83a21ae30c37a390d06f7aafb55d6d81
Bench CategoryDEC-SMALLINT-LIN (no optimisation, small integers, linear constraints)
Best result obtained on this benchmark
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Has Objective FunctionNO
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables22848
Total number of constraints23234
Number of constraints which are clauses23042
Number of constraints which are cardinality constraints (but not clauses)192
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint1
Maximum length of a constraint192
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 193
Number of bits of the biggest sum of numbers8
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.01	c bsolo beta version 3.0.17 - 27/04/2007 : 1600 GMT
0.00/0.01	c Developed by Vasco Manquinho IST/UTL - INESC-ID
0.00/0.01	c type "bsolo -h" for help
0.00/0.01	c Time Limit set via environment variable PBTIMEOUT to 1800
0.00/0.01	c Memory Limit set to 15500 MB
0.00/0.01	c Instance file HOME/instance-3742347-1338770203.opb
0.00/0.01	c File size is 1585017 bytes.
0.08/0.15	c Highest Coefficient sum: 193
0.08/0.15	c Parsing was ok!!
0.08/0.15	c Total parsing time: 0.14 s
0.08/0.16	c Var: 22848 Constr: 23234 23042/192/0 Lit: 149530 Watch. Lit: 68930
0.08/0.16	c Obj. Vars: 0 (0 % of total variables)
10.28/10.31	c Pre-processing Time: 10.299 s
10.28/10.31	c    Confl   Vars     Ctrs     Lits  LPC   W.Lits     Max Learnt Conf/s Time
10.28/10.31	c        0  21277    21769   142892    6    65580  195625      0      0 10.30
10.28/10.31	c Switching off LPR mode.
1800.01/1800.50	c      104  21277    21873   153394    7    65788  215188    104     10 10.34
1800.01/1800.50	c      254  21277    22023   185354    8    66088  236707    254     24 10.44
1800.01/1800.50	c      479  21277    22248   202750    9    66538  260378    479     44 10.66
1800.01/1800.50	c      817  21277    22586   226473   10    67214  286416    817     73 11.06
1800.01/1800.50	c     1327  21277    23096   283426   12    68234  315058   1327    113 11.69
1800.01/1800.50	c     2090  21277    23859   338618   14    69760  346564   2090    165 12.61
1800.01/1800.50	c     3236  21277    25005   494932   19    72052  381221   3236    232 13.89
1800.01/1800.50	c     4957  21277    26726   658195   24    75494  419344   4957    313 15.81
1800.01/1800.50	c     7541  21277    29310   897271   30    80662  461279   7541    387 19.48
1800.01/1800.50	c    11420  21277    33189  1237171   37    88420  507407  11420    439 25.97
1800.01/1800.50	c    17242  21277    39011  2195233   56   100064  558148  17242    507 33.95
1800.01/1800.50	c    25979  21277    47748  4726117   98   117538  613963  25979    546 47.54
1800.01/1800.50	c    39089  21277    60858  7720004  126   143758  675360  39089    510 76.52
1800.01/1800.50	c    58760  21277    80529 12566621  156   183100  742897  58760    454 129.41
1800.01/1800.50	c    88270  21277   110039 24433500  222   242120  817187  88270    410 214.87
1800.01/1800.50	c   132540  21277   154309 39139583  253   330660  898906 132540    321 412.24
1800.01/1800.50	c   198955  21277   220724 70567119  319   463490  988797 198955    252 787.89
1800.01/1800.50	c   298575  21277   320344 117287606  366   662730  500000 298575    190 1563.57
1800.01/1800.50	c Caught signal: 15
1800.01/1800.50	c  Rst    Confl    NCB      Dec     Ctrs     Lits Learnt Conf/s Time
1800.01/1800.50	c   18   319487 204610 26535964   341256 125537095 319487    177 1800.02
1800.01/1800.50	s UNKNOWN
1800.01/1800.50	c Total time: 1800.02 s

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3742347-1338770203/watcher-3742347-1338770203 -o /tmp/evaluation-result-3742347-1338770203/solver-3742347-1338770203 -C 1800 -W 1900 -M 15500 bsolo3.0.17 -m15500 HOME/instance-3742347-1338770203.opb 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.96 2.00 2.00 3/176 13502
/proc/meminfo: memFree=24688820/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=11812 CPUtime=0 cores=0,2,4,6
/proc/13502/stat : 13502 (runsolver) D 13500 13502 11604 0 -1 4202560 40 0 0 0 0 0 0 0 20 0 1 0 343146264 12095488 51 33554432000 4194304 5436572 140733358540304 140733358536600 4883175 0 0 4096 24578 18446744071579943789 0 0 17 2 0 0 0 0 0
/proc/13502/statm: 2953 51 31 304 0 2645 0

[startup+0.0143961 s]
/proc/loadavg: 1.96 2.00 2.00 3/176 13502
/proc/meminfo: memFree=24688820/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=14052 CPUtime=0 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 486 0 0 0 0 0 0 0 20 0 1 0 343146264 14389248 423 33554432000 134512640 135840783 4288628592 4288625024 10978599 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 3513 424 327 325 0 2062 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 14052

[startup+0.100145 s]
/proc/loadavg: 1.96 2.00 2.00 3/176 13502
/proc/meminfo: memFree=24688820/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=10264 CPUtime=0.08 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 1740 0 0 0 8 0 0 0 20 0 1 0 343146264 10510336 1637 33554432000 134512640 135840783 4288628592 4288627280 10978599 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 2566 1637 572 325 0 1115 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 10264

[startup+0.300205 s]
/proc/loadavg: 1.96 2.00 2.00 3/176 13502
/proc/meminfo: memFree=24688820/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=13372 CPUtime=0.28 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 2990 0 0 0 28 0 0 0 20 0 1 0 343146264 13692928 2500 33554432000 134512640 135840783 4288628592 4288353108 10965258 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 3343 2500 342 325 0 2279 0
Current children cumulated CPU time (s) 0.28
Current children cumulated vsize (KiB) 13372

[startup+0.700125 s]
/proc/loadavg: 1.96 2.00 2.00 3/176 13502
/proc/meminfo: memFree=24688820/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=13372 CPUtime=0.68 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 2990 0 0 0 68 0 0 0 20 0 1 0 343146264 13692928 2500 33554432000 134512640 135840783 4288628592 4288353392 134551326 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 3343 2500 342 325 0 2279 0
Current children cumulated CPU time (s) 0.68
Current children cumulated vsize (KiB) 13372

[startup+1.50012 s]
/proc/loadavg: 1.96 2.00 2.00 3/177 13503
/proc/meminfo: memFree=24679116/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=13372 CPUtime=1.48 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 2991 0 0 0 148 0 0 0 20 0 1 0 343146264 13692928 2501 33554432000 134512640 135840783 4288628592 4288353552 134552003 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 3343 2501 342 325 0 2279 0
Current children cumulated CPU time (s) 1.48
Current children cumulated vsize (KiB) 13372

[startup+3.10012 s]
/proc/loadavg: 1.96 2.00 2.00 3/177 13503
/proc/meminfo: memFree=24675520/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=13372 CPUtime=3.08 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 2991 0 0 0 308 0 0 0 20 0 1 0 343146264 13692928 2501 33554432000 134512640 135840783 4288628592 4288353108 10965346 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 3343 2501 342 325 0 2279 0
Current children cumulated CPU time (s) 3.08
Current children cumulated vsize (KiB) 13372

[startup+6.30024 s]
/proc/loadavg: 1.96 2.00 2.00 3/177 13503
/proc/meminfo: memFree=24672428/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=13372 CPUtime=6.28 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 2992 0 0 0 627 1 0 0 20 0 1 0 343146264 13692928 2502 33554432000 134512640 135840783 4288628592 4288353532 5342256 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 3343 2502 342 325 0 2279 0
Current children cumulated CPU time (s) 6.28
Current children cumulated vsize (KiB) 13372

[startup+12.7003 s]
/proc/loadavg: 1.96 2.00 2.00 3/177 13503
/proc/meminfo: memFree=24664888/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=16172 CPUtime=12.68 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 3964 0 0 0 1266 2 0 0 20 0 1 0 343146264 16560128 3249 33554432000 134512640 135840783 4288628592 4288627952 134550631 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 4043 3249 347 325 0 2979 0
Current children cumulated CPU time (s) 12.68
Current children cumulated vsize (KiB) 16172

[startup+25.5002 s]
/proc/loadavg: 1.97 2.00 2.00 3/177 13503
/proc/meminfo: memFree=24641684/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=28356 CPUtime=25.48 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 7002 0 0 0 2545 3 0 0 20 0 1 0 343146264 29036544 6287 33554432000 134512640 135840783 4288628592 4288626080 134575118 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 7089 6287 347 325 0 6025 0
Current children cumulated CPU time (s) 25.48
Current children cumulated vsize (KiB) 28356

[startup+51.1002 s]
/proc/loadavg: 1.98 2.00 2.00 3/177 13503
/proc/meminfo: memFree=24564832/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=79344 CPUtime=51.07 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 19761 0 0 0 5103 4 0 0 20 0 1 0 343146264 81248256 19046 33554432000 134512640 135840783 4288628592 4288627904 134550872 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 19836 19046 347 325 0 18772 0
Current children cumulated CPU time (s) 51.07
Current children cumulated vsize (KiB) 79344

[startup+102.306 s]
/proc/loadavg: 1.99 2.00 2.00 3/177 13503
/proc/meminfo: memFree=24432068/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=142796 CPUtime=102.27 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 35607 0 0 0 10218 9 0 0 20 0 1 0 343146264 146223104 34892 33554432000 134512640 135840783 4288628592 4288627952 134551084 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 35699 34892 347 325 0 34635 0
Current children cumulated CPU time (s) 102.27
Current children cumulated vsize (KiB) 142796

[startup+162.3 s]
/proc/loadavg: 1.95 1.98 1.99 3/177 13542
/proc/meminfo: memFree=27231900/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=224856 CPUtime=162.23 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 56154 0 0 0 16208 15 0 0 20 0 1 0 343146264 230252544 55416 33554432000 134512640 135840783 4288628592 4288627944 1419469 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 56214 55416 347 325 0 55150 0
Current children cumulated CPU time (s) 162.23
Current children cumulated vsize (KiB) 224856

[startup+222.3 s]
/proc/loadavg: 1.98 1.98 1.99 3/178 13546
/proc/meminfo: memFree=27021468/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=341188 CPUtime=222.19 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 85304 0 0 0 22198 21 0 0 20 0 1 0 343146264 349376512 84521 33554432000 134512640 135840783 4288628592 4288627892 10983969 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 85297 84521 347 325 0 84233 0
Current children cumulated CPU time (s) 222.19
Current children cumulated vsize (KiB) 341188

[startup+282.3 s]
/proc/loadavg: 1.99 1.98 1.99 3/177 13546
/proc/meminfo: memFree=26885476/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=404536 CPUtime=282.15 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 101156 0 0 0 28190 25 0 0 20 0 1 0 343146264 414244864 100331 33554432000 134512640 135840783 4288628592 4288627952 134551312 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 101134 100331 347 325 0 100070 0
Current children cumulated CPU time (s) 282.15
Current children cumulated vsize (KiB) 404536

[startup+342.3 s]
/proc/loadavg: 1.99 1.98 1.99 3/177 13546
/proc/meminfo: memFree=26750220/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=456524 CPUtime=342.11 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 114188 0 0 0 34183 28 0 0 20 0 1 0 343146264 467480576 113326 33554432000 134512640 135840783 4288628592 4288627952 134551175 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 114131 113326 347 325 0 113067 0
Current children cumulated CPU time (s) 342.11
Current children cumulated vsize (KiB) 456524

[startup+402.3 s]
/proc/loadavg: 1.99 1.98 1.99 3/177 13546
/proc/meminfo: memFree=26627120/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=514016 CPUtime=402.07 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 128606 0 0 0 40175 32 0 0 20 0 1 0 343146264 526352384 127703 33554432000 134512640 135840783 4288628592 4288625464 1419469 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 128504 127703 347 325 0 127440 0
Current children cumulated CPU time (s) 402.07
Current children cumulated vsize (KiB) 514016

[startup+462.3 s]
/proc/loadavg: 1.99 1.98 1.99 3/177 13546
/proc/meminfo: memFree=26488396/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=583916 CPUtime=462.04 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 146139 0 0 0 46168 36 0 0 20 0 1 0 343146264 597929984 145202 33554432000 134512640 135840783 4288628592 4288627952 134550779 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 145979 145202 347 325 0 144915 0
Current children cumulated CPU time (s) 462.04
Current children cumulated vsize (KiB) 583916

[startup+522.3 s]
/proc/loadavg: 1.99 1.98 1.99 3/177 13547
/proc/meminfo: memFree=26352024/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=661828 CPUtime=522 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 165643 0 0 0 52159 41 0 0 20 0 1 0 343146264 677711872 164668 33554432000 134512640 135840783 4288628592 4288627620 10965346 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 165457 164668 347 325 0 164393 0
Current children cumulated CPU time (s) 522
Current children cumulated vsize (KiB) 661828

[startup+582.3 s]
/proc/loadavg: 1.99 1.98 1.99 3/177 13547
/proc/meminfo: memFree=26194076/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=715536 CPUtime=581.98 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 179084 0 0 0 58154 44 0 0 20 0 1 0 343146264 732708864 178092 33554432000 134512640 135840783 4288628592 4288612848 134586540 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 178884 178092 347 325 0 177820 0
Current children cumulated CPU time (s) 581.98
Current children cumulated vsize (KiB) 715536

[startup+642.3 s]

################
# More data... #
################

[startup+882.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 13551
/proc/meminfo: memFree=25588860/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=1003252 CPUtime=881.94 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 251006 0 0 0 88130 64 0 0 20 0 1 0 343146264 1027330048 250014 33554432000 134512640 135840783 4288628592 4288627952 134550764 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 250813 250014 347 325 0 249749 0
Current children cumulated CPU time (s) 881.94
Current children cumulated vsize (KiB) 1003252

[startup+942.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 13551
/proc/meminfo: memFree=25417784/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=1078452 CPUtime=941.93 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 269809 0 0 0 94125 68 0 0 20 0 1 0 343146264 1104334848 268817 33554432000 134512640 135840783 4288628592 4288627952 134550804 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 269613 268817 347 325 0 268549 0
Current children cumulated CPU time (s) 941.93
Current children cumulated vsize (KiB) 1078452

[startup+1002.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 13551
/proc/meminfo: memFree=25262076/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=1109536 CPUtime=1001.92 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 277582 0 0 0 100122 70 0 0 20 0 1 0 343146264 1136164864 276590 33554432000 134512640 135840783 4288628592 4288627952 134550904 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 277384 276590 347 325 0 276320 0
Current children cumulated CPU time (s) 1001.92
Current children cumulated vsize (KiB) 1109536

[startup+1062.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 13551
/proc/meminfo: memFree=25143064/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=1159000 CPUtime=1061.92 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 289962 0 0 0 106119 73 0 0 20 0 1 0 343146264 1186816000 288970 33554432000 134512640 135840783 4288628592 4288627904 134551084 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 289750 288970 347 325 0 288686 0
Current children cumulated CPU time (s) 1061.92
Current children cumulated vsize (KiB) 1159000

[startup+1122.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 13552
/proc/meminfo: memFree=25021704/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=1227460 CPUtime=1121.9 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 307080 0 0 0 112114 76 0 0 20 0 1 0 343146264 1256919040 306088 33554432000 134512640 135840783 4288628592 4288627952 134550893 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 306865 306088 347 325 0 305801 0
Current children cumulated CPU time (s) 1121.9
Current children cumulated vsize (KiB) 1227460

[startup+1182.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 13552
/proc/meminfo: memFree=24930404/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=1263208 CPUtime=1181.9 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 316019 0 0 0 118111 79 0 0 20 0 1 0 343146264 1293524992 315027 33554432000 134512640 135840783 4288628592 4288627952 134550779 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 315802 315027 347 325 0 314738 0
Current children cumulated CPU time (s) 1181.9
Current children cumulated vsize (KiB) 1263208

[startup+1242.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 13552
/proc/meminfo: memFree=24849844/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=1304292 CPUtime=1241.89 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 326266 0 0 0 124107 82 0 0 20 0 1 0 343146264 1335595008 325274 33554432000 134512640 135840783 4288628592 4288627952 134551321 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 326073 325274 347 325 0 325009 0
Current children cumulated CPU time (s) 1241.89
Current children cumulated vsize (KiB) 1304292

[startup+1302.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 13552
/proc/meminfo: memFree=24777100/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=1338524 CPUtime=1301.89 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 334848 0 0 0 130105 84 0 0 20 0 1 0 343146264 1370648576 333856 33554432000 134512640 135840783 4288628592 4288627952 134551052 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 334631 333856 347 325 0 333567 0
Current children cumulated CPU time (s) 1301.89
Current children cumulated vsize (KiB) 1338524

[startup+1362.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 13552
/proc/meminfo: memFree=24705340/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=1388324 CPUtime=1361.87 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 347289 0 0 0 136100 87 0 0 20 0 1 0 343146264 1421643776 346297 33554432000 134512640 135840783 4288628592 4288627952 134550893 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 347081 346297 347 325 0 346017 0
Current children cumulated CPU time (s) 1361.87
Current children cumulated vsize (KiB) 1388324

[startup+1422.31 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 13555
/proc/meminfo: memFree=24605680/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=1458480 CPUtime=1421.86 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 364817 0 0 0 142096 90 0 0 20 0 1 0 343146264 1493483520 363825 33554432000 134512640 135840783 4288628592 4288627952 134550804 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 364620 363825 347 325 0 363556 0
Current children cumulated CPU time (s) 1421.86
Current children cumulated vsize (KiB) 1458480

[startup+1482.3 s]
/proc/loadavg: 2.03 2.01 2.00 3/179 13578
/proc/meminfo: memFree=24547592/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=1486832 CPUtime=1481.85 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 371896 0 0 0 148093 92 0 0 20 0 1 0 343146264 1522515968 370904 33554432000 134512640 135840783 4288628592 4288627952 134550902 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 371708 370904 347 325 0 370644 0
Current children cumulated CPU time (s) 1481.85
Current children cumulated vsize (KiB) 1486832

[startup+1542.3 s]
/proc/loadavg: 2.01 2.00 2.00 3/178 13578
/proc/meminfo: memFree=24475832/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=1523988 CPUtime=1541.84 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 381199 0 0 0 154088 96 0 0 20 0 1 0 343146264 1560563712 380207 33554432000 134512640 135840783 4288628592 4288627952 134551048 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 380997 380207 347 325 0 379933 0
Current children cumulated CPU time (s) 1541.84
Current children cumulated vsize (KiB) 1523988

[startup+1602.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/178 13578
/proc/meminfo: memFree=24397092/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=1558096 CPUtime=1601.83 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 389737 0 0 0 160085 98 0 0 20 0 1 0 343146264 1595490304 388745 33554432000 134512640 135840783 4288628592 4288627952 134551330 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 389524 388745 347 325 0 388460 0
Current children cumulated CPU time (s) 1601.83
Current children cumulated vsize (KiB) 1558096

[startup+1662.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/178 13578
/proc/meminfo: memFree=24318728/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=1594576 CPUtime=1661.82 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 398861 0 0 0 166081 101 0 0 20 0 1 0 343146264 1632845824 397869 33554432000 134512640 135840783 4288628592 4288627952 134550779 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 398644 397869 347 325 0 397580 0
Current children cumulated CPU time (s) 1661.82
Current children cumulated vsize (KiB) 1594576

[startup+1722.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/178 13578
/proc/meminfo: memFree=24249660/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=1620848 CPUtime=1721.81 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 405429 0 0 0 172078 103 0 0 20 0 1 0 343146264 1659748352 404437 33554432000 134512640 135840783 4288628592 4288627728 10976158 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 405212 404437 347 325 0 404148 0
Current children cumulated CPU time (s) 1721.81
Current children cumulated vsize (KiB) 1620848

[startup+1782.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/178 13579
/proc/meminfo: memFree=24190512/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=1637332 CPUtime=1781.8 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 409520 0 0 0 178076 104 0 0 20 0 1 0 343146264 1676627968 408528 33554432000 134512640 135840783 4288628592 4288627824 134574878 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 409333 408528 347 325 0 408269 0
Current children cumulated CPU time (s) 1781.8
Current children cumulated vsize (KiB) 1637332



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1800.5 s]
/proc/loadavg: 2.00 2.00 2.00 3/178 13579
/proc/meminfo: memFree=24176748/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=1642496 CPUtime=1800.01 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 410819 0 0 0 179896 105 0 0 20 0 1 0 343146264 1681915904 409827 33554432000 134512640 135840783 4288628592 4288627952 134551048 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 410624 409827 347 325 0 409560 0
Current children cumulated CPU time (s) 1800.01
Current children cumulated vsize (KiB) 1642496

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1800.5 s]
/proc/loadavg: 2.00 2.00 2.00 3/178 13579
/proc/meminfo: memFree=24176748/32873648 swapFree=7112/7112
[pid=13502] ppid=13500 vsize=1642496 CPUtime=1800.01 cores=0,2,4,6
/proc/13502/stat : 13502 (bsolo3.0.17) R 13500 13502 11604 0 -1 4202496 410819 0 0 0 179896 105 0 0 20 0 1 0 343146264 1681915904 409827 33554432000 134512640 135840783 4288628592 4288627952 134551048 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/13502/statm: 410624 409827 347 325 0 409560 0
Current children cumulated CPU time (s) 1800.01
Current children cumulated vsize (KiB) 1642496

Child status: 15
Real time (s): 1800.57
CPU time (s): 1800.08
CPU user time (s): 1798.97
CPU system time (s): 1.11383
CPU usage (%): 99.9728
Max. virtual memory (cumulated for all children) (KiB): 1642496

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1798.97
system time used= 1.11383
maximum resident set size= 1639308
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 410822
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 298
involuntary context switches= 1844

runsolver used 3.46047 second user time and 9.03563 second system time

The end

Launcher Data 

Begin job on node126 at 2012-06-04 02:36:44
IDJOB=3742347
IDBENCH=90163
IDSOLVER=2318
FILE ID=node126/3742347-1338770203
RUNJOBID= node126-1338747103-11620
PBS_JOBID= 14636777
Free space on /tmp= 70784 MiB

SOLVER NAME= PB07: bsolo 3.0.17
BENCH NAME= PB12/normalized-PB12/DEC-SMALLINT-LIN/sroussel/ShortestPathNG/normalized-NG_K192.opb
COMMAND LINE= bsolo3.0.17 -mMEMLIMIT BENCHNAME            
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3742347-1338770203/watcher-3742347-1338770203 -o /tmp/evaluation-result-3742347-1338770203/solver-3742347-1338770203 -C 1800 -W 1900 -M 15500  bsolo3.0.17 -m15500 HOME/instance-3742347-1338770203.opb            

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 83a21ae30c37a390d06f7aafb55d6d81
RANDOM SEED=1641214583

node126.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.456
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.456
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.456
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.456
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.456
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.456
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.456
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.456
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873648 kB
MemFree:        24689688 kB
Buffers:          445676 kB
Cached:          4037900 kB
SwapCached:         9484 kB
Active:          4401780 kB
Inactive:        2900228 kB
Active(anon):    2808396 kB
Inactive(anon):    12264 kB
Active(file):    1593384 kB
Inactive(file):  2887964 kB
Unevictable:        7112 kB
Mlocked:            7112 kB
SwapTotal:      67108856 kB
SwapFree:       67095016 kB
Dirty:              8172 kB
Writeback:             0 kB
AnonPages:       2817328 kB
Mapped:            11028 kB
Shmem:                 0 kB
Slab:             738380 kB
SReclaimable:     182796 kB
SUnreclaim:       555584 kB
KernelStack:        1456 kB
PageTables:         9504 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545680 kB
Committed_AS:    2931616 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346400 kB
VmallocChunk:   34341988052 kB
HardwareCorrupted:     0 kB
AnonHugePages:    401408 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7680 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70776 MiB
End job on node126 at 2012-06-04 03:06:46